rate-limiting-explained-with-a-wedding-hall-example

--- title: Rate Limiting Explained with a Wedding Hall Example date: 2026-07-11 category: System Design description: A practical explanation of rate limiting using a wedding dining hall analogy, covering throttling types, placement options, state management, and common algorithms. readTime: 7 min read ---

Rate limiting is easier to understand when we map it to a real-world situation.

Imagine a wedding function where 2,000 guests are invited. The dining hall is ready, the food is served, but the hall can safely handle only 100 guests at a time.

If there is no gatekeeper, a large crowd may rush into the dining area as soon as food is ready. Too many people entering at once can overload the hall, block movement, exhaust food counters, break furniture, and create a poor experience for everyone.

Now imagine there are one or more gatekeepers at the entrance. They allow only 100 guests into the dining hall for each round. Everyone inside can eat peacefully, use the available resources properly, and leave before the next group enters.

That is the basic idea behind rate limiting.

Mapping the wedding example to software systems

In software, the dining hall is your service or API. The guests are incoming requests. The hall capacity is the maximum number of requests your system can safely process within a specific time window. The gatekeeper is the rate limiter.

A rate limiter controls how many requests are allowed to reach a service within a defined time period. For example, if an API allows 500 requests per minute, the rate limiter permits requests up to that limit and blocks or delays the extra requests.

Without rate limiting, too many requests can overload the service, consume system resources, increase latency, and cause failures for legitimate users.

Why rate limiting is needed

Rate limiting helps protect systems from traffic spikes and abusive behavior.

It is commonly used to:

• Protect APIs from denial-of-service attacks.
• Reduce brute-force login attempts.
• Prevent one client from consuming all resources.
• Keep backend services stable during traffic bursts.
• Enforce fair usage across users, tenants, or applications.
• Control cost when downstream services are expensive.

The goal is not always to reject users. The goal is to keep the system healthy and predictable.

Hard, soft, and elastic throttling

There are different ways to enforce limits depending on the requirement.

Hard throttling is strict. If the limit is 500 requests per minute, request number 501 is rejected. This is useful when the system must not exceed a fixed capacity.

Soft throttling allows a small buffer beyond the configured limit. For example, if the limit is 500 requests per minute and the buffer is 5 percent, the system may allow up to 525 requests. This is useful when small bursts are acceptable.

Elastic or dynamic throttling allows extra requests when the system has available capacity. Instead of using a fixed upper buffer, the system makes decisions based on current resource usage such as CPU, memory, queue depth, or downstream availability.

In the wedding example, hard throttling means exactly 100 guests enter per round. Soft throttling may allow 105 guests if there is enough space. Elastic throttling may allow more guests only if the hall is not crowded and the food counters can handle it.

Handling priority users

Sometimes not all requests are equal.

In the wedding example, there may be VIP guests. We may reserve part of the dining hall capacity for them. For example:

• 95 percent of capacity for normal guests.
• 5 percent of capacity for VIP guests.

If there are no VIP guests waiting, the system can temporarily use that unused quota for normal guests.

The same idea applies in software. A rate limiter can apply different rules based on user type, subscription plan, API key, tenant, region, or endpoint. Premium customers may get a higher limit. Internal services may get a reserved quota. Public endpoints may get stricter limits.

This is why rate limiting should usually be based on identity, not only on the global system limit.

Where to place the rate limiter

A rate limiter can be placed in different parts of the system.

Client-side rate limiting can reduce unnecessary requests before they leave the client application. However, it cannot be fully trusted because clients can be modified or bypassed.

Server-side rate limiting runs inside the backend service. It gives the service direct control, but every request still reaches the application before the limit is checked.

Middleware or gateway-level rate limiting is usually placed before requests reach the application servers. This can be implemented in an API gateway, reverse proxy, load balancer, edge service, or dedicated middleware layer.

The best placement depends on the organization, architecture, traffic pattern, and security requirement. In many systems, rate limiting is applied at multiple layers.

Managing rate limit state

Rate limiters need to track counters. For example, how many requests has this user made in the current minute?

When traffic is small, this can be simple. But at large scale, one rate limiter cannot handle millions of users alone. We usually need multiple rate limiter nodes. This creates an important design question: where should the request counters live?

One approach is to use a centralized store such as Redis or a database. All rate limiter nodes read and update the same counters. This gives better consistency, but it adds network latency and can create contention under high traffic.

Another approach is to let each node track counters locally or use a distributed store. This is faster, but it may be less accurate because different nodes may temporarily have different views of the same user's usage.

There is always a tradeoff between accuracy, latency, availability, and operational complexity.

High-level rate limiter flow

A typical rate limiter design may include these components:

• Rule database: Stores rate limit rules defined by the service owner.
• Rules retriever: A background process that reads rule changes and updates cache.
• Cache: Keeps frequently used rules close to the decision layer.
• Client identifier: Extracts a key such as user ID, API key, IP address, or tenant ID.
• Decision maker: Applies the selected rate limiting algorithm.
• Request processor: Handles the request if the rate limiter allows it.

When a request arrives, the system identifies the client, reads the applicable rule, checks the current usage, and decides whether to allow, reject, delay, or queue the request.

For hard throttling, the decision maker rejects requests immediately after the limit is reached. For soft or elastic throttling, it may allow some additional requests or queue them for later processing.

In distributed systems, synchronizing counters and rules can be difficult. Some systems update counters synchronously for accuracy. Others return the response quickly and update counters asynchronously for lower latency. The right choice depends on how strict the limit needs to be.

Common rate limiting algorithms

Token bucket is like prepaid mobile balance. Tokens are added at a fixed rate. Each request consumes a token. If enough tokens are available, the request is allowed. This algorithm handles bursts well because tokens can accumulate up to a configured bucket size.

Leaky bucket is like an airport security line. Many people may arrive at once, but they are processed at a constant speed. Requests enter a queue and are served steadily. If the queue is full, new requests are rejected.

Fixed window counter divides time into fixed windows, such as one minute. If a user is allowed 500 requests per minute, the counter resets at the start of each minute. The drawback is that users can send many requests near the end of one window and many more at the start of the next window.

Sliding window log stores timestamps for each request and removes old entries outside the time window. It is accurate, but it can consume more memory because each request timestamp must be stored.

Sliding window counter is a more efficient variation. It estimates usage across the current and previous windows, giving smoother behavior than a fixed window while using less storage than a full request log.

Final thoughts

Rate limiting is not just about blocking traffic. It is about protecting system reliability, improving fairness, and making sure one client or traffic spike does not degrade the experience for everyone.

The wedding hall example gives us a simple way to remember the idea: when capacity is limited, a gatekeeper protects the system by controlling how many people enter, who gets priority, and how overflow is handled.

The same thinking applies to APIs and backend services. A good rate limiter understands capacity, identity, priority, and tradeoffs.

If you liked this post, subscribe to the newsletter and let me know what system design topic you would like me to cover next.