What you'll learn

By the end of this you'll understand what rate limiting actually is in a distributed system, why the algorithm you pick matters more than the number you pick, how to implement it correctly with Redis, and what to do when a request is rejected. You'll also have a mental model for where to place rate limiting in your architecture — because putting it in the wrong place is almost as bad as not having it.

This is the design-level explanation, not a library quickstart. The goal is that you can reason through rate limiting in a system design interview or in a production architecture conversation without sounding like you memorized an answer.

Who this is for

  • Backend engineers who've implemented express-rate-limit or similar and want to understand what's actually happening underneath
  • Engineers preparing for system design interviews where rate limiting comes up as a constraint in API design problems
  • Anyone who's had their API show up in someone's abuse report and needed to understand their options

You can skip this if you already understand the difference between token bucket and sliding window counter algorithms and have built a distributed rate limiter in Redis across multiple service instances.

What is rate limiting?

Rate limiting controls how many requests a client can make to your API within a given time window. When a client exceeds the limit, the server rejects the request — typically with HTTP 429 Too Many Requests — instead of processing it.

Plain English: Rate limiting is a bouncer for your API. Each client gets a budget of requests per minute (or hour, or second). Spend the budget and you're turned away until it refills. Legitimate users rarely notice; abusive or buggy clients get stopped before they knock your database over.

Simple idea: count requests per client over time. If the count exceeds the limit, return 429 with a Retry-After header telling the client when to try again.

Prerequisites

  • Basic HTTP knowledge (status codes, headers)
  • Familiarity with Redis or another shared key-value store
  • A running API (any framework) where you can add middleware

Setup from zero

Step 1 — Pick your algorithm

The algorithm determines the shape of what's allowed and what isn't. The three you need to know:

Fixed window counter — count requests in a fixed time window (like per minute). Reset the counter at the start of each window. Simple to implement, fast to check, but has a burst problem: a client can send half their allowed requests at the end of one window and half at the start of the next, doubling their effective rate at the boundary.

Sliding window counter — store request counts for the current and previous fixed windows, then calculate a weighted count based on how far you are into the current window. Much cheaper in storage than a full log while still avoiding the fixed-window burst problem.

Token bucket — each client has a bucket that fills at a steady rate up to a maximum capacity. Each request consumes one token. If the bucket is empty, the request is rejected. Tokens accumulate when the client is idle, allowing natural bursts up to the bucket size.

For most API rate limiting use cases, token bucket or sliding window counter is the right choice.

Step 2 — Decide what to limit on

Rate limiting on IP address is simple but imprecise. One IP might represent many users (corporate NAT) or the same user might appear from different IPs (mobile networks).

Better options depending on your context:

  • Authenticated user ID — the most accurate for APIs that require authentication
  • API key — for public or partner APIs where each key represents an integration
  • Tenant or organization ID — for multi-tenant SaaS where you limit by customer account

In practice you often want multiple layers: a broad IP-based limit to catch unauthenticated abuse, plus a finer user or key-based limit for authenticated traffic.

Step 3 — Implement with Redis and return the right response

A single-server in-memory rate limiter breaks as soon as you run more than one instance of your service. Redis is the standard backing store for distributed rate limiting because it's fast, supports atomic operations, and has native TTL support.

When a request is rate limited, return HTTP 429 Too Many Requests — not 400, not 403. Include informative headers so clients can adapt:

HTTP/1.1 429 Too Many Requests
Retry-After: 45
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1751040000

> Little tip: Use Redis pipelining or Lua scripts when you need multiple Redis operations in a rate-limit check to be atomic. Non-atomic multi-command sequences can admit slightly more traffic than your limit allows under high concurrency.

The mental model

The mental model for rate limiting is budget allocation over time.

Every client gets a budget of requests per time window. The algorithm you choose determines the shape of that budget — whether the client can spend the whole budget in the first second (fixed window burst) or whether spending is smoothed across the window (sliding window, leaky bucket) or whether they can save up unused budget for occasional bursts (token bucket).

The number you set — 100 requests per minute, 1000 per hour — is less important than the shape. A fixed-window limit of 100/minute with a burst vulnerability is a weaker guarantee than a token-bucket limit of 60/minute that prevents bursting. Think about the shape before you think about the number.

Rate limiting is also a circuit breaker signal. When a client consistently hits their limit, that tells you something about their usage pattern — either they have a legitimate need for a higher tier, or they're building something incompatible with your API's design.

Key terms

429 Too Many Requests — the HTTP status code for a rate-limited response.

Token bucket — clients accumulate tokens at a steady rate and spend one per request. Allows natural bursting up to bucket capacity.

Fixed window counter — count requests within a fixed time interval, reset at each boundary. Simple, but vulnerable to burst attacks at window boundaries.

Sliding window — count requests within a rolling time interval rather than a fixed one. More accurate than fixed window.

Leaky bucket — requests are queued and processed at a fixed rate. Smooths bursts into a steady output rate.

Retry-After — HTTP response header telling a rate-limited client how long to wait before retrying.

Distributed rate limiting — rate limiting that works correctly across multiple instances of a service, typically backed by a shared store like Redis.

Burst — a short spike of requests above the sustained rate. Whether bursts are allowed depends on the algorithm you choose.

Step-by-step: token bucket in Redis

A token bucket implementation using a Lua script for atomicity:

-- KEYS[1] = bucket key
-- ARGV[1] = max tokens (bucket capacity)
-- ARGV[2] = refill rate (tokens per second)
-- ARGV[3] = current timestamp

local bucket = redis.call("HMGET", KEYS[1], "tokens", "last_refill")
local tokens = tonumber(bucket[1]) or tonumber(ARGV[1])
local last_refill = tonumber(bucket[2]) or tonumber(ARGV[3])

local elapsed = tonumber(ARGV[3]) - last_refill
local refill = elapsed * tonumber(ARGV[2])
tokens = math.min(tonumber(ARGV[1]), tokens + refill)

if tokens < 1 then
  return 0  -- rejected
end

tokens = tokens - 1
redis.call("HMSET", KEYS[1], "tokens", tokens, "last_refill", ARGV[3])
redis.call("EXPIRE", KEYS[1], 3600)
return 1  -- allowed

Wire it into Express middleware:

async function rateLimitMiddleware(req, res, next) {
  const key = `ratelimit:${req.user?.id ?? req.ip}`;
  const allowed = await redis.eval(TOKEN_BUCKET_SCRIPT, 1, key, 100, 1.67, Date.now() / 1000);

  res.setHeader("X-RateLimit-Limit", "100");
  res.setHeader("X-RateLimit-Remaining", String(Math.max(0, allowed)));

  if (!allowed) {
    res.setHeader("Retry-After", "60");
    return res.status(429).json({ error: "Too many requests" });
  }
  next();
}

Working examples

Sliding window with Redis sorted sets:

ZADD key timestamp timestamp       -- record the request
ZREMRANGEBYSCORE key 0 (now - window)  -- remove old entries
ZCARD key                          -- count current requests in window

Two-layer production setup:

Client → API Gateway (IP limit: 1000/min)
       → App middleware (user limit: 100/min)
       → Handler

Patterns / when to use each algorithm

Use fixed window when: simplicity matters more than precision, and burst at window boundaries is acceptable (internal tools, low-stakes APIs).

Use sliding window when: you need fair, accurate limits without storing every timestamp. Good default for public APIs.

Use token bucket when: you want to allow legitimate bursts — a user clicking through several pages quickly — while still capping sustained abuse.

Use gateway-level limits when: you need to reject abuse before it consumes application server resources (TLS handshake, auth lookup).

> Little tip: If you're running on Cloudflare, Fastly, or AWS API Gateway, use the built-in rate limiting primitives rather than rolling your own. They handle distributed enforcement across edge nodes for you.

Common mistakes

Returning 403 instead of 429 — 403 means "you're not allowed." 429 means "slow down." Clients handle them differently. Always use 429 for rate limits.

In-memory rate limiting in a multi-instance deployment — each instance tracks its own counter, so a client can send N × limit requests by hitting different instances. Use a shared store.

No Retry-After header — without it, well-behaved clients retry immediately and amplify the problem. Always tell clients when they can try again.

Same limit for all endpoints — a search endpoint and a health check have different cost profiles. Apply tighter limits to expensive operations.

Troubleshooting

Legitimate users hitting limits unexpectedly — check whether you're limiting on IP when many users share one (corporate NAT, university network). Switch to user ID or API key for authenticated traffic.

Limits not enforced consistently across instances — you're using in-memory counters or non-atomic Redis operations. Move to Lua scripts or Redis transactions.

Clients retrying immediately after 429 — your response is missing Retry-After. Add it, and document the header in your API docs so SDK authors implement exponential backoff.

Rate limit counters never reset — keys are missing TTL or the window logic has an off-by-one error at window boundaries. Inspect key TTLs with TTL key in Redis.

Checklist

  • [ ] Chose token bucket or sliding window over fixed window for public APIs
  • [ ] Rate limiting backed by Redis (or equivalent shared store) for multi-instance deployments
  • [ ] Limits applied on user ID or API key for authenticated traffic, IP for anonymous
  • [ ] Rejected requests return HTTP 429 with Retry-After header
  • [ ] Successful responses include X-RateLimit-Limit and X-RateLimit-Remaining
  • [ ] Gateway-level IP limits in place before application middleware
  • [ ] Expensive endpoints have tighter limits than general API endpoints
  • [ ] Rate limit hit logs reviewed periodically as a product signal

Practice task

Add distributed rate limiting to one API endpoint using Redis and a token bucket Lua script. Set a limit of 20 requests per minute per user. Verify that two concurrent requests from the same user on different server instances share the same counter. Trigger a 429, confirm the Retry-After header is present, and verify the client can succeed again after waiting.

FAQ

What's the difference between rate limiting and throttling?

Rate limiting rejects requests over the limit (hard stop). Throttling slows requests down (queues them, delays responses). Rate limiting is simpler and more common for APIs. Throttling appears more in network hardware and streaming systems.

Should I rate limit authenticated and unauthenticated traffic differently?

Yes. Unauthenticated traffic gets IP-based limits to catch bots and scanners. Authenticated traffic gets per-user or per-key limits that are more generous but more precise. Two layers together cover both abuse vectors.

Can rate limiting replace authentication?

No. Rate limiting slows attackers; it doesn't stop them. Layer it with authentication, authorization, and input validation. A determined attacker with valid credentials can still abuse an API within their rate limit.

What to learn next

With rate limiting in place: caching to reduce origin load so fewer requests hit expensive paths, message queues to absorb traffic spikes asynchronously, and CDN edge rules to reject abuse before it reaches your origin servers.

  • System design: caching — reduce origin load so rate limits protect fewer expensive operations
  • System design: queues — absorb traffic spikes that would otherwise trigger mass 429 responses
  • System design: CDN and edge — enforce IP-based limits at the network edge before requests reach your app

Takeaways

Rate limiting protects your API from abuse, fair-use violations, and runaway costs. The algorithm shape matters more than the number — token bucket and sliding window beat fixed window for most public APIs. Distributed enforcement requires a shared store like Redis. Always return 429 with Retry-After.

If you remember only one thing: rate limiting is budget allocation over time — pick the algorithm that matches the spending pattern you want to allow, not just the number of requests per minute.