MediumExperience level: 3-5 years · 8-15 years1 min read

How to Design a Distributed Rate Limiter

Reviewed by Gurusankar M. · Updated Aug 24, 2026

Asked inAmazonGoogleMicrosoft
#rate limiter#token bucket#sliding window#redis#distributed
Report issue

Short Answer

Use a token-bucket or sliding-window-log/counter algorithm, keyed per client, with counters in a shared low-latency store (Redis) so the limit is enforced globally across all app instances. Update atomically (Lua/INCR+EXPIRE), and decide fail-open vs fail-closed if the store is down.

On this page

Concept Made Simple

Coffee Chat Question

“How would you design a distributed rate limiter?”

Remember It Faster

Mind Map Answer

Algorithm
token bucket / sliding window
State
Redis (shared, atomic)
Key
per client/API key
On store down
fail-open vs fail-closed

Think Beyond the Expected

What If?

Why not just keep the counter in each app instance's memory?

Per-instance counters let a client multiply the limit by the instance count and reset on deploys/scaling — the global limit isn't enforced. A shared atomic store (Redis) gives one consistent count across instances; the gateway is a common enforcement point.

Real World

Distributed rate limiting lives at the gateway with Redis-backed counters and atomic Lua scripts; the design questions are algorithm choice (burst vs smoothness) and fail-open/closed behavior.

Interviewer's Expectation

Keywords they're listening for:

  • token bucket vs sliding window
  • shared atomic store (Redis)
  • per-key global enforcement
  • atomicity (Lua/INCR)
  • fail-open/closed

Common Mistakes

  • Per-instance counters (limit multiplied)
  • Non-atomic read-modify-write races
  • No plan for the limiter store failing

Best Practices

  • Shared atomic counters (Redis Lua)
  • Enforce at the gateway
  • Choose algorithm for burst vs smoothness; decide fail mode

Follow-up Questions

  1. Token bucket vs sliding-window-log vs counter?
  2. Fail-open or fail-closed when Redis is down?
  3. How do you handle bursts vs sustained rate?

Continue Learning with AI

Take this question deeper with your favourite AI assistant. Pick a depth, copy the prompt, or open it directly — AI is your learning companion, not a shortcut.

Plain-language foundations

I'm preparing for a software engineering interview and want to understand this from scratch, as a beginner.

Topic: Scaling (System Design)
Interview question: "How would you design a distributed rate limiter?"

Please:
1. Explain the core idea in simple, plain language, using an everyday analogy.
2. Define any technical terms you use.
3. Walk through one small, concrete example.
4. Finish with a single sentence I can easily remember.

Keep the tone friendly and assume I'm new to this topic.
Open inChatGPTGeminiClaude

Was this answer helpful?

Affiliate links

Tools for Your Developer Workspace

Equipment for a comfortable, productive setup.

As an Amazon Associate I earn from qualifying purchases.