Rate limiting is one of those backend concerns that feels solved until your API gets hammered at 11:59 PM on the last second of a billing window and half your users get throttled for doing nothing wrong. Picking the right algorithm is not a matter of preference — it is a matter of your API contract, your user expectations, and your infrastructure budget.
This article skips the basics and gets into the real trade-offs.
Why the Algorithm Choice Actually Matters
Most developers reach for a rate limiter the first time they get abused — a scraper, a misconfigured client hammering endpoints in a tight loop, or a DDoS-lite situation. The instinct is to slap on a simple counter and call it done.
The problem is that different algorithms produce very different user experiences, even at the same configured limit. A user allowed 1,000 requests per hour can have a smooth experience or a frustrating one depending entirely on how that limit is enforced across time.
Fixed Window: Simple, But Dangerously Naive
The fixed window algorithm resets a counter at the start of each time window — say, every 60 seconds. It is the easiest to implement and the easiest to abuse.
// Simple fixed-window counter in Node.js (Redis-backed)
const key = `rate:${userId}:${Math.floor(Date.now() / 60000)}`;
const count = await redis.incr(key);
await redis.expire(key, 60);
if (count > LIMIT) return res.status(429).json({ error: 'Too Many Requests' });
The boundary attack problem: A client can send LIMIT requests at 11:59:59 and another LIMIT requests at 12:00:01 — effectively doubling the allowed throughput at window boundaries. For most public APIs, this is a real vulnerability.
When to use it anyway: Internal services with trusted clients, admin tooling, or cases where implementation simplicity genuinely outweighs precision. If you control both sides of the API call, fixed window is often fine.
Sliding Window Log: Precise, But Memory-Hungry
The sliding window log stores a timestamp for every request in a sorted set. When a new request arrives, it prunes timestamps older than the window duration and checks the count.
This eliminates the boundary problem completely — the window is always the last N seconds relative to now, not a wall-clock interval.
The cost: For high-throughput APIs, storing per-request timestamps becomes expensive fast. A user making 10,000 requests per hour means 10,000 entries in memory per user. At scale with thousands of users, your Redis memory budget balloons quickly.
When to use it: APIs where fairness and precision are non-negotiable — financial data feeds, metered billing APIs, or any contract where users pay per request and expect deterministic enforcement.
Sliding Window Counter: The Practical Middle Ground
This is the algorithm most production systems should default to. It approximates the sliding window by combining two fixed-window counters — the current window and the previous one — weighted by how far into the current window the request arrives.
Estimated count = previous_window_count × (1 − elapsed_ratio) + current_window_count
This trades perfect precision for dramatically lower memory usage. The approximation error is small enough for nearly all practical use cases and eliminates the boundary attack vulnerability of the naive fixed window.
Major rate-limiting infrastructure (including Cloudflare's documented approach) uses this pattern because it scales horizontally without per-request storage.
When to use it: Public REST APIs, SaaS product APIs, mobile backend APIs — essentially the default choice when you need correctness without the memory overhead of the full sliding window log.
Token Bucket: Controlled Bursting with Grace
The token bucket model maintains a bucket of tokens that refills at a steady rate. Each request consumes one token. If the bucket is empty, the request is rejected or queued.
The key differentiator is burst tolerance. A user who has been idle accumulates tokens and can legitimately fire a burst of requests without being throttled — which mirrors realistic usage patterns far better than a hard per-second cap.
Trade-offs to understand:
- Burst size is a separate configuration from refill rate. Getting this wrong either makes the limiter useless (bucket too large) or frustrating (bucket too small for legitimate bursts).
- Token bucket does not prevent the boundary problem if your bucket can fully refill across a window reset.
- It requires tracking both current token count and the last refill timestamp — slightly more state than a simple counter.
When to use it: APIs consumed by human-driven clients (dashboards, mobile apps), SDKs where bursty-then-idle patterns are normal, or anywhere you want to reward well-behaved clients with accumulated capacity.
Leaky Bucket: Smoothing Output, Not Controlling Input
The leaky bucket is often confused with the token bucket, but the intent is different. Instead of controlling how fast requests come in, it controls how fast they are processed — requests queue up and drain at a fixed rate.
This is fundamentally a traffic shaping tool, not an abuse prevention tool. It guarantees a smooth output rate to downstream services but does nothing to prevent a malicious client from flooding the queue itself.
When to use it: Rate-limiting calls to outbound third-party APIs from your backend — SMS providers, payment gateways, email services — where you need to respect their rate limits and smooth your own traffic. It is the wrong choice for protecting your own API surface from external abuse.
Choosing the Right Strategy: A Decision Framework
| Scenario | Recommended Algorithm |
|---|---|
| Internal microservice calls | Fixed Window |
| Public SaaS API, metered billing | Sliding Window Counter |
| Financial / per-request billing APIs | Sliding Window Log |
| Mobile / dashboard clients, bursty usage | Token Bucket |
| Outbound calls to third-party APIs | Leaky Bucket |
A few additional considerations that rarely appear in tutorials:
- Distributed systems: All of these algorithms require atomic operations when running multiple API server instances. Redis with Lua scripts or Redis transactions is the standard approach.
- Headers matter: Always return
X-RateLimit-Limit,X-RateLimit-Remaining, andRetry-Afterheaders. Clients that can backoff gracefully reduce your abuse surface significantly. - Per-endpoint limits: Global user-level limits miss abuse patterns. Combining a global limit with tighter per-endpoint limits (e.g., authentication endpoints) is standard practice for any API handling sensitive operations.
Why This Matters for Your Project
If you are building a SaaS product or exposing an API to third-party developers, your rate limiting strategy is part of your API contract — and changing it later breaks integrations and erodes trust. Taking twenty minutes to choose the right algorithm before launch is significantly cheaper than migrating a live system under pressure. The sliding window counter covers the majority of use cases well; layer in token bucket behaviour for client-facing APIs where burst tolerance improves perceived performance, and reserve the full sliding window log for contexts where billing precision is legally or contractually required.
