Caching is where good APIs become fast APIs — but choosing the wrong layer can quietly destroy consistency, waste memory, or introduce latency bugs that only surface under load. Most tutorials pick a tool, show a GET/SET example, and move on. This article doesn't do that.
Here is exactly when Redis wins, when in-process memory wins, and what most engineers get wrong about eviction policies.
The Two Layers Every Backend Engineer Should Know
Before comparing tools, it helps to be precise about what "caching layer" means in a production API context.
In-process (in-memory) caching lives inside your application's own runtime — think node-cache in Node.js, Guava Cache in Java, or a plain Python dictionary wrapped with a TTL decorator. Data lives in the same heap as your application code. Access is measured in microseconds.
Redis is an external, networked key-value store. It runs as a separate process or cluster, and your application connects over TCP. Access is measured in sub-millisecond to low-millisecond round trips — fast, but not the same order of magnitude.
This distinction matters more than most teams realize.
When In-Process Memory Is the Right Choice
In-process caching is underused in modern backends, largely because Redis has excellent tooling and is the default recommendation almost everywhere. But for certain workloads, it is definitively faster and simpler.
Use in-process caching when:
- Your data is read-heavy and rarely invalidated — feature flags, config values, country code lookups, permission matrices.
- Your API runs as a single instance or you can tolerate per-instance inconsistency (i.e., cache misses across replicas don't break correctness).
- Latency is critical at the sub-millisecond level — for example, middleware that runs on every request.
- You want zero infrastructure overhead — no Redis cluster to provision, monitor, or pay for.
A concrete benchmark to anchor expectations: retrieving a cached value from an in-process store like Guava Cache in a JVM application typically runs at 200–800 nanoseconds. A Redis GET over a local loopback interface typically runs at 200–500 microseconds — roughly 300–600x slower. Over a network hop in a cloud environment, add another 1–3ms.
For middleware that executes on every single HTTP request, that difference compounds fast.
When Redis Is the Right Choice
Redis earns its place when your architecture scales beyond a single process. The moment you run more than one replica of your API — which is nearly every production deployment — in-process caches become isolated silos. Two requests hitting two different pods may get two different answers, or trigger two separate expensive database queries for the same key.
Use Redis when:
- You run multiple instances of your API (horizontal scaling, Kubernetes pods, serverless functions).
- You need cache invalidation that propagates instantly across all instances.
- You are caching user session data, rate-limit counters, or any state that must be consistent cluster-wide.
- Your cache needs to survive an application restart — Redis persistence (RDB snapshots or AOF) gives you durability that in-process caches simply cannot.
- You need atomic operations — incrementing a counter, checking and setting a value, or managing distributed locks all require the atomicity Redis provides.
Redis Cluster also gives you horizontal scaling of the cache itself, which no in-process solution can match without significant custom engineering.
Eviction Policies: The Part Most Tutorials Skip
Both layers support TTL-based expiration, but Redis exposes a far richer set of eviction policies that are worth understanding before you hit memory limits.
Redis offers eight eviction policies. The two most consequential for API caching:
allkeys-lru— evicts the least recently used key across all keys when memory is full. This is the right default for most API response caches, where recent data is more likely to be requested again.volatile-lfu— evicts the least frequently used key, but only among keys that have a TTL set. Useful when you mix persistent config data (no TTL) with transient response cache (TTL set), and you only want eviction to touch the transient data.
A common mistake: teams set noeviction (Redis's default in some configurations) and then wonder why their API starts returning errors when Redis memory fills up. Under noeviction, Redis rejects write commands when it hits the memory limit. In a high-traffic scenario, that means cache writes fail silently and every request falls through to the database — exactly the thundering herd problem caching was meant to prevent.
In-process caches have their own eviction nuances. Guava Cache and Caffeine (Java) use window TinyLFU, a frequency + recency hybrid that outperforms plain LRU in benchmarks by 30–50% on skewed access patterns. If you're using a basic LRU map in Node.js, consider switching to a library like lru-cache that implements proper size-bounded eviction.
A Layered Approach: The Architecture Worth Considering
For high-traffic APIs, the most resilient pattern isn't choosing one layer — it's layering both.
Request → In-Process Cache (L1) → Redis (L2) → Database
The application checks the local in-process cache first. On a miss, it checks Redis. On a Redis miss, it hits the database and populates both caches on the way back. This gives you:
- Microsecond reads for hot keys at L1
- Consistent cross-instance caching at L2
- Significantly reduced Redis network traffic (L1 absorbs the majority of hits)
The trade-off is invalidation complexity. When data changes, you must invalidate both layers — and L1 invalidation across multiple pods requires either a short TTL or a pub/sub broadcast (Redis itself can serve this role via its pub/sub channels).
Keep L1 TTLs short (5–30 seconds) and L2 TTLs longer (minutes to hours). Accept that L1 may serve slightly stale data within that window, and design your product accordingly.
Choosing Based on Your Scaling Stage
| Scenario | Recommended Approach |
|---|---|
| Single-instance API, low complexity | In-process only |
| Multi-instance API, shared state needed | Redis |
| Multi-instance, latency-critical hot path | Layered L1 + L2 |
| Serverless / stateless functions | Redis only (no persistent process) |
| Rate limiting, distributed counters | Redis always |
Why This Matters for Your Project
If you are building or scaling a SaaS product, caching decisions made early become architectural constraints later. A team that hard-codes Redis as their only cache layer will pay unnecessary network latency on every request. A team that leans exclusively on in-process caches will hit consistency bugs the moment they scale past one server. The discipline is in matching the caching layer to the specific access pattern, consistency requirement, and deployment topology — not defaulting to whatever the last tutorial recommended.
