Caching, the Honest Version

Caching is one of those topics that sounds simple in an interview and turns into a series of unpleasant surprises in production. "Just put Redis in front of it" is not a strategy. Where the cache lives, how it stays fresh, and what happens when it lies to you are the actual problems.

This is a grounded tour of the layers that matter, the trade-offs each one forces on you, and a few real-world examples of how they fit together.

Why bother caching

Most user-facing systems are not bottlenecked by CPU. They are bottlenecked by I/O and by the cost of recomputing the same answer for thousands of users a second. Caching trades a bit of staleness and memory for a large drop in latency and load. Done well, it is the single biggest performance lever you have.

The catch is consistency. A cache is, by definition, a copy of the truth. The instant the truth changes, the cache is lying. Most of caching's complexity is in deciding how long that lie is acceptable.

Where caching happens

A request from a browser to your database can pass through four or five caches before it ever reaches your code. Each one has a different cost profile and a different failure mode.

Browser cache. The browser holds DNS responses, static assets, and anything your headers told it to keep. Free, and the fastest cache in the chain, because the request never leaves the device.

CDN. Cloudflare, Fastly, CloudFront and friends serve static and cacheable dynamic content from a point of presence near the user. Latency drops from tens or hundreds of milliseconds to single digits. Cost shifts from your origin to the edge.

Application-server local cache. An in-process or on-disk cache on each backend node. Great for read-heavy data that does not change often (feature flags, config, lookup tables). The trap is that every server has its own copy and they go out of sync independently.

Centralised cache (Redis, Memcached). A shared, in-memory store that every backend talks to. One source of truth for hot data, at the cost of a network hop. This is where most "the cache" lives in modern systems.

Database buffer pool. Postgres, MySQL, and friends already cache pages in RAM. This is not your cache, but it is one. Tuning it matters before you reach for an external one.

A useful instinct: when you find yourself adding a new cache, ask which of the existing layers should have handled this. The answer is often one of them.

Cache invalidation, ranked by how badly it can hurt you

Phil Karlton's old joke ("There are only two hard things in computer science: cache invalidation and naming things") is right because invalidation is a correctness problem dressed up as a performance optimisation.

A few common strategies, with their actual trade-offs:

TTL (Time To Live). Every entry expires after N seconds. Simple, predictable, and the right answer 80 percent of the time. The hard part is picking N. Too short and your cache stops paying for itself. Too long and users see stale data for an embarrassingly long time.

Write-through invalidation. When the underlying data changes, the writer also updates the cache (or deletes the entry). Clean in theory, awkward in practice because every writer of the data needs to know about every cache that holds it.

Metadata-based validation. Store a version number, timestamp, or hash with the cached value. On read, cheaply compare against the source. Useful when the source is fast to check but expensive to fully fetch. Coding platforms do this with problem files: key the cache as <problem_id>_<updated_at> so any update naturally produces a new cache key and the old entries fall out.

Event-driven invalidation. Publish change events on a stream (Kafka, Pub/Sub, change-data-capture from the DB) and have caches subscribe. Most accurate, most operationally heavy.

The honest hierarchy: start with TTL. Move to metadata-based validation when staleness becomes a complaint. Reach for event-driven only when nothing else will do.

Eviction is not invalidation

Eviction is what the cache does when it runs out of room. Invalidation is what you do when the truth changes. People conflate these and end up with both the wrong policy and the wrong expectations.

The common eviction policies:

• LRU (Least Recently Used). Evicts whatever has not been touched in the longest. The default for a reason. Matches almost every realistic access pattern.
• LFU (Least Frequently Used). Evicts whatever has been touched the fewest times. Good when popularity is stable. Bad during a traffic spike on a new item.
• FIFO. Evicts the oldest entry regardless of usage. Cheap. Rarely the right answer.
• Random. Surprisingly hard to beat in workloads with no clear pattern, and trivially cheap.

Redis's default allkeys-lru is the answer for most teams. Reach for the others only if your access pattern actually demands it.

Write strategies: where the cache and database disagree

You are about to write a new value. Where do you write first?

Write-through. Update the cache, then the database, synchronously. Reads after the write see fresh data. Writes get slower. Cache and DB stay in lockstep as long as both writes succeed (which is itself an interesting failure mode).

Write-back (or write-behind). Update the cache, return to the caller, and flush to the database asynchronously. Fastest writes, but a cache crash before flush loses data. Acceptable for some metrics and view counters, dangerous for anything you cannot recompute.

Write-around. Write directly to the database and let the cache load lazily on the next read. Avoids cache pollution from writes that nobody ever reads back. Pairs naturally with TTL.

There is no universal right answer. Pick per use case. A counter for "likes on a post" is fine as write-back. A user's billing address is not.

Three patterns from the wild

Local caching for a coding platform. When a user submits a solution, the judge needs the test input file. Fetching it from object storage on every submission adds seconds of latency. Solution: cache the file on disk on each judge node, key it as <problem_id>_<updated_at>_input.txt, and refresh only when the metadata changes. The cache key encodes the freshness check, so invalidation is automatic and per-server staleness becomes impossible.

Global leaderboards with Redis. During a contest, every refresh of the leaderboard wants the current ranking. Computing that from the submissions table per request will kill the database. Solution: compute the ranking on a schedule (every few seconds) and store it as a single Redis sorted set. Every backend reads from the same set. The database sees one writer instead of thousands.

Facebook-style newsfeed. Loading a user's feed by joining across millions of friend posts in real time is hopeless. The trick is to split the storage: only the most recent N days of posts live in a fast, lean store optimised for WHERE user_id IN (friend_ids) ORDER BY created_at DESC LIMIT 50. Older posts are archived elsewhere. The "cache" here is really a denormalised hot store, but it plays the same role: keep the working set small enough to be fast.

SELECT *
FROM recent_posts
WHERE user_id IN (:friend_ids)
ORDER BY created_at DESC
LIMIT 50 OFFSET :cursor;

The query is boring. The fact that recent_posts is a tightly scoped subset of all posts is the actual optimisation.

Things that will bite you

A few patterns worth knowing about because they will eventually happen to you:

• Thundering herd. A hot cache entry expires. A thousand requests miss the cache simultaneously, all hit the database, all try to repopulate. Defence: use a lock, a single-flight pattern, or staggered expirations.
• Cache stampede on cold start. A fresh deploy or a Redis restart leaves the cache empty. Your origin gets hit with full read load. Defence: warm critical entries on boot, or roll deploys slowly.
• Negative caching. Forgetting to cache "not found" answers means every lookup of a missing key keeps hitting the source. Cache the absence too, with a short TTL.
• Cache key drift. Two services build the same key from the same inputs in slightly different ways. They never read each other's writes. Defence: centralise key construction in one place.

Takeaways

1. Caching is a hierarchy. Browser, CDN, local, central, database buffer. Reach down only when the layer above cannot do the job.
2. TTL is the default for a reason. Reach for fancier invalidation only when staleness becomes a real complaint.
3. Eviction policy is separate from invalidation policy. Pick both, deliberately.
4. Pick a write strategy per use case. There is no global right answer.
5. The interesting failures (stampedes, cold starts, key drift) all happen at the seams, not inside Redis itself.

Caching is the art of telling a small, well-managed lie so that your users get a fast, mostly-true answer. The skill is in deciding which lies you can live with.