Caching architecture: from CDN to application layer
An old line calls cache invalidation one of the two hard problems in computer science. It holds up because caching itself is easy: store a copy of something expensive closer to where it's needed. Every layer of a system has a cache somewhere. The design problem is what happens when the cached value stops being true.
1. The layers a request passes through
A request from client to database passes several places that could hold a cached answer, and the earlier the hit, the cheaper the request. A CDN, geographically distributed and closest to the client, can serve a cached response without the request reaching origin infrastructure. A reverse proxy or edge cache in front of the application can serve a response without application code running. An application-level cache (an in-memory store or a dedicated service like Redis) can serve a value without querying the database. The database's internal buffer cache is the last layer: it keeps recently accessed pages in memory, so a miss at every layer above doesn't necessarily mean a disk read.
Each layer trades hit rate against the range of content it can safely cache. A CDN caches whole responses well, but only content that's the same for every requester (or per a small number of variants). An application-level cache can hold something specific to one user's session, but every request past the CDN still reaches the application to check it.
2. CDN caching: keys and TTLs
A CDN decides whether to serve a cached response using a cache key, typically the URL plus whichever headers the origin says matter (most commonly via a Vary header), and a time-to-live that sets how long a cached response is served before the CDN re-fetches from origin. A wrong cache key fails in one of two opposite ways. Too broad, and one user's personalized or region-specific content is served to another user who requested the same URL. Too narrow (varying on a header that doesn't change the response), and one shared entry fragments into many, tanking the hit rate for nothing.
TTL trades freshness against origin load. A short TTL keeps content current but hits the origin more often. A long TTL protects the origin but leaves clients seeing stale content longer after a change. Avoid one global value by setting TTL per content type: long for a versioned static asset whose filename changes with its content, short for anything that changes on its own schedule without a URL change.
3. Application-level caching patterns
Cache-aside (lazy loading): the application checks the cache, and on a miss reads from the database and writes the result into the cache before returning it, so the next request hits. It's the simplest pattern and the most common default. Its key property: after a write, the cache and the database can disagree until the entry expires or is explicitly invalidated.
Write-through: the application writes to the cache and the database in the same write path, so the cache is never behind the latest write, but every write pays the latency of both operations. Write-behind (write-back): the write goes to the cache immediately and to the database asynchronously. It's fast for the caller but risks durability. A crash between the cache write and the deferred database write loses data the caller was told was saved. Use write-behind only when that risk is acceptable for the data in question.
4. Invalidation: the hard problem
TTL expiry is invalidation by default. The entry goes stale on schedule whether or not the data changed. It's simple and safe, but every cached value is wrong for up to a full TTL after an update. Explicit invalidation (deleting or updating the entry when the data changes) closes that window. The cost: the invalidation logic must run correctly every time, on every code path that writes the data, including paths added later by someone who forgot the cache exists.
The production failure is usually an invalidation path that covers the common write path and misses a secondary one: a bulk import, an admin tool, a background job that updates the same data through code nobody taught to touch the cache. Remembering every write path doesn't scale. Derive invalidation from something every write already does: a database change-data-capture stream, or a version or timestamp on the cached value checked against the source on read.
5. Cache stampede
A cache stampede happens when one heavily requested key expires and a burst of concurrent requests miss at almost the same instant. Each one independently recomputes or re-fetches the same expensive value from origin, because none can see another request already doing the work. For a hot key (a popular product page, a frequently read configuration value), an ordinary expiry becomes a multiplied load spike on the origin at the moment the cache was supposed to protect it.
The standard mitigations make sure one request pays for recomputation while the rest wait or serve something slightly stale. With locking, the first request to miss acquires a lock, recomputes, and populates the cache; concurrent requests wait briefly for the lock or, under a stale-while-revalidate policy, serve the stale value. With probabilistic early expiration, each request has a chance of recomputing a hot key before its TTL expires, and that chance rises as expiry approaches, so one early request refreshes the key well before the hard deadline that would make every concurrent request miss at once.
6. Consistency risk: stale reads and their real cost
Any cache that isn't write-through can return a value that's no longer true. Ask explicitly how expensive a stale read is for each piece of data. A product description that's stale for a few seconds after an edit is a non-event. A stale account balance, or a stale permission check still showing access that was just revoked, is a correctness and security problem. Choose the caching pattern and TTL from that cost. Don't apply one caching policy across all data regardless of what being wrong costs.
7. Worked example: a stampede-safe cache-aside implementation
A product catalog page is served from a cache-aside layer in front of the database, with a five-minute TTL per product. Normal traffic is fine. During a promotional spike, one product's entry expires while thousands of concurrent requests are hitting that page. They all miss within a few milliseconds and send thousands of identical database queries at once, briefly degrading the database for every other product too.
The fix adds a short-lived lock keyed to the product ID. The first request to miss acquires the lock, queries the database, and repopulates the cache. Every other concurrent request for that key sees the lock held and either waits a short, bounded interval and reads the fresh value, or, if a slightly stale previous value exists and staleness is acceptable for this data, serves it immediately. Either way the expired key causes one database query instead of thousands. Choosing between waiting and serving stale is the freshness-cost question from Section 6, made explicitly for this data instead of left to how the cache happened to expire.
See cloud cost optimization for how much origin load, and therefore cost, a well-designed cache layer removes, and database architecture in the cloud for read-replica lag, the same staleness tradeoff one layer further down.