Caching
Caching stores copies of frequently accessed data in fast, temporary storage closer to the consumer. It is one of the highest-impact techniques in system design: a well-placed cache can reduce latency by orders of magnitude and protect databases from thundering herds.
Caching is also one of the hardest techniques to get right. Stale data, cache misses, invalidation races, and stampeding herds can introduce subtle bugs. The art of caching lies in choosing what to cache, where to cache it, and how to keep it consistent.
This guide covers caching layers from browser to database, caching patterns, invalidation strategies, eviction policies, and failure handling. By the end, you will know how to design a cache that accelerates your system without becoming a liability.
Caches exist at multiple layers between the user and the origin datastore. Each layer has different latency, capacity, and consistency characteristics. The closer a cache is to the user, the faster it is and the less load it places on origin systems.
| 1 | User |
| 2 | │ |
| 3 | ├─▶ Browser cache (fastest, private, short TTL) |
| 4 | │ |
| 5 | ├─▶ CDN edge cache (regional, shared, static assets) |
| 6 | │ |
| 7 | ├─▶ Reverse proxy cache (Varnish / Nginx, full responses) |
| 8 | │ |
| 9 | ├─▶ Application cache (Redis / Memcached, objects) |
| 10 | │ |
| 11 | ├─▶ Database buffer pool (frequently read pages) |
| 12 | │ |
| 13 | └─▶ Origin datastore (source of truth) |
| Layer | Typical TTL | Scope | Best For |
|---|---|---|---|
| Browser | Minutes to years | Single user | Static assets, immutable files |
| CDN | Hours to days | Regional users | Images, JS, CSS, video |
| Reverse proxy | Seconds to minutes | All users, per URL | Idempotent GET responses |
| Application | Minutes to hours | All users, per object | Database objects, computed results |
| Database | Managed by engine | Instance-local | Hot pages and indexes |
info
A Content Delivery Network (CDN) caches static content at geographically distributed edge locations. When a user requests an asset, the CDN serves it from the nearest edge, reducing latency and offloading origin servers.
Modern CDNs also support dynamic content acceleration, edge computing, and security features such as DDoS mitigation and WAF. For system design interviews, focus on cache invalidation, origin shielding, and TTL strategies.
| 1 | CDN architecture: |
| 2 | |
| 3 | User in Tokyo ──▶ CDN edge in Tokyo ──▶ Origin in Virginia |
| 4 | │ |
| 5 | └── cache hit: serve immediately |
| 6 | └── cache miss: fetch from origin, cache, serve |
| 7 | |
| 8 | Best practices: |
| 9 | - Use versioned filenames (app.v2.js) for long TTLs |
| 10 | - Set Cache-Control headers explicitly |
| 11 | - Use origin shield to reduce origin load |
| 12 | - Purge by tag or URL when content changes |
best practice
How the application interacts with the cache and database determines consistency, latency, and write behavior. The four main patterns are cache-aside, read-through, write-through, and write-behind.
Cache-Aside (Lazy Loading)
The application checks the cache first. On a miss, it reads from the database, populates the cache, and returns the result. This is the most common pattern because it is simple and gives the application full control over cache keys and TTLs.
| 1 | Cache-aside read: |
| 2 | |
| 3 | 1. app.get(key) from cache |
| 4 | 2. if hit: return value |
| 5 | 3. if miss: |
| 6 | value = db.query(key) |
| 7 | cache.set(key, value, ttl) |
| 8 | return value |
| 9 | |
| 10 | Cache-aside write: |
| 11 | |
| 12 | 1. db.update(key, value) |
| 13 | 2. cache.delete(key) |
| 14 | 3. next read repopulates cache |
Read-Through
The cache itself knows how to fetch missing data from the database. The application always talks to the cache. Read-through simplifies application code but couples the cache to the data model.
Write-Through
Data is written to the cache and the database at the same time. Reads are fast because the cache is always up to date. The downside is that writes are slower and cache capacity limits write throughput.
Write-Behind (Write-Back)
Data is written to the cache first and flushed to the database asynchronously. This maximizes write throughput but risks data loss if the cache fails before persistence. It is common in high-throughput analytics and logging systems.
| Pattern | Read Latency | Write Latency | Consistency |
|---|---|---|---|
| Cache-aside | Fast on hit | Fast (DB only + invalidation) | Eventual, simple |
| Read-through | Fast on hit | Same as DB | Eventual |
| Write-through | Fast | Slower (cache + DB) | Strong |
| Write-behind | Fast | Fastest | Risk of loss |
best practice
There are only two hard things in computer science: cache invalidation and naming things. Invalidation keeps cached data consistent with the source of truth. The three main strategies are TTL expiration, active invalidation, and write-time update.
| 1 | Invalidation strategies: |
| 2 | |
| 3 | 1. TTL (Time-To-Live) |
| 4 | - Set an expiration on each key |
| 5 | - Simple, but data can be stale until TTL expires |
| 6 | - Best for data where brief staleness is acceptable |
| 7 | |
| 8 | 2. Active invalidation |
| 9 | - Delete or update cache key on write |
| 10 | - Keeps cache fresh but adds write latency |
| 11 | - Risk: race conditions between read and write |
| 12 | |
| 13 | 3. Write-time update |
| 14 | - Update cache at the same time as database |
| 15 | - Used in write-through caching |
| 16 | - Cache and DB stay in sync |
warning
When a cache reaches capacity, it must evict existing entries to make room. The eviction policy determines which entries to remove. The right policy depends on access patterns and the cost of a miss.
| Policy | Behavior | Best For |
|---|---|---|
| LRU | Evict least recently used | General workload, temporal locality |
| LFU | Evict least frequently used | Stable popularity distributions |
| FIFO | Evict oldest inserted | Streaming data |
| TTL-based | Evict expired entries first | Time-sensitive data |
| Random | Evict random entry | Simple, low overhead |
info
Caches fail too. A cache outage can suddenly redirect all traffic to the database, causing a cascading failure. Understanding failure modes helps you build resilient systems.
Cache Stampede
A cache stampede occurs when many requests simultaneously hit a missing cache key, all trying to recompute or refetch the value. This can overwhelm the database. Solutions include per-process locking, probabilistic early expiration, and single-flight request coalescing.
| 1 | Stampede mitigation with single-flight: |
| 2 | |
| 3 | value = cache.get(key) |
| 4 | if not value: |
| 5 | value = singleflight.do(key, () => db.query(key)) |
| 6 | cache.set(key, value, ttl) |
| 7 | |
| 8 | The single-flight pattern ensures only one |
| 9 | request computes the value; others wait. |
Cache Penetration
Cache penetration happens when requests repeatedly ask for keys that do not exist. Since the cache cannot store a null result, every request hits the database. Store a sentinel value for missing keys with a short TTL, or validate keys before lookup.
Cache Avalanche
A cache avalanche occurs when many keys expire at the same time, causing a flood of database queries. Mitigate by adding jitter to TTLs, using staggered expiration, and pre-warming critical keys.
warning
Redis is the most common in-memory data structure store used for caching. It supports strings, hashes, lists, sets, sorted sets, bitmaps, and streams. Its sub-millisecond latency makes it ideal for hot data and real-time use cases.
| Feature | Use Case |
|---|---|
| Strings with TTL | Simple key-value cache |
| Hashes | Caching object fields |
| Sets / Sorted Sets | Leaderboards, unique visitors |
| Bitmaps | Daily active user tracking |
| Streams | Event sourcing, logs |
pro tip