Skip to content

Cache

Why cache?

Caching improves page-load times and reduces load on servers and databases. The dispatcher first checks if a request has been served before and returns the previous result, saving the actual execution.

Databases benefit from uniform read/write distribution. Popular ("hot") items skew the distribution and cause bottlenecks — a cache in front of a database absorbs uneven loads and traffic spikes.

Where caches live

  • Client caching — browser or OS.
  • CDN caching — CDNs are a type of cache.

  • Web server caching — reverse proxies and caches like Varnish serve static/dynamic content directly.

  • Database caching — DBs include some caching by default; tune it for your usage patterns.

  • Application caching — in-memory key-value stores (Memcached, Redis) between the app and data storage.

  • Data in RAM is far faster than disk-backed DBs.

  • RAM is limited, so use eviction algorithms like LRU (least recently used).
  • Redis adds persistence and built-in data structures (sorted sets, lists).

Levels you can cache

Two broad categories — database queries and objects:

  • Row level
  • Query level
  • Fully-formed serializable objects
  • Fully-rendered HTML

Avoid file-based caching — it makes cloning and auto-scaling harder.

Caching at the database query level

Hash the query as a key and store the result. Downside — expiration issues:

  • Hard to delete a cached result for complex queries.
  • If one cell changes, you must delete all cached queries that might include it.

Caching at the object level

Treat data as objects (like in app code); assemble the DB result into a class instance or data structure:

  • Remove the object from cache when its underlying data changes.
  • Enables async processing: workers assemble objects from the latest cached object.

Good things to cache: user sessions, fully-rendered pages, activity streams, user graph data.

When to update the cache

Cache-aside (lazy loading)

The application is responsible for reading/writing storage; the cache doesn't touch storage directly.

  1. Look for entry in cache → cache miss.
  2. Load entry from database.
  3. Add entry to cache.
  4. Return entry.
def get_user(self, user_id):
    user = cache.get("user.{0}".format(user_id))
    if user is None:
        user = db.query("SELECT * FROM users WHERE user_id = {0}".format(user_id))
        if user is not None:
            cache.set("user.{0}".format(user_id), json.dumps(user))
    return user

Used by Memcached. Only requested data is cached (no filling cache with junk).

Disadvantages: each miss = 3 trips (latency); data can go stale (mitigate with TTL or write-through); a failed node is replaced by an empty one.

Write-through

The app uses the cache as the main data store; the cache writes through to the DB.

  1. App adds/updates entry in cache.
  2. Cache synchronously writes to data store.
  3. Return.

Slow overall (write op), but subsequent reads are fast, and cache data is not stale.

Disadvantages: a new node won't cache entries until they're updated in the DB (combine with cache-aside); much written data is never read (use TTL).

Write-behind (write-back)

  1. App adds/updates entry in cache.
  2. Cache asynchronously writes to data store → faster writes.

Disadvantages: data loss risk if the cache dies before flushing to the store; more complex to implement.

Refresh-ahead

Cache automatically refreshes recently-accessed entries before they expire. Reduces latency vs read-through if the cache can predict future needs.

Disadvantage: poor prediction makes performance worse.

Strategy comparison

Strategy Write path Read freshness Risk
Cache-aside App writes DB directly Can be stale (TTL) Miss latency, stale data
Write-through Sync via cache Fresh Slow writes, cold cache
Write-behind Async via cache Fresh-ish Data loss on crash
Refresh-ahead — Proactive refresh Bad predictions

Disadvantages of caching overall

  • Must maintain consistency between cache and source of truth (invalidation).
  • Cache invalidation is hard — deciding when to update adds complexity.
  • Requires app changes (adding Redis/Memcached).

Key takeaways

  • Caching is the highest-leverage performance tool, but its cost is staleness and invalidation complexity.

  • "There are only two hard things in Computer Science: cache invalidation and naming things."

Common Interview Questions

Design a cache system like Memcached

Why it's asked here: the direct cache question.

Key points to discuss:

  • In-memory KV, eviction policy (LRU), and sharding across nodes.
  • Consistent hashing to minimize reshuffling when nodes change.
  • Cache-aside (lazy loading) flow and the cache-miss cost.
  • Invalidation and TTL to manage staleness.
flowchart LR
    A[App] -->|get| C[(Cache cluster)]
    C -->|miss| DB[(Database)]
    DB --> C
    C -->|consistent hashing| N1[Node 1]
    C -->|consistent hashing| N2[Node 2]
Design a key-value store for a search engine (query cache)

Why it's asked here: caching query results is a classic query-level cache.

Key points to discuss:

  • Cache query → results; hit ratio drives value.
  • Query-level vs object-level caching trade-offs.
  • Invalidation: hard to delete cached results for complex queries.
  • Eviction + TTL for the long tail of queries.
flowchart LR
    Q[Query] --> C[(Query cache)]
    C -->|hit| R[Result]
    C -->|miss| E[Search engine]
    E --> C
Design a news feed / Twitter timeline (cache fan-out)

Why it's asked here: timelines are served from cache via fan-out.

Key points to discuss:

  • Precompute and cache each user's timeline (fan-out on write).
  • Redis sorted sets / lists for timeline storage.
  • Cache update strategies: write-through vs write-behind vs cache-aside.
  • Staleness vs performance for celebrity (hot) users.
flowchart LR
    P[Post] --> F[Fan-out on write]
    F --> T[(Timeline cache)]
    U[User] --> T
    T -->|miss| D[(Sharded tweets)]

Further reading