Caching: the hardest easy thing
The same answer, 50,000 times
Priya's dashboard shows a popular garden's page read 50,000 times an hour, though it changes only a few times. Every read asks the database again.
A cache keeps a copy of recent answers in fast memory, so repeated reads skip the database.
Cache-aside
Microsoft's architecture guide describes the most common pattern, cache-aside:
- 1. Look in the cacheIf the item is there (a cache hit), return it.
- 2. On a miss, read the databaseA cache miss means the item is not in the cache.
- 3. Store it, then return itThe next read is a hit.
Quick check
What is a cache miss?
- The item is not in the cache, so the app reads it from the database
- The cache returned the wrong garden
- The database is down
Show the answer
The item is not in the cache, so the app reads it from the database
A miss is normal: the first read of anything is a miss. The app fetches it once and stores it for the next reads.
Two hard things
Programmers often quote Phil Karlton: "There are only two hard things in Computer Science: cache invalidation and naming things."
Invalidation means removing a copy once it is out of date. Miss one, and players see stale data: an old garden name, a price that changed.
How long to keep a copy
Cached items usually expire after a set time. Microsoft's guide warns both ways: too short, and reads keep going back to the database. Too long, and the copy goes stale.
It adds that caching "works best for relatively static data or data that applications read frequently".
Quick check
A longer expiry time is always better, because it saves more database reads.
- True
- False
Show the answer
False
Longer expiry also means older copies. The right time depends on how stale the data may safely be.
Bug hunt
1async function rename(2 id, name) {3 await cache.delete(4 `garden:${id}`);5 await db.update(id,6 { name });7}
After Remy's rename code runs, the old name now and then comes back. Why?
- The cache is cleared before the database update, so a read in between caches the old name again
- Names cannot be cached
- The cache key is spelled wrong
Show the answer
The cache is cleared before the database update, so a read in between caches the old name again
Microsoft's guide says to update the database first, then remove the cached copy. The other order leaves a gap where a read stores the old value.
True story: Fortnite at 3.4 million
In early 2018, Fortnite grew from a peak of 60,000 players online at once to 3.4 million "in just a few months". One weekend brought 6 outages, and Epic published a postmortem.
In front of its account service sat a proxy that checked sign-in tokens against a cache, memcached. Under the load, memcached began to fail. The proxy kept waiting 100 ms on each call, ran out of workers, and could serve nothing else, "including health-check calls". Epic gave memcached far more capacity, then took that cache out of the path.
Source: Epic Games: Postmortem of Service Outage at 3.4M CCU (2018), fortnite.com
Quick check
Why did a failing cache stop far more than sign-in checks?
- Calls waited on the cache, and the waiting used up every worker
- The cache deleted the accounts
- Players typed their passwords wrong
Show the answer
Calls waited on the cache, and the waiting used up every worker
A cache is meant to make things faster. When every call waits 100 ms on a failing one, the waits pile up until nothing else gets through.
What not to cache
Microsoft's guide says sensitive or security data should always be read from its real source. Neither should a cache hold answers that differ for every player, like a private save.
Bolt says: Next: CDNs, caches spread around the world.
Fact-checked 12 October 2026
- Microsoft: Cache-Aside pattern learn.microsoft.com
- Martin Fowler: TwoHardThings martinfowler.com
- Epic Games: Postmortem of Service Outage at 3.4M CCU (2018) fortnite.com