Postmortem: The Day Our Cache Stampeded

At 14:02 a single hot cache key expired. Within 200ms, every in-flight request missed the cache and hammered the database simultaneously — a textbook stampede.

The fix: request coalescing

We added a single-flight lock so only the first miss recomputes; everyone else waits for that result.

const inflight = new Map<string, Promise>();
 
function getCached(key: string): Promise{
  const existing = inflight.get(key);
  if (existing) return existing;
  const p = compute(key).finally(() => inflight.delete(key));
  inflight.set(key, p);
  return p;
}