Designing Data-Intensive Applications

A tour of how databases and distributed systems behave under the hood, and which guarantees you give up when you scale out.

Ideas worth keeping

• Most systems are data-intensive, not compute-intensive. The hard parts are the amount of data, how complex it is and how fast it changes.
• Replication comes in three shapes: single-leader, multi-leader and leaderless. Each one fails in its own way.
• Isolation levels are where the bugs hide. Snapshot isolation still allows write skew.

Write skew, concretely

Two doctors are on call. Each checks that someone else is still on call, then goes off call. Under snapshot isolation both transactions succeed, and nobody is on call:

BEGIN;
SELECT count(*) FROM doctors WHERE on_call = true AND shift_id = 1234;
-- both transactions see 2, so both continue
UPDATE doctors SET on_call = false WHERE name = 'Alice' AND shift_id = 1234;
COMMIT;

The fix is serializable isolation, or locking the rows you read with SELECT ... FOR UPDATE.