Step 7: Detailed Component Design

Alice (user 123) likes video XYZ1 in the minute it goes viral, along with thousands of other people. Her click stores her reaction in UserLikes, the table of who reacted to what, and raises the total of XYZ1 in ContentStats, the table of totals. The second write is the hard one, because thousands of likes a second all change the same row.

This lesson compares three approaches to that second write, each fixing the weakness of the one before:

1. Write the like and the count together. Exact, but every like waits for one row.
2. Record the like now and count it later. Batching removes the hot row, but reads still hit the database.
3. Add a cache for reads. Reads move to memory. This is the approach we choose.

1. Synchronous updates: write the like and the count together

One database transaction writes Alice's row and adds one to the total of XYZ1, so the total is exact the moment the request returns, and there is nothing else to run.

The trouble starts when XYZ1 goes viral. Every like updates the same counter row, so transactions wait for its lock or fail and retry, and throughput falls exactly when traffic rises.

This suits a forum with a few hundred reactions a second, but at the 11,500 writes a second from Step 3, the single row is the limit, as the diagram below shows.

2. Asynchronous counting: record the like now, count it later

The second approach stores Alice's like now and counts it later. The service writes her row and publishes an event, "user 123 liked XYZ1", to Kafka, a durable log that consumers read in order while recording how far they have read (their offset). Workers read these events and update ContentStats in batches.

Batching removes the hot row. A worker collects up to 100 events or one second of them, groups them by item, and writes once per item. For XYZ1, a hundred likes become one update.

def flush(batch):
    changes = defaultdict(lambda: {"likes": 0, "dislikes": 0})
    for event in batch:
        changes[event.content_id]["likes"] += event.likes_delta
        changes[event.content_id]["dislikes"] += event.dislikes_delta

    for content_id, change in changes.items():      # one write per item, not per event
        row = content_stats.get(content_id)
        content_stats.put(content_id,
                          likes=row.likes + change["likes"],
                          dislikes=row.dislikes + change["dislikes"])

    consumer.commit_offsets()                        # only after the totals are written

The topic is partitioned by content_id, so one worker owns XYZ1 and needs no lock. It commits its offset only after writing, so a crash replays events instead of losing them, and Step 8 keeps a replay from counting twice.

Alice waits only for one insert and one append, and a burst of 50,000 likes waits in Kafka instead of overloading the database.

The cost is a total that lags by a second or two, and every count read, 250,000 a second by the Step 3 estimate, still goes to the database, as the diagram below shows.