Step 3: Capacity Estimation

Most reads in this system are not for video likes. They come from the comments under each video, and the arithmetic below shows why. The same numbers give two more results: reads outnumber writes about a hundred to one, and the record of who reacted to what is far too large for one machine. The diagram summarizes the estimates.

Traffic Estimates

Users. Assume about 2 billion monthly active users (MAU), of whom about 200 million use the site on a given day (daily active users, DAU).

Write volume. If 20 percent of daily users react, at an average of 5 reactions each, the system receives about 200 million reactions a day. Each reaction is one write.

Write QPS. QPS means queries per second. Spread evenly over 86,400 seconds, 200 million writes is about 2,300 QPS. Traffic is never even, so we assume a peak of five times the average, about 11,500 write QPS, with bursts above 20,000. We design for the burst, which arrives when a video goes viral.

Read QPS from video views. About 1 billion video views a day, each fetching one video count, gives about 11,500 reads per second.

Read QPS from comment views. Each view also shows the top 20 comments, each with its own count. That adds 11,500 x 20, about 230,000 reads per second.

Total read QPS. Together, the system serves about 250,000 count reads per second, roughly 100 for every average write. A database cannot serve that many reads directly, so the design depends on a cache. Each view also checks whether this viewer has reacted to each item on screen, and those per-user lookups grow at the same rate.

Storage Estimates

Storage divides into two problems of very different size.

Count storage. One record per video or comment holds an ID and two integers. Hundreds of millions of items need only gigabytes, which fits in memory in Redis or Memcached.

Reaction storage. Enforcing one reaction per user per item, with undo, requires storing every reaction, and nearly all the storage goes here. 200 million new reactions a day is about 6 billion a month, and over five years that reaches hundreds of billions of records.

At about 64 bytes per record (two IDs, a timestamp, and a status), 6 billion records add roughly 384 GB a month, or about 23 TB over five years before replication. One machine cannot hold that, so the records need a database that spreads its data across many machines (a sharded database), like Cassandra, DynamoDB, or sharded SQL.