YouTube Likes Counter: Capacity Estimation

Step 3: Back-of-the-Envelope Estimation

Before we design the architecture, we estimate the scale, because the numbers decide the design. They tell us how much storage we need, and how many requests per second we must handle (throughput). The diagram summarizes these estimates.

Traffic Estimates

Users. About 2 billion people use the site each month (monthly active users, MAU). We assume about 200 million of them use it on a given day (daily active users, DAU). Every number below follows from these two.

Write volume. Assume 20 percent of daily users react, at an average of 5 reactions each. Each like or dislike is one write, so that gives about 200 million write events per day.

Write QPS. QPS means queries per second. On average, 200M / 86,400 seconds = about 2,300 QPS. Traffic is rarely even across the day, so assume a 5x peak during viral events or peak viewing hours. Then we must handle about 11,500 write QPS, with bursts up to 20,000 QPS or more. The burst is the number to design for, because load peaks exactly when a video goes viral.

Read QPS from video views. Assume about 1 billion video views per day. Each view fetches the video like count. That is about 11,500 reads per second.

Read QPS from comment views. An average view also fetches counts for the top 20 comments. That adds 11,500 x 20 = about 230,000 read QPS. Comments, not videos, produce most of the read traffic.

Read QPS from user state. On every view, the client also asks "did I like this?".

Total read QPS. The system must sustain about 250,000 read QPS or more. Reads outnumber writes by about 100 to 1. A ratio this high means the design must depend on caching.

Storage Estimates

Storage splits into two different problems. The counts are small. The record of who reacted to what is enormous.

Count storage. We store the total likes and dislikes per item. There are hundreds of millions of videos and comments, so there are hundreds of millions of counter records. Each record is small, an ID plus a few integers. So the whole set of active counts is in the gigabyte range. It fits easily in memory, in a store like Redis or Memcached.

Reaction storage. To enforce one like per user, and to allow unliking, we must store every user-to-item reaction. This is the user-entity relationship, and it is where almost all the storage goes.

Volume. 200 million new likes a day is about 6 billion new records a month. Over 5 years that grows to hundreds of billions of records.

Size. Assume each record is about 64 bytes: the IDs, a timestamp, and a status. Then 6 billion records is about 384 GB of new data per month. Over five years that is roughly 23 TB. One machine cannot hold that, so we need a database that spreads its data across many machines (a sharded database). Cassandra, DynamoDB, or sharded SQL all fit.