Design YouTube

2. Clarify functional requirements

Functional requirements list what the system must do for its users.

Must Have:

• Upload Video: Users can upload a video file. The system then converts it so that it plays on any device.
• Stream Video: Users can watch a video with little or no pausing to load (buffering).
• Adaptive Quality: The picture quality, for example 360p, 720p or 4K, changes automatically to match the viewer's internet speed.
• Search: Users can find videos by title or tags.
• Stats: Users can like a video and see how many views and likes it has.

Nice to Have:

• Resumable Uploads: If an upload stops halfway, it continues from where it stopped instead of starting again.
• Watch History: A viewer can return to a video and continue from where they left off.

Notice that the two most important requirements have opposite needs. Uploads can be slow as long as they are reliable, while streaming must be fast every time.

3. Clarify non-functional requirements

Non-functional requirements describe how well the system must work: how big, how fast, and how reliable.

• Scale: The system must support 1 billion daily active users (DAU), meaning 1 billion different people use it on a normal day.
• Read vs. Write: The system is extremely read-heavy. There is roughly 1 upload for every 1,000 views, a ratio of 1:1000.
• Latency: Latency is the time a user waits for a response.
  • Uploads: A delay of several minutes is acceptable. Processing happens in the background (asynchronously), so the creator does not have to wait on the page.
  • Streaming: Playback must start quickly. The time to first byte (TTFB), which is the time until the first piece of video data arrives, should be under 200 ms.
• Availability: 99.99%. Playback is the core product, so it must almost never be down. For reference, 99.99% still allows about 52 minutes of downtime a year, so the design must contain failures rather than assume none.
• Consistency:
  • Metadata: The creator must see their own new video and its title as soon as they save it. This is called read-your-own-writes consistency. Other viewers can see the change a moment later.
  • View Counts: Eventual consistency is fine. If a view count is a few minutes behind, nobody is harmed.
• Data Retention: Videos are kept forever unless the creator deletes them.

The 1:1000 ratio is the most important number on this list. It tells us that any expensive work should happen once, when a video is uploaded, and not again each time someone watches it.

4. Back of the envelope estimates

These estimates are rough on purpose. Their job is to show which part of the system needs the most attention.

• Storage (Writes):

  • Starting figure: 500 hours of video are uploaded every minute.
  • Per day: 500 hours × 60 minutes × 24 hours = 720,000 hours of new video a day.
  • Assume one hour of uploaded video is about 1 GB. This is a simple average for the estimate; a phone recording at 1080p is often 3 to 5 GB per hour, so real numbers can be several times higher.
  • Raw storage: 720,000 hours × 1 GB = 720 TB per day.
  • Transcoding: We keep several versions of each video (360p, 480p, 720p, 1080p and 4K). Together with the original, they take about 3 times the raw size.
  • Total new storage: 720 TB × 3 ≈ 2 PB (petabytes) per day.

• Bandwidth (Reads):

  • Starting figure: 5 billion views per day, which is about 5 views for each of the 1 billion daily users.
  • Assume an average view downloads 100 MB, a mix of short mobile views and longer desktop views.
  • Total data sent to viewers (egress): 5 billion × 100 MB = 500 PB per day.
  • Throughput: 500 PB ÷ 86,400 seconds ≈ 5.8 TB per second, which is about 46 terabits per second. Busy evening hours run higher than this average.

• Checking the ratio: If an average upload is about 7 minutes long, 720,000 hours a day is about 6 million videos. Against 5 billion views, that is close to the 1:1000 ratio from section 3.

• Conclusion: No single data center can send 5.8 TB every second to viewers spread across the world. Video must be served from a global Content Delivery Network (CDN), which is a large set of cache servers placed close to viewers in many cities.

5. API design

The API is the set of requests the app sends to our servers. We use a REST API, which means plain HTTP requests on named URLs. One rule shapes the whole design: the video file itself never passes through the API servers. The API only handles small text requests.

1. Initiate Upload

• POST /videos/init
• Request: {"title": "My Cat", "size": 1073741824, "format": "mp4"} (the size is in bytes, here 1 GB)
• Response:

{
  "video_id": "vid_123",
  "upload_url": "https://s3-upload-signed-url..."
}

• Note: The response contains a presigned URL. This is a temporary link, signed by our server, that lets the app upload this one file straight into cloud storage.

2. Stream Video

• GET /videos/{video_id}
• Response:

{
  "title": "My Cat",
  "stream_url": "https://cdn.youtube.com/vid_123/master.m3u8",
  "views": 10500,
  "likes": 200
}

• Note: stream_url points to a manifest file, a small text file that lists every available quality and where its pieces are stored. It does not point to one large video file. Section 8 explains how the player uses it.

3. Search

• GET /search?q=cat&page=1
• Response: A list of short video summaries: ID, title, thumbnail, and view count.

4. Like Video

• POST /videos/{video_id}/like
• Response: 200 OK
• Note: The server records each like as one pair of user and video. If the user taps twice, the second request finds the pair already there and changes nothing, so a like is never counted twice.

6. High-level architecture

We split the system into two parts. The control plane handles small text requests: video details, users, search and likes. The data plane moves the video bytes themselves: uploading, processing, storing and streaming them. The two parts have very different traffic, so we scale them separately.

Architecture Flow:

1. Client (the mobile app or web browser)
2. Load Balancer, which spreads requests across many API servers
3. API Gateway, which checks who the user is (authentication) and limits how many requests one user can send (rate limiting)
4. Microservices, small services that each own one job:
   • Video Metadata Service
   • User Service
   • Search Service
5. Databases & Cache:
   • SQL database for video details
   • Redis, an in-memory cache for the most requested details
   • Elasticsearch, a search engine that holds the search index
6. Async Processing:
   • Message Queue (Kafka), which holds processing jobs until a worker is free
   • Transcoding Workers, servers that convert videos, often with GPUs or dedicated video chips
7. Storage:
   • Object Storage (such as Amazon S3), the permanent home of every video file
   • CDN, the edge servers that stream video to viewers

Component Roles:

• CDN: Serves about 99% of all video traffic. Most viewers never reach our own storage.
• Object Storage (S3): Stores very large video files cheaply and safely. It is the source of truth, meaning that if any copy disagrees, this one is correct.
• Transcoding Workers: Convert each uploaded video into the formats and sizes that players can stream (HLS or DASH, explained in section 8).
• Metadata DB: Stores the text about each video: title, description, uploader and processing status.

10. Storage, indexing and media

• Blob Storage (S3): Blob storage means storage for large files (binary large objects).

  • We use storage tiers, which are classes of storage with different price and speed.
  • Hot: New and popular videos stay in standard storage, which is fast to read.
  • Cold (for example, S3 Glacier): The original uploaded files of old videos that nobody watches move to slower, much cheaper storage. This is safe because playback never reads the original file; it reads only the transcoded segments.

• Search Indexing:

  • When a video becomes active, a background service copies its title and tags into Elasticsearch.
  • Elasticsearch keeps an inverted index, which maps each word to the list of videos that contain it. That makes word search fast, which a SQL database does not do efficiently at this size.

11. Scaling strategies

• Database Sharding:

  • We split the metadata database across many machines by video_id.
  • A simple rule is shard_id = hash(video_id) % total_shards, which spreads videos evenly.
  • The weakness of this rule is that changing the number of shards moves almost every row. Real systems use consistent hashing or a fixed set of logical shards, so adding machines moves only a small part of the data.

• Parallel Transcoding (Split & Merge):

  • Converting a long 4K video on one server can take hours.
  • Split: We cut the uploaded video into pieces of about a minute each. Each cut is made at a keyframe, a frame that does not depend on earlier frames, so every piece can be converted on its own.
  • Fan-out: We send the pieces to many workers, for example 50, which convert them at the same time.
  • Merge: A final step joins the results and writes the manifest. Processing time drops from hours to minutes, and if one piece fails, only that piece is redone.

• Edge Servers Inside Internet Providers:

  • We place CDN servers inside the networks of internet service providers (ISPs), and we connect our network directly to theirs (peering). Video then travels a much shorter path to the viewer, which lowers both delay and our cost of sending data.

12. Reliability, failure handling and backpressure

Backpressure means slowing down the intake of work when the parts that process it are full. The message queue from section 8 is our main form of backpressure.

• Dead Letter Queues (DLQ): If a video makes a worker crash, for example a corrupted file, we retry it 3 times. If it still fails, we move the job to a separate queue (a dead letter queue) for an engineer to inspect. This way one bad file cannot block every video behind it.

• Thundering Herd Protection: A thundering herd is a large number of requests for the same thing at the same moment. When a very popular creator publishes a video, a million viewers may request it before any edge server has it. Each edge server uses request coalescing: the first request fetches the segment from the origin, and all other requests for that segment wait for that one response. A middle layer of regional caches (an origin shield) does the same between the edge servers and the origin, so the origin receives one request per region, not one per viewer.

• Circuit Breakers: A circuit breaker stops calling a service that is failing and returns a fallback instead. If the view and like counter service is slow or down, the watch page still loads and plays the video, just without the counts. The core watching feature does not depend on it.

13. Security, privacy and abuse

• Presigned URLs: Users never get general write access to our storage. Each upload receives a presigned URL that works for one file at one location and expires after a short time.

• Signed Playback URLs: For private and age-restricted videos, the manifest and segment URLs carry a signed token that expires. A copied link stops working after a short time, so the permission check in Flow B cannot be skipped by sharing a URL.

• Digital Rights Management (DRM): For paid content, the segments are encrypted. The player must request a decryption key from a license server, and only approved players receive it.

• Content ID: During processing, we create audio and video fingerprints, which are short summaries of the content, and compare them against a database of copyrighted material. A match can block the upload or apply the rights owner's policy.

14. Bottlenecks and next steps

• Bottleneck: Bandwidth cost. Sending video is the largest cost of the whole system.

• Next Step: Use per-title encoding. Analyze each video's complexity and choose its bitrates to match. A simple cartoon needs fewer bits than a fast action scene for the same visible quality. Netflix reported savings of about 20% in bandwidth from this approach.

• Bottleneck: Search freshness. A new video can take minutes to appear in the main search index.

• Next Step: Keep a small real-time index for videos uploaded in the last hour, search both indexes together, and merge the small one into the main one later.

The complete picture

Every diagram above shows one part of the system. This one shows all of them together: the upload path, the background processing, the metadata path and the streaming path, with the caches, queues and scaling strategy of each one marked. Notice that the upload path and the streaming path meet in only one place, object storage.

Summary

Say this first. YouTube is two systems behind one website. The control plane handles small text requests, like titles, likes and search, and it works like a normal read-heavy web service. The data plane moves the video itself, about 5.8 terabytes a second, and that traffic never passes through our application servers. Once you separate those two, every other decision follows from it.

The decisions that mattered

The numbers that forced them

What breaks first

1. Bandwidth cost. This is a money problem, not an outage. Better codecs and per-title encoding are the main tools, and both are worth mentioning before the interviewer asks.
2. A corrupted upload blocking the pipeline. Three retries and then a dead letter queue, so one bad file cannot stop every other video behind it.
3. A very popular new upload. A million people request a video that no edge server has yet. Request coalescing and an origin shield mean the origin serves each segment once per region, not a million times.

If the interviewer pushes

• "Why not store one high-quality file and convert it to lower quality as people watch?" Converting on demand costs computing power for every view. Converting once costs it for every upload. At 1,000 views for every upload, converting once is far cheaper.
• "How does the player know when to switch quality?" It does not ask our servers. It downloads a manifest that lists every rendition, measures its own download speed between segments, and chooses the quality of the next segment itself.
• "What happens when the search index falls behind?" A new video cannot be found for a few minutes. The fix is a small real-time index for the last hour, searched together with the main index and merged into it later.
• "Where would this design stop working?" Live streaming. Everything here assumes the whole file exists before anyone watches it, while live video has to be processed and delivered seconds after it is recorded.