Introduction

Moving Data: How to Read This Module

Every system design diagram is made of boxes and arrows. The boxes are your services: the API, the database, the email sender. The arrows show data moving between them.

Most engineers spend their attention on the boxes. This module is about the arrows, because every arrow is a decision. When service A needs to tell service B something, you have to choose how that message travels. That one choice decides three things:

• Does someone have to wait for the answer?
• What happens when B is down?
• What happens if the message arrives twice?

Choose lazily, and you get the disasters these lessons open with: a flash sale that crashed a store which had plenty of capacity, and a "real-time" dashboard that cost a fortune just to keep hearing "nothing new yet."

The Map: Seven Patterns, Seven Everyday Ideas

Here's the good news: you already understand these patterns, because people use all seven in daily life. The whole module fits in one table.

Two rules sit under this whole table.

Rule 1: the phone call is the default. If the user is waiting for the answer, use request-response. The other six patterns exist for when waiting is wasteful or unnecessary. They are upgrades with costs, not free improvements. A common junior mistake is using a fancy pattern where a simple call was right.

Rule 2: anything that retries can deliver twice. Queues, events, and webhooks all resend messages when they're not sure a delivery worked. That's a feature, not a bug: it's how they avoid losing data. But it means your code must handle the same message showing up twice without doing the work twice. This idea is called idempotency, and it has its own lesson later in the course. For now, just remember: duplicates are normal. Plan for them.

Read In Order

These lessons tell one continuous story about one online store. Its flash sale breaks the synchronous design (queue). The queue works so well that five other teams want the data (pub/sub). The events are too thin, so everyone calls back for details (event-driven architecture). Then the payment provider starts calling into the store from outside (webhooks). That story runs from Request-Response through Webhooks, so those five lessons make the most sense read in order. The last two, Server-Sent Events and Bidirectional Streaming, are a pair about live streams, and the capstone uses all seven patterns to design one notification system.

In a hurry before an interview? Request-Response, Message Queue, Pub/Sub, and Webhooks are the four that come up most.

Four Terms You'll See Everywhere

• Job vs fact. A job is work to do once: "resize this image." A fact is news: "an order was placed." Jobs go on queues. Facts get announced. Mixing these up is the most common arrow mistake.
• At-least-once delivery. The system promises your message won't be lost, but it might be delivered more than once. See Rule 2.
• Push vs pull. Pull means the client keeps asking "anything new?" Push means the server speaks up when there's news. Pull wastes questions; push holds connections open. Neither is free.
• Synchronous vs asynchronous. Synchronous: the caller stops and waits for the answer. Asynchronous: the caller drops off the message and moves on.

Before You Start

Try answering these for a system you've worked on. "I don't know" is a fine answer; the module exists to change it.

1. Find your slowest API call. Is a human actually waiting for its result?
2. When your code puts a message on a queue, what happens if that message gets delivered twice?
3. Does anything in your product refresh on a timer? Out of every 100 refreshes, how many actually find something new?

Start with Request-Response: the phone call. It's the shortest lesson, and every other pattern is a variation on choosing not to make one.