0% completed
Planning for Year One and Year Three
On This Page
What an Evolution Plan Is
The Load in Year Three
Decide Now, Build Later
The First Version's Capacity
The Stages and Their Triggers
What Is Deliberately Temporary
When a Simpler Plan Is Enough
Using This in an Interview
Key Takeaways
Practice Questions
Suppose you are designing the marketplace's new scheduler. Today, about 12,000 of its 42,000 daily jobs start in the first 10 minutes after midnight, so the peak is about 1,200 job starts a minute. The number of jobs has grown by about 40 percent a year, and it may keep growing at that rate. How much capacity would you build into the first version?
If you build for today's peak, the scheduler may be too small within a year, and the team rebuilds it in the middle of a migration. If you build for the peak three years from now, the team spends an extra quarter on capacity nobody needs yet. Meanwhile, six incidents a quarter continue. So how do you plan for both years at once? The answer has four parts: what to build first, what to decide now but build later, what to mark as temporary, and which measurements start each later stage.
What an Evolution Plan Is
An evolution plan is a written sequence of stages for a system, in which each later stage starts on a measured trigger, not a date. For every part of the design, it chooses one of three answers. The part is built now, prepared now and built later, or built now and removed later.
The plan exists because the first version of a shared system is never its final version. Writing the later stages down lets the team keep the first version small without making the later stages expensive. It also explains the first version to engineers who join after the original team has moved on.
The Load in Year Three
Estimate the load at three years first, because it shows which choices must be made now. At 40 percent growth a year, the load multiplies by 1.4 each year:
| Today | Year one | Year two | Year three | |
|---|---|---|---|---|
| Jobs a day | 42,000 | about 59,000 | about 82,000 | about 115,000 |
| Peak starts a minute | 1,200 | about 1,700 | about 2,350 | about 3,300 |
The year-three figures come from 1.4 x 1.4 x 1.4, which is about 2.74. So 42,000 jobs become about 115,000, and a peak of 1,200 starts a minute becomes about 3,300. Across the whole day, the average is only about 29 starts a minute. The capacity must therefore be set by the midnight peak, not by the daily total.
The growth rate is itself a guess. It could be 20 percent or 60 percent. The plan must therefore work if the guess is wrong, which is the reason for triggers rather than dates.
Decide Now, Build Later
A choice belongs in the first version if changing it later would force every owning team to change something. Capacity is not such a choice. More machines can be added later without any team noticing. Other choices affect every job and are expensive to change once 26 teams depend on them.
The Staff engineer sorted the design's choices into two groups.
| Decided in the first version | Built when a trigger is reached |
|---|---|
| How a job is identified, including a key that lets the work be split across several dispatchers later | Splitting the dispatcher into several parts |
| The run identifier passed to every job | Workflow support |
| The retry rule and how a job declares itself safe or unsafe | Cheaper long-term storage for old run records |
| The interface teams use to define and change jobs | Any further capacity beyond the first version |
The first version therefore stores each job with a key that can later be divided into ranges. At first, one dispatcher handles every range. The dispatcher is the part of the scheduler that finds jobs that are due and sends their run requests. When the load grows, the ranges can be split among several dispatchers without moving any job and without any owning team doing anything.
Building for year three now would have added about one quarter to the first version, mostly for running and testing several dispatchers during the migration. Advanced System Design Fundamentals shows how an estimate tests a design option in When Estimates Change the Design. Here, the estimate decided which choices to fix early.
The First Version's Capacity
The first version is built for 2,400 starts a minute, twice today's peak. That covers about two years of growth at 40 percent, since the peak is expected to reach about 2,350 in year two.
A trigger must fire early enough to finish the next stage before the current capacity is used up. Splitting the dispatcher takes about one quarter. The Staff engineer set the trigger at 1,800 starts a minute, which is 75 percent of the first version's capacity. At 40 percent growth, the peak takes about 10 months to grow from 1,800 to 2,400. That leaves time for one quarter of work, with some margin if growth is faster.
Notice that the trigger sits well below the capacity line. The gap between them is the time the team needs to build the next stage, not spare capacity for its own sake.
The Stages and Their Triggers
The full plan has four stages. Only the first has a date. Each later stage has a trigger.
| Stage | What it adds | Trigger |
|---|---|---|
| 1. One scheduler | One dispatcher, the retry rule, run records, all 26 teams migrated | Starts now; three quarters |
| 2. Split dispatch | Several dispatchers, each owning a range of job keys | Peak reaches 1,800 starts a minute for two weeks |
| 3. Cheaper run history | Run records older than 30 days move to cheaper storage | Jobs a day pass 80,000 |
| 4. Workflows | One job starts after another finishes | 5 teams with a written use case |
The trigger for workflows comes from the non-goal agreed in Choosing What to Leave Out. The plan does not decide again whether workflows are needed. It records what the system needs before they can be added, which is the run record that already exists in stage 1.
Each trigger is measured by something the scheduler already reports. The platform team reviews all four once a quarter, in a 15-minute item at its planning meeting. A trigger that nobody measures never starts its stage.
Notice that stages 2, 3, and 4 do not depend on each other. Any of them may start first, and any of them may never start if its trigger is never reached.
What Is Deliberately Temporary
A temporary part of a design exists only to support the migration or an early stage, and it needs a written removal condition and an owner. Without both, temporary parts stay, because removing them is never anyone's most urgent work.
The scheduler had four temporary parts:
| Temporary part | Why it exists | Removed when | Owner |
|---|---|---|---|
| Converters for A's and B's job formats | Copy job definitions without each team rewriting them | Each old scheduler is retired | Platform team |
| The ownership table, with its per-job switch back to the old scheduler | Keeps each job on one scheduler, and returns a job to its old scheduler if the new one runs it incorrectly | The switch back to each old scheduler ends when that scheduler is retired; the table goes 30 days after the last one is retired | Platform team |
| A daily report comparing old and new schedules | Shows that both schedulers agree during the migration | The last job moves | The Staff engineer |
| Fixed-time chaining of data pipeline steps | Replaces workflows for the data team | Stage 4 starts, if it ever does | Data team |
The converters are the most dangerous item on the list. While they exist, teams can keep writing jobs in the old formats. Every new job written that way is one more job that depends on a converter. The plan therefore turns off new definitions in each old format one month before that scheduler is retired.
The last item may never be removed, and the table says so. Marking it temporary still matters, because it tells the data team that a better option is planned and what will start it.
When a Simpler Plan Is Enough
A four-stage plan with measured triggers is worth its cost for a system that 26 teams depend on and that will run for years. For a tool inside one team, a short note listing what is temporary is usually enough. The team can change its own tool whenever it needs to. The test is the same one used throughout this lesson: how many teams must change something if the first version turns out to be wrong.
Using This in an Interview
When you present a design, say which choices you are fixing now and why. Then name what you are not building yet, and the number that would start it.
"I would build for twice today's peak, which is 2,400 starts a minute and about two years of growth. The job key allows the dispatch to be split later without moving any job. I would start that split when the peak reaches 1,800 for two weeks. The split takes a quarter, and growth from 1,800 to 2,400 takes about 10 months."
Expect two follow-up questions:
- "What if growth is much faster than 40 percent?" Explain that the trigger is measured, not dated, so faster growth simply fires it earlier. Say how much warning the 75 percent level gives at a faster rate.
- "Why not build the split dispatcher now?" Give the cost: about one extra quarter during a migration, while six incidents a quarter continue.
Key Takeaways
- An evolution plan lists stages, and every stage after the first starts on a measured trigger rather than a date.
- Estimate the load at three years to decide which choices must be fixed now, then build the first version for a shorter period.
- Fix early the choices that every owning team would have to change later: identifiers, the job interface, and the retry rule. Add capacity later.
- Set each trigger early enough that the next stage can be finished before the current capacity is used up.
- Give every temporary part a removal condition and an owner, or it will stay.
The plan describes how the system grows, but not what it costs to run at each stage or who pays. The lesson Cost and Operational Load in Numbers counts both the bill and the work of keeping the scheduler running.
Practice Questions
1. Why is the first version built for 2,400 starts a minute rather than today's 1,200 or year three's 3,300?
<details><summary>Show answer</summary>Because 2,400 covers about two years at the expected growth rate, without the extra quarter that year-three capacity would cost. Building for 1,200 would make the system too small within the migration. Building for 3,300 would delay the first version while incidents continue.
</details>2. Why is the job key decided in the first version, while the split dispatcher is built later?
<details><summary>Show answer</summary>Because changing the key later would affect every job and every owning team, while adding dispatchers would not. A choice that affects every job belongs in the first version. Capacity can be added later without anyone outside the platform team noticing.
</details>3. The trigger for splitting the dispatcher is 1,800 starts a minute. What would go wrong if it were set at 2,300?
<details><summary>Show answer</summary>The team would not have time to build the next stage. At 40 percent growth, the peak moves from 2,300 to 2,400 in about seven weeks, while the split takes about a quarter. The scheduler would reach its limit before the work was finished.
</details>4. Why does the plan turn off new job definitions in each old format a month before that scheduler is retired?
<details><summary>Show answer</summary>Because every new job written in an old format adds a dependency on the converter. If teams keep writing jobs that way, the converter cannot be removed, and a temporary part becomes permanent.
</details>Reading Progress
0%
On This Page
What an Evolution Plan Is
The Load in Year Three
Decide Now, Build Later
The First Version's Capacity
The Stages and Their Triggers
What Is Deliberately Temporary
When a Simpler Plan Is Enough
Using This in an Interview
Key Takeaways
Practice Questions