0% completed
Choosing Compute: Servers, Containers, and Functions
On This Page
Three Ways to Run Code
How Capacity Grows
One Day of Traffic
Choosing by Workload Shape
The Online Store Decision
Using This in an Interview
Key Takeaways
Practice Questions
At 4 a.m., the retailer's online store receives about 30 orders an hour. At 8 p.m., its busiest normal hour, it receives about 400. On the first morning of the holiday sale, it receives about 2,000 orders in the hour after the sale opens at 9 a.m. and 2,400 by 10 a.m.
In the old data center, three fixed servers carried this traffic. Each server handles about 200 orders an hour within the target response time, so the three together served 600 orders an hour. That was enough for any normal day, but it was only a quarter of the sale peak.
This lesson compares the three main ways to run code in the cloud against that traffic pattern. For each one, it shows the billing, the speed of growth, and the work left to the team.
Three Ways to Run Code
A cloud platform runs code on virtual machines, in containers, or as functions, and each option moves a different part of the work to the provider.
A virtual machine (VM) is a server that the provider creates from shared hardware in a few minutes. You choose its size and operating system. You also install and patch the software, and you pay for each second it runs. On AWS, VMs are EC2 instances.
A container runs an application from a packaged file (an image) that holds the application and its libraries, so that it starts the same way on any server, usually in seconds. A managed orchestrator is a service that places containers on servers, restarts failed ones, and adds more with traffic. On AWS, ECS and EKS are orchestrators. With Fargate, an option for both, the provider also runs the servers underneath, so you pay for the processor and memory each container reserves.
A serverless function is code that the provider runs only when an event arrives, such as a web request or a new file. You pay for each request and for each millisecond of run time at the memory size you choose, and you pay nothing when no event arrives. On AWS, functions run on Lambda, and one call can run for at most 15 minutes. The first request to a new copy of a function waits while the provider starts it, which is called a cold start.
Azure and Google Cloud offer equivalents of all three options under different names.
| Option | Billed by | Typical start time | Main limit | Team effort |
|---|---|---|---|---|
| Virtual machine | Second of run time | 3 to 5 minutes | You size and patch it | Highest |
| Container on Fargate | Reserved processor and memory per second | Under a minute | Application must be packaged as an image | Medium |
| Function on Lambda | Request and millisecond of run time | Under a second, longer on a cold start | 15 minutes per call | Lowest |
The start times are typical values for a small application. A large image or a slow application start makes every option slower.
How Capacity Grows
An auto scaling group is a set of VMs that the platform keeps between a minimum and a maximum count, adding or removing VMs by a rule. The most common rule is target tracking: the group keeps a measured value near a target, such as average processor use at 50 percent.
New capacity does not arrive at once. With one-minute monitoring turned on, the rule usually waits for a few readings, about 2 minutes in all. The VM then takes 3 to 5 minutes to start the store application. Finally, the load balancer, the service that spreads requests across the VMs, checks that it answers. In total, a new VM usually serves its first order 5 to 7 minutes after traffic rises. A container waits for the same readings but starts in under a minute, so it serves in about 3 minutes.
On a normal evening, traffic rises from 200 to 400 orders an hour over several hours, so a delay of 6 minutes does not matter. At the sale, traffic rises from 600 to 2,000 orders an hour within minutes. A group that reacts only to processor use would run short of capacity during the busiest minutes of the year.
The fix is scheduled scaling, a rule that changes the group's size at a set time. The retailer's group raises its minimum to 12 VMs at 8 a.m. on each sale day, because 2,400 divided by 200 is 12. Target tracking still runs above that minimum and adds VMs if traffic exceeds the plan.
One Day of Traffic
The diagram compares the capacity of each approach on the first sale day.
The fixed servers lack capacity from 9 a.m. to about 11 p.m. At the last sale, this shortage made the store unavailable for 3 hours on the first morning. The auto scaling group is ready one hour before the sale opens.
Choosing by Workload Shape
The shape of the load over time decides the compute option more than the application's language or size.
| Workload shape | Retailer example | First choice | Reason |
|---|---|---|---|
| Steady | ERP system, similar load all day | VMs, rehosted | No scaling needed; a discount for committing to a year or more of use lowers the price |
| Spiky | Online store, 6 times normal in sale week | Auto scaling VMs or containers | Capacity follows traffic, with scheduled scaling for known peaks |
| Event-driven | Resizing each new product image | Functions | Runs only when a file arrives, nothing to pay between uploads |
| Long-running batch | Nightly sales report, about 40 minutes | Containers or VMs | One function call stops at 15 minutes |
Functions become expensive when traffic is high and steady, because per-request billing then costs more than a VM that is busy all day.
The Online Store Decision
The SA compared two ways to provide sale capacity. Fixed VMs sized for the sale would run 12 VMs all year: 12 times 8,760 hours is 105,120 VM-hours. The auto scaling group runs about 2.5 VMs on an average normal day and 12 during the sale week.
Normal days: 358 days x 24 hours x 2.5 VMs = 21,480 VM-hours Sale week: 7 days x 24 hours x 12 VMs = 2,016 VM-hours Total: 23,496 VM-hours 23,496 / 105,120 = 0.22, about 22 percent of the fixed option
VM-hours are not cost, because prices and discounts differ, but the gap explains most of the saving in the retailer's estimate.
The SA chose VMs in an auto scaling group and rejected the other two options. Containers would scale faster, but the store was not packaged as images. The 12 engineers would also have had to learn an orchestrator before the sale in month 9. Functions would have required rewriting the store, and cold starts at 9 a.m. would affect the first customers of the sale. The proposal recorded containers as a later step, after the third wave.
In the sale in month 9, the busiest hour reached about 2,600 orders. Target tracking added a 13th VM above the scheduled 12, because 2,600 divided by 200 is 13.
Using This in an Interview
When you choose compute, name the workload shape first, then the option, then the option you rejected and why.
"The store runs 400 orders an hour normally and 2,400 in sale week, and each server handles about 200. I would use an auto scaling group with a minimum of 2 VMs and a scheduled rise to 12 before the sale opens, because metric-based scaling takes about 6 minutes. I would not use functions, because the store would need a rewrite and the team has a fixed deadline."
A mid-level answer chooses a sensible option. A senior answer also states the capacity arithmetic, the scaling delay, and the team's ability to run the choice.
Expect two follow-up questions:
- "Why not containers?" They start faster, but the team must first package the application and learn an orchestrator. Plan them for after the migration.
- "What if the sale is twice as big as planned?" Target tracking adds VMs above the scheduled minimum, so set the maximum well above 12, such as 20, and test it.
Key Takeaways
- VMs, containers, and functions differ in billing, start time, limits, and the work left to the team.
- New VMs take about 5 to 7 minutes to serve traffic, so known peaks need scheduled scaling.
- Choose by workload shape: steady, spiky, event-driven, or long-running batch.
- The retailer's store uses an auto scaling group because the team could run it before the sale.
The next lesson, Choosing Storage and Databases, matches each kind of retailer data to the store that suits it.
Practice Questions
1. Why does target tracking alone fail at the start of the sale?
<details><summary>Show answer</summary>Because new VMs take about 5 to 7 minutes to serve traffic. At 9 a.m., traffic rises from 600 to about 2,000 orders an hour within minutes. Scheduled scaling starts the 12 VMs an hour before the sale opens.
</details>2. The nightly sales report takes about 40 minutes. Can it run as one Lambda function call?
<details><summary>Show answer</summary>No, because one call can run for at most 15 minutes. Run it as a container task or on a VM, or split it into parts that each finish within the limit.
</details>3. The sale grows to 3,600 orders an hour. How many VMs does the store need?
<details><summary>Show answer</summary>About 18, because 3,600 divided by 200 is 18. The scheduled minimum should rise to 18, and the maximum must be above that number. A new load test should confirm that the database also handles the higher rate.
</details>Reading Progress
0%
On This Page
Three Ways to Run Code
How Capacity Grows
One Day of Traffic
Choosing by Workload Shape
The Online Store Decision
Using This in an Interview
Key Takeaways
Practice Questions