Grokking the Google Forward Deployed Engineer Interview
0% completed
Scaling Agents: Concurrency, Quotas, and Cost
The freight carrier's exceptions arrive Monday to Friday, about 900 a day. Monday is the peak at about 1,400, and its busiest hour brings about 220.
An average exception uses about 6 model requests, each with about 12,000 input tokens. For sizing, this lesson uses 8 requests per exception, which leaves a margin for longer runs. In the busiest hour, that is 220 × 8 = 1,760 requests, or about 29 a minute. It is also 1,760 × 12,000 = 21,120,000 input tokens, or about 352,000 a minute.
Averages do not cause failures
.....
.....
.....
Like the course? Get enrolled and start learning!
Reading Progress
0%