Grokking the Google Forward Deployed Engineer Interview
Vote

0% completed

​

Long Context, Context Caching, and Cost per Request

The freight carrier's exceptions agent makes about 6 model calls per exception: the coordinator plans and checks, and the three sub-agents do their tasks. Each call sends about 12,000 tokens of input and receives about 500 tokens of output. These are this course's working numbers, not measured values.

One exception therefore sends 6 × 12,000 = 72,000 input tokens. At 4,500 exceptions a week, that is 4,500 × 72,000 = 324 million input tokens a week, plus 13.5 million output tokens.

.....

.....

.....

Like the course? Get enrolled and start learning!

Reading Progress

0%


Vote for new content