Grokking the AI System Design Interview
Vote

0% completed

​

Capstone: The Inference Bill

Know the Vocabulary

This lesson uses the terms below many times. Read them once now, and come back here when a later section needs one.

  • Token: The unit of text a large language model (LLM) reads, writes, and is billed in. One token is about three quarters of an English word.
  • Input tokens and output tokens: Input tokens are everything sent to the model for one request. Output tokens are the tokens it writes back. Providers price the two separately, and output tokens usually cost more.
  • Frontier model: The most capable and most expensive class of model a provider offers.

.....

.....

.....

Like the course? Get enrolled and start learning!

Reading Progress

0%

Vote for new content