Inference Economics

Know the Vocabulary

This lesson uses the terms below many times. Read them once now, and come back here when a later section needs one.

• Inference: Running a trained model on an input to get an output. For a large language model (LLM), the output is generated text.
• Token: The unit of text an LLM reads, writes, and is billed in. In English, one token is about three quarters of a word.
• Input tokens: The tokens you send to the model: instructions, conversation history, documents, and the new question.
• Output tokens: The tokens the model writes back as its answer.