Introduction: LLM Serving & Inference

Know the Vocabulary

This lesson uses the terms below many times. Read them once now, and come back here when a later section needs one.

• Large language model (LLM): A model trained on very large amounts of text to write text. Given some text, it predicts what comes next, one small piece at a time.
• Token: The unit of text an LLM reads and writes. In English, a token is about three-quarters of a word on average, or about four characters.
• Prompt: All the text sent to the model in one call. It can hold instructions, the conversation so far, and documents.