LLMs at Interview Altitude

1. The Component View

For system design purposes, think of a large language model as a function. You put tokens in, you get tokens out, one token at a time. A token is the unit of text the model reads and writes, roughly three-quarters of an English word. You send the model a sequence called the prompt: your instructions, the conversation so far, maybe some documents. The model then writes a continuation by predicting the next token, adding it to the text, and doing it again.

Remember that one sentence: it writes one token per step