Introduction: LLM Serving & Inference

1. What This Module Covers

Up to now, every system in this course predicted something: a score, a label, a yes or no. Will this user click? Is this payment fraud? Starting now, the system writes something instead: an answer, a summary, a first draft. That one shift, from predicting to generating, flips the economics of the whole design. Inference, by the way, is just the word for running a trained model to get an answer.

Here is the difference in one line, carried over from Module 2. There, the refrain was "tabular inference is nearly free; the pipelines are the bill