Capstone: Design an LLM Inference API

This lesson gives the full 45-minute answer to:

Design an API for model inference.

The hard part is not drawing one box labeled Model. It is serving three callers that value different things while publishing one contract they can all understand.

"I will design a multi-tenant inference API that accepts text input and returns text, structured output, or tool calls. It supports streaming interactive responses, asynchronous batch work, live model capability and price discovery, token estimation, usage reporting, idempotent billing, and compute limits. I will not design training, mod