LLM Gateway

1. The Incident

Eighteen months of AI enthusiasm, and the store now has fourteen services calling model providers directly. There is the assistant, search, summarization, three agent experiments, a fraud explainer, and eight things nobody has inventoried. The bills arrive together, as they always do.

A key leaks: a team embedded a provider API key in a client bundle, and the key later appears on a public paste site. Rotation requires finding every use across fourteen services, three languages, and an unknown number of copies, then redeploying every affected service