9.2 Operating a Semantic Cache

Which answers may a semantic cache reuse, and for how long? The previous lesson built the mechanism: each question becomes an embedding, Redis runs a filtered nearest-neighbor search over the stored questions, and the stored answer is returned when its cosine distance is under a threshold tuned on labeled pairs. For the support assistant at 1 million requests per day, a 40 percent hit rate cuts the daily cost from $10,000 to about $6,300.

A wrong cached answer costs more than a slow model call, so this lesson covers what keeps the cache correct and safe in production