Reference Architecture for a GenAI Feature

Here is the full design of a production GenAI feature on one page.

Read it from top to bottom, following one question from an online store's help page: "How many days do I have to return an item?" The API gateway checks who is asking and how often. The input guardrail screens the text, and the cache looks for a saved answer. If there is none, the model router picks a model, and retrieval finds the help passages. The prompt layer puts the passages and the question into a template. The model call sends that request, with a backup model ready if the first model fails