What Is Agentic System Design?
Short answer: agentic system design is the practice of designing systems where the model chooses its own next step at runtime. It does not follow steps an engineer fixed in advance. Most of the design work is not about making the model smarter. It is about bounding that freedom. Which tools it may call, how much context it carries, when the loop must stop, and which actions need a human to approve.
The term sounds new, but the engineering is familiar. You are designing a component that makes decisions under uncertainty, and then putting limits around it.
The one test that separates an agent from a workflow
Remove the marketing language and one question decides it. Who chose the next step, and when?
In a workflow, the engineer chose the steps at design time. The model fills in the blanks along a fixed path. Transcribe, then summarize, then extract action items. That is three model calls and zero decisions about what happens next.
In an agent, the model chooses the next step at runtime, from what it has seen so far. You say "resolve this customer's issue" and nothing more. It decides on its own to look up the order, then check the refund policy, then issue a credit. Nobody scripted that order in advance.
That loop is the whole idea. Reason, act, observe, repeat.
Autonomy is a spectrum, not a switch
Being able to place a design on this spectrum is worth more in an interview than any framework name.
| Level | Who decides | Typical use |
|---|---|---|
| Fixed workflow | Engineer, at design time | Known, repeatable tasks |
| Routed workflow | Model picks a branch | Triage, classification |
| Agent with approval | Model picks tools, human approves writes | Anything with money or data loss |
| Bounded agent | Model acts within limits, reviewed afterward | Well-understood, reversible work |
Most production systems that are marketed as agents sit in the middle two rows. Saying so is credibility, not a concession.
The five parts you actually design
Every agent system has the same parts, and each one is a design decision.
The model. Which tier, and where a cheaper one is good enough. Planning and summarizing rarely need the same model.
The tools. What the agent may do to the world. Tool names and descriptions are an interface for the model, in the same way an API is an interface for a developer. Vague descriptions make the model guess.
Memory. What the agent carries between steps, and what it deliberately drops. Long accumulated history makes judgment worse and costs more per step.
The orchestrator. What runs the loop, enforces budgets, and decides when to stop.
The environment. What the agent can see and touch, and the boundary it cannot cross.
Why bounded loops are not optional
Nothing about a model guarantees that a loop ends, makes progress, or stays affordable while it runs. That is the central failure mode, and it is arithmetic rather than opinion.
A step that succeeds 95 percent of the time is fine once. Across twenty steps, success falls to about 36 percent. Compounding error is the reason nearly every pattern in this field exists.
So every real agent carries hard limits. A maximum number of steps, a token or cost budget, a timeout, and a rule for what happens on failure. Designing those limits is the job.
What this is not
Agentic system design is not prompt engineering, and it is not picking a framework. Frameworks change every few months. The decisions above do not.
It is also not the same as a chatbot with a search tool attached. If the sequence of steps is fixed, you have a workflow, and a workflow is usually the better choice. Build the workflow first, then add autonomy only where the task genuinely branches.
How to Prepare
- Learn the single-agent design first. Multi-agent is the hyped topic and the wrong starting point. See how do you design a multi-agent system for why the default answer is one agent.
- Get the tool layer right. Most agent failures are tool design failures, not model failures. See what is MCP (Model Context Protocol).
- Know the retrieval piece. Agents read from stores constantly, so the retrieval answers apply directly: retrieval-augmented generation and vector databases.
- Be able to cost it. Interviewers ask what an agent run costs. Practice with how to scale an LLM-based application.
- Study the full chapter. Grokking the AI System Design Interview has an Agentic Systems chapter. It covers the loop, tool use, orchestration, memory, and evaluation, ending in a customer support capstone. Read what an agent actually is first.
GET YOUR FREE
Coding Questions Catalog
$123
$197
$72
