0% completed
The Agent Loop
On This Page
How the Loop Works
The Model Does Not Remember
What Each Step Costs
How the Loop Stops
The Six Jobs Around the Loop
Key Takeaways
Practice Questions
Here is a complete agent, written as pseudocode:
const history = [instructions, customerMessage] for (let step = 1; step <= MAX_STEPS; step++) { const reply = callModel(history, tools) history.push(reply) if (reply.toolCalls.length === 0) { return reply.text // The model has finished } for (const call of reply.toolCalls) { const result = runTool(call) history.push(result) } } return handToEmployee(history) // The step limit stopped the run
That short program is the agent loop, and every agent contains some version of it. The program is correct, yet it says nothing about what the model should see, where progress is saved, or whether a refund really happened. This lesson covers how the loop works, what it costs, how it stops, and which jobs the code around it must perform.
How the Loop Works
Each pass through the loop is one step, and a step has three parts:
- The model chooses. Your code sends the history and the tools, and the model replies with text and possibly some tool calls.
- Your code acts. The model only describes the call it wants, so no action happens until your code runs the tool.
- The result joins the history. The next step begins with that longer history.
Two exits end the loop: the model stops requesting tools because it considers the task finished, or the step count reaches MAX_STEPS and your code ends the run.
Notice that the two questions lead to the only two ways out of the loop. Only the second is under your code's control.
The Model Does Not Remember
The model does not remember earlier calls. Each call is separate, and the model knows only what that call contains. When Desk reads the refund policy at step 4, it knows about the broken replacement only because your code sent the results of steps 1 to 3 again. The history is therefore the agent's working record of the task. If your code removes the order details, they no longer exist for the model.
Notice that nothing is ever replaced. Every step sends everything from the earlier steps, plus one more choice and one more result.
What Each Step Costs
Resending the full history on every step has a direct cost. Suppose the instructions, tool descriptions, and customer message take 6,000 tokens, and each step adds 1,800 tokens: 300 for the model's choice and 1,500 for the tool result.
| Step | Tokens sent on this step | Tokens sent so far |
|---|---|---|
| 1 | 6,000 | 6,000 |
| 2 | 7,800 | 13,800 |
| 3 | 9,600 | 23,400 |
| 4 | 11,400 | 34,800 |
| 5 | 13,200 | 48,000 |
| 6 | 15,000 | 63,000 |
The total grows faster than the number of steps, because each new step sends all the earlier ones again. With the same inputs, a 20-step run is about 3 times as long as the 6-step run but sends 462,000 tokens, about 7 times as many. A request that takes twice the usual number of steps can therefore cost far more than twice as much.
How the Loop Stops
A production agent needs more exits than the two in the short program, and each one answers a different failure:
| Stop condition | What triggers it | What Desk does next |
|---|---|---|
| The model finishes | A reply with no tool calls | Sends the reply to the customer |
| Step limit | The step count reaches the maximum | Hands the case to an employee |
| Budget limit | Time, tokens, or money reach a maximum | Hands the case to an employee |
| Repeated failure | The same tool fails several times | Hands the case to an employee |
| Human stop | An employee cancels the run | Saves its progress and ends |
A limit is required because the model can repeat itself without making progress. If the order system is down, the model may retry the same failing lookup on every step, and without a limit the run has no upper bound. With a limit of 25 steps, the same arithmetic gives a maximum of 690,000 tokens. That amount is wasteful, but it is known in advance.
What follows a limit matters as much as the limit itself. A run that ends with an error leaves the customer with nothing, while handToEmployee(history) passes along everything the agent has learned, so the employee does not need to start again.
The first stop condition deserves the most suspicion, because it is only the model's own claim. A reply with no tool calls means that the model considers the task finished, not that the task succeeded. Desk could write "Your refund has been issued" after the refund tool returned an error.
The Six Jobs Around the Loop
A survey of agent system design, "From Question Answering to Task Completion", divides the code around the loop into six responsibilities: observation, context, control, action, state, and verification. Each one is a separate design decision.
Read each card as a question your design must answer. Notice that the model appears only at the top, while all six answers are ordinary code that you write and test.
| Job | The question it answers | If it is missing |
|---|---|---|
| Observation | What does the model see from each tool? | A raw payment export fills the history with unreadable data |
| Context | What does this call contain? | A long run loses the customer's original request |
| Control | Should the run continue, stop, or ask? | The agent loops without limit, or refunds without approval |
| Action | How is a requested tool call carried out? | A refund with an invalid amount reaches the payment system |
| State | Where is progress kept? | A crash at step 5 restarts the run, and the customer may be refunded twice |
| Verification | Did the action really happen? | The agent reports a refund that failed |
The short program covers only part of two jobs: it runs tools (action) and counts steps (control). Most patterns in this course exist to perform these six jobs well, so these cards return in later lessons with the relevant job highlighted.
Key Takeaways
- An agent loop repeats three actions: the model chooses, your code acts, and the result joins the history.
- The model does not remember earlier calls, so your code sends the full history on every step.
- Because the history is sent again each time, total tokens grow faster than the number of steps.
- Every loop needs limits on steps and budget, and each limit needs a defined next action, such as handing the case to an employee.
- A reply with no tool calls is the model's claim that it has finished, not evidence that the task succeeded.
- The code around the loop has six jobs: observation, context, control, action, state, and verification.
The six jobs say what the surrounding code must do, but not which part of the system should own each rule. The next lesson, Anatomy of an Agent System, names those parts and assigns the rules.
Practice Questions
1. At step 4, Desk refers to a broken replacement that it found at step 2. How does the model know about it?
<details><summary>Show answer</summary>Your code sent that result again. The model keeps nothing between calls. It knows the result only because the step 2 tool result is part of the history sent at step 4.
</details>2. Using the inputs in this lesson, how many tokens does a 10-step run send in total?
<details><summary>Show answer</summary>141,000 tokens. Each of the 10 steps sends the base 6,000 tokens, which gives 60,000. Each also sends 1,800 tokens per earlier step, and 0 + 1 + ... + 9 is 45 earlier steps, which gives 81,000.
</details>3. The refund tool returns an error, and the model then replies "Your refund has been issued" with no tool calls. Which of the six jobs should catch this?
<details><summary>Show answer</summary>Verification. The loop stopped correctly by its own rules. Only code that reads the payment system can show that the refund does not exist.
</details>4. Why is handToEmployee(history) better than ending the run with an error when the step limit is reached?
It preserves the work and gives the customer a next step. The employee receives everything the agent has read and tried, while an error leaves the customer without an answer and forces someone to repeat the lookups.
</details>Reading Progress
0%
On This Page
How the Loop Works
The Model Does Not Remember
What Each Step Costs
How the Loop Stops
The Six Jobs Around the Loop
Key Takeaways
Practice Questions