How to Read an Agent Execution Trace
An alert tells you a run went wrong. The trace tells you why. Here's how to read one — step by step — when a run is slow, expensive, or just produced the wrong answer.
A run is a black box until you open the trace. The trace is the ordered list of steps the agent took — each LLM call, tool call, and retrieval — with timing and token counts attached. Read it well and most failures explain themselves. Here's the mental model.
Step types, and what each tells you
- LLM call — a request to the model. Watch its token count and duration. This is usually where both cost and latency concentrate.
- Tool call — the agent invoking an external function or API. Slow tools and failing tools live here.
- Retrieval — a fetch from a vector store or database. A retrieval that returns junk is the quiet cause of a lot of wrong answers.
- Reasoning step — the agent's intermediate planning. Useful for understanding why it chose the next action it did.
Diagnosing a slow run
Scan the per-step duration_ms and find the one that dominates. Latency is almost never spread evenly — it's one step. If it's an LLM call, you're likely sending too much context or asking for too much output; check the token count on that step. If it's a tool call, the problem is downstream of you — an API that's slow or timing out. If it's retrieval, your index or query is the bottleneck. The trace turns "the run was slow" into "step 3 took 80% of the time," which is the difference between guessing and fixing.
Diagnosing an expensive run
Sort the steps by tokens. A run that cost more than its peers usually has one bloated LLM call — a context window that grew because you stuffed in the whole conversation history, or an output that rambled. Watch especially for the same step type repeating: three near-identical LLM calls in a row is the signature of a retry loop or an agent re-planning because a tool kept failing. That pattern is what quietly triples a bill.
Diagnosing a wrong answer
A run can succeed — status success, no error — and still be wrong. For these, read the trace as a story. Did the retrieval return relevant context, or did it pull the wrong documents and the model dutifully reasoned over garbage? Did a tool return an error the agent swallowed and worked around incorrectly? The wrong answer is almost always upstream of the final LLM call; the trace is how you walk back to the step where the run actually went off the rails.
From trace back to a rule
The best outcome of reading a trace is a rule that catches the next one automatically. If you found a retry loop, that's a cost or failed-runs alert. If you found a slow tool, that's a latency alert. The trace is the investigation; the alert rule is how you avoid having to investigate it twice.
Alerts find the run. Traces explain it. Rules make sure you only have to learn each lesson once.