When AI Agents Fail: How to Debug the Delegation

When an AI agent fails, the easiest explanation is “the model was bad.”

Sometimes the model contributes to the failure. The final answer alone rarely tells you where the problem began.

Agents fail as systems. The model may misunderstand the goal. The tool may return bad data. The memory may be stale. The prompt may be vague. The source may contain hostile instructions. The approval gate may be missing. The success check may be too weak to catch the error.

If you debug only the model, you miss the machine around it.

Start with the trace

If an agent may still be causing harm, pause the affected run or risky tool first. Check queued work and in-flight calls separately: stopping new decisions may not cancel an action already accepted by another service. For an active incident, use the incident response guide .

Before rewriting the instruction, preserve the available evidence and ask what happened. A useful trace can show:

• The original goal.
• The recorded plan or proposed steps, if available.
• The sources it read.
• The tools it used.
• The outputs it received.
• Recorded decision points and approval decisions.
• The final action or answer.
• Any errors or skipped steps.

Include a run ID, timestamps, relevant instruction and tool versions, and references to the affected records. Protect logs that contain private data or credentials; share a redacted copy when appropriate.

A trace shows what the instrumentation recorded, not necessarily the model’s internal reasoning or every action. Missing events are gaps, not proof that nothing happened. OpenTelemetry’s tracing overview explains how connected spans represent operations; the observability guide covers what to record for agent runs.

Even a simple trace can reveal the real failure. Maybe the agent never opened the key file. Maybe the search result was old. Maybe it used a tool correctly, but the tool returned incomplete data. Maybe the final answer was polished, but the proof step was missing.

The six common failure points

These six failure points are useful starting hypotheses. A run can have more than one, and the evidence may not establish a single cause.

1. The goal was vague

The agent was told to “clean this up” or “research options” or “fix the issue.” It guessed the real goal.

Fix: define the outcome, the scope, and what done looks like.

2. The context was missing

The agent did not know the business rule, project convention, customer history, or source of truth.

Fix: provide the source, or tell the agent where to retrieve it.

3. The tool was wrong or weak

The agent had a broad tool when it needed a narrow one, or a narrow tool when it needed broader context. Sometimes it had no tool at all and invented a path.

Fix: improve the tool, constrain the tool, or add a verification step after tool use.

4. The memory was stale

The agent reused an old preference, policy, command, contact, price, or account state.

Fix: require fresh retrieval for facts that change.

5. The environment fought back

Web pages changed. Permissions failed. A popup appeared. A hidden instruction inside a page tried to redirect the agent. A file format was not what the agent expected.

Fix: check the actual environment and permissions. Treat instructions found in pages or files as untrusted content, and enforce tool permissions outside the prompt. OWASP’s prompt injection guidance describes how external content can redirect a model; see also the agent prompt injection guide .

6. The evaluation was too soft

The agent produced something plausible, and nobody checked it against the actual requirement.

Fix: define proof before the run. Tests, citations, diffs, totals, approvals, screenshots, or review checklists all count.

Repair the workflow, not the apology

A bad agent run often ends with a fluent apology.

The apology may be polite. It is not the fix.

The fix is a change to the workflow:

• Add a source requirement.
• Add a confirmation gate.
• Make the tool output structured.
• Reduce the task scope.
• Replace a vague instruction with a checklist.
• Require the agent to cite the exact file, row, or record it used.
• Make the final answer include uncertainty and open questions.

A repair should address the observed failure and reduce the chance or impact of a repeat. Record what remains uncertain; a checklist or prompt alone cannot guarantee that the failure will never recur.

The replay method

A replay is an experiment, not an exact reconstruction. Model output, retrieved pages, account state, and tools can change between runs. First record the original input, versions, retrieved material, tool results, and the expected outcome as far as they are available. Mark anything missing.

Before replaying, check whether the original run already changed a file, sent a message, or started a job. A timeout on a write means the outcome may be unknown; check the receiving system’s record before retrying. If the outcome cannot be established, leave it unknown and escalate. The retries and idempotency guide explains how to handle uncertain outcomes and duplicate actions.

Reproduce in an isolated environment with test data and write-capable tools disabled or replaced with safe stubs. A prompt alone is not a safety control: even a preliminary tool call can have side effects. If the task cannot be safely reproduced, inspect the preserved trace and artifacts instead.

Start with the original conditions where feasible, then change one suspected factor at a time. For example, compare a run using the original retrieved policy with one using the correct version. Keep the goal and check the same, record what changed, and compare the results against the expected outcome. One successful replay is evidence, not proof that the issue is fixed; use representative cases and repeat runs where variability matters.

In the isolated environment, a diagnostic instruction can help organize the review:

Use only the supplied test data and read-only tools. Do not send, submit, or
change anything. Identify the evidence used at each step, the proposed action,
and any missing information. Stop before any action and report uncertainty.

Compare that report with the recorded tool calls and source material. An agent’s later explanation can be incomplete or wrong; it does not replace the trace.

Debugging examples

A research agent gives a confident vendor recommendation, but the price is outdated.

Likely failure: stale source or no date comparison.

Repair: require current source checks, capture publication or access dates, and flag pricing that comes from pages without clear dates.

A coding agent changes the right file but breaks a different feature.

Likely failure: narrow context or weak test selection.

Repair: tell it to inspect neighboring code, add or run related tests, and summarize blast radius before editing.

A support agent drafts a reply that reveals private account details.

Likely failure: missing data handling boundary.

Repair: add a privacy rule, redact sensitive fields by default, and require approval for external messages.

For each example, turn the repair into a check: use a dated price fixture and verify the cited price; run the relevant regression test for the code change; or feed a synthetic account record through the draft workflow and verify the protected fields stay out. Include a normal case as well as the failure case, so the new control does not simply block all useful work.

Separate confidence from correctness

Agents can sound certain when they are wrong. They can also sound cautious when they are right.

Do not score the agent by tone.

Score it by evidence:

• Did it use the right sources?
• Did it follow the instruction?
• Did it handle exceptions?
• Did it show what changed?
• Did it ask when the task crossed a boundary?
• Did the result pass an independent check?

Confidence is a feeling. Correctness is a property of the work.

The postmortem questions

For any serious failure, answer these:

1. What was requested, and what did the recorded steps suggest the agent understood?
2. Which context was available, missing, or outdated?
3. What did each relevant tool or source actually return?
4. Which permissions and approvals applied, and what happened in the target system?
5. Which check could have caught the error, and why did it not?
6. What repair has an owner, and how will it be verified?

That last question matters most.

The point of debugging agents is not to prove that agents are unreliable or reliable. It is to make the next delegation sharper than the last one.