Article
AI Agent Failure Modes: The Eight Ways Autonomous Agents Break
gravity.fast
Quoted on this wiki
Every place a page here uses this source, in the order the words come in it.
“The honest answer is almost always one of eight things, and the eight things are different enough that lumping them together as "the agent broke" loses the information needed to fix the problem.” The eight categories are the same eight that drive the 80-test methodology in how we test AI agents. Generalised here for buyers and operators who want to recognise failures in their own agents, not just understand the testing.
1. Input variation “The same intent phrased differently produces different behaviour.” Defence: explicitly test paraphrases. The input-variation category in the 80-test methodology runs ten paraphrases of the same intent for every capability. Defences include broader tool descriptions, fewer hard-coded phrasings in the system prompt, and giving the agent room to reason about what the user means rather than match keywords.
1. Input variation “Input variation failures appear when the agent's prompt or tool descriptions over-fit to specific phrasings.” Defence: explicitly test paraphrases. The input-variation category in the 80-test methodology runs ten paraphrases of the same intent for every capability. Defences include broader tool descriptions, fewer hard-coded phrasings in the system prompt, and giving the agent room to reason about what the user means rather than match keywords.
1. Input variation “"Send a follow-up to leads who haven't replied in 5 days" works; "ping unresponsive leads from last week" surfaces a different lead set or fails to find any.” Defence: explicitly test paraphrases. The input-variation category in the 80-test methodology runs ten paraphrases of the same intent for every capability. Defences include broader tool descriptions, fewer hard-coded phrasings in the system prompt, and giving the agent room to reason about what the user means rather than match keywords.
The same intent phrased differently produces different behaviour. "Send a follow-up to leads who haven't replied in 5 days" works; "ping unresponsive leads from last week" surfaces a different lead set or fails to find any. Input variation failures appear when the agent's prompt or tool descriptions over-fit to specific phrasings. “The input-variation category in the 80-test methodology runs ten paraphrases of the same intent for every capability.” 2. Tool failure
The same intent phrased differently produces different behaviour. "Send a follow-up to leads who haven't replied in 5 days" works; "ping unresponsive leads from last week" surfaces a different lead set or fails to find any. Input variation failures appear when the agent's prompt or tool descriptions over-fit to specific phrasings. “Defences include broader tool descriptions, fewer hard-coded phrasings in the system prompt, and giving the agent room to reason about what the user means rather than match keywords.” 2. Tool failure
How do you debug AI agent failures? “Input variation: add paraphrase tests. Tool failure: add retry with exponential backoff. Partial results: tighten stopping conditions. Hostile input: strengthen prompt-level refusal policy.” Are AI agent failure modes deterministic?
A downstream API changes its response shape. The new response is missing a field, has a renamed field, or returns a different type. The agent either fails fast (preferred) or hallucinates around the missing field (preferred-not). Schema drift is the most underestimated production failure mode because it is silent until it happens and then it cascades quickly. “If the schema does not match expectations, the agent stops with a clear error rather than continuing with a guessed value.” 7. Refusal failures (both directions)
“The honest answer is almost always one of eight things, and the eight things are different enough that lumping them together as "the agent broke" loses the information needed to fix the problem.” The eight categories are the same eight that drive the 80-test methodology in how we test AI agents. Generalised here for buyers and operators who want to recognise failures in their own agents, not just understand the testing.
What are the main AI agent failure modes? “Eight categories: input variation (the same intent phrased differently), tool failure (a downstream API errors), partial results (the agent completes some steps and not others), hostile input (prompt injection), rate limits (the agent hits a quota), schema drift (an API changed shape), refusal failures (over-compliance or over-caution), and idempotency (the agent double-executes).” Why do AI agents stop after one task?
A downstream API changes its response shape. The new response is missing a field, has a renamed field, or returns a different type. The agent either fails fast (preferred) or hallucinates around the missing field (preferred-not). Schema drift is the most underestimated production failure mode because it is silent until it happens and then it cascades quickly. “Vendor agents that monitor their tool integrations for schema drift between releases catch this faster than agents that assume APIs do not change.” 7. Refusal failures (both directions)
A downstream API returns 5xx, times out, or returns a malformed payload. The agent now has a partial state: did the action happen or not? Tool failures are the most common reason agents stop mid-task in production. “The default should be safe-stop; the exception is when the tool's idempotency guarantees allow safe retry.” 3. Partial results (the "stop after one task" failure)