Article
AI Agent Failure Rate: Why 70-95% Fail in Production
fiddler.ai
Quoted on this wiki
Every place a page here uses this source, in the order the words come in it.
Verification Strategies: Match the Check to the Risk “The right choice depends on task type and how much risk you can tolerate.” Approach Latency Impact Cost Best For Schema Validation Minimal Low Structured outputs Assertion Tests Low Low Defined constraints LLM-as-Judge High High (external API) Open-ended reasoning
The architectural choice between co-pilot and autonomous patterns directly affects failure rates. Autonomous agents execute multi-step workflows without human checkpoints, meaning a single reasoning error can cascade through downstream actions before anyone notices. “When an agent's confidence score falls below a defined threshold, the task routes to a human reviewer instead of executing automatically.” OWASP identifies "Excessive Agency" as a critical vulnerability, breaking it into three root causes:
Tool and Interface Errors “Multi-tool workflows introduce fragile integration points.” When agents operate independently across multiple applications, reduced human oversight becomes the primary vulnerability, enabling unintended file manipulation and unauthorized transactions. State drift is a related problem: an agent loses track of what it has already done and calls the same API three times, or skips a critical step because it incorrectly believes the task is complete.
LLM-as-judge patterns use a second model to evaluate quality for open-ended tasks like summarization, where schema validation cannot capture nuance. The tradeoff is external API costs and added latency; at scale, that cost compounds significantly. “Retry strategies with exponential backoff and circuit breakers handle transient failures.” For orchestrated workflows, step function retry logic means a failed step retries independently rather than restarting the entire pipeline from the beginning.
Verification Strategies: Match the Check to the Risk “The right choice depends on task type and how much risk you can tolerate.” Approach Latency Impact Cost Best For Schema Validation Minimal Low Structured outputs Assertion Tests Low Low Defined constraints LLM-as-Judge High High (external API) Open-ended reasoning