The pitch for agentic AI is compelling: systems that plan, use tools, loop until the task is done, and handle the kind of multi-step work that flat LLM calls can't. We've been building these systems in production. Here is the honest accounting.
The failure mode no one talks about
Agents fail at the seams. The same seams that break every distributed system — state management, error propagation, retry logic, timeout handling — except now you have an LLM making decisions about how to proceed when things go wrong. That's a non-deterministic decision-maker sitting at the most critical point in your error-handling path. Plan for it.
The second failure mode is hallucinated tool calls. An agent that can call your APIs will occasionally call them with confidently wrong parameters, at the wrong time, on the wrong resource. We've seen agents attempt to update records that don't exist, call deprecated endpoints because they appeared in the context, and construct authentication headers from pattern-matched memory rather than the actual credentials provided. These are not edge cases. They happen in the first week of production.
What actually works
Structured outputs with validation. Every tool call result should be validated against a schema before the agent acts on it. Every agent output destined for a downstream system should be typed. The LLM is good at generating structure; you are responsible for enforcing it. Don't rely on the model being consistent across calls.
Narrow tool surfaces. The broader the tool surface, the more ways an agent can go wrong. A tool that does one specific thing — not a general API wrapper — limits the blast radius of hallucinated calls. We've found that agents with 3–5 well-scoped tools outperform agents with 15 general ones, in both reliability and task completion rate.
Explicit state, not implicit memory. Agents that track their own state through conversation history alone accumulate errors. Context grows stale. Earlier summaries become authoritative and wrong. Pass state explicitly — structured, typed, outside the context window if the task runs long. Treat the LLM as a reasoning engine, not a database.
Deterministic fallbacks for high-stakes branches. Not everything in an agentic workflow should be decided by the LLM. When the outcome of a decision has a large blast radius — deleting data, sending communications, calling external APIs with side effects — write a deterministic guard. The agent can recommend; the guard decides.
Evaluation loops before production. Agents need to be evaluated the same way you'd evaluate any probabilistic system: with a test suite that covers edge cases, a benchmark of expected outputs, and regression testing when the underlying model changes. Model updates silently break agentic behavior in ways that unit tests won't catch. Build your eval suite before your first production deploy.
What doesn't work
Long single-agent chains. A chain of 15 steps where each step's output feeds the next is a compounding error machine. By step 10, a small hallucination at step 3 has propagated into a confident wrong conclusion. Break long chains into checkpoints with human-in-the-loop review or deterministic validation between stages.
Trusting the agent to know when to stop. Agents will loop when they should stop. They will also stop when they should loop. Maximum step limits are not optional. Distinguish between "task complete" and "I've run out of ideas" in your termination logic — they require different responses.
Multi-agent systems without message contracts. Two agents passing free-form natural language to each other is not a system design. It's a conversation. Define message schemas. Define what each agent is responsible for producing. Treat inter-agent communication like an internal API — because that's what it is.
Skipping security review. An agent with access to your database, your email, and your CRM is a significant trust boundary. Prompt injection — where malicious content in a tool's response hijacks the agent's next action — is real and documented. Security review of agentic systems is not a formality. If your codebase is getting agentic AI features, get the security surface audited before you ship. That's exactly what Eloquent Lens does →
The honest summary
Agentic AI is not new magic. It's distributed systems with a probabilistic component in the control path. The same engineering rigor applies — tight interfaces, explicit state, deterministic fallbacks for critical paths, test coverage, monitoring. The teams winning with this are not the teams that moved fastest in 2024. They're the teams that treated production reliability as a first-class requirement from the start.
If you're building production agentic systems and want a second opinion on the architecture: ai@eloquentix.com →