You deploy your new AI agent. A user asks it to find a contract. The agent calls a tool, then another, and another. It spins its wheels for a dozen iterations, burns through the context window, and either times out or confidently returns a completely fabricated answer.
Your engineering team sighs and says, “The model is just unstable right now.” But as a product manager, you feel it in your gut: the problem isn’t the model. Your gut is right.
60% of AI agent failures happen before a single line of code is ever written.
The root cause of unreliable agent behavior isn’t a lack of reasoning ability. It’s a lack of explicit design in how tools are defined, described, and expected to fail. If you want to stop your agent from inventing plausible-looking lies, you don’t need a better model. You need better product design.
Here are the three design decisions you must fix immediately.
1. Tool Granularity: The Goldilocks Trap
Engineers love to build tools, but they usually build them wrong. They fall into two extremes:
Too Coarse: A single tool like processOrder() handles validation, inventory deduction, messaging, and logging. When the agent calls it and something breaks, it has no idea which step failed, and it certainly can’t retry just the broken part.
Too Fine: The tools are shattered into a dozen micro-steps. To complete one task, the agent has to make 10+ consecutive calls, flooding the context window with raw data. Once an agent makes more than 8 tool calls, task completion rates drop by over 20%.
The rule is simple: A tool must perform one atomic operation with a verifiable result. If an agent can’t independently verify a tool’s result, your granularity is dead on arrival. If the output of Tool A requires calling Tool B just to make sense of it, you’ve failed the design phase.
2. Tool Descriptions: The Model Can’t Read Your Mind
Many product managers think tool descriptions are an engineering afterthought. They aren’t. The tool description is the only thing the model looks at when deciding what to call. It doesn’t see your elegant code; it only sees the text prompt.
Here is a terrible description: search(query) — “Searches for relevant content.”
When you have 10+ tools, vague descriptions cause the model to randomly guess between similar functions, or trigger tools when it shouldn’t. The model doesn’t read your code. It reads your descriptions. And your descriptions are a mess.
A good description needs four elements: the action, when to use it, when NOT to use it, and the structure of the return value. Changing “search contracts” to “Searches the contract database. Use when the user provides a keyword or vendor name. Do NOT use for contract status (use get_status). Returns a list of matches with IDs and dates.” will instantly boost accuracy by 30-40%.
3. Failure Handling: The Silent Hallucination Engine
This is the most dangerous blind spot in AI products. Tool calls fail in three ways: hard failures (network timeouts), soft failures (successful call, but empty/unexpected results), and hallucinated calls (the model thinks it called a tool that doesn’t exist).
Soft failures are the silent killer. When a tool returns an empty list, the LLM doesn’t stop and say, “I couldn’t find it.” It assumes the absence of data means the absence of constraints, and it continues reasoning on top of a void. An empty result isn’t an absence of data; to an LLM, it’s a blank canvas for hallucination.
You must design explicit failure semantics. Never return an empty array. Return a failure object: {“success”: false, “reason”: “no_match”, “suggestion”: “try broader keywords”}. Tell the model directly in the tool description: “If success=false, do not assume data exists. Inform the user.”
When your AI agent goes off the rails, don’t blame the inference engine. Look at your tool design. The fix for hallucination isn’t more compute—it’s a better PRD.
FAQ
Q: Isn't this just an engineering problem? Why should product managers care?
A: Because engineers build tools based on technical logic, not user-facing logic. A PM must define the behavioral semantics—when a tool should be used, what happens when it fails, and how the agent should respond to empty data. This is product design, not code.
Q: How do I know if my tool descriptions are good enough?
A: Give the description to a non-technical team member (like a customer support rep). If they can't tell you exactly when to use the tool and when not to use it, the description is broken and needs a rewrite.
Q: Doesn't this mean LLMs are just too dumb to handle ambiguity?
A: No, it means LLMs are highly literal pattern matchers. Giving them vague descriptions and silent soft-failures is like giving a human employee a broken compass and blaming them for getting lost. Good design makes smart models look brilliant.