10decoders
The Answer Was Fluent, Confident, and Wrong
A finance operations lead asks an internal agent for last quarter's net revenue retention by region. The agent replies in two seconds, in clean prose, with a tidy table. The numbers are wrong. Nothing was invented in the usual sense. The agent found a retention figure in a dashboard export, calculated the way the sales team defines it, and presented it as the finance definition. Nobody noticed until the board pack was half built.
That is the failure pattern enterprises are now reporting. In the July survey, 37% of respondents said it had happened more than once, and another 32% said it had happened once. The model was not the cause. It did what language models do with the information in front of them, which is to produce the most reasonable-sounding answer. When the information is incomplete or contradictory, the most reasonable-sounding answer is often a wrong one stated with full confidence.
Business context is everything the agent needs that is not in the question. Which definition of a metric applies. Which system is the source of record. Which exceptions the policy allows. Which records this user may see. How recent the data is. People carry this in their heads and in side conversations. An agent gets whatever the retrieval step returns, and it cannot tell that something is missing. That is why teams that respond to a bad answer by upgrading the model often get a bad answer with better grammar.
A wrong answer that sounds sure of itself is a data problem wearing a language model's voice.
Six Context Gaps Behind Confident Wrong Answers
These are the gaps we look for first when an agent's answers are not trusted. Severity reflects how likely the gap is to produce a wrong answer that reaches a decision.
| Context gap | What the agent does | Severity |
|---|---|---|
| Conflicting metric definitions | Sales, finance, and operations each define the same term differently, and the agent quotes whichever document it retrieved first | Critical |
| Stale reference data | The agent answers from a cached export or an old policy PDF and gives no sign that the figure is weeks out of date | Critical |
| Missing business rules and exceptions | The agent applies the general policy and misses the carve-out that every analyst knows, so the answer is correct for the rule and wrong for the case | High |
| Retrieval that ignores permissions | Answers are built from records the requester should not see, or exclude records they should, and the result is incomplete either way | High |
| Unresolved entity identity | The same customer appears under three IDs, so totals are split, doubled, or attached to the wrong account | Moderate |
| Undocumented tribal knowledge | Rules that live in people's heads never reach the agent, which fills the gap with a plausible guess | Lower |
Not sure where your agent context gaps are?
10decoders traces a sample of your agent's wrong answers back to the sources, definitions, and rules it was missing. You get a ranked list of context gaps and a plan to close the top ones before the next release.
Book a Free AI Assessment →More Governance, More Reported Failures: Read the Number Carefully
The survey's headline looks like bad news for context layers. Enterprises running or building a governed layer reported recurring failures at 50%, against 21% for those without one. Read literally, governance doubles the problem. The more careful reading, which VentureBeat's own analysis leans toward, is that detection explains most of the gap. A team with a governed layer has lineage, evaluation sets, and review queues, so it sees wrong answers. A team without them mostly does not.
The same survey supports that reading. Twenty-two percent of enterprises reported no context failure at all, and the report describes that group as the least likely to be checking, not the best governed. Size shows up as well. Enterprises with more than 1,000 employees reported recurring failures at 55%, against 30% for those with 101 to 1,000, and only 24% of the larger group had a governed layer in production, compared with 37% of the smaller one. Bigger estates have more systems, more definitions of the same word, and more places for context to disagree. The sample is small, so treat all of this as direction, not proof.
The practical point is to measure before you optimize. A flat zero in an agent's error log usually means nobody is looking. A team that can say “we found 14 wrong answers last month and traced 9 to stale reference data” is in a better position than a team that cannot say anything, even though the first number looks worse.
Context Lives in the Prompt
Definitions and rules sit in a system prompt or a shared folder. Wrong answers are found by users, by accident, and nobody counts them.
Sources Are Chosen on Purpose
Each metric has a source of record, retrieval respects permissions, and answers carry citations. A small golden question set runs before releases, though ownership is still informal.
Context Is Owned, Versioned, and Measured
Definitions, rules, and freshness limits have named owners and version history. Every wrong answer is traced to a gap, and the monthly count goes to the business owner.
Checklist: Does Your Agent Have the Context It Needs to Be Right?
Pick one agent that people already use for decisions and test it against this list. Anything you cannot show is a place where a confident wrong answer can get through.
Fix the places the agent reads from before you tune the way it writes. Sources are cheaper to correct than apologies.
What to do this week
01 Collect ten wrong answers and trace each one
Ask the people who use your most-trusted agent for the ten worst answers they remember, plus any flagged in the last month. For each, write down the question, the answer, the correct answer, and the single piece of context that was missing or wrong. Group the results into the six gaps in the table above. Most teams find that two or three gaps account for most of the failures, and that tells you where to spend the first month.
02 Name the source of record for five metrics
Choose the five figures your agent is asked about most. For each, write one definition, name the system that owns it, name a person who can settle a dispute, and record how often the data refreshes. Then check what the agent actually retrieves for each question. If it can return a second source with a different number, either remove that source from the agent's reach or label it so the answer says which one it used.
03 Make the agent show its sources and its data age
Change the agent's response format so every answer lists the records and definitions it used, and the date each was last loaded. Set a freshness limit for the five metrics from the previous step and have the agent decline or warn when data is older than that. Reviewers can then check a claim by opening one record, and users learn quickly to distrust answers that do not show their sources.
04 Build a golden question set and run it every week
Write thirty questions with known correct answers, drawn from the ten failures and the five metrics above. Run them against the agent on a schedule and after every change to data, prompts, or the model, and record the pass rate. Agree a threshold with the business owner below which a release is held. Expect the number to look worse the first week as hidden problems surface, and expect it to improve once the top gaps are closed.
Let 10decoders Audit the Context Behind Your AI Agents
We take one production agent, trace a sample of its wrong answers to the missing sources, definitions, and rules behind them, and score its context against the checklist above. You leave with a ranked gap list, a golden question set, and a plan for a governed context layer sized to your estate.