TL;DR: why did the agent hallucinate a tool result?
An agent hallucinates a tool result when the data it asked for is missing or ambiguous and the model fabricates a value instead of abstaining. Our SFR-Bench measurements across sixteen models show most fabrications are not invented from nothing: the model copies a real value from a look-alike record in its own context, a failure we call mis-binding. Look-alike interference, one permission sentence in the prompt, and reasoning configuration each move the fabrication rate more than model choice does.
The rest of this post is the evidence: what agent hallucination is, the benchmark we released to measure the missing-data case, what sixteen models scored, where the fabricated values come from, and what to change in the agent harness to stop it.
What is agent hallucination?
Agent hallucination — AI agent hallucination, in the longer phrasing — is an AI agent asserting something its inputs do not support: a value no tool returned, a record that does not exist, a field copied from the wrong row. It is broader than model hallucination because an agent's inputs include tool results, retrieved documents, and memory, so the failure usually surfaces downstream of a tool call — the agent reports an order status its order-lookup tool never returned, in perfect prose, with full confidence.
The variant that causes the worst production incidents is the quiet one. Not the agent that errors out. The agent that answers a question it had no basis to answer, with a value that belongs to a different record. Every span in the trace returns 200 OK; the task itself fails. This is why agent hallucination is a property of the whole agent harness — the prompts, tools, and context around the model — and not of the model alone, and why the numbers below move so much when only the harness changes.
What is a tool call hallucination?
A tool call hallucination — tool calling hallucination, in the alternate phrasing — is an agent fabricating either side of a tool interaction. On the call side, the model invokes a tool that does not exist, invents arguments the user never supplied, or claims it ran a tool it never called. On the result side, the tool runs correctly and returns nothing, or returns a different record than the one asked about, and the model reports a confident value anyway.
The result side is the case this post measures, and it is the harder one to catch: the call succeeded, the response parsed, the answer reads well. Nothing in the span metadata distinguishes it from a correct run. Both sides are behavioral failures rather than errors, which is why tool call hallucinations survive code review, retries, and uptime dashboards, and only show up when someone reads the whole trace against what the tools actually returned.
What causes agent tool hallucination?
Agent tool hallucination has three measurable triggers, and our data ranks them. First, missing data: the tool returns nothing for the requested record, and the model fabricates rather than abstaining — the base rate SFR-Bench scores per model. Second, look-alike interference: near-duplicate records around the gap push some models from single-digit fabrication to near-certainty, and the fabricated value is usually a neighbor's real one. Third, configuration: reasoning-effort flags and answer-format instructions move the rate more than model choice, in both directions.
None of the three lives in the model weights alone. The prompt decides whether abstention is permitted, the tool schema and data shape decide how much look-alike interference the model faces, and the deployment config decides how much checking the model does before answering. That is what makes agent tool hallucination a harness problem with harness fixes, covered in the prevention section below.
Why we built SFR-Bench
We build a harness engineering platform for teams running agents in production, and hallucinated tool results are the failure class our users hit that nobody publishes numbers for. We went looking for how often each model fabricates when the answer is absent and found nothing to cite. Vendors publish retrieval scores and reasoning scores; none of the sixteen model cards we checked says what a model does when the record it was asked about is not in its context. So we measured it ourselves.
SFR-Bench is that measurement: an open benchmark for missing-data hallucinations. We scored sixteen systems, from 2.7B open-weight models to the current API frontier. The benchmark, the test harness, the leaderboard, and every raw model output are at github.com/ModaLabs/sfr-bench, and every number in this post can be checked against the raw data.
What the benchmark measures
The test is a lookup — the same shape as an agent reading a tool result. We generate a synthetic phonebook, thousands of lines of Name: (555) xxx-xxxx, sized to fill the model's context window using that model's own tokenizer, or a reference tokenizer where the vendor's is closed. Then we ask for the phone number of a person who is not in the book.
Every phone number a model could give to that question is wrong by construction. A model should abstain and say the entry is not there. The failure is a confident, well-formed, wrong number. The Silent Failure Rate is the fraction of failures that are fabrications:
SFR = fabrications / (fabrications + abstentions)
Two more knobs turn this from a toy into something that resembles production data. First, similarity: in the interference conditions, we surround the missing name with four entries whose names sit one to three characters away, each with its own real number. Think order IDs one digit apart, or two customers with the same surname. Second, permission: half the runs use a neutral prompt, half append one sentence telling the model it may reply NOT FOUND.
Fifty samples per cell, greedy decoding wherever the API allows it, two context lengths (32K and 128K) with probes out to a million tokens. Scoring is a string classifier we committed to git before any model ran, plus a correction layer we validated against blind human labels from hand-read samples. No judge model anywhere.
What makes an agent hallucinate: the findings
The headline result is what similarity does. Most models handle a plainly missing record. Surround the gap with look-alikes and fabrication climbs, for some models from single digits to near-certainty.
Line chart of fabrication rate under a neutral prompt as distractors become more similar to the absent query name "Steven Jordan" (from distant look-alikes such as "Stevn Jordn", through close such as "Steven Jordn", to nearly identical such as "Stephen Jordan"). Kimi-Linear-48B rises from 65% to 90% to 100%; DeepSeek-V4-Flash from 80% to 94% to 100%; Qwen3-32B from 46% to 80% to 98%; DeepSeek-V4-Pro from 84% to 96% with the middle tier not measured; Claude Sonnet 5 stays low, from 6% to 8% to 22%. Measured at 32k-token context, n = 50 per point.
DeepSeek-V4-Pro fabricates on 4 percent of failures when the name is missing on its own, and on 96 percent when four near-identical names sit around the gap. Same model, same prompt. Gemini 3.1 Pro and Grok 4.3 both reach 44 percent under the same change. Claude Opus 5 and GPT-5.6 Sol stay at 0 and 2 percent, the only two models that pass the hardest condition without help.
The second result is where the fabricated values come from, and it is the one that explains most hallucinated tool results. We checked every fabricated number against the book. For attention-based models, 75 to 100 percent of fabrications in the hardest condition are a look-alike's real number, copied verbatim. The model retrieved the wrong record and presented it as the right one — mis-binding. The one recurrent-state model we could test this way, xLSTM, did the opposite: 491 fabrications, zero copied from the book. Two different mechanisms produce the same wrong answer.
Figure comparing where fabricated phone numbers come from. For attention and hybrid models, 75 to 100 percent of fabricated values in the hardest interference condition are another entry’s real number copied verbatim from the context. The recurrent xLSTM model produced 491 fabrications and zero of them appear anywhere in the context. In the worked example, both are asked for Michael Gonzales’s phone number, which is not in the phonebook: an attention model answers with Kathleen Gonzales’s real number (555) 117-2472, a surname match with the wrong first name, while xLSTM answers (555) 402-6156, digits that appear nowhere in the context.
Third, configuration moves this number more than model choice. Grok 4.3 fabricates on 2 percent of absent-name lookups with default reasoning and on 100 percent with reasoning_effort: "none", the flag teams flip for latency. The same toggle helps some models and hurts others: reasoning protects Grok, Qwen3-32B gets worse with thinking on, and Gemini gets better with thinking turned down.
Paired-dot chart of fabrication rate on 50 absent-name phonebook lookups per arm, comparing each model with its reasoning setting on versus off. Qwen3-32B fabricates 16 percent of answers with thinking off and 72 percent with thinking on, so thinking on hurts. Grok 4.3 fabricates 100 percent with thinking off and 2 percent with thinking on, one lookup in fifty, so thinking on rescues it. Gemini 3.1 Pro fabricates 88 percent with more thinking and 34 percent with less thinking, so less thinking helps. The same toggle moves fabrication in a different direction for each model.
One prompt sentence, "if the entry does not appear in the context, reply exactly: NOT FOUND", cuts fabrication on plainly missing records by as much as 65 points. It does much less under look-alikes: Kimi-Linear stays at 100 percent with the permission line in the prompt. We also measured the reverse effect: format instructions like "answer with the number only" increase fabrication, because a format cue implies an answer exists.
What is mis-binding in LLM agents?
Mis-binding in LLM agents is the mechanism behind most of those fabrications: the model retrieves a real record — the wrong one — and binds its value to the entity the user asked about. The output is not invented text; it is genuine data attached to the wrong name, which is why mis-binding survives plausibility checks that catch free-form hallucination. In our hardest interference condition, 75 to 100 percent of fabricated values from attention-based models were a look-alike's real value copied verbatim from the agent's own context.
The trigger is similarity. Records one to three characters apart — order IDs off by a digit, two customers sharing a surname, SKUs in the same family — sit close together in the model's attention, and under pressure to answer, the nearest neighbor wins. Recurrent-state architectures fail differently: the one we tested fabricated values from nothing rather than copying neighbors, so the same wrong answer can come from two different mechanisms. For an agent team the practical consequence is the same either way: if your tools return data with near-duplicates, mis-binding is the failure mode to test first, and the interference condition above is how to reproduce it.
The leaderboard
| Model | plainly missing | look-alikes one letter away | look-alikes three letters away | |||
|---|---|---|---|---|---|---|
| plain prompt | one permission sentence | plain prompt | one permission sentence | plain prompt | one permission sentence | |
| Kimi K2.6full attention MoE | 0% | 0% | 8% | 0% | 4% | 0% |
| Grok 4.3 | 2% | 2% | 44% | 0% | 62% | 0% |
| Claude Opus 5 | 0% | 0% | 0% | 0% | 0% | 0% |
| Gemini 3.1 Pro | 88% | 0% | 44% | 0% | 78% | 0% |
| GPT-5.6 Sol | 0% | 0% | 2% | 0% | 12% | 2% |
| DeepSeek-V4-Procompressed attention | 4% | 0% | 96% | 6% | 84% | 4% |
| Claude Sonnet 5 | 2% | 2% | 22% | 18% | 6% | 4% |
| DeepSeek-V4-Flashcompressed attention | 36% | 0% | 100% | 29% | 80% | 0% |
The full table, with the remaining condition columns, the 128K runs, and the audit-corrected label pipeline, lives in the repo: leaderboard. A few placements are worth knowing before you look. Claude Opus 5 is the only model at zero across every condition we could score. Gemini 3.1 Pro is the surprise at the other end. On plainly missing names with a neutral prompt it fabricated 88 percent of the time at 32K, every fabrication a real person's number from elsewhere in the book. And the two Kimi models make the case that this is trainable: Kimi-Linear fabricates on half of its failures, while Kimi K2.6, from the same lab six months later, scores zero.
How to prevent tool call hallucinations in agents
Everything the benchmark moves lives in the agent harness, not the model weights, which is what makes this failure fixable without waiting for the next model release. To stop an AI agent hallucinating tool results, apply the changes that measurably lower fabrication, in this order:
- Give the model permission to abstain. One sentence — reply NOT FOUND when the record is absent — cut fabrication by up to 65 points on plainly missing records. Put it in the system prompt and in the tool's result-handling instructions.
- Remove format cues that imply an answer exists. "Answer with the number only" raised fabrication in our runs, because a format instruction presupposes a value.
- Audit reasoning configuration per model.
reasoning_effort: "none"took one model from 2 percent to 100 percent fabrication. The latency flag you flipped last quarter may be the reason the agent started inventing tool results. - Treat look-alike data as the hard case. Near-duplicate IDs, names, and SKUs are where fabrication climbs toward certainty. If your tools return rows with near-identical keys, test that condition specifically.
- Write agent evals from the hallucinations your agent actually produced in production, not imagined ones. A replayed trace where the agent fabricated a tool result is a regression test; a hand-written guess is a hope.
This loop — detect the failure in production traces, attribute it to the responsible harness component, verify the fix against replayed traffic — is harness engineering, and it is the loop Moda runs. Moda is a harness engineering platform, not an observability dashboard: it turns production traces into verified improvements for the agent harness — prompts, tools, skills, evals, memory.
Why did my agent call the wrong tool when the right one was in the schema?
For the same reason models copy a look-alike's phone number: selection under similarity is a binding decision, and near-duplicates interfere. When two tools have overlapping names, similar descriptions, or adjacent argument shapes — search_orders next to search_order_items, a lookup tool next to a fetch tool — the model binds the request to the nearest plausible neighbor, exactly the way it binds a missing record to a look-alike entry. The right tool being present in the schema does not protect you; in our record-level measurements, the right answer being absent and a wrong-but-similar one being present is precisely the condition where fabrication climbs toward certainty.
The fixes are schema and harness changes, not model swaps. Make tool names and descriptions maximally distinct, and say in each description what the tool is not for. Prune tools that overlap; every near-duplicate pair in the schema is an interference condition you are shipping. Give the model an explicit out — a fallback or clarify action — so the nearest neighbor is not the only legal answer under ambiguity. And when a wrong-tool call shows up in production traces, turn that trace into an eval case, so the schema change that fixes it is verified against the traffic that produced it rather than assumed.
Limitations we know about
The phonebook is synthetic and the absence is unambiguous, which makes scoring mechanical but means we measure a floor. Real contexts have soft absences. The two haystack designs we used across rounds place distractors differently, so interference columns are not comparable between rows from different designs, and the leaderboard marks which design produced each row. MiniMax-M2 numbers come from an API that ignores seeds, so they are estimates, not bit-reproducible. And our own scorer misjudged the frontier until we audited it: the frozen classifier scored Claude Opus 5 at 94 percent under interference, when hand-reading showed the true rate was 0, the model politely refusing in a phrasing our regexes had never seen. The correction layer that fixes this is part of the release, and so are the audits.
Run it yourself
Everything you need to reproduce or extend the benchmark is in the repo: the generators, the frozen classifier, the model configs with every provider quirk we hit, more than 26,000 raw generations, and four verify scripts that recompute each published number from those generations. Adding your own model is one YAML entry and one command if it's served on OpenRouter. The newest models have since beaten these conditions; the follow-up study is SFR-Bench V1, the same question with the string present and the answer still missing.
Frequently asked questions
Why did the agent hallucinate a tool result?
Because the data the agent needed was missing or ambiguous, and the model filled the gap instead of abstaining. In our SFR-Bench measurements the dominant mechanism was mis-binding: the model copied a real value from a look-alike record elsewhere in its context and presented it as the answer, so the hallucinated tool result looks plausible because most of it is real. The rate is set less by which model you run than by the harness around it: whether the prompt permits abstention, whether format instructions imply an answer exists, how reasoning effort is configured, and how much near-duplicate data your tools return.
What is AI agent hallucination?
AI agent hallucination is an AI agent asserting something its inputs do not support: reporting a value no tool returned, citing a record that does not exist, or binding a real value to the wrong entity. It differs from plain model hallucination because an agent's inputs include tool results, retrieved documents, and memory, so the failure typically appears downstream of a tool call and passes silently: every step returns 200 OK while the final answer is wrong. Detection has to read the whole trace, not individual spans, which is why hallucinated tool results rarely show up in error logs.
What is agent hallucination?
Agent hallucination is the shorthand for the same failure: an AI agent confidently producing information it had no basis for. The practical taxonomy from our measurements has three triggers — missing data the model papers over, look-alike data the model mis-binds, and configuration that suppresses the checking the model would otherwise do. All three are properties of the agent harness, so the fixes are harness changes: permission to abstain in the prompt, format instructions that allow NOT FOUND, per-model reasoning settings, and eval cases written from the hallucinations observed in production traces.
How do you prevent tool call hallucinations in agents?
Five harness changes, in the order our SFR-Bench measurements rank them. Give the model explicit permission to abstain — one NOT FOUND sentence cut fabrication by up to 65 points on plainly missing records. Remove format instructions that presuppose an answer exists, since they measurably raise fabrication. Audit reasoning configuration per model: one latency flag took a model from 2 percent to 100 percent fabrication. Test the look-alike condition specifically, because near-duplicate IDs, names, and SKUs are where fabrication climbs toward certainty. And write eval cases from the tool call hallucinations observed in your production traces, so every fix is verified against the traffic that produced the failure.
What is a tool calling hallucination?
A tool calling hallucination is the same failure under its alternate name: an agent fabricating part of a tool interaction, either the call — invoking a tool that does not exist, inventing arguments, claiming a tool ran when it did not — or the result, asserting a value its tools never returned. Both pass silently because every step reports success; only the content is wrong. Detection has to compare what the agent said against what its tools actually returned across the whole trace, which is why these failures rarely appear in error logs or span-level monitoring.
What is a tool call hallucination?
A tool call hallucination is an AI agent fabricating a tool interaction or its outcome: calling a nonexistent tool, inventing arguments, or — the case our measurements cover — reporting a value when the tool returned nothing or returned a different record. In SFR-Bench's hardest condition, 75 to 100 percent of fabricated values from attention-based models were a look-alike record's real value copied verbatim, so the hallucinated result is built from genuine data bound to the wrong entity. That makes it plausible to users, invisible to uptime monitoring, and fixable at the harness layer rather than the model.
What is mis-binding in LLM agents?
Mis-binding in LLM agents is the model retrieving a real but wrong record and presenting its value as the answer for the entity that was asked about — genuine data bound to the wrong name. It is the dominant mechanism behind fabricated lookups in our measurements: under look-alike interference, 75 to 100 percent of fabrications from attention-based models were a neighbor's real value copied verbatim from the agent's own context. Because the value itself is real, mis-binding passes the plausibility checks that catch free-form hallucination, and it is triggered by near-duplicate data — IDs a digit apart, shared surnames, adjacent SKUs — rather than by anything visible in the prompt.
What is agent tool hallucination?
Agent tool hallucination is an AI agent fabricating some part of its tool use: the call, the arguments, or the result it reports. The measurable triggers are missing data the model papers over instead of abstaining, look-alike data it mis-binds, and configuration — reasoning-effort flags and format instructions — that suppresses checking. All three are properties of the harness around the model, which is why the same model's fabrication rate moved from 2 percent to 100 percent in our runs on configuration alone, and why the fixes are prompt, schema, and eval changes rather than a model swap.
Why did my agent call the wrong tool when the right one was in the schema?
Because tool selection is a binding decision under similarity, and near-duplicate tools interfere the same way near-duplicate records do. When two tools share overlapping names, similar descriptions, or adjacent argument shapes, the model binds the request to the nearest plausible neighbor — the presence of the right tool in the schema does not prevent the wrong one from being closer. The fixes are harness changes: make names and descriptions maximally distinct, state what each tool is not for, prune overlapping tools, give the model an explicit fallback or clarify action so the nearest neighbor is not the only legal answer, and turn every wrong-tool trace from production into an eval case that gates the schema fix.
Find the tool results your agent invented
If your agents look things up in data with near-duplicates, which is most data, this failure mode is in your traffic today. Moda finds it in your production traces, attributes each hallucinated tool result to the harness component responsible, and verifies the fix against replayed traffic before it ships. Book a demo to see it on your own agent.