AI Hallucination Rates by Model: A Reference Table

There is no such thing as the AI hallucination rate. Anyone who quotes you a single figure — “9.6%,” “18%,” “over 90%” — has thrown away the three things that make the number mean anything: which model produced it, what task it was doing, and what the researchers counted as a hallucination in the first place. This page is a reference table, not an argument. It collects the measurements that have actually been published, keeps each one attached to its methodology, and deliberately refuses to average them, because averaging figures that measured different things is how a real 18% and a real 91% become a fictional 55%.

Read it the way you would read a spec sheet. Each row below is a specific model, on a specific task, in a specific domain, scored against a specific definition. Two rows that look comparable usually are not.

Why one number is always a lie

Before the tables, the fault line that runs under all of them. “Hallucination” is not one measurable thing. Published studies are measuring at least three distinct failures, and they produce wildly different percentages because they are not looking at the same object:

• Fabricated bibliographic citations — the model invents a reference to a paper that does not exist. This is measured by generating references and checking each against real databases. It is the harshest test, and it produces the highest numbers.
• Factual-claim error rate — the model asserts something false inside otherwise fluent prose. This is measured by decomposing an answer into individual claims and labelling each true or false. Farquhar and colleagues, writing in Nature, narrow the term further to what they call confabulations: cases where the model is “fluently” wrong and arbitrary, meaning the answer flips if you rerun the same prompt with a different random seed (Farquhar et al., 2024, p. 625). A confabulated fact has no truth underneath it, only a probability distribution.
• Summarization unfaithfulness — given a source document, the model’s summary states something the document does not. This is the mildest test because the model is handed the facts, and it produces the lowest numbers.

A model can score 1.5% on the third test and 47% on the second, in the same week, and both figures are correct. They are not in tension. They are answers to different questions. That is the entire reason this page exists, and the reason it will not hand you a headline number.

Table 1 — Fabricated citations (open-ended generation)

This is the hardest test: ask the model to produce references from memory, then check whether the cited works exist. Every figure here comes from hand-verification against real databases.

Study	Model	Domain	What was measured	Rate
Walters & Wilder, 2023	GPT-3.5	42 multidisciplinary topics	Share of references that are fabricated	55%
Walters & Wilder, 2023	GPT-4	42 multidisciplinary topics	Share of references that are fabricated	18%
Chelli et al., 2024	GPT-3.5	Rotator-cuff systematic reviews	Hallucinated references (≥2 of title/author/year wrong)	39.6%
Chelli et al., 2024	GPT-4	Rotator-cuff systematic reviews	Hallucinated references	28.6%
Chelli et al., 2024	Bard	Rotator-cuff systematic reviews	Hallucinated references	91.3%
Safran & Çalı, 2025	ChatGPT	Musculoskeletal rehab	Fully fabricated (score 0) on first pass	42.5%

The single most important pair of numbers in this literature is the first two rows. William Walters and Esther Wilder had GPT-3.5 and GPT-4 write short literature reviews on 42 topics, then hand-checked all 636 references in the 84 resulting papers. 55% of the GPT-3.5 citations were fabricated, against just 18% of the GPT-4 citations (Walters & Wilder, 2023, p. 1). One generation of the same product family, a threefold difference. This is why “does ChatGPT hallucinate citations” has no answer without a version number attached.

The Bard row is the one that gets misquoted most. Mikaël Chelli and colleagues, in the Journal of Medical Internet Research, ran GPT-3.5, GPT-4, and Google’s Bard on the same systematic-review task in the same study. Bard failed to correspond to a real paper 91.3% of the time (95 of 104 references), while GPT-4 — the best of the three — sat at 28.6% (Chelli et al., 2024, pp. 7, 9). When someone cites “over 90%” as ChatGPT’s rate, they have taken Bard’s number and pinned it on the wrong model. Note also the definition in the fourth column: Chelli’s team counted a reference as hallucinated only if at least two of title, first author, or year were wrong. A looser or stricter definition would move every number in that block.

Ertuğrul Safran and Adem Çalı add the finding that survives even when the totals look moderate: on their hardest prompt — rotator-cuff repair rehabilitation — 6 of 10 references (60%) were completely fabricated, using real authors bolted onto invented titles and non-resolving DOIs (Safran & Çalı, 2025, p. 698). Across all their prompts, only 7.5% of references were fully accurate on the first pass. Re-prompting the model to verify its own work raised that share sharply, but it is a repair step, not a starting condition.

Table 2 — Factual-claim and QA error rates

Here the model is not asked for citations but for facts, and each claim is graded individually. The numbers are lower than Table 1, and they swing hard on one variable: whether the model was allowed to retrieve.

Source	Model	Condition	What was measured	Rate
OpenAI, 2025 (GPT-5 System Card)	gpt-5-main	Browsing enabled	Claim-level factual error rate, production traffic	~9.6%
OpenAI, 2025 (GPT-5 System Card)	gpt-5-main	Closed-book (SimpleQA)	Answers wrong on hard factual questions	~half
Farquhar et al., 2024	LLaMA / Falcon / Mistral (7B–70B)	Biography QA, closed-book	Factual claims labelled incorrect	45 of 150 claims
Alkaissi & McFarlane, 2023	ChatGPT (GPT-3.5)	Clinical case, closed-book	Provided references with real PMIDs pointing to unrelated papers	All 5 wrong

The GPT-5 rows carry the lesson that the entire “did the new model fix it?” debate turns on. OpenAI’s own system card reports gpt-5-main producing a claim-level error rate around 9.6% — but the evaluation it comes from is titled for production traffic with browsing enabled (OpenAI, 2025, p. 1). The model was allowed to look things up. Take retrieval away and put the same model on SimpleQA, a benchmark of deliberately hard questions answered from parameters alone, and it misses roughly half the time. Same model, same day; the gap between the two rows is the search box, not the neural network. Quoting the 9.6% without the words “browsing enabled” describes a system the reader is not using.

Alkaissi and McFarlane’s case is the mechanism made visible. Asked for references on bone metabolism, ChatGPT returned citations with real, correctly formatted PubMed IDs — every one of which pointed to an unrelated paper. A “reference” on homocysteine and bone metabolism carried a PMID that actually belonged to a urology paper about titanium surgical staples (Alkaissi & McFarlane, 2023, p. 2). The number was real, the paper was real, the connection was invented. The same model, incidentally, scored between 43% and 68% on open-ended USMLE questions (Alkaissi & McFarlane, 2023, p. 4) — a reminder that a model can be a competent test-taker and an unreliable librarian at once.

Table 3 — Summarization faithfulness (grounded task)

This is a different universe of numbers, and it is the one most likely to mislead if you line it up against the others. Vectara’s public Hallucination Leaderboard measures how often a model introduces an unsupported claim when summarizing a document you have handed it — a grounded task, scored automatically with their HHEM evaluation model (Vectara, 2026).

Source	Model class	Task	Typical rate
Vectara Leaderboard, 2026	Best-performing frontier models	Summarize a supplied short document	under 2%
Vectara Leaderboard, 2026	Most current frontier models	Summarize a supplied short document	~1–3%

Do not read this table as “AI hallucinates 2% of the time.” It says that when the facts are placed directly in front of a strong 2026 model and it only has to restate them, it stays faithful roughly 97–99% of the time. That tells you almost nothing about what the same model does when asked to recall a citation from memory, which is Table 1’s job — and where the same class of model can be wrong a third of the time or more. The leaderboard is the strongest existing evidence that hallucination is a retrieval-and-grounding problem, not a fixed property of the model. It is not a licence to quote a low single-digit number for tasks it never measured.

The legal domain is its own worst case

Every table above gets worse when the question turns to case law, which is why lawyers keep ending up in the news. Magdalena Łągiewska, surveying AI in arbitration, cites the running tally kept by researcher Damien Charlotin: by 30 June 2025 he had documented 161 cases involving hallucinated content produced by generative AI in real legal filings (Łągiewska, 2025, p. 116). She also draws on Matthew Dahl and colleagues to separate three distinct kinds of legal hallucination — output that contradicts the query, output that contradicts its own training corpus, and output that simply departs from real-world fact regardless of how the model was trained or prompted (Łągiewska, 2025, p. 117). That taxonomy matters for the same reason the tables do: a single “legal hallucination rate” would blur three failure modes that need to be counted separately.

How to read this page without misusing it

A few rules follow directly from the numbers above, and they are the reason the tables are not merged into one:

• Never compare across tables. An 18% fabricated-citation rate (Table 1) and a 1.5% summarization rate (Table 3) are not evidence that one model is ten times better than another. They are two different tests. The only honest comparisons are within a column, within a study.
• Always carry the condition. “gpt-5-main, 9.6%” is meaningless; “gpt-5-main, browsing enabled, claim-level, production traffic, 9.6%” is a datum. Strip the condition and you have a slogan.
• Watch the definition. Chelli counted a citation as hallucinated at two wrong fields out of three; another study might count one; Walters and Wilder separated outright fabrication from real-but-erroneous citations and found that even among real GPT-4 references, 24% still carried substantive errors (Walters & Wilder, 2023, p. 1). Where the threshold sits changes the headline.
• Verify, don’t trust. Because fabricated references look identical to real ones, the only reliable defence is to check whether each cited work actually resolves to a real paper — the existence check behind free tools like cytado.com’s bibliography checker (disclosure: cytado.com is operated by this site’s owner). Alex Glynn’s proposal for full-text reference deposit — requiring authors to lodge the sources they cite — is the institutional version of the same instinct (Glynn, 2025, p. 1).

The numbers on this page will age; new models will push Table 1 down and crowd Table 3 near zero. What will not change is the discipline the tables are built on. A hallucination rate is a measurement, and a measurement without its model, its task, its domain, and its definition is not data — it is a rumour with a decimal point. For the narrative behind the citation figures specifically, see our companion piece on whether ChatGPT makes up sources; for the wider set of verification workflows these numbers feed into, the fact-checking hub collects them, and the retrieval-versus-recall distinction that drives Table 2 is largely a question of how you prompt.