Fabricated references are not an edge case — they are a measured, reproducible failure mode. This is the pillar where we put numbers on it and show you how to check.
Verification workflows, model-by-model error rates, and what to do when a source does not resolve.
Yes — a follow-up verify prompt measurably improves citation accuracy (7.5% to 77.5% in one study). No — self-checking still leaves a fifth wrong, because a model can't reliably audit itself.
7 cited sources
A fake citation isn't a botched copy of a real record — the model never had one. It assembles the most probable author, title, and journal, so plausibility is baked in.
5 cited sources
A sourced table of measured AI hallucination rates by model, task, and domain — with the methodology behind each figure kept intact, not averaged away.
9 cited sources
A minute is enough to catch a fabricated citation — if you check the fields in the order the errors actually sit. Start with the DOI and PMID, not the title.
8 cited sources
A made-up citation now has a price you can read off a docket: $5,000 in Mata v. Avianca, up to $109,700 by 2026 — plus a quieter cost when no one catches it.
10 cited sources
Yes, ChatGPT fabricates citations. Five peer-reviewed studies measured how often: 18% for GPT-4, 55% for GPT-3.5, and up to 88% for legal questions.
9 cited sources
Each of these comes from a peer-reviewed study, cited in full on the post that covers it.
In a controlled evaluation of AI-generated scientific writing, the majority of references were either partially incorrect or entirely fabricated — plausible authors, real-looking journals, invalid DOIs.
Citation hallucination runs from roughly 18% in GPT-4 to over 70% in other frontier models. In legal questions about federal cases, published rates reached 58–88%.
Asking the model to verify its own bibliography converts partially correct references into accurate ones at a measurable rate. It does not fix everything — but it is not folklore either.
Entropy-based methods published in Nature detect a subset of hallucinations — confabulations — by measuring uncertainty at the level of meaning rather than wording.
Yes, measurably and repeatedly. The phenomenon has a name in the literature — citation hallucination — and published rates range from about 18% to over 70% depending on the model. The references usually look right: real author names, plausible journals, well-formed DOIs that resolve to nothing or to a different paper.
Partly. Structured follow-up prompts that explicitly ask the model to verify bibliographic accuracy do improve reference validity — that effect is documented. But the model cannot reliably check what it never had access to, so a follow-up prompt narrows the problem rather than removing it. Resolve the DOI yourself.
Resolve every DOI and confirm the title, first author and year match. Studies that verified AI-generated references against Crossref, PubMed and Google Scholar found that mismatches cluster in exactly those three fields, which makes them a cheap, high-yield check.