Your References Could Be Fake — And You Need to Know
You’re not being paranoid if you feel uneasy about every citation you’ve never verified. You’re being responsible.
Here’s what most students don’t realize until it’s too late: AI writing tools routinely generate citations that don’t exist. They’re not random typos. They’re structured, plausible, and formatted perfectly — right down to the DOI.
Stefano Spennemann, a researcher who studied AI hallucinated references for arXiv, put it bluntly: these are “not random errors but predictable, pattern-driven artifacts.” The AI reconstructs citations from real authors, real journals, and topical keywords. It stitches together a completely fake paper that looks legitimate at a glance.
And now you have to submit something with those references in it.
- The problem is real and growing: Over 146,000 hallucinated citations appeared in 2025 alone across 2.5 million papers — a 12× increase from 2023.
- A fake DOI is not proof of a real paper: AI generates DOIs in the correct format that resolve nowhere. Visual inspection cannot tell the difference.
- The 5-step workflow works: Resolve DOI → Check metadata → Search Google Scholar → Verify in PubMed or WorldCat → Cross-check every field
- One famous case: “Frankencitations” — a completely fictitious paper (“Education governance and datafication”) has been cited 140 times despite never existing.
This guide teaches you the actual 5-step verification workflow — the same process researchers use to check if a citation is real. No vague advice. No tool links that don’t explain the method. Just the steps, why each matters, and what to watch for.
Why Checking References Matters Now More Than Ever
The scale of the hallucinated-citation problem is staggering.
A study by Zhao et al. (arXiv:2605.07723, May 2026) audited 111 million references across 2.5 million papers on arXiv, bioRxiv, SSRN, and PubMed Central. The result: 146,932 hallucinated citations in 2025 alone. The fabrication rate was 12 times higher than in 2023.
That means more than one in seven references in recently published preprints contained a citation that did not actually exist.
And it’s not just AI researchers. A massive portion of student work is now written with AI assistance — and every hallucinated citation from an AI becomes one more fake reference floating in the academic literature.
The Frankencitation Problem
Ben Williamson, a researcher at the University of Edinburgh, coined the term “Frankencitation” to describe citations that combine real authors and real journals with invented titles. His most famous example is a completely fictitious paper called “Education governance and datafication” — it has been cited 140 times by other papers, yet the paper has never existed.
The pattern is telling: the authors are real people in the field, the journal is real, the title sounds plausible, and the DOI would be perfectly formatted if it existed. It’s convincing at a glance. It’s also completely fake.
Google Scholar indexes these fake citations and treats them as “real” because they appear in indexed papers. That creates a feedback loop: an LLM generates a fake citation → Google Scholar indexes it → another LLM retrieves it from Scholar and treats it as a legitimate source. The cycle repeats.
Your 5-Step Reference Verification Checklist
This checklist is copyable. Save it. Use it every time you submit a paper.
## ✅ 5-Step Verification Checklist
**Step 1: Resolve the DOI**
- [ ] Copy the DOI (starts with "10.")
- [ ] Go to https://doi.org/[DOI]
- [ ] Does it resolve to a real paper page? (Not a "Page Not Found" error)
**Step 2: Check the Metadata**
- [ ] Does the title match?
- [ ] Do the authors match?
- [ ] Does the year match?
- [ ] Is the journal name correct?
**Step 3: Search in Google Scholar**
- [ ] Search the exact paper title at scholar.google.com
- [ ] Does the paper appear in search results?
**Step 4: Verify in a Database**
- [ ] For biomedical papers: check pubmed.ncbi.nlm.nih.gov
- [ ] For books: check worldcat.org
- [ ] For cross-disciplinary: search Crossref at crossref.org
**Step 5: Cross-Check Everything**
- [ ] Author names spelled correctly?
- [ ] Journal name not similar to a real journal?
- [ ] Publication year makes sense?
- [ ] DOI resolves AND metadata matches?
The 5-Step Workflow Explained
Step 1: Resolve the DOI
Every valid DOI starts with 10. and resolves at https://doi.org/[DOI].
Here’s how it works: go to the DOI resolver at doi.org and paste the DOI you’re checking. The resolver has handled over 158 billion resolutions with 99.97% uptime. If the DOI is real, you’ll land on the paper’s publisher page.
The catch: a correctly formatted DOI is not proof of a real paper. AI tools generate DOIs that match the 10.XXXX/YYYY format perfectly. They resolve nowhere. Visually, you cannot tell whether a DOI is real or fabricated — you have to actually try resolving it.
When a DOI fails, AiCitationChecker (aicitationchecker.org/fake-doi-checker) explains three categories of failure:
- The DOI is not registered at all
- The DOI resolves but to the wrong paper
- The DOI resolves but the metadata mismatches (wrong authors, wrong journal, wrong year)
Categories 2 and 3 are especially dangerous — the DOI “works,” but the paper it points to is not the one you cited.
Step 2: Check the Metadata
Once the DOI resolves, compare the information on the publisher’s page against what your citation says. This is where you catch Frankencitations.
Check every single field:
- Title: Does it match exactly? Not “similar” — exact.
- Authors: Are the names spelled correctly? Same initials? Same full names?
- Year: Does the publication year match what’s in your bibliography?
- Journal name: Is it the exact journal, or a very similar name?
This step catches the vast majority of hallucinated citations. AI tools are good at constructing plausible titles. They are less consistent at matching real metadata across databases.
Step 3: Search in Google Scholar
Google Scholar indexes billions of papers and is completely free. Search the exact paper title at scholar.google.com.
If the paper actually exists, you should find it in the results. If you can’t find it — especially after confirming the DOI resolved correctly — something is wrong. The paper may be behind a paywall that Google Scholar doesn’t index, but for the majority of academic papers, a real paper will appear in Scholar when you search by title.
This step also catches mismatches: if the DOI points to a paper with a completely different title than what you cited, Google Scholar will reveal the discrepancy immediately.
Step 4: Verify in a Database
Different databases cover different fields. Here’s what to use:
PubMed (pubmed.ncbi.nlm.nih.gov) covers 40+ million biomedical citations. If you’re writing in medicine, biology, or health sciences, this should be your primary verification tool.
WorldCat (search.worldcat.org) catalogs library holdings worldwide with over 500 million records. Use it to verify books. Enter the title or ISBN and see which libraries actually hold the book.
Crossref (crossref.org) is the global DOI registry. It registers 37,000 records daily and receives 2.1 billion API calls monthly. While you typically check DOIs through the resolver, Crossref’s API provides authoritative metadata for papers registered with a DOI.
Step 5: Cross-Check Everything
After running through Steps 1-4, do one final pass. Look for:
- Author names spelled correctly (real vs. hallucinated names can look identical at a glance)
- Journal names that are suspiciously similar to real journals (a common hallucination pattern)
- Publication years that don’t make chronological sense (e.g., citing a journal article from a year when the journal didn’t yet exist)
7 Red Flags of Fake AI-Generated Citations
| What the Citation Says | The Red Flag | How to Verify |
|---|---|---|
| DOI resolves but metadata doesn’t match | Fake DOI pointing to wrong paper | Compare title, authors, year on the resolver page against your citation text |
| Journal name sounds similar to a real journal | Mismatched or fabricated journal | Search the journal name at Google Scholar or the publisher’s website |
| Real author cited on unrelated topic | Author on completely unrelated research | Search the author’s recent publications; see if they actually wrote about the claimed topic |
| Publication year predates the journal | Impossible timeline | Check when the journal was first published; if it started in 2018, a “2015” citation is fake |
| Title has AI-ish phrasing | Frankencitation pattern | Search the exact title in Google Scholar; if nothing appears, it doesn’t exist |
| Multiple citations to same author in one paper | Authorship inflation | Cross-check author names across the bibliography; AI often over-uses the same names |
| Citation has no accessible source | Dead link or broken reference | Try resolving DOI, searching Google Scholar, and checking the journal’s archive |
What to Do When a Citation Fails Verification
If a citation fails any step of the 5-step workflow, don’t panic — just follow the process. Here’s what to do:
1. Do not keep the citation. A citation that fails verification is not a citation you can defend. It’s a citation that will look like academic misconduct if a professor or reviewer checks it.
2. Find a real alternative. Search Google Scholar, your library database, or PubMed for the topic. Find a real paper on the same subject and replace the fake citation with it. This is the standard workflow for academic research.
3. Document what happened. If you’re worried about being penalized for using AI-generated citations, keep a brief note showing that you verified the citations and replaced any that failed. Most professors would rather you acknowledge a mistake than hide it.
4. Check for retractions. If a citation you verified turns out to be a retracted paper (which happens), verify it against the Retraction Watch Database at retractiondatabase.org before citing it again.
Related Guides
If you’re using AI tools to write academic papers, you need to understand both the tools and the verification process:
- Best AI Tools for Academic Writing — A comparison guide that covers citation risks and mentions fake citation detection as a core issue.
- How to Cite AI in Academic Writing — The guide that says “Always verify every source the AI cites” — this article expands that instruction into a full 5-step workflow.
- How to Write an Engineering Research Paper — Includes citation-specific guidance for engineering and STEM fields.
Bottom Line
Checking if a reference is real is the single most important skill an AI-era student can develop. The 146,932 hallucinated citations in 2025 alone aren’t theoretical — they’re already in published literature, indexed by Google Scholar, and being cited by other LLMs.
The 5-step workflow (resolve DOI → check metadata → search Google Scholar → verify in database → cross-check everything) takes about 5 minutes per citation. It takes less time than it takes to panic about whether your references are real.
Use the checklist above. Run it for every citation. Your grade — and your integrity — will thank you.