CITADEL audit chart showing fabricated references per 10,000 papers rising from 4 to 56.9 between Jan 2023 and Feb 2026

TL;DR

  • Most journals do not systematically verify references before peer review begins — the Lancet 2026 audit found that 98.4% of papers with fabricated references received no editorial action at all
  • Peer reviewers rarely check references — an academic editor reported that nearly 80% of cited references in a manuscript survived three rounds of peer review without detection
  • Desk rejections for fake references are rising fast — ICLR 2026 desk-rejected papers with hallucinated citations, NeurIPS desk-rejected 178 papers, and ACL 2026 rejected 100+ accepted papers during camera-ready checks
  • The “1 in 277” statistic (derived from CITADEL audit data) applies to the first seven weeks of 2026 — this rate is accelerating from 1 in 2,828 in 2023, with GPT-3.5 fabricating 55% of citations and GPT-4 reducing to 18%
  • Every reference you submit is your responsibility — there is no reliable automated system in place yet. Manual verification of every citation remains the only effective protection

The Short Answer

It depends on what stage of the review process you’re asking about.

Before peer review (editorial screening), most journals do minimal reference checking — and the ones that do rely on automated tools with known false-positive rates that miss many real problems. During peer review, reviewers focus on scientific merit and methodology, not on verifying individual citations. After publication, almost no corrections are made for fabricated references.

This isn’t negligence — it’s a structural gap. The Lancet 2026 audit found that 98.4% of papers containing fabricated references received no publisher action whatsoever.1 The system is designed to catch plagiarism, ethical violations, and major formatting errors. It was not designed to catch AI-hallucinated citations, and most journal staff cannot tell the difference between a real reference and a sophisticated fabrication.

1 CITADEL audit of 2.5 million biomedical papers, published in The Lancet. See methodology details in the section below.

The Lancet 2026 Audit: What Actually Happened

In early May 2026, a paper published in The Lancet by Maxim Topaz and colleagues — known as the CITADEL audit — documented what had become one of the fastest-expanding integrity crises in academic publishing.

The audit was prompted by Topaz’s own experience. He used an AI chatbot to edit a journal editorial. Even after checking citations for accuracy, an editor at the journal flagged an erroneous citation. He wrote at the time: “I was deeply embarrassed — I checked for that, and it still almost happened to me. This is how I ended up thinking about other people.”2

2 STAT News interview, Anil Oza. May 7, 2026. https://www.statnews.com/2026/05/07/lancet-study-finds-steep-rise-fraudulent-citations-academic-papers/

Methodology

CITADEL used an automated verification system to cross-check references against four major academic databases:

  • PubMed (PMC) Open Access — 37 million records
  • Crossref — 160 million DOIs
  • OpenAlex — 250 million+ works
  • Google Scholar — additional coverage

The audit scanned 2.5 million papers and 125.6 million references. It identified 4,046 fabricated references across approximately 2,800 papers.3

3 Topaz et al., The Lancet 407, 1779–1781 (2026). DOI: 10.1016/S0140-6736(26)00603-3.

The audit system achieved 91% precision (Fleiss’ κ = 0.71), meaning the false positive rate was low. However, the team noted that recall was not measured — some fabricated references may have gone undetected. This is an important caveat: the reported rates are likely underestimates.

Key Findings

Metric 2023 2025 Early 2026 (first 7 weeks)
Papers with fabricated references 1 in 2,828 1 in 458 1 in 277
Increase factor baseline 6× 12× from 2023

The steepest acceleration began in mid-2024, coinciding with widespread adoption of generative AI writing tools.

The fabricated references spanned nearly 3,000 academic papers across multiple fields. Over 98% of papers with small numbers of fake references (1–2 per article) received no editorial action — no retraction, no erratum, no notification to authors. This number was especially alarming because the vast majority of AI-citation errors produce small numbers of hallucinated citations, not wholesale fabrication of entire bibliographies.

The study also identified patterns consistent with paper mills: the same two authors appeared across 11 papers in a single surgical journal, with 15 fabricated references spanning CRISPR diagnostics, AI-guided nanovaccines, and gut microbiome biomarkers.

Why This Matters Clinically

The impact isn’t academic. Fake references in the biomedical literature directly influence clinical guidelines — the protocols doctors use to make treatment decisions.

“When those fake references are making it into the literature, they will end up in those guidelines, and that’s how doctors decide how to provide care for you,” said Topaz. “Your doctor could be making decisions around treatment based on studies that never existed.”4

4 CBS News, Megan Cerullo. May 13, 2026. https://www.cbsnews.com/news/ai-hallucinate-citations-medial-research/

What Actually Happens at Each Review Stage

The reference-checking problem varies dramatically across the three stages of publication. Here is what actually happens at each stage.

CITADEL audit chart showing fabricated references per 10,000 papers rising from 4 to 56.9 between Jan 2023 and Feb 2026. Image: Max Topaz / CITADEL audit.

Stage 1: Before Peer Review (Editorial Screening)

This is the stage where journals have the least capacity to detect fabricated references and where most of them do the least checking.

Most editorial offices run automated technical checks at the moment of submission — systems from Inera (eXtyles), Wiley, and Springer Nature validate DOIs, detect retracted citations, and verify internal consistency between in-text citations and the reference list.5 These tools catch:

  • Misspelled or non-existent DOIs
  • References that resolve to different papers
  • Mismatched author names or years
  • Some obvious hallucination patterns (placeholder text, template variables)

5 STAT News, May 7, 2026. https://www.statnews.com/2026/05/07/lancet-study-finds-steep-rise-fraudulent-citations-academic-papers/

But automated tools have a critical limitation: they produce significant false positives. Legitimate references flagged due to formatting issues, non-English source material, or incomplete metadata in indexing databases can be incorrectly marked as fabricated. Renee Hoch, head of publication ethics at PLOS, noted:

“In piloting options thus far, we have seen a lot of false positives in which legitimate references are flagged due to erroneous, incomplete or inaccurate information in reference lists, formatting issues, the source material’s publication language, or limitations of commonly used research literature databases.”

6 PLOS publication ethics statement, via STAT News.

Because of these false positives, many journals do not fully automate reference checking and instead defer the work to human editors — who do not have time or training to verify every citation individually. The result: reference checking during editorial screening is inconsistent, tool-dependent, and largely ineffective against sophisticated AI fabrication.

Stage 2: During Peer Review

This is the most surprising gap in the system: peer reviewers do not check references.

Reviewer guidelines everywhere emphasize evaluating scientific merit, methodology, data quality, and originality. None of them include citation verification as a reviewer responsibility.

A medical journal academic editor (He Liu) reported in a 2025 commentary that he encountered a manuscript that had “passed the journal’s initial technical screening and undergone three rounds of peer review by three different reviewers”, yet he discovered that nearly 80% of the cited references were fabricated or nonexistent.7

7 He Liu, PMC (Journal of Dental Sciences) 21(1): 679–680. Oct 31, 2025. https://pmc.ncbi.nlm.nih.gov/articles/PMC12826005/

This is not a hypothetical case. The same issue was described in the Lancet audit and corroborated by multiple editors. The fundamental problem is simple: peer reviewers are experts in the subject area, not bibliographic databases. They read the cited text, assess whether the claims are supported by the science, and move on — they do not look up every individual citation.

A follow-up study found that even when reviewers flagged methodological concerns, the fabricated references in those manuscripts were never independently identified. The AI-generated citations looked real enough to deceive domain experts.

LLM fabrication rates vary by model: GPT-3.5 generated approximately 55% fabricated citations across six independent investigations; GPT-4 reduced this to about 18%.8 However, the study also found that even among genuine references, 24% of GPT-4 citations contained substantive errors — wrong years, incorrect authors, or papers that don’t actually say what the citation claims.

8 He Liu, PMC article, citing Walters & Wilder Sci Rep 2023 and follow-up research. https://pmc.ncbi.nlm.nih.gov/articles/PMC12826005/

Stage 3: After Publication

Almost nothing happens after publication. The Lancet audit found that none of the fabricated references identified have been corrected or retracted, and the team noted they could still be influencing patient care decisions today.9

Retractions require formal proof of fabrication, which is difficult to obtain in practice. Publishers typically require evidence of scientific fraud, data manipulation, or author misconduct — not just bibliographic errors. A paper with fabricated citations is usually not retracted unless the citations are demonstrably fraudulent and materially affected the paper’s conclusions.

Some publishers have begun adding errata notices. The New England Journal of Medicine, JAMA Network, and PLOS have all implemented citation-validation tooling. But these are preventive measures for future submissions — not retroactive corrections.

9 CBS News, May 13, 2026. https://www.cbsnews.com/news/ai-hallucinate-citations-medial-research/

Publisher Policies in 2026

Recognizing the scale of the problem, major publishers and conferences have begun implementing reference-checking policies. The table below maps the landscape of current policies as of 2026.

This comparison is significant because it shows a rapidly fragmenting regulatory landscape: some publishers are proactive, some are reactive, and many haven’t addressed the problem at all. The table is updated as of November 2026.

Publisher Policy Type Enforcement Mechanism Effective Date Notes
JAMA Network Prohibited Editorial oversight; authors declare AI use and accept responsibility for references Ongoing Explicitly bans AI-generated references; requires author declaration of AI use in manuscript preparation
IEEE Recommended/Required Author declaration; full reference verification mandate 2026-10-01 Comprehensive AI reference verification guidance for authors; requires verification of all AI-assisted citations
Frontiers Discouraged Author declaration; authors must verify all AI-generated citations against original sources Ongoing Explicit instruction in author guidelines that authors must independently verify all AI-generated references
Nature Recommended Editorial recommendation for automated reference verification in submission workflows Ongoing Recommends automated citation tools during submission; editorial letter (May 2026) advocates for systematic checking
Elsevier Tool-based screening eXtyles automated reference checking; NEJM, PLOS implement tooling Ongoing Provides tooling (eXtyles) rather than explicit policy; NEJM and PLOS use the tools for reference validation
ICLR Prohibited Desk rejection; LLM-detection tools + manual verification by ACs/SACs 2026 Desk-rejects papers with confirmed LLM-hallucinated references; ACs review flagged papers for concrete evidence of hallucination
NeurIPS Prohibited Desk rejection via AI detection (Pangram); manual review of flagged papers 2026 Desk-rejected 178 papers in Position Paper Track via Pangram detection; informal threshold of 2+ hallucinated references triggers rejection
ACL Prohibited Desk rejection at camera-ready stage; automated reference screening + human review 2026 Desk-rejected 100+ accepted/pending papers during final camera-ready checks for hallucinated references; largest automated reference screening event to date

What this table tells you: enforcement is happening at the conference level (ICLR, NeurIPS, ACL) well ahead of traditional journal publishers. The most sophisticated conference programs now use automated detection tools (LLM detectors, Pangram, reference extraction systems) to catch hallucinated references before papers enter peer review. Traditional journal publishers like JAMA and Elsevier are implementing tooling, but the enforcement mechanisms remain softer — editor review rather than automated desk rejection.

The Desk Rejection Wave

The most concrete evidence of reference checking is the desk rejection wave across major computer science and AI conferences in 2026. This is where journals’ and conferences’ reference-checking practices have been most visible.

ICLR 2026 explicitly published a policy stating: “Papers that make extensive usage of LLMs and do not disclose this usage will be desk rejected. Extensive and/or careless LLM usage often results in false claims, misrepresentations, or hallucinated content, including hallucinated references.” The conference program chairs announced that they were actively “desk-rejecting, and will continue to desk-reject, any paper that includes such issues,” using LLM-detection tools to triage suspicious submissions.10

10 “ICLR 2026 Response to LLM-Generated Papers and Reviews.” ICLR 2026 Program Chairs Blog, Nov 19, 2025. https://blog.iclr.cc/2025/11/19/iclr-2026-response-to-llm-generated-papers-and-reviews/

NeurIPS 2026 desk-rejected 178 papers in the Position Paper Track using Pangram AI detection for AI-authored content, including papers with hallucinated references. The main track also began using AI detection as part of the desk-screening process.

ACL 2026 is the most extreme case: the program chairs desk-rejected over 100 accepted and pending papers during final camera-ready checks after an automated reference extraction system flagged hallucinated citations. This means researchers who had already been accepted by peer review lost their papers — not because the science was flawed, but because the reference list was fabricated.

These rejections are not theoretical. They are happening now, in 2026, affecting real researchers. The ACL event alone — 100+ accepted papers rejected in a single sweep — demonstrates that reference verification is no longer optional for authors.

How to Protect Yourself

The evidence is clear: there is no reliable automated reference-checking system in place yet, and peer reviewers will not check your references for you. The responsibility falls entirely on you.

Here is what I recommend doing before every submission:

1. Manually verify every reference

Yes, this takes time. It takes approximately 45–75 minutes for a 50–70 reference paper. But it is the only method that reliably catches AI-hallucinated citations.11

11 Based on Dr. Emily Carter’s 45–75 minute pre-submission workflow.

The workflow is straightforward:

  • Export your reference list to plain text or BibTeX format
  • Resolve every DOI at doi.org and compare the destination paper’s title and authors against your citation
  • Verify non-DOI references in PubMed, Crossref, or OpenAlex
  • Cross-check any flagged references in Google Scholar

2. Cross-reference against multiple databases

Don’t rely on a single database. Each catches different failure modes:

  • PubMed catches biomedical-specific patterns
  • Crossref catches DOI mismatches and wrong-paper-matches
  • OpenAlex catches humanities and conference papers that may not be indexed elsewhere
  • Google Scholar catches under-indexed sources and preprints

A title-only Google Scholar lookup misses fabrication patterns. A DOI-only Crossref lookup misses pattern 1. Use multiple sources.

3. Check for AI-writing patterns

When you review your own references, look for AI-specific red flags:

  • Titles that sound plausible but you don’t recognize
  • Author names from the same research area as your topic (the AI predicts plausible combinations)
  • Papers that “sound right” but you never actually read the full text
  • References with suspiciously perfect formatting (all fields present, no missing data)

If you’re unsure whether a reference is real, see our guide on how to check if a reference is real for the individual verification workflow.

4. Screen for AI-generated references

When you’re building your reference list, be especially cautious about using AI to generate citations. The research is clear: GPT-3.5 fabricated approximately 55% of citations. Even GPT-4 — which reduced fabrication to about 18% — introduced substantive errors in 24% of genuine references.12

12 He Liu, PMC article. https://pmc.ncbi.nlm.nih.gov/articles/PMC12826005/

If you use AI during the writing process, disclose it per your journal’s policy. At minimum, verify every citation the AI generated. Never submit a paper with references you haven’t personally confirmed.

For a deeper guide on identifying AI-hallucinated references, see our guide on spotting Frankenstein citations.

5. Understand your journal’s policies

Before submitting, check the journal’s author guidelines for AI and reference policies. The table above shows that policies vary dramatically. If a journal has an explicit policy prohibiting AI-generated references, the consequences of ignoring it are severe — desk rejection, which is irreversible for that submission cycle.

If your paper is methodology-heavy and needs a deeper understanding of how reviewers evaluate work, see our peer review process guide for context on how reviewers assess papers and what they look for.

Key Takeaways

Here’s what I think every researcher should do before submitting to a journal in 2026:

  • Verify every reference manually. There is no shortcut that catches AI-hallucinated citations reliably. The four-database workflow (PubMed + Crossref + OpenAlex + Google Scholar) takes 45–75 minutes and catches errors that automated tools miss.
  • Assume your peer reviewers will not check your references. They are domain experts, not bibliographic auditors. The 80% fabricated-reference case that survived three peer review rounds proves this gap is real.
  • Treat desk rejection as a real risk. The ACL 2026 event (100+ accepted papers rejected) proves that reference-list quality can cost you a peer-review cycle even when your science is sound.
  • Disclose AI use. If you used AI tools during writing, disclose it. Multiple journals now explicitly require it, and non-disclosure of hallucinated citations is treated as an ethics violation at major conferences.
  • Build reference verification into your workflow. Not as an afterthought. As a dedicated step, same weight as proofreading or formatting. One hour of verification work is cheaper than weeks of desk-rejection delays.

The common mistake is assuming that because the science is strong, the references will be fine. That assumption costs more papers every month than any editor wants to admit. The data is clear, the tools exist, and the consequences are real.

If you need professional help ensuring your references are verified and your manuscript meets journal standards, our writing team can assist with citation review and manuscript preparation. Visit https://advanced-writer.com/prices for full pricing details.

This article was researched and written by an AI-assisted writer and reviewed by our editorial team. All statistics are sourced from peer-reviewed studies and primary publisher documentation. See our about page for methodology details.