[ Evidence-led insight · human-approved release ]

How to tell whether an AI assistant's cited sources and references are real before I rely on or republish them

A guide on mapping AI citation failure modes to MPS/0.1 claim tags and using the free MPS Auditor to generate claim-by-claim provenance records before publication.

Published
Updated
Editorial review
Mayone Maha Rajan

[ Reader's answer ]

Direct answer

The reader leaves with a concrete, repeatable way to separate real, checkable citations from plausible-but-fabricated ones, and knows which claims to hold as unverified until a human confirms the primary source.

[ Method ]

How this insight was assembled

Maha maps common AI-citation failure modes (fabricated references, real-but-wrong DOIs, uncited statistics) to the MPS/0.1 claim tags and pairs each with a reviewer action, then runs a real AI passage through the free MPS Auditor to produce a claim-by-claim provenance record. The Auditor labels each claim's evidentiary status for a human to check; it does not determine truth or certify accuracy.

Maha's editorial workflow separates candidate material, drafting, evidence review, and human release. A published insight is an accountable synthesis, not a substitute for primary sources or professional advice.

[ Evidence record ]

Sources used for this release

  1. 01

    Fabrication and errors in the bibliographic citations generated by ChatGPT

    Peer-reviewed Scientific Reports study finding 55% of GPT-3.5 and 18% of GPT-4 generated citations were fabricated, with many real citations still containing substantive errors; supports the claim that AI-produced references cannot be trusted without checking the primary source.

  2. 02

    Hallucinating Law: Legal Mistakes with Large Language Models are Pervasive

    Stanford RegLab research summary reporting legal hallucination rates of 69%–88% on specific queries to leading models; supports that AI confidently produces unverifiable or wrong citations, a high-stakes case for pre-publication verification.

  3. 03

    AI Search Has a Citation Problem

    Tow Center study of eight AI search tools finding over 60% of responses cited sources incorrectly and often misattributed content; supports that even 'grounded' AI search mislabels its sources, so citations must be independently checked.

Open supporting artifact: Free MPS Auditor Tool ↗

[ Limits and decision boundary ]

What this does not establish

The MPS Auditor labels a claim's evidentiary status for human review; it does not determine factual truth, certify overall accuracy, or eliminate the need for manual verification of primary sources.