AI MODELS CITE WRONG SOURCES DESPITE RIGHT ANSWERS

Major language models demonstrate a troubling disconnect between answer accuracy and source validity. When analyzing documents, these systems often cite passages that are irrelevant or contradictory to their stated conclusions—even when the final answer proves correct. The phenomenon, termed "attribution hallucination" by Peking University researchers, poses significant risks in regulated industries. Legal professionals relying on AI for case research could receive accurate conclusions paired with fabricated or misapplied citations. Medical practitioners using AI diagnostic tools face similar hazards if recommendations lack proper evidential grounding. To address this gap, researchers developed CiteVQA, the first benchmark designed to systematically test citation accuracy in AI models. The tool measures whether models can reliably point to supporting evidence when answering questions based on documents—a critical requirement for trustworthy AI deployment in high-stakes fields. The discovery highlights a broader challenge in AI reliability. While models have become increasingly capable at generating correct information, their reasoning pathways remain opaque. A correct answer paired with incorrect attribution is functionally problematic; users cannot verify the model's logic or identify where errors occurred. This distinction matters particularly in professional contexts. A lawyer cannot cite an AI-generated brief if the sources don't check out, regardless of whether the legal analysis is sound. A doctor cannot defend a diagnosis based on sources the AI hallucinated. The research suggests that improving AI systems requires more than optimizing for answer correctness. Future development must ensure models cite legitimate, relevant sources that actually support their conclusions. As AI tools become embedded in professional workflows, attribution accuracy will be as important as content accuracy. The CiteVQA benchmark provides a foundation for measuring progress on this problem. Its development signals growing recognition that trustworthy AI demands transparent, verifiable reasoning—not just reliable outputs.