Ask AI Research to Show What Would Change Its Answer

A real source can still be the wrong evidence

GPTZero’s account of the report shows three different ways a citation can fail.

• The source is real but says something else. The report appears to cite a 2019 press release from the Japanese railway JR East as evidence that it uses AI agents; the release predates agentic AI products and does not mention AI.
• Two organizations are merged into one claim. A case study describes the Austrian utility Verbund using AI agents; the closest cited source is a 2025 Verbund release about its venture unit funding a startup, which does not mention AI agents.
• A figure may drift from its origin. The report cites “KPMG research” for 55% of CEOs ranking AI as “their top investment priority”; KPMG’s own 2025 CEO Outlook, released the same month, says 71% call AI “a top investment priority”. GPTZero takes the CEO Outlook to be the likely source and allows that the gap may have an explanation; the two statistics are not worded the same way.

GPTZero’s view is that an AI research tool was probably asked to find examples of companies using agentic AI, conflated sources and exaggerated claims, and that no one checked the citations before publication. That is GPTZero’s inference, not KPMG’s account. KPMG’s own guidelines, its spokesperson said, expect “human oversight to validate content and verify independent sources”.

The problem is measured, not anecdotal

In a study published in October 2025, 22 public service media organizations coordinated by the European Broadcasting Union and the BBC had journalists assess 2,709 answers to news questions from the consumer versions of ChatGPT, Copilot, Perplexity and Gemini. Forty-five percent of the answers had at least one significant issue, and 31% had significant sourcing problems, such as a cited source that did not support the claim. The study quotes one participating broadcaster’s term for the most basic sourcing failure, “ceremonial citations”: references added to give an impression of thorough research that do not support the claim when checked. In one answer Perplexity appended 19 links and referred to three of them, and participants warned that an abundance of sources creates a misleading air of authority. The assistants declined to answer only 0.5% of the time.

The Tow Center for Digital Journalism found the same pattern in February 2025 tests of eight AI search tools. Together the tools answered more than 60% of its queries incorrectly. ChatGPT identified 134 articles incorrectly and signaled uncertainty only 15 times in 200 answers. The tools often cited the wrong article, and sometimes a syndicated copy rather than the original. Both studies note that the assistants change and that results will move with new models. The checking does not depend on which model is current.

Five pages can be one source

A claim that appears in many places looks corroborated. Often it is repeated. GPTZero said the KPMG report’s statistics had already been recycled in trade publications and a major Czech newspaper, and that AI tools were citing the report, which can strip claims of their context.

The same thing happens inside organizations. In an invented example, five articles say a product cuts processing time by 30%. All five repeat one vendor announcement, and none describes an independent comparison or the work that was measured. An AI briefing calls the benefit “widely reported”. The accurate summary is one vendor claim, repeated five times, and untested for this team’s work.

A court and a regulator now expect the check

A court and an audit regulator have begun to set the standard. In June 2025 the High Court of England and Wales said that freely available generative AI tools cannot conduct reliable legal research and may cite sources that do not exist or quote passages a real source does not contain. It said lawyers who use AI for research, or who rely on others’ work that used it, must check it against authoritative sources before relying on it. The court was addressing lawyers in England and Wales; the principle, that relying on someone else’s research means checking it, applies to anyone.

The UK Financial Reporting Council’s March 2026 guidance on generative and agentic AI lists, among controls on deficient output, rules such as requiring outputs to cite their sources. In one of its illustrative examples, the auditor still corroborates a cited summary by reading the source documents. A citation makes a claim checkable. Only reading the source checks it.

How to commission research you can challenge

1. Name the decision and the claims that could change it. Ask for those first. Background can wait.
2. Set the boundary. State the period, geography, population and source types the research covers, and when to stop. Ask the tool to say what it did not search. A bounded scan is not a systematic review and should not be presented as one.
3. Ask for a claim register, not a source list. For each decision-critical claim: the exact claim, its original source, the source’s own date, what population or entity it covers, and whether it is a measured result, a vendor statement or an opinion.
4. Open the origin of every decision-critical claim. Confirm it says what the briefing says, about the same organization and period. A working link proves only that the page exists.
5. Count origins, not pages. Trace repeated claims back to where they started.
6. Ask what would change the conclusion, and look for that evidence, including results that cut against the preferred answer.
7. Publish the gaps with the findings. A briefing that says what remains unconfirmed is easier to rely on than one that sounds complete.

Where this applies across the centers

• Finance: market sizes, benchmarks and “industry average” savings in a business case.
• People teams: research behind an assessment method or a workforce trend, checked for the population it actually studied.
• Buying AI: vendor capability and adoption claims, which are often press releases repeated across trade coverage.

Are you at risk?

• Do AI-assisted briefings reach decision-makers with a source list but no record of who opened which source?
• Would anyone notice if two of the five sources behind a key claim were the same announcement?
• Does published research carrying your name get a check against the original sources before release?
• Can the team say what evidence would change the recommendation?

Trace the claims that could change the decision

AI makes research fast and its output convincing. Require the claims that matter to be traced to their origins and read, count origins rather than pages, and ask what would change the answer before you act on it.