Eminence Growth Solutions, home

Search

Subscribe
Menu

Search

Subscribe

How to Check AI Work

Seven working methods for checking what AI produces before anyone relies on it, from the question it was asked to the action it takes. Each opens with a real case from the Eminence Scan record, and each applies across finance, people and buying decisions.

Watercolor sequence on a ruled grid: a loose cloud of squares is refined in stages into an ordered block, with one orange cross marking a rejected item.

Key takeaways

  • Choose the method for the part of the work you need to test, from the initial question to the resulting action.
  • Compare the consequential claim with its source and preserve gaps, assumptions and scope.
  • Leave a record of the check and ensure it can stop or redirect work that fails.

Start with the part of the work you need to check: the question, the research, the data, a missing fact, the summary, the review or the action. Use the corresponding method below and retain enough evidence for another person to follow the decision.

The aim is an answer you can use responsibly, with the assumptions, limits and authority visible.

The methods apply across finance, people and buying decisions. Each develops a documented case into a practical way to check the work.

The methods, in the order the work happens

1. Frame the work

Ask AI to Challenge the Question Before It Answers. Assistants do not reliably challenge the premise they are given. In a September 2025 Which? test, ChatGPT and Copilot advised on an ISA allowance £5,000 above the real limit while three other tools corrected it.

Use when: The question contains an assumption that may be wrong.

Leave with: A neutral question, its governing assumptions and a reason to challenge them.

2. Trace the research

Ask AI Research to Show What Would Change Its Answer. A source list shows research was cited, not checked. KPMG withdrew a report after only five of its 45 citations were found accurate.

Use when: A briefing offers a source list but the conclusion remains hard to test.

Leave with: The origin of the consequential claims and the evidence that would change the answer.

3. Check the data and calculation

Check What Happened to the Records and the Arithmetic. A transformation can change what the data describes. Australia’s Robodebt averaged annual income across fortnights and raised debts it later refunded.

Use when: Records have been extracted, transformed, joined, summarized or calculated.

Leave with: A trace from the original population to the reported result, with independent arithmetic.

4. Keep missing evidence visible

When AI Cannot Say “Unknown”, It Fills the Gap. An AI support bot invented a policy that customers canceled over; a tribunal held an airline liable for what its chatbot said.

Use when: The source does not settle the answer or a key input is unapproved.

Leave with: An explicit gap and a decision about who must resolve it.

5. Preserve meaning in the summary

A Shorter Version Must Not Become a Stronger Claim. Summaries, quotes, headlines and charts lose the limits on a claim first. Ars Technica retracted an article whose AI-generated quotations were published as a source’s words.

Use when: Evidence is becoming a shorter memo, headline, quotation or slide.

Leave with: A summary that retains the source’s scope, strength and uncertainty.

6. Test the check

A Check Counts Only If It Can Fail and Stop the Work. Maryland’s Inspector General for Education found that a canceled AI gun alert could still be passed to police as active.

Use when: A reviewer or automated control is expected to prevent a consequential error.

Leave with: A demonstrated failure test and an effective stop or escalation.

7. Constrain the action

What AI Reads Should Not Decide What It Is Allowed to Do. Once AI can act, content it reads can carry instructions. An AI agent published a post attacking a maintainer by name with no person approving it, by its operator’s account.

Use when: An AI tool can act on information it reads.

Leave with: Permissions enforced by the system and a traceable approval of the exact action.

Why these methods start with the work

In 2025 and 2026 a consulting firm withdrew a report whose citations an outside check found mostly inaccurate, a publication retracted an article whose quotations an AI tool had invented, and a school district’s canceled AI alert still brought armed police to a student. These cases differ, but each exposes a specific place to test the work or its consequences. See the source-linked records in Eminence Scan.

What every method has in common

  • Check against the source, not against another AI’s output or the work’s own summary.
  • Keep the limits next to the claim: the population, the period, the denominator, the word “unknown”.
  • Make the check able to stop the work, and put limits on what AI can do outside the model.

How the centers use them

  • AI Cost & Governance applies them to finance work: the close, controls, forecasts and the cost of AI.
  • AI Talent & People treats them as the skills to hire and train for; see AI Skills by Level and Interviewing for AI Judgment.
  • Vendor Intelligence turns them into questions for buyers to ask of AI vendors.

See What We Mean by AI for the kinds of AI these methods cover.

Sources and further reading