Review Is a Control Only When It Can Catch the Error

The reviews that passed the error

The most instructive recent failures happened at organizations with review processes, policies and training in place.

• Sullivan & Cromwell, April 2026. An emergency motion the firm filed in a US bankruptcy court contained AI-generated fabricated citations, which another law firm in the case found. The firm said its lawyers must complete two tracked training modules before using AI, that its rules require them to check all AI output independently, and that its citation review still did not identify the errors.
• Butler Snow, July 2025. A partner added ChatGPT-generated citations to court motions without checking them; the lawyer who signed and filed them reviewed his supervisor’s additions only for grammar and typography. The firm had warned all its lawyers in 2023 that AI output “can appear perfectly researched and logical while in fact it is wholly inaccurate”. The court described the lawyers as well-trained and experienced, with repeated warnings and firm policies behind them.
• Latham & Watkins, May 2025. In an expert declaration, a citation formatted by an AI tool had the wrong title and authors; the firm’s manual cite-check confirmed that the source supported the point and corrected its page numbers, but did not notice the title and authors were wrong.
• Deloitte, August to October 2025. Deloitte repaid the final instalment of A$97,587.11 on a contract initially worth A$439,142, and the department tested all the citations itself before publishing a corrected report in October 2025.

These accounts describe different breakdowns: a narrow check, a missed check, or a required procedure not followed. The lesson is to specify both what a review should detect and what evidence shows the procedure ran. More such cases, with the review each passed, are set out in When a Report Cites Sources That Do Not Exist.

The pattern predates AI. In August 2020 a Citibank payment passed the bank’s three-person approval process, and almost $900 million of the bank’s own money went out with $7.8 million of interest, because all three people shared the same mistaken belief about how the payment system worked; the ruling on who kept the money was later overturned on appeal, but not the court’s account of the error. A review that rests on the same assumption as the work it checks cannot catch the error.

Why this matters in the close

In Gartner’s 2025 survey of 183 CFOs and senior finance leaders, 59% said their finance function used AI, and among those that had implemented it, accounts payable automation and error and anomaly detection were among the most common uses. The natural response is to have a person review the output before it is accepted.

That is a useful starting point, but consider an illustrative example (fictional):

An AI assistant reviews accounts payable data and reports that all supplier balances reconcile. The controller approves the result after checking the list of identified exceptions. The AI never received transactions from one business unit. Every listed match is correct; the reconciliation is still incomplete.

A reviewer who only looks at the AI’s exceptions cannot detect a missing population that the tool never considered. The UK audit regulator gives the same warning from the other side of the audit: an auditor who treats an AI summary of lease sections as a complete summary of the contracts, and performs no other procedures, can miss obligations the summary left out. The reviewer needs an independent check of the population or a reliable completeness control upstream.

What makes a review an effective control?

PCAOB auditing standards distinguish between whether a control is appropriately designed and whether it actually operates as designed. A control is designed effectively when, operated as prescribed by people with the necessary authority and competence, it can prevent or detect errors or fraud that could result in material misstatement. PCAOB staff guidance on management review controls, which the PCAOB still lists among its staff guidance, says that evaluating such a review involves its objective, the factors that set its precision (expectations, aggregation and criteria for investigation) and evidence of the steps actually performed.

PCAOB staff’s March 2025 inspection update shows the same gaps in practice: audit teams that did not test a control’s design against incomplete information, did not evaluate the review procedures the control owner actually performed, and did not test the accuracy and completeness of the system reports a control relied on. Neither the standard nor the staff guidance mentions artificial intelligence; the application below is CoE | AI Cost & Governance analysis.

A practical review design answers six questions.

The depth of review depends on risk. Routine, low-impact suggestions do not need the same evidence as a material accrual, reporting adjustment or external disclosure.

The AI-specific problem: polished output can hide weak evidence

The UK Financial Reporting Council’s March 2026 guidance on generative and agentic AI sorts the errors of today’s language models into five kinds: hallucinations, omissions, distortions, faulty reasoning and inconsistencies. It explains that output can sound coherent while distorting meaning or applying unsound logic, and that some risk of such errors will always remain. An ICAEW practice article said in September 2026 that AI output can be remarkably convincing while inaccurate, outdated or fabricated, and that accountants should apply to it the same professional skepticism they apply to any information they review.

People are inclined to accept it. The International AI Safety Report 2026, published in February 2026, notes users’ tendency to trust confident-sounding output uncritically and says evidence of automation bias in AI-assisted work has accumulated since January 2025. In one experiment it cites, with 2,784 participants, people were less likely to correct wrong AI suggestions when correcting them took extra effort. We found no study that measures how often human review catches errors in AI-prepared close work.

The response is to examine what the system used and did, not merely what it says it concluded. The FRC’s guidance says a reviewer needs the competence to identify these errors, and that a review may mean forming an independent view, for example by reading the source material to confirm a summary captured it. For example:

• Accuracy: Reperform selected calculations and tie significant balances to authoritative records.
• Completeness: Reconcile source populations, dates, entities and transaction counts before reviewing AI-generated matches.
• Classification: Check high-risk or unusual matches against transaction attributes and source documents.
• Explanation: Validate causal statements in variance commentary against independent records, rather than accepting fluent prose as evidence.
• Change: Confirm whether prompts, retrieval sources, models or configuration changed since the prior reporting period.

A prompt log can help establish what was requested, but it does not prove that the model read complete data or reached a correct conclusion.

Train people on what AI errors look like, and keep training them

Learning to use an AI tool is not the same as learning when to distrust its output. The studies below make that distinction concrete.

• Tool training is not error training. In a large consulting experiment, the group given a prompt-engineering overview did worse than the group with AI alone on a task outside the AI’s capabilities, a difference significant at the 10% level. In a 2025 randomized trial of 44 physicians, reported in a preprint that has not been peer-reviewed, those who had completed 20 hours of AI-literacy training still scored an adjusted 14 points lower on diagnostic reasoning when the AI’s recommendations contained deliberate errors. The authors concluded that AI-literacy training alone was insufficient. Sullivan & Cromwell’s tracked training did not prevent its errors either.
• Seeing the errors helps. In a 24-person process-control simulation, people who experienced the aid’s misses during training relied less on its alarms, and missed fewer events, than people who were only told failures could happen; the effect did not extend to the aid’s diagnoses. In a 2026 trial of 72 physicians by the same group, also a preprint, those shown cues about the AI’s accuracy alongside flawed recommendations scored 7.6 points higher than those without them.
• It has to be ongoing. The FRC’s guidance, written for audit firms, says training on reviewing AI tools may cover automation bias and tools that sound confident when wrong, and that staff education “is an ongoing activity, rather than something which is only relevant at the launch of a tool”. It adds that monitoring a tool’s patterns of failure can show what further training is needed. For covered high-impact uses, US federal agencies must give the people operating the AI periodic training so they can interpret and act on its output. The International AI Safety Report 2026 notes proposals for periodic testing of employees, or “reliance drills”, to watch for over-reliance.
• Once a year is not enough. In a 2025 study of more than 19,500 employees at a US healthcare organization, recent completion of annual cybersecurity awareness training had no significant relationship with whether people fell for simulated phishing. That study concerns phishing, not AI, but it is a warning about compliance-style annual modules. We found no study showing how quickly training on AI errors wears off.

In practice: brief reviewers on the specific errors each tool in the close is known to make; run short, regular exercises in which they review the team’s own AI outputs with errors planted in them; record what they catch; and use the misses to decide what to train next. See What Good AI Use Looks Like at Each Level of a Team for how this varies by role.

Make the review specific enough to test

Start with the AI-assisted close activities that could materially affect booked amounts, financial statements or control evidence.

1. Inventory the review points. Identify where an AI-produced output becomes a figure booked, a reconciliation accepted, a variance explanation approved, or evidence relied upon.
2. State the specific failure modes. Include both wrong outputs and missing outputs; distinguish accuracy, completeness, classification and authorization.
3. Set the precision of the review. Define comparisons, quantitative tolerances and qualitative triggers before the reporting period closes.
4. Confirm the source evidence. Ensure critical input populations and system reports are complete and accurate, or that compensating procedures address known limitations.
5. Train the reviewers on the errors, and repeat it. Use the tool’s own past mistakes and planted errors, not only instructions on how to use it.
6. Record the actual performance. Retain exceptions, investigation steps, resolutions and approvals, not just a check mark.
7. Test and refine. Use seeded defects, past exceptions or independent re-performance to see whether the procedure would detect the errors it is meant to catch.

What human oversight can and cannot do

Human review remains important, particularly where material judgment, exceptions or authorization are involved. But human involvement does not substitute for missing source data, weak permissions, inadequate configuration controls or a poorly designed check.

Nor should every AI use require two or three independent approvers. Segregation of duties should be designed proportionately and supported by system-enforced restrictions and compensating controls when staffing makes full separation impracticable.

Demonstrate what the review can catch

The question for a controller is not simply “Did someone review the AI’s work?” It is “What error was the review designed to find, were the reviewers trained to see it, and what evidence shows they could find it?”

If finance can answer those questions, human oversight becomes a meaningful control rather than an approval step added to a workflow.