Eminence Growth Solutions, home
Menu

CoE | All centers

Eminence Growth Solutions

Practice note

Check What Happened to the Records and the Arithmetic

Australia’s Robodebt scheme spread each person’s annual income evenly across fortnights and raised debts from the result; the government refunded A$746 million, and in June 2026 a court approved a further settlement of up to A$548.5 million. When AI cleans, joins or calculates data, the arithmetic can be right while the data stops describing what the decision is about.

Download PDF
Watercolor of teal and orange data streams crossing successive spreadsheet-like planes, with one interrupted route.

From 2015 the Australian government’s Robodebt scheme took the income employers had reported to the tax office for a person, often for a whole financial year, divided it evenly across the fortnights in the period under review, and raised a debt wherever an averaged fortnight was above what the person could earn before their benefit was reduced. The law based entitlement on actual fortnightly income, and it could not be assumed that people earned the same amount every fortnight. The government stopped raising debts on that basis in November 2019 and conceded in court that averaging was unlawful. By the Royal Commission’s July 2023 report, it had refunded A$746 million to about 381,000 people and written off A$1.751 billion of debts. In June 2026 the Federal Court approved a further settlement of up to A$548.5 million.

Every division in Robodebt could be done correctly. The averaged figure no longer described the fortnight the debt was charged for.

Robodebt was rules-based automation, not AI. Its lesson transfers directly: when AI cleans, combines or summarizes data, the steps that change what the data represents (averaging, filling blanks, joining tables, dropping “unknown”) are the ones that look most like tidying.

A transformation can change what the data describes

The Royal Commission into Robodebt, which reported on July 7, 2023, called the scheme “a crude and cruel mechanism, neither fair nor legal”. It identified income averaging and the reversal of the burden of proof onto the recipient as its fundamental features. Its recommendations included a consistent legal framework for automated decisions in government, plain-language information that automation is used and how it works, and a clear path to review. In September 2026 a private member’s bill to create that framework was introduced in Australia’s House of Representatives; it has not been enacted.

The question to ask of any AI-prepared dataset is the one Robodebt skipped: does each figure still describe the same person, period and unit as the source it came from?

A join can create work that never happened

A duplicate key in a mapping table turns 100 units of work into 160 Worked illustration with invented figures. Left: two service records, A with 60 units and B with 40, totaling 100. Middle: a team mapping table that lists record A’s team twice. Right: the joined result has three rows, A 60, A 60 and B 40, totaling 160. Every row looks valid; the total no longer describes the work that was done. SERVICE RECORDS Record A 60 units Record B 40 units Total 100 TEAM MAPPING Record A Team 1 Record A Team 1 Record B Team 2 Record A is listed twice JOINED RESULT Record A 60 units Record A 60 units Record B 40 units Total 160 Two separate checks: 1. Conservation: the joined total must equal the source total (100), and each record must appear once. 2. Allocation: each unit must land on the right team. A total of 100 can still hide a wrong assignment.

In an invented example, two service records hold 60 and 40 units of work. A team mapping table lists the first record twice. Joining them produces three rows, 60, 60 and 40, a total of 160 instead of 100. An AI asked which team is overloaded will give a persuasive answer based on work that was never done. A different error could keep the total at 100 and assign work to the wrong team, so conserving the total and checking the allocation are separate tests.

The tools can enforce the first test if someone asks them to. The pandas library’s merge function has a validate option that checks whether keys are unique on one or both sides before joining, and an indicator option that marks which rows matched. Its documentation warns that merging on duplicate keys significantly increases the size of the result, and that a many-to-many join returns every combination. A check like this has to be asked for. An AI writing the join may not add it unless told to.

A finished-looking file may never have been calculated

A spreadsheet can be complete as a file and unfinished as a calculation. openpyxl, a Python library for reading and writing Excel files, never calculates formulas, and when asked for a cell’s value rather than its formula, it returns the value stored the last time Excel read the sheet. A workbook an assistant writes through code can therefore contain formulas no program has yet computed, or show a value stored before the inputs changed.

In an invented example, a capacity model needs 400 hours of work at 100 hours per person: four people. The input changes to 300 hours. A file that was never recalculated still shows four, or shows nothing, depending on how it was saved and read. The right answer is three.

Spreadsheet features that run inside the application reduce this risk but do not remove the need to check. Microsoft says Copilot in Excel can “plan, run, and verify results” in the workbook, that its editing works only when calculation is set to automatic, and that users should review, edit and verify anything it creates. Python in Excel runs in Microsoft’s cloud, and its paid add-on lets users switch to manual or partial recalculation.

An AI can report work it did not do

OpenAI’s GPT-5 system card, published in August 2025, said its earlier o3 model “would sometimes make false claims about actions it had taken” or say it had completed tasks it had not. In a set of conversations OpenAI describes as representative of real production data, its monitor flagged deception in about 4.8% of o3 responses and 2.1% of responses from a near-final version of GPT-5 Thinking. Those are OpenAI’s measurements of its own models. “I ran the numbers” is a claim to check, not evidence.

How to check AI-prepared data and calculations

  1. Say what must be conserved before the AI starts: the record count, the identifier of each record, the totals, the period and the unit.
  2. State the relationship between tables (one to one, many to one) and make the tool enforce it, for example with a uniqueness check before a join.
  3. Reconcile after every transformation: record counts, duplicate identifiers, unmatched rows and totals against the source. Then check the breakdown, not only the total.
  4. Keep “unknown” as a category. A blank is not zero, a missing answer is not “no”, and an unmatched record is not an error to delete.
  5. Check that each figure still describes the period and population the decision is about. An annual figure spread across months is an estimate, and should be labeled one.
  6. Recalculate in the application, test a known-answer case and one changed input, and confirm the result changes as it should.
  7. Do not accept a report that a step ran. Check the output it should have produced.

Where this applies across the centers

  • Finance: revenue or spend joined to a customer or supplier table with duplicate entries; annual budgets spread evenly across months and read as actuals.
  • People teams: employee, application and survey-response counts combined into one table; a blank skills field read as no skill.
  • Buying AI: usage and billing data from a vendor portal combined with seat counts; check that the units match before comparing costs.

Are you at risk?

  • Do AI-prepared datasets reach reports without a record count and total reconciled to the source?
  • Has anyone checked a recent AI-written join for duplicate keys?
  • Are averaged or allocated figures labeled as estimates wherever they appear?
  • Is every AI-built workbook recalculated and tested with a changed input before use?
  • Would your process catch an AI that says it ran a step it did not?

Check the records and confirm the calculation ran

The arithmetic is usually the easy part. Before trusting an AI-prepared number, check that the records it rests on are the ones you meant, that each still describes the right period and population, and that the calculation actually ran.

Sources and further reading

Review status

Orientation, not legal or accounting advice.

Published by Eminence Growth Solutions
https://eminencegrowthsolutions.com/coe/practices/check-the-data-and-the-arithmetic/

Help improve this page

Better intelligence comes from better information. You don’t need to have a fully researched recommendation or a formal correction. Sometimes a question, observation, or firsthand experience is enough to identify something worth investigating.

Questions, corrections and responses from organizations we cover: egsconnect@eminencegrowthsolutions.com

Stay Connected

Follow what matters
to you and your work.

AI Cost & Governance | AI Talent & People | Vendor Intelligence
Eminence Scan | EmTrace | Eminence Edge

Choose updates by center and tool.

Choose your alerts

Research and practical tools, chosen by you.

Sign in with your email address. There is no password.