Eminence Growth Solutions, home

Search

Subscribe
Menu

Search

Subscribe

AI failures · Incorrect automated decision

Australia's Robodebt Scheme — Income-Averaging Debts Found Unlawful; $746m Refunded to 381,000 People

Verified Checked against its sources on 3 sources quoted

Where it happened
Australia
Organization
Australian Government (Department of Human Services / Services Australia; Department of Social Services)
Date
Scheme ran 2015 to November 2019; class action settled November 2020; Royal Commission report July 7, 2023
Function
Government / Social Services
Where it happened
Debt assessment; benefits administration
System
Automated data-matching and income-averaging debt calculation
Technology
Rules-based automation (not AI)
Type of failure
Incorrect automated decision
Caught by
The person affected, A court

Summary

From 2015 the Australian Government's Robodebt scheme raised welfare overpayment debts by averaging annual tax-office income data across fortnights, a method that did not comply with the Social Security Act and shifted the burden of proof to recipients. After legal challenges, debts based on averaging were stopped in November 2019; the government refunded $746 million to about 381,000 people, wrote off debts totalling $1.751 billion, and settled a class action in November 2020. The Royal Commission, reporting on July 7, 2023, called the scheme crude, cruel and not legal, and made referrals in a sealed chapter. This was rules-based automation, not machine learning.

Consequence

  • Refunds of $746 million to about 381,000 people, write-off of $1.751 billion in debts, and settlement of the class action in November 2020.
  • Royal Commission found the scheme crude, cruel and not legal (report of July 7, 2023).

Who caught it The person affected, A courtFound outside the organization

Analysis

What could have been done

Testing before use

The system is tested for accuracy, bias and failure modes on representative cases before use.

Debts were raised from 2015 by averaging annual tax-office income across fortnights, a method that did not comply with the Social Security Act, and it ran until legal challenges stopped it in November 2019.

Controls over input data

Inputs and source data are complete, current and fit for the use before the AI acts on them.

Annual income data was spread evenly across fortnights to calculate fortnightly overpayments, so the input did not match the period the debt was charged for.

A way for affected people to challenge the decision

People affected by an automated decision can challenge it and get a human re-review.

The scheme shifted the burden of proof to recipients, and the error was surfaced by recipient challenges and Federal Court proceedings; the government refunded $746 million to about 381,000 people and wrote off $1.751 billion in debts.

What would likely have prevented it

  • Obtain and document a legal review confirming that an automated calculation method complies with the governing law before it raises any debt or charge.
  • Do not raise a debt from averaged or estimated data alone; require actual data for the period before an amount becomes payable.
  • Keep the burden of establishing a debt with the organization, and give each person a route to have a person re-check the calculation against actual records.

What would likely have caught it earlier

  • Track how often challenged debts are reduced or cancelled on review, and treat a high reversal rate as a signal that the calculation itself is wrong, escalating it to the scheme owner.

Related reading Treat AI Output Like an Unsupported Journal EntryMost Documented AI Failures Were Found by Someone ElseWhen the System Is Presumed Right

Key dates

  1. July 7, 2023Event
  2. October 16, 2026Checked against sources
  3. October 18, 2026Checked against sources
  4. October 21, 2026Checked against sources

Developments are added when the record is reviewed. Next review: October 16, 2027.

Evidence

Each passage is quoted from its source. The quoted passages are re-checked against the live pages every month.

  1. This involved reimbursement of $746 million to some 381,000 affected individuals and writing off debts amounting in total to $1.751 billion. In November 2020, on the day it was set for trial, the government settled the class action.

    Supports: refund amount and count, write-off, class action settlement

    Report of the Royal Commission into the Robodebt Scheme, Volume 1 Royal Commission into the Robodebt Scheme Primary source

  2. Commissioner Catherine Holmes AC SC presented the Report of the Royal Commission into the Robodebt Scheme to the Governor-General, His Excellency, General the Honourable David Hurley AC DSC (Retd) on 7 July 2023.

    Supports: report date

    Report — Royal Commission into the Robodebt Scheme Royal Commission into the Robodebt Scheme Primary source

  3. Robodebt was a crude and cruel mechanism, neither fair nor legal, and it made many people feel like criminals.

    Supports: Royal Commission finding (quoting the report)

    Crude, cruel and unlawful: Robodebt Royal Commission findings Law Society Journal (NSW) Reporting or analysis

Suggested citation. Eminence Scan, “Australia's Robodebt Scheme — Income-Averaging Debts Found Unlawful; $746m Refunded to 381,000 People”, Eminence Growth Solutions, last verified October 16, 2026, https://eminencegrowthsolutions.com/tools/eminence-scan/record/fail-047/.

Using our research: you may read and quote Eminence Scan with credit to “Eminence Scan, Eminence Growth Solutions” and a link; all other rights reserved. Terms: Using our research.

Spotted an error, or a development we have missed? Report a correction. How we check records: research methodology.