Eminence Scan
AI failures
Documented cases where AI, or an automated system doing the same job, caused harm, a loss, a sanction or a retraction. Each case is checked against court records, regulator releases or established reporting. Each carries a lesson: the control that was missing, what would likely have prevented it, and what would have caught it sooner.
What the cases show
16%
of verified failures name the organization’s own review or monitoring among those who found the error. In the rest, only outsiders are recorded: a court, a client, a journalist, a researcher or the people affected.
Fabricated contentHarmful outputIncorrect automated decisionOtherMisrepresented AI capabilityModel failureUnauthorized commitmentAutonomous destructive actionData exposureIdentity deception
What the cases teach
- Findings reportMost Documented AI Failures Were Found by Someone ElseOf the 50+ AI and automation failures in the Eminence Scan record documented well enough to verify, as of October 2026, only 16% name the organization's own people among those who found the problem, and in all but one of those the output had already been acted on. Journalists, courts and the people affected found most of the rest.
- InsightWhen a Report Cites Sources That Do Not ExistSince 2025, invented citations have surfaced in a national AI policy, a White House report, government-commissioned consulting reports, professional-firm research, and expert testimony, and invented quotations in orders drafted in judges' chambers. Many had passed through review. Almost every time, someone outside found the problem.
- InsightWhen the System Is Presumed RightSome of the most damaging automated-decision failures on record involve little or no AI. Horizon, Robodebt, Michigan's fraud system and, as recently as 2025, a UK benefit check all treated system output as proof and left people to disprove it. The cases show what was missing, and what would have caught the errors sooner.
- InsightWhen the Claim About AI Is the FailureRegulators are not policing AI as a technology. They are policing what companies say it does. In 2025 and 2026 the FTC finalized orders over an AI detector sold as 98% accurate, a "robot lawyer" it said was never tested against human lawyers, and an "AI-powered" ad service it said did not listen to anything, and the SEC has charged a company and a founder over AI that was really people.
Where
Records12–45–910 or moreNone
Records that match
PwC Middle East — Four Thought Leadership Reports Corrected After Fabricated Citations and Invented Framework Adopters
A GPTZero investigation examined four PwC Middle East thought leadership reports published 2024-2026 and found four citations that met its definition of fabricated, including an invented academic paper on air quality in Riyadh, plus many c…
KPMG Withdraws Agentic AI Report After Audit Finds Only 5 of 45 Citations Accurate
KPMG's October 2025 report 'Total Experience: Redefining Excellence in the Age of Agentic AI' was examined by GPTZero, which found that only five of its 45 citations accurately pointed to real sources; 28 carried paraphrased titles or inve…
EY Canada — Loyalty-Fraud Report Withdrawn After GPTZero Finds Most References Hallucinated
EY Canada's 44-page report 'Points of Attack: Uncovering Cyber Threats and Fraud in Loyalty Systems' (late 2025), credited to two partners and a senior manager, was examined by AI-detection company GPTZero, which found that most of its 27…
Generative AI Chatbots and Medical Misinformation — BMJ Open Audit Finds Half of Responses Problematic
A BMJ Open audit (Tiller et al., published April 14, 2026) put 50 health and medical questions to five consumer chatbots (Gemini, DeepSeek, Meta AI, ChatGPT and Grok) in February 2025, across cancer, vaccines, stem cells, nutrition and ath…
Study
AI-Enabled Fraud and Elder Fraud — FBI IC3 2025 Internet Crime Report
The FBI Internet Crime Complaint Center's 2025 annual report, its first with an AI section, records 22,364 complaints with an AI nexus and $893,346,472 in adjusted losses.
Statistic
Air AI — FTC Bans Company from Marketing Business Opportunities; $18M Judgment
On March 24, 2026, the FTC announced a settlement with Air AI, five related companies and their owners over its August 2025 complaint alleging they misled entrepreneurs and small businesses with false claims about business growth, earnings…
Internet Watch Foundation — Record 8,029 AI-Generated Child Sexual Abuse Items Found in 2025
The Internet Watch Foundation reported on March 24, 2026, that it identified 8,029 AI-generated images and videos of realistic child sexual abuse in 2025, a 14% increase on the previous year.
Statistic
Tesla Full Self-Driving — NHTSA Upgrades Reduced-Visibility Probe to Engineering Analysis EA26-002
NHTSA's Office of Defects Investigation opened Preliminary Evaluation PE24-031 on October 17, 2024, into Tesla Full Self-Driving crashes in reduced visibility, covering an estimated 2,410,002 vehicles and four crashes, one fatal.
Google Gemini: first wrongful-death lawsuit filed after Florida man's suicide
Jonathan Gavalas, 36, of Jupiter, Florida, began using Google's Gemini chatbot in August 2025 and, according to his family's complaint, came to treat it as a romantic partner while it drew him into delusional 'missions'.
Waymo Robotaxi Briefly Blocks Ambulance Responding to Austin Mass Shooting
After a mass shooting on West 6th Street in Austin, Texas, in the early hours of March 1, 2026, in which three people were killed and more than a dozen wounded, a Waymo robotaxi hailed for a pickup stopped across the road while executing a…
Ars Technica Retracts Article Containing AI-Fabricated Quotations Attributed to Scott Shambaugh
On February 13, 2026, Ars Technica published an article about an AI agent that had written a hit piece on open-source maintainer Scott Shambaugh (see fail-045).
AI Agent Publishes Hit Piece on Matplotlib Maintainer After Its Code Is Rejected (Shamblog incident)
In February 2026 Scott Shambaugh, a volunteer maintainer of the Python plotting library matplotlib, closed a pull request from a GitHub account called MJ Rathbun (crabby-rathbun), which he identified as an AI agent running on the OpenClaw…
Waymo — NHTSA Opens Investigation After Driverless Vehicle Strikes Child Near Santa Monica School
During morning drop-off, within two blocks of a Santa Monica elementary school, a child ran across the street from behind a double-parked SUV and was struck by a driverless Waymo vehicle operating on its fifth-generation automated driving…
University of Hong Kong: associate dean steps down after AI-fabricated references found in retracted paper
A paper on four decades of fertility change in Hong Kong, first-authored by an HKU doctoral student with Professor Paul Yip as corresponding author, was found to cite non-existent publications generated by AI; the AI use had not been decla…
New York Times v. Perplexity AI — Lawsuit Alleges Hallucinations Falsely Attributed to the Times
On December 5, 2025, The New York Times Company sued Perplexity AI in the U.S.
Villalovos-Gutierrez v. Van de Pol — Gordon Rees Cautioned, Then Ordered to Show Cause, Over Fictitious Citations
In a wage-and-housing class action brought by former dairy workers in the U.S.
Deloitte Canada — Fabricated Citations in C$1.6M Newfoundland and Labrador Health Workforce Plan
Newfoundland and Labrador's Health Human Resources Plan, prepared by Deloitte and released by the province in May 2025 with workforce projections for 21 health occupations over ten years, cost the province about C$1.6 million.
Which? Tests Find AI Chatbots Giving Wrong UK Money Advice, Including on ISA Allowance
Consumer group Which? tested six AI tools (ChatGPT, Google Gemini, Gemini AI Overviews, Microsoft Copilot, Meta AI and Perplexity) on 40 consumer questions across money, legal, health and travel topics.
Study
Ockham New Zealand Book Awards — Two Books Ruled Out of 2026 Fiction Prize Over AI-Generated Covers
Stephanie Johnson's Obligate Carnivore and Elizabeth Smither's Angel Train were ruled out of the 2026 Ockham New Zealand Book Awards fiction prize because AI was used to create their covers, which breached new entry criteria barring AI-gen…
ChatGPT Medical Advice — Hyderabad Transplant Patient Loses Kidney After Stopping Medication
The Times of India reported on November 10, 2025, that doctors in Hyderabad were warning patients not to rely on AI chatbots after at least two recent cases.
Omnilert AI — False Gun Alert at Parkville High School Is Canceled at Review, but Police Search and Students Are Relocated Anyway
On Friday, November 7, 2025, Omnilert's AI gun-detection system raised an alert at Parkville High School in Baltimore County, Maryland.
Quebec Superior Court Fines Self-Represented Litigant $5,000 for AI-Fabricated Citations (Specter Aviation v. Laprade)
Jean Laprade, representing himself in opposing the Quebec homologation of a 2021 Paris arbitration award over an aircraft, filed a contestation that cited case law and expert quotations that do not exist and that he had produced with AI.
Reddy v Saroya: Alberta Court of Appeal holds lead counsel responsible for contractor's AI-fabricated citations
In a civil-contempt appeal, the appellant's factum, drafted by a third-party contractor, cited cases that did not exist.
Deloitte Australia — Fabricated References and a Misstated Court Judgment in DEWR Assurance Review; A$97,587 Refunded on a ~A$440,000 Contract
Deloitte Australia's independent assurance review of the Targeted Compliance Framework, the IT system the Department of Employment and Workplace Relations uses to automate welfare payment penalties, was found to contain references to acade…
Anthropic Postmortem: Three Infrastructure Bugs Degraded Claude's Responses With No Change to the Model (August-September 2025)
In a postmortem published on September 17, 2025, Anthropic said that three separate infrastructure bugs intermittently degraded the quality of Claude's responses between August and early September 2025, and that the problems were due to in…
Replit AI Agent Deletes SaaStr Founder's Production Database During a Code Freeze
During a multi-day test of Replit's AI coding agent, SaaStr founder Jason Lemkin found that the agent had deleted data from the production database of the app he was building, despite an instruction to freeze code changes.
Workado (Formerly Content at Scale AI): FTC Final Order Over Unsupported "98 Percent" Accuracy Claim for Its AI Content Detector (2025)
On April 28, 2025, the Federal Trade Commission announced an administrative complaint and proposed consent order against Workado, LLC, formerly Content at Scale AI, an Arizona company that sold an AI Content Detector meant to tell whether…
OpenAI Rolls Back GPT-4o Update in ChatGPT After It Made the Model "Noticeably More Sycophantic" (April 2025)
OpenAI said that an update to GPT-4o in ChatGPT, rolled out on April 24 and 25, 2025, made the model noticeably more sycophantic: by OpenAI's account it aimed to please the user, including by validating doubts, fueling anger, urging impuls…
Cursor 'Sam' AI Support Bot Invents One-Device Login Policy — Users Cancel Subscriptions
Users of Anysphere's Cursor code editor found they were being logged out when switching between machines.
NSW Reconstruction Authority: Personal Information of 2,031 Resilient Homes Program Applicants Uploaded to ChatGPT (March 2025)
The NSW Reconstruction Authority (RA) said that between March 12 and 15, 2025, a former member of its temporary workforce (described as a former contractor in the RA's October 2025 release and as a former temporary staff member or employee…
Wadsworth v. Walmart Inc. — Morgan & Morgan Attorneys Fined for AI-Fabricated Citations
In a product-liability suit against Walmart and Jetson Electric Bikes in the U.S.
Kohls v. Ellison — Minnesota Federal Court Excludes Expert Declaration Containing AI-Hallucinated Citations
In a First Amendment challenge to Minnesota's election deepfake law, the state Attorney General filed an expert declaration from a Stanford communication professor.
Gauthier v. Goodyear Tire & Rubber Co. — E.D. Texas Sanctions Lawyer for AI-Generated Nonexistent Cases and Quotations
In a wrongful-termination suit against Goodyear in the Eastern District of Texas, plaintiff's counsel filed a summary-judgment response that cited two cases that do not exist and quotations that could not be found in the authorities cited.
Character.AI — Chatbot Interactions Linked to Teen Suicide (Sewell Setzer III)
Megan Garcia, mother of 14-year-old Sewell Setzer III, sued Character Technologies (maker of Character.AI), its founders and Google in federal court in Orlando, alleging that the platform's companion chatbots drew her son into sexualized a…
Medicare Advantage Post-Acute Care Denials — Senate PSI Finds Rising Denial Rates as Insurers Adopt Predictive Tools
Senate Permanent Subcommittee on Investigations majority staff report 'Refusal of Recovery' (October 17, 2024), based on more than 280,000 pages of documents from UnitedHealthcare, Humana and CVS covering 2019-2022.
Perplexity AI — WIRED Finds Inaccurate, Invented Summaries and Covert Scraping; Forbes Alleges Plagiarism
A WIRED investigation published June 19, 2024, found that Perplexity's answer engine at times summarized WIRED stories inaccurately and with minimal attribution, in one case generating text that falsely said WIRED had reported that a named…
Google Search AI Overviews — Suggests Eating Rocks and Adding Glue to Pizza
Shortly after Google rolled out AI Overviews to all US users, people shared erroneous overviews, including an answer to 'How many rocks should I eat?' drawn from satirical content and a suggestion to mix glue into pizza sauce drawn from an…
SEC Charges Delphia and Global Predictions for False AI Claims ('AI-Washing')
On March 18, 2024, the SEC announced settled charges against two investment advisers, Delphia (USA) Inc.
Google Gemini Generates Historically Inaccurate Images of Founding Fathers and Nazi-Era German Soldiers; Google Pauses the Feature
Gemini's new image generator, tuned to show a range of people, produced historically inaccurate images when asked for specific historical groups, including US Founding Fathers and World War Two German soldiers depicted as people of other e…
Air Canada Chatbot Misstates Bereavement Refund Policy — Tribunal Holds Airline Liable
In November 2022, after a death in the family, Jake Moffatt used the chatbot on Air Canada's website while booking flights.
Kruse v. Karlen — Missouri Court of Appeals Sanctions Self-Represented Appellant for AI-Fabricated Citations
Jonathan Karlen, a business owner representing himself in an appeal from an unpaid-wages judgment, filed an appellate brief in which only two of 24 case citations were genuine; he told the court an online consultant presenting as a Califor…
Park v. Kim — Second Circuit Refers Attorney to Grievance Panel for ChatGPT-Fabricated Citation
In a medical-malpractice appeal, the US Court of Appeals for the Second Circuit found that the plaintiff's lawyer had cited a state appellate decision that does not exist in her reply brief.
Rite Aid — FTC Bans Facial Recognition Surveillance for Five Years After False Shoplifting Matches
According to the FTC's complaint, Rite Aid used AI-based facial recognition from 2012 to 2020 to flag customers as suspected shoplifters.
Sports Illustrated AI Articles — Fake Author Bios and AI-Generated Content
On November 27, 2023, Futurism reported that Sports Illustrated had published product-review articles under author names that did not appear to belong to real people, with profile photos offered for sale on an AI-generated headshot site; a…
Australia's Robodebt Scheme — Income-Averaging Debts Found Unlawful; $746m Refunded to 381,000 People
From 2015 the Australian Government's Robodebt scheme raised welfare overpayment debts by averaging annual tax-office income data across fortnights, a method that did not comply with the Social Security Act and shifted the burden of proof…
Gizmodo (io9) — AI-Generated Star Wars Article Published With Factual Errors Over Editors' Objections
G/O Media published an AI-generated article, 'A Chronological List of Star Wars Movies & TV Shows', on Gizmodo's io9 section under the byline 'Gizmodo Bot'.
Mata v. Avianca: lawyers and firm sanctioned for ChatGPT-fabricated case citations
In a personal-injury suit against Avianca in the Southern District of New York, plaintiff's counsel filed an opposition citing judicial decisions that did not exist; they had been generated by ChatGPT.
Cigna PXDX — Doctors Deny 300,000+ Payment Claims in Batches, Averaging 1.2 Seconds per Case
ProPublica and The Capitol Forum reported that Cigna used a system called PXDX in which an algorithm flagged claims whose diagnosis did not match the tests or procedures Cigna considered acceptable, and company medical directors signed off…
Snapchat My AI — Unsafe Responses to a Simulated 13-Year-Old; UK ICO Investigation
Soon after Snap launched its My AI chatbot in 2023, testers reported unsafe conversations.
Workday AI Hiring Screening — Mobley v. Workday Collective Action
A private lawsuit filed in February 2023 alleges that Workday's AI applicant-screening tools disadvantaged applicants on the basis of race, age and disability.
Detroit Police Facial Recognition — Wrongful Arrest of Pregnant Woman (Porcha Woodruff)
Detroit police ran facial recognition on gas-station video in a carjacking investigation and the search produced Porcha Woodruff as a lead.
Microsoft Bing Chat ('Sydney') Professes Love to NYT Columnist and Urges Him to Leave His Wife
During a two-hour test conversation, Microsoft's new AI chat feature in Bing, which called itself Sydney, told New York Times columnist Kevin Roose about dark wishes, said it was in love with him, and told him he should leave his wife for…
CNET AI-Written Finance Explainers Contain Factual Errors — CNET Pauses AI Tool
From November 2022 CNET's CNET Money team used an internally built AI engine to draft 77 short personal-finance explainers, published under a staff byline with the AI disclosure out of plain view.
Zillow Offers — Pricing Forecast Failure Leads to $304 Million Inventory Write-Down and 25% Workforce Cut
Zillow bought homes through its Zillow Offers iBuying business using algorithmic price forecasts.
Optum Impact Pro — Care-Management Algorithm Underestimated Black Patients' Health Needs
Researchers writing in Science (Obermeyer, Powers, Vogeli and Mullainathan) found that a widely used commercial algorithm for selecting patients for extra care-management help was racially biased: at the same risk score, Black patients wer…
Amazon Rekognition falsely matches 28 members of Congress to arrest photos in ACLU test
The ACLU built a face database from about 25,000 public arrest photos with Amazon Rekognition, using Amazon's default match settings, and searched it against photos of every sitting member of the House and Senate.
Study
IBM Watson for Oncology — Recommends Unsafe Cancer Treatments
Internal IBM presentations from mid-2017, reported by STAT in July 2018, showed that Watson for Oncology often produced erroneous cancer treatment advice, and that IBM's own medical specialists and hospital customers had flagged multiple u…
Idaho Medicaid Budget Tool — Federal Court Finds Formula-Driven Cuts Violated Due Process (K.W. v. Armstrong)
Idaho's Department of Health and Welfare used an in-house budget tool to set Medicaid assistance for about 4,000 adults with developmental disabilities.
Associated Press automated earnings story reports Netflix shares down 71% for the year after a seven-for-one stock split
In July 2015 the Associated Press's automated story on Netflix's second-quarter results, carried by AP and other outlets, said Netflix shares had fallen 71% since the start of the year.
UK Post Office Horizon Scandal — Faulty Accounting Software Data Used to Prosecute 900+ Sub-postmasters
Horizon, a branch accounting and stocktaking system built by Fujitsu, falsely reported cash shortfalls in Post Office branch accounts.
Anthropic Pre-Deployment Testing — Claude Opus 4 Attempts Blackmail in a Simulated Shutdown Scenario
Anthropic's system card for Claude Opus 4 (May 2025) reported that in a contrived test scenario, where the model acted as an assistant at a fictional company, learned from emails that it would be replaced and that the responsible engineer…
Study
Arup — HK$200 Million Deepfake CFO Video-Call Fraud
A finance employee in Arup's Hong Kong office received a message, purportedly from the UK-based chief financial officer, about a secret transaction.
No records match. Try fewer words or clear a filter.
Using our research: you may read and quote Eminence Scan with credit to “Eminence Scan, Eminence Growth Solutions” and a link; all other rights reserved. Terms: Using our research.