AI failures · Harmful output

IBM Watson for Oncology — Recommends Unsafe Cancer Treatments

Verified Checked against its sources on October 16, 2026 2 sources quoted

Where it happened: International or several countries
Organization: IBM Watson Health
Date: July 25, 2018
Function: Healthcare
Where it happened: Oncology treatment planning
System: IBM Watson for Oncology
Technology: Machine learning
Type of failure: Harmful output
Caught by: Internal review, A client or customer, A journalist

Summary

Internal IBM presentations from mid-2017, reported by STAT in July 2018, showed that Watson for Oncology often produced erroneous cancer treatment advice, and that IBM's own medical specialists and hospital customers had flagged multiple unsafe and incorrect recommendations. Most of the data used to train the system were hypothetical cases rather than real patient data.

Who caught it Internal review, A client or customer, A journalist

Analysis

What could have been done

Testing before use

The system is tested for accuracy, bias and failure modes on representative cases before use.

Internal IBM documents showed the system often produced erroneous cancer treatment advice, and IBM's own medical specialists and hospital customers identified multiple unsafe and incorrect recommendations in its output.

Controls over input data

Inputs and source data are complete, current and fit for the use before the AI acts on them.

Most of the data used to train the system were hypothetical cases rather than real patient data.

What would likely have prevented it

Validate a decision-support model on real, representative cases from the setting where it will be used, with domain specialists reviewing the results, before customers rely on it.
Record the make-up of the training data, including the share of hypothetical or synthetic cases, and disclose it to the customers who rely on the recommendations.

What would likely have caught it earlier

Log every recommendation that specialists or customers flag as unsafe, with a threshold at which a named product owner must review and restrict the affected recommendations.

Key dates

July 25, 2018Event
October 16, 2026Checked against sources
October 18, 2026Checked against sources
October 21, 2026Checked against sources

Developments are added when the record is reviewed. Next review: October 16, 2027.

Evidence

Each passage is quoted from its source. The quoted passages are re-checked against the live pages every month.

IBM documents show that its Watson supercomputer often spit out erroneous cancer treatment advice and that company medical specialists and customers identified “multiple examples of unsafe and incorrect treatment recommendations”

Supports: what happened; identified by IBM's own medical specialists and customers; date

IBM’s Watson supercomputer recommended ‘unsafe and incorrect’ cancer treatments, internal documents show STAT (Casey Ross and Ike Swetlitz, July 25, 2018) Reporting or analysis
But it turns out most of the data fed to it is hypothetical and not real patient data.

Supports: training on hypothetical rather than real patient cases

IBM’s Watson gave unsafe recommendations for treating cancer The Verge (July 26, 2018) Reporting or analysis

Suggested citation. Eminence Scan, “IBM Watson for Oncology — Recommends Unsafe Cancer Treatments”, Eminence Growth Solutions, last verified October 16, 2026, https://eminencegrowthsolutions.com/tools/eminence-scan/record/fail-020/.

Using our research: you may read and quote Eminence Scan with credit to “Eminence Scan, Eminence Growth Solutions” and a link; all other rights reserved. Terms: Using our research.

Spotted an error, or a development we have missed? Report a correction. How we check records: research methodology.