Eminence Growth Solutions, home

Search

Subscribe
Menu

Search

Subscribe

AI failures · Incorrect automated decision

Optum Impact Pro — Care-Management Algorithm Underestimated Black Patients' Health Needs

Verified Checked against its sources on 3 sources quoted

Where it happened
United States
Organization
Optum / UnitedHealth Group
Date
Published in Science, October 2019 (vol. 366, pp. 447-453); press coverage 2019-10-24
Function
Healthcare
Where it happened
Care management; patient prioritization
System
Impact Pro (Optum) risk-prediction algorithm
Technology
Machine learning
Type of failure
Incorrect automated decision
Caught by
A researcher

Summary

Researchers writing in Science (Obermeyer, Powers, Vogeli and Mullainathan) found that a widely used commercial algorithm for selecting patients for extra care-management help was racially biased: at the same risk score, Black patients were considerably sicker than White patients. The algorithm predicted health-care costs rather than illness, and because less money is spent caring for Black patients with the same needs, it understated their needs. Fixing the disparity would raise the share of Black patients selected for extra help from 17.7% to 46.5%. The paper describes the algorithm as affecting millions of patients; press coverage identified it as Optum's Impact Pro (UnitedHealth Group). On October 25, 2019, New York's financial-services and health regulators wrote to UnitedHealth asking it to show the algorithm was not discriminatory or stop using it.

Consequence

  • New York's Department of Financial Services and Department of Health called on UnitedHealth to investigate and show the algorithm was not discriminatory, or stop using Impact Pro

Who caught it A researcherFound outside the organization

Analysis

What could have been done

Controls over input data

Inputs and source data are complete, current and fit for the use before the AI acts on them.

The algorithm predicted health-care costs rather than illness, and because less money is spent caring for Black patients with the same needs, cost understated their needs.

Testing before use

The system is tested for accuracy, bias and failure modes on representative cases before use.

At the same risk score Black patients were considerably sicker than White patients, a disparity found by outside researchers; correcting it would raise the share of Black patients selected for extra help from 17.7% to 46.5%.

What would likely have prevented it

  • Before a score is used to allocate help, confirm that the quantity the model predicts measures the need the decision is about, and document where the two diverge.
  • Test the score by group against a direct measure of the outcome that matters (here, illness at a given score) before use, with acceptance criteria set in advance.

What would likely have caught it earlier

  • Periodically compare people selected and not selected at the same score on an independent measure of need, by group, and escalate unexplained gaps to the model owner.

Related reading Most Documented AI Failures Were Found by Someone Else

Key dates

  1. October 24, 2019Event
  2. October 16, 2026Checked against sources
  3. October 18, 2026Checked against sources
  4. October 21, 2026Checked against sources

Developments are added when the record is reviewed. Next review: October 16, 2027.

Evidence

Each passage is quoted from its source. The quoted passages are re-checked against the live pages every month.

  1. Scientists detected racial bias in a product sold by Optum, but the problem likely extends to algorithms used by major health systems and insurers

    Supports: Product sold by Optum; dated 24 Oct 2019 (page title and lede only; body behind paywall)

    Racial bias in a medical algorithm favors white patients over sicker black patients The Washington Post Reporting or analysis

  2. The bias arises because the algorithm predicts health care costs rather than illness, but unequal access to care means that we spend less money caring for Black patients than for White patients.

    Supports: Authors, finding, mechanism (cost as proxy), 17.7% to 46.5% figure, 'millions of patients'

    Dissecting racial bias in an algorithm used to manage the health of populations (abstract record) Science, via Europe PMC Primary source

  3. We call on you to immediately investigate these reports and demonstrate that this algorithm is not racially discriminatory or to cease using Impact Pro

    Supports: NY DFS and DOH letter of 25 Oct 2019 to UnitedHealth; product is Impact Pro sold by Optum

    NY Regulators Probe for Racial Bias in Health-Care Algorithm Government Technology (syndicated) Reporting or analysis

Suggested citation. Eminence Scan, “Optum Impact Pro — Care-Management Algorithm Underestimated Black Patients' Health Needs”, Eminence Growth Solutions, last verified October 16, 2026, https://eminencegrowthsolutions.com/tools/eminence-scan/record/fail-018/.

Using our research: you may read and quote Eminence Scan with credit to “Eminence Scan, Eminence Growth Solutions” and a link; all other rights reserved. Terms: Using our research.

Spotted an error, or a development we have missed? Report a correction. How we check records: research methodology.