AI failures · Other
Anthropic Postmortem: Three Infrastructure Bugs Degraded Claude's Responses With No Change to the Model (August-September 2025)
Verified Checked against its sources on 1 source quoted
- Where it happened
- International or several countries
- Organization
- Anthropic
- Date
- Bugs from August 5 to mid-September 2025; postmortem published September 17, 2025
- Function
- AI model serving infrastructure
- Where it happened
- Infrastructure changes to request routing, server configuration and token sampling
- System
- Claude (Sonnet 4, Opus 4 and 4.1, Haiku 3.5, Opus 3), first-party API and Claude Code, with some effects on Amazon Bedrock and Google Cloud Vertex AI
- Technology
- Generative AI
- Type of failure
- Other
- Caught by
- A client or customer
Summary
In a postmortem published on September 17, 2025, Anthropic said that three separate infrastructure bugs intermittently degraded the quality of Claude's responses between August and early September 2025, and that the problems were due to infrastructure bugs alone, not to any reduction in model quality. By Anthropic's account: from August 5 some Sonnet 4 requests were misrouted to servers configured for an upcoming 1M-token context window, rising after a routine load-balancing change on August 29 to 16% of Sonnet 4 requests at the worst hour on August 31, and about 30% of Claude Code users had at least one misrouted message; an August 25 misconfiguration on TPU servers corrupted output, for example inserting Thai or Chinese characters into English answers or syntax errors into code; and an August 25 change to token selection triggered a latent compiler bug affecting Claude Haiku 3.5 and possibly some Sonnet 4 and Opus 3 requests. Anthropic said users began reporting degraded responses in early August, that the reports were hard to tell from normal variation, and that it opened an investigation in late August. It said its evaluations did not capture the degradation, that it did not at first connect the August 29 spike in reports to the load-balancing change, and that privacy controls limited engineers' access to affected conversations. It rolled back or fixed each change between September 2 and September 18 and said it would run quality evaluations continuously on production systems. These facts are as Anthropic reported them.
Consequence
- Anthropic said it rolled back the output-corruption change on September 2 and added detection tests for unexpected character outputs to its deployment process
- Anthropic said it deployed the routing fix from September 4, completing the rollout by September 18
- Anthropic said it will run quality evaluations continuously on production systems
Who caught it A client or customerFound outside the organization
Analysis
What could have been done
Error rates, overrides, complaints and drift are monitored, with thresholds that trigger escalation.
Users began reporting degraded responses in early August, but Anthropic said the reports were hard to distinguish from normal variation, it opened an investigation only in late August, and it did not at first connect the August 29 spike in reports to the load-balancing change.
The system is tested for accuracy, bias and failure modes on representative cases before use.
Anthropic said the evaluations it ran did not capture the degradation users reported, and an August 25 misconfiguration that produced unexpected characters reached users before detection tests for such output were added to the deployment process.
What would likely have prevented it
- Test each infrastructure change, not only each model change, for its effect on output quality, using checks able to tell a working implementation from a broken one.
- Record every change to routing, configuration and serving code with a quality check attached, so the change can be rolled back when that check moves.
What would likely have caught it earlier
- Run quality checks continuously on the production system and compare any rise in complaints with the log of recent changes, with a threshold that opens an investigation.
Key dates
- August 2025Event
- October 21, 2026Checked against sources
Developments are added when the record is reviewed. Next review: October 21, 2027.
Evidence
Each passage is quoted from its source. The quoted passages are re-checked against the live pages every month.
-
Between August and early September, three infrastructure bugs intermittently degraded Claude's response quality. We've now resolved these issues and want to explain what happened.
Supports: what and when: three infrastructure bugs degraded Claude's response quality, August to early September 2025
To state it plainly: We never reduce model quality due to demand, time of day, or server load. The problems our users reported were due to infrastructure bugs alone.
Supports: Anthropic's statement that the problems were due to infrastructure bugs alone, not a reduction in model quality
In early August, a number of users began reporting degraded responses from Claude. These initial reports were difficult to distinguish from normal variation in user feedback.
Supports: how caught: user reports from early August, hard to distinguish from normal variation
By late August, the increasing frequency and persistence of these reports prompted us to open an investigation that led us to uncover three separate infrastructure bugs.
Supports: the reports prompted an investigation in late August that found the bugs
On August 5, some Sonnet 4 requests were misrouted to servers configured for the upcoming 1M token context window
Supports: bug 1: context-window routing error from August 5
At the worst impacted hour on August 31, 16% of Sonnet 4 requests were affected.
Supports: scale: 16% of Sonnet 4 requests at the worst hour
Approximately 30% of Claude Code users who made requests during this period had at least one message routed to the wrong server type, resulting in degraded responses.
Supports: scale: about 30% of Claude Code users had at least one misrouted message
On August 25, we deployed a misconfiguration to the Claude API TPU servers that caused an error during token generation.
Supports: bug 2: output corruption from an August 25 misconfiguration
for example producing Thai or Chinese characters in response to English prompts, or producing obvious syntax errors in code.
Supports: symptoms of the output corruption
On August 25, we deployed code to improve how Claude selects tokens during text generation. This change inadvertently triggered a latent bug in the XLA:TPU
Supports: bug 3: token-selection change triggered a latent XLA:TPU compiler bug
Resolution: We identified the issue and rolled back the change on September 2. We've added detection tests for unexpected character outputs to our deployment process.
Supports: consequence: output-corruption change rolled back September 2; detection tests added
We deployed the fix on September 4. Rollout to our first-party platform and Google Cloud's Vertex AI was completed by September 16, and to AWS Bedrock by September 18.
Supports: consequence: routing fix deployed from September 4, completed by September 18
The evaluations we ran simply didn't capture the degradation users were reporting, in part because Claude often recovers well from isolated mistakes.
Supports: Anthropic's account of why its evaluations missed the degradation
When negative reports spiked on August 29, we didn't immediately make the connection to an otherwise standard load balancing change.
Supports: the August 29 spike was not immediately linked to the load-balancing change
Our internal privacy and security controls limit how and when engineers can access user interactions with Claude, in particular when those interactions are not reported to us as feedback.
Supports: privacy controls limited engineers' access to the affected interactions
Although we run regular evaluations on our systems, we will run them continuously on true production systems to catch issues such as the context window load balancing error.
Supports: commitment: continuous quality evaluations on production systems
Published Sep 17, 2025
Supports: postmortem publication date
Resolution: We first observed the bug affecting Haiku 3.5 and rolled it back on September 4. We later noticed user reports of problems with Opus 3 that were compatible with this bug, and rolled it back on September 12.
Supports: bug 3 rollbacks: Haiku 3.5 on September 4, Opus 3 on September 12
We also believe this could have impacted a subset of Sonnet 4 and Opus 3 on the Claude API.
Supports: bug 3 may also have affected some Sonnet 4 and Opus 3 requests
Suggested citation. Eminence Scan, “Anthropic Postmortem: Three Infrastructure Bugs Degraded Claude's Responses With No Change to the Model (August-September 2025)”, Eminence Growth Solutions, last verified October 21, 2026, https://eminencegrowthsolutions.com/tools/eminence-scan/record/fail-100/.
Using our research: you may read and quote Eminence Scan with credit to “Eminence Scan, Eminence Growth Solutions” and a link; all other rights reserved. Terms: Using our research.
Spotted an error, or a development we have missed? Report a correction. How we check records: research methodology.
