Welcome to our status page. If you are looking for help, please check our documentation guides or contact us on our community forum. All products listed below have a target availability of 99.9%.
Uptime over the past 90 days. View historical uptime.
On August 12, 2026, between approximately 15:10 and 18:15 UTC, Communications Mining (IXP) users in the United States region experienced intermittent request failures.
The incident was confined to one of the region's deployment units, where users on it saw failures in bursts of five to ten minutes, with up to 5–7% of their requests failing with 5xx errors at peak.
Between bursts the service operated normally, retried requests generally succeeded, and no data was lost.
A storm of requests to a rarely used API feature that computes machine-learning predictions on demand coincided with repeated retraining of the requested model. Each retraining invalidated cached predictions, turning each request into a multi-minute computation.
The API placed no time limit on how long a request could wait for this computation, so these long-running requests progressively occupied all request-processing capacity, causing unrelated requests to fail. Automatic scaling reached its maximum capacity quickly and could not compensate.
Automated monitoring detected the failures at 15:15 UTC, about five minutes after impact began, and paged the on-call engineer.
The incident was initially attributed only to the client generating the request storm, but client reports and further investigation showed a wider set of users was affected during the failure bursts.
The on-call engineer traced the failures to the unbounded wait in the on-demand prediction path. Service capacity was repeatedly restored by automatic instance replacement while a code fix was developed.
The fix is a strict timeout on the contributing request so it fails fast without affecting other requests, and was deployed to the affected region at approximately 18:00 UTC, and resolution was confirmed at 18:15 UTC.