Customer Impact

Between 11:13 UTC and 20:30 UTC on 10 April 2026, customers using email integrations in UiPath IXP experienced a disruption to email ingestion across all regions. During this time, a subset of new emails were not being ingested into IXP Communications Mining, causing a delay in the overall ingestion of emails.

Root Cause

At ~ 11:13 UTC 10 April 2026, a code change was deployed to IXP that modified how certain types of Microsoft exchange connection errors were handled. The change was intended to improve service resilience by ensuring that a single failing email integration would not disrupt other healthy integrations running on the same server. Due to the nature of the change being around error handling, this did not cause immediate issues.

This change inadvertently removed a recovery mechanism for a specific category of connection error. Previously, when the email sync service encountered a corrupt connection state (for example due to an internal server error from the mail server) the service would automatically restart, clearing the broken connection and resuming normal operation. After the change, the service instead retried the operation against the same broken connection, entering an unrecoverable loop. Affected email integrations stalled and were unable to ingest new emails until a fix was deployed.

Detection

UiPath maintains automated monitoring across all production environments for IXP. This includes real-time alerts for email integration lag, unhandled error rates in the email ingestion mechanism, and synthetic health checks that continuously probe the availability of our services.

The nature of the broken connection error was such that service for a given integration would degrade slowly over time; there was minimal impact when the change was deployed at 11:13 UTC however error rates increased slowly throughout the day. While these kind of connection errors are expected in the normal operations, more finely targeted signal monitoring would have detected these issues (with work underway to implement this). At 17:23 UTC engineers started to investigate the alerts triggered by the automated monitoring systems and were able to start mitigating the root cause by 18:42 UTC before resolving the incident fully and restoring full service at 20:30 UTC.

Response

The engineering team prioritised investigation of this issue and began examining cluster and application logs. The root cause was identified as the code change deployed earlier that day.

A fix reverting the problematic change was prepared and deployed:

• 18:42 UTC, 10 April: The service was manually restarted to refresh broken client states
• 19:51 UTC, 10 April: Fix deployed to first cluster; email ingestion restored
• 20:30 UTC, 10 April: Fix deployed across all main clusters

Due to the design of IXP’s email ingestion mechanism, the system quickly recovered to a state equivalent to if the bug had not been introduced (i.e. all emails that were held up due to the connection errors were eventually ingested for all customers).

Follow-Up

To address this incident and prevent recurrence, we are taking the following steps.

Short term:

• A targeted fix has been developed to re-introduce the resilience improvement from the original change in a way that preserves the existing recovery mechanism for connection errors, and is currently the focus of an in-depth review.
• Additional dedicated testing for the affected error-handling code paths to prevent similar regressions from reaching production.

Long term:

• We are investing in improved observability and telemetry for the email integration and other critical asynchronous infrastructure. This will make it easier to correlate alerts from a common root cause and identify regressions more quickly after a deployment.
• We are also investigating improvements to automatically initiate the status page update process, so that customers can be made aware of activate incidents sooner.