Between March 26, 2025, at 14:47 UTC, and March 26, 2025, at 20:00 UTC, two UiPath Automation Cloud services had partial outages in the U.S. region. Document Understanding customers may have experienced errors or increased latency while using the digitization capabilities. In Orchestrator, a subset of the tenants saw latency and some failed requests.
Azure declared an incident with their Virtual Machine service at 13:30 UTC. This did not immediately impact Automation Cloud. But later, our traffic increased during U.S. business hours. Document Understanding and Orchestrator were unable to scale up to handle this extra traffic. By 14:47 UTC, this lack of capacity had caused latency and errors.
The increased latencies alerted us at 15:02 UTC. We immediately started investigating. During the investigation, customers reached out to Support to report degraded performance.
Azure's incident prevented the allocation of new resources.
Document Understanding moved traffic to existing resources that had spare capacity. It was running at full capacity by 17:45 UTC, and had finished processing the backlog.
Orchestrator re-created their infrastructure using resources that Azure's issue did not affect. They then switched traffic to the new instance. The Orchestrator latency issues were mitigated by 20:00 UTC.
We are working with Azure to see what can be done to avoid these sorts of incidents. We are also looking at our own tools and processes to make it faster to detect and fix infrastructure problems.