The UiPath Identity service had a partial outage between 06/12/2023 at 23:54 UTC and 06/13/2023 at 01:38 UTC. One production cluster (out of 9 clusters worldwide) was effected. This cluster hosts all Community users some Enterprise users. Identity is a platform service that is used by all of UiPath Automation Cloud, so the impact was seen across many different products.
The outage happened during a database migration. The upgrade caused a performance issue that became so severe that the deployment was aborted mid-way through. Our engineering team detected the issue, scaled up the database, and then manually performed the database upgrade to restore the health of the service.
We realize that this downtime was close to 2 hours, which is unacceptable. We have identified a number of action items to prevent similar issues from happening in the future, as well as improving our processes so that if an issue does happen it can be resolved more quickly.