Customer impact

Between July 3, 2026 at 02:19 UTC and July 3, 2026 at 10:00 UTC, a subset of customers experienced incorrect execution behavior for Orchestrator API trigger jobs. Jobs initiated through API triggers did not consistently use their configured execution target, including the intended template, account, and machine settings.

The issue affected customers in the U.S. and Japan regions. Once identified, the fix was deployed to those regions first and then rolled out to the remaining affected regions. Service was fully restored after deployment was completed across all environments.

Root cause

The incident was caused by a recent change to Orchestrator’s job allocation logic that unintentionally altered how execution targets were resolved for API-triggered jobs. As a result, API-triggered jobs did not consistently honor their configured template, account, and machine settings, and could instead be allocated to other eligible machines and accounts within the same folder.

Detection

The issue was first identified through a customer report on July 3, 2026 at 02:19 UTC. The incident was acknowledged immediately, and engineering quickly confirmed the underlying issue. A public status page update was published at 02:24 UTC to inform customers that the issue had been identified and that remediation was in progress.

Response

Following the customer report, engineering initiated an incident response and began mitigating the issue by reverting the affected environments to the previous stable version while simultaneously preparing a permanent fix.

After the corrective changes were developed, reviewed, and validated, they were deployed first to the U.S. and Japan regions, where service was confirmed to be operating normally. The deployment was then completed across the remaining affected regions. Following validation that API-triggered jobs were once again honoring their configured execution targets, the incident was marked mitigated and subsequently resolved.

Follow-Up

Completed

• Added automated regression tests to verify that API-triggered jobs consistently preserve and honor their configured execution targets, including template, account, and machine settings.