Added latency in working with Data Service in US Region

Customer impact

Users experienced added latency for DataService in US region between 2025-04-08 0723 UTC and 2025-05-07 1930 UTC. All other regions were unaffected.

Root Cause

This was an infrastructure issue where user requests were routed to the farthest compute rather than the closest compute. The issue occurred upon infrastructure upgrades specific to DataService in US region.

Detection

The issue was discovered by our routine review of service performance on 2025-05-07 1128 UTC.

Response

Once identified, we manually overrode the routing configuration to route traffic to the closest compute in US region. Gradual recovery of system performance was observed after this change.

Follow-up

To prevent similar issues in the future, we are taking the following steps:

• Improving Detection Mechanism: Fine-tune our detection mechanism to be more sensitive to latency changes.
• Improving checks: Additional validation checks for safe infrastructure updates.