2026-July-15 Resolved Service Incident - Error Reporting (Backtrace)

Between July 14th 23:45 UTC and July 15th 23:05 UTC, we experienced an issue where symbol archive uploads to Backtrace projects failed with HTTP 400 errors. Incident has been resolved and all services are operational.

### Dates:

Tuesday July 14th 2026, 23:45 UTC – Wednesday July 15th 2026, 23:05 UTC

### What happened:

Symbol archive uploads to Error Reporting (Backtrace) projects failed with HTTP 400 errors. All regions were affected.

### Why it happened:

The credentials our symbol processing service used to write to cloud storage were no longer valid, so uploads could not be stored. Failed uploads were retried repeatedly and accumulated on local disk until the service ran out of space, at which point it also began rejecting new uploads.

### How we fixed it:

We reissued the storage credentials, increased disk capacity on the affected workers, and restarted the service. The queued uploads then processed successfully, and we confirmed recovery with affected customers.

### What we are doing to prevent it from happening again:

We've added disk-utilization monitoring and alerting to this service so we detect the condition ourselves before it affects uploads, rather than relying on customer reports.