Skip to content
OutageDeck

Incident detail

ORD data center network outage

Resolved incidentMajor1 affected service

Timeline window

to

Outage alerts

Get alerted the next time Bitrise breaks

Free email alerts for up to 5 providers — no card, live in about a minute. Paid plans add Slack, Discord, and webhook delivery across your whole stack, plus higher API quotas.

Set up free alertsHow Bitrise alerts work

Timeline

Incident updates

Updates are normalized from the official source chronology so timeline changes remain easy to scan.

  1. Investigating

    We are currently investigating a network outage affecting our ORD data center. Builds running in this data center may have failed or been delayed during the outage. Our team identified an issue with Proxmox storage in the affected data center and is working on a fix.

    We will provide a further update as soon as more information is available.

  2. Monitoring

    The network outage affecting our ORD data center has been mitigated. Builds running in this data center may have failed or been delayed during the outage window. Our team identified and addressed an issue with Proxmox storage in the affected data center, and job scheduling has resumed. We are continuing to monitor stability as we investigate the underlying root cause.

    We will provide a further update as soon as more information is available.

  3. Resolved

    This incident has been resolved. We experienced a network outage in our ORD data center caused by dropped packets between nodes, which led to failed health checks and a cluster-wide disruption. The outage was mitigated on July 14, and we've since applied network configuration changes across all our data centers to address the underlying cause and prevent recurrence. We apologize for the disruption it caused.