Skip to content

Incident detail

Increased Error Rates in SFO3

Active incidentMinor

Timeline window

Started

Get alerted the next time DigitalOcean breaks

Free email alerts for the handful of vendors you cannot afford to miss. No card, live in about a minute. Paid plans add Slack, Teams, Discord, and webhook delivery across your whole stack, plus higher API quotas.

Timeline

Incident updates

Every update on the official status source, oldest to newest, exactly as it appeared there.

  1. Investigating

    As of 02:53 UTC, our Engineering team is investigating increased error rates impacting multiple services in the SFO3 region. At this point, users with resources in SFO3 may experience errors, timeouts, or delays when creating or managing Droplets, Volumes, Load Balancers, Reserved IPs, Cloud Firewalls, Managed Databases, and Managed Kubernetes clusters through the API or Cloud Control Panel. Some users may also experience connectivity issues with Load Balancers and with newly created or restarted Droplets.

    We apologize for the inconvenience and will share an update once we have more information.

  2. Monitoring

    Our Engineering team has mitigated the increased error rates impacting multiple services in the SFO3 region and is monitoring the situation. The impact was caused by a network routing issue that disrupted connectivity to one of our core database systems after a load balancer node became unavailable.

    As of 04:32 UTC, users should no longer experience errors or timeouts when creating or managing resources in SFO3. Operations that failed during the incident may need to be retried.

    We will post an update as soon as the issue is fully resolved.

  3. Monitoring

    Our Engineering team has mitigated the issue, and services in the SFO3 region are currently operating normally. The team is continuing to monitor the situation to ensure stability given the scope of the impact.

    We will continue monitoring and share another update once the situation is fully resolved. In the meantime, if you encounter any further errors or timeouts, please retry the affected operations. If the issue persists, please submit a ticket through your Cloud Support page so our team can assist you further: https://cloudsupport.digitalocean.com/s/createticket

Keep exploring

More from DigitalOcean

Neighboring incidents on DigitalOcean's timeline and the rest of their record on OutageDeck.