Provider
JFrogIncident detail
Azure Services - Australia East - Customer on this region might experience intermittent service degradations due to cooling issue in a section of a data centrer
Timeline window
to
Get alerted the next time JFrog breaks
Free email alerts for the handful of vendors you cannot afford to miss. No card, live in about a minute. Paid plans add Slack, Teams, Discord, and webhook delivery across your whole stack, plus higher API quotas.
Timeline
Incident updates
Every update on the official status source, oldest to newest, exactly as it appeared there.
Investigating
From Azure:
Summary of Impact: Starting at 08:30 UTC on 30 August 2023, a subset of customers with workloads hosted in the Australia East may be experiencing difficulties accessing and managing resources deployed in this region.
Current Status: Azure are actively working on a cooling event which has impacted a specific datacenter in the Australia East region. The result of this event is that resources might experience unexpected restarts. Further updates will be provided in an hour or as events warrant.
Identified
The issue has been identified and a fix is being implemented.
Identified
Update from Azure (region azprod-aue-202009)
Current Status: We do not have an exact ETA at this time, but temperature in the impacted datacenter have been stabilized. The Azure service recovery process has commenced and is expected to progressively return over a number of hours. Due to the nature of this issue our storage scale units are expected to require additional recovery efforts to ensure all resources return in a consistent state. Note that any new allocations for resources will automatically avoid the impacted scale units. If your workloads are protected by Azure Site Recovery or Azure Backup, we recommend to either initiate a failover to the recovery region or recover using Cross Region Restore.
Monitoring
Current Status: Having successfully recovered 99% of storage services and 99% of impacted Virtual Machines, we are actively investigating individual downstream services to confirm their recovery status and mitigate remaining issues. At this stage, we believe most downstream services that are still experiencing impact are the result of dependencies on one of three services with investigations ongoing.
Power has been restored to all clusters hosting Cosmos DB. Accounts that were not failed over should see recovery in Australia East. Accounts failed over to alternate regions will continue to operate in these alternate regions; the Cosmos DB team has begun onlining the Australia East region for accounts that were failed over earlier.
Over 99% of impacted Azure storage accounts are now fully recovered, a small number resources are currently being replaced or repaired due to thermal damage in order to bring the remaining impacted storage accounts back online.
Azure SQL is continuing to recover impact to a single backend cluster, impacting a small subset of databases instances. Additionally, we are taking steps to reduce the overall zonal load to facilitate improved resource allocation and recovery.
Further updates on these remaining recovery efforts will be provided in 60 minutes, or as events warrant.
Resolved
Azure Successfully recovered all standard storage tenants
Keep exploring
More from JFrog
Neighboring incidents on JFrog's timeline and the rest of their record on OutageDeck.