Skip to content
OutageDeck

Incident detail

Test Engine services unavailable

Resolved incidentMajor2 affected services

Timeline window

to

Outage alerts

Get alerted the next time Buildkite breaks

Free email alerts for up to 5 providers — no card, live in about a minute. Paid plans add Slack, Discord, and webhook delivery across your whole stack, plus higher API quotas.

Set up free alertsHow Buildkite alerts work

Timeline

Incident updates

Updates are normalized from the official source chronology so timeline changes remain easy to scan.

  1. Investigating

    A third party database provider is experiencing an outage which is resulting in Test Engine services being unavailable. We will report updates as we receive them.

  2. Investigating

    We have been notified by the third party provider that they are currently restoring services. We will update when we have more information.

  3. Monitoring

    Our third party database provider has recovered their systems and we are have worked through our backlog of test executions. We are continuing to monitor to ensure our systems are healthy.

  4. Resolved

    We believe this issue is now resolved and all Test Engine systems are functioning. Ingestion of test data was delayed during this incident but we believe there has been no data loss.

    Customers using Buildkite Test Engine Client (bktec) to run tests will have experienced slower builds during the incident due to bktec using a sub-optimal test planning strategy while Test Engine APIs were unavailable.