Provider
BuildkiteIncident detail
Buildkite service disruption
Resolved incidentMinor1 affected service
Timeline window
to
Outage alerts
Get alerted the next time Buildkite breaks
Free email alerts for up to 5 providers. No card, live in about a minute. Paid plans add Slack, Teams, Discord, and webhook delivery across your whole stack, plus higher API quotas.
Timeline
Incident updates
Every update Buildkite posted, oldest to newest, exactly as it appeared on their official status page.
Investigating
We've spotted that something has gone wrong. We're currently investigating the issue, and will provide an update soon.
Identified
We're seeing increased latency and error rates for a subset of our customers. We're currently addressing the cause and will provide status updates as they become available.
Identified
For a subset of customers, <1% of their Agent API traffic is experiencing issues. We are rolling back the change that caused this issue while we investigate it further.
Monitoring
We have reverted the change which introduced the elevated error rate. We believe the issue to be resolved but will continue to monitor.
Resolved
For the last week we have been internally testing our agent traffic running on our new EKS infrastructure successfully.
At 23:01 11 August 2026 UTC we started routing <1% of agent traffic for a small subset of our customer to this EKS infrastructure. Within that subset of customers, any that were using an agent version < 3.97.0 saw an error "failed to decode JSON response: invalid character '<' looking for beginning of value". These older agents crashed once they received that error, which for some customers triggered repeated agent cycling.
We reverted the routing to EKS infrastructure at 03:25 UTC after being alerted to the issue.
Keep exploring
More from Buildkite
Neighboring incidents on Buildkite's timeline and the rest of their record on OutageDeck.