Provider
GitLabIncident detail
CI pipelines not starting on merge requests.
Incident has now been mitigated and runner backlogs are now back to normal. CI pipelines are now resuming. Engineers will continue to monitor this recovery.
Resolved incidentMinor
Timeline window
to
Outage alerts
Get alerted the next time GitLab breaks
Free email alerts for up to 5 providers — no card, live in about a minute. Paid plans add Slack, Discord, and webhook delivery across your whole stack, plus higher API quotas.
Timeline
Incident updates
Updates are normalized from the official source chronology so timeline changes remain easy to scan.
Investigating
We are currently investigating an issue affecting CI pipelines not starting on merge requests. Our team is actively investigating the root cause.
Identified
We have identified that the sidekiq workers processing the jobs are getting deferred. The team is working on identifying the remediation steps.
Identified
No material updates to report. We are continuing to investigate the current issue and engineers are working to determine remediation steps.
Identified
Our engineers are still investigating the issue but jobs are starting to be processed with a delay. More details can be found here: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/22412
Monitoring
Queues and runner backlog are trending back towards normal, while systems continue to work through the remaining backlog. Engineers are continuing to monitor this.
Resolved
Incident has now been mitigated and runner backlogs are now back to normal. CI pipelines are now resuming. Engineers will continue to monitor this recovery.