Skip to content
OutageDeck

Incident detail

Elevated Queue Times on Linux Machines

Resolved incidentMinor1 affected service

Timeline window

to

Outage alerts

Get alerted the next time Bitrise breaks

Free email alerts for up to 5 providers — no card, live in about a minute. Paid plans add Slack, Discord, and webhook delivery across your whole stack, plus higher API quotas.

Set up free alertsHow Bitrise alerts work

Timeline

Incident updates

Updates are normalized from the official source chronology so timeline changes remain easy to scan.

  1. Investigating

    <b>We are currently investigating elevated build queues on Bitrise. Some users may experience longer-than-usual wait times for builds to start, as queued jobs take longer to be picked up.</b>

    <b>Services:</b>

    Build start times and queue wait times; build execution itself remains normal once a worker is allocated.

    <b>Scope:</b>

    Queues on standard, elite, and elite-xl machines

    <b>Severity:</b>

    [SEV-2]

    <b>What we’re doing?</b>

    Our Engineering teams are actively investigating causes, reviewing worker pool utilization, autoscaling events, recent deploys, and any upstream provider issues. We are analyzing metrics and logs to identify bottlenecks like unusually high build volume, throttling from third-party providers, etc. We are working on immediate actions to remediate this issue. We will perform validated mitigations as soon as they are determined safe.

    <b>When will the next update be available?</b>

    You can expect a status update approximately <b>1 hour</b> from now.

  2. Identified

    <b>We identified the root cause of elevated build queues on Bitrise. Some users may experience longer-than-usual wait times for builds to start, as queued jobs take longer to be picked up.</b>

    <b>Services:</b>

    Build start times and queue wait times; build execution itself remains normal once a worker is allocated.

    <b>Scope:</b>

    Queues on standard, elite, and elite-xl machines

    <b>Severity:</b>

    SEV-2

    <b>What we’re doing?</b>

    Our Engineering teams identified the root cause and are now actively working on mitigations to restore stability across all affected services. We will perform validated mitigations as soon as they are determined safe.

    <b>When will the next update be available?</b>

    You can expect a status update approximately <b>1 hour</b> from now.

  3. Resolved

    <b>This incident is now resolved, and impacted services are now operating without issues.</b>

    Our teams identified the root cause, released a fix, and were monitoring our systems continuously to ensure the mitigation is safe. We'll conduct an internal post-mortem to understand what happened and learn how to avoid a similar outage in the future.

    Thank you for your patience and understanding. If you encounter any issues, please reach out to our Support team:

    https://support.bitrise.io/en/