Skip to content
OutageDeck

Incident detail

Delays when checking out GitHub repos via SSH

Resolved incidentMinor1 affected service

Timeline window

to

Outage alerts

Get alerted the next time Semaphore breaks

Free email alerts for up to 5 providers — no card, live in about a minute. Paid plans add Slack, Discord, and webhook delivery across your whole stack, plus higher API quotas.

Set up free alertsHow Semaphore alerts work

Timeline

Incident updates

Updates are normalized from the official source chronology so timeline changes remain easy to scan.

  1. Identified

    We identified slow communication between Semaphore jobs and GitHub. We are continuing to work with our upstream provider to resolve this. For impacted jobs, please replace the normal checkout command with the following:

    retry --times 5 "timeout 40 bash -lc 'source ~/.toolbox/toolbox; checkout'"

    And afterwards, use cd to change the directory to the cloned repository (replace with your project).

  2. Monitoring

    A fix has been implemented and we are monitoring the results.

  3. Identified

    We identified slow GitHub operations related to problems with the network, and we deployed a fix to re-route those operations to a healthier server. Some HTTPS operations started to fail because of that. To disable the fix, you should use the command:

    sudo sed -i 's/^([0-9].*github.com)/#\1/' /etc/hosts

    And to re-enable, you can use:

    sudo sed -i 's/^#([0-9].*github.com)/\1/' /etc/hosts

  4. Monitoring

    A fix has been implemented, and everything should be back to normal. The previously shared snippets can be removed.

  5. Resolved

    This incident has been resolved.