Skip to content

Incident detail

Harness Artifact Registry upload is failing from pipeline - EU1 region

Resolved incidentMinor

Timeline window

to

Get alerted the next time Harness breaks

Free email alerts for the handful of vendors you cannot afford to miss. No card, live in about a minute. Paid plans add Slack, Teams, Discord, and webhook delivery across your whole stack, plus higher API quotas.

Timeline

Incident updates

Every update on the official status source, oldest to newest, exactly as it appeared there.

  1. Investigating

    We are currently investigating this issue.

  2. Identified

    The issue has been identified and a fix is being implemented.

  3. Monitoring

    A fix has been implemented and we are monitoring the results.

  4. Resolved

    This incident has been resolved.

  5. Resolved

    # Summary

    On July 31, 2026, artifact uploads performed through pipeline in the EU1 cluster began failing with an authentication error. Uploads initiated manually (outside of a pipeline) were not affected, and the ability to retrieve existing artifacts (downloads) was also unaffected — this was isolated to the specific pipeline upload path in one cluster.

    # Impact

    • Artifact uploads performed through pipeline in the EU1 cluster failed with an authentication error for approximately 4 hours and 34 minutes.
    • Retrieving existing artifacts (downloads) was not affected.
    • Manually uploading artifacts outside of a pipeline was not affected.
    • Other clusters/regions were not affected by this issue.

    # Root Cause

    The component responsible for handling pipeline-based artifact uploads is distributed as a container image. In the EU1 cluster, this image is retrieved from an internal registry that mirrors a public image source; in other clusters, the same image is retrieved directly from the public source.

    A publishing error in our release process caused a new build of this component to be published using a version label that was already in use, rather than being assigned a new, unique version. As a result, two different images ended up associated with the same version label in the public source.

    Our internal registry mirrors images from the public source via an automated replication process. Because of how that replication was triggered, it copied the original (earlier) image associated with that version label rather than the corrected one. This meant the EU1 cluster — which pulls from the internal mirror — ended up running a different, defective image than other clusters, which pull directly from the public source and therefore received the corrected image. The defective image contained an authentication issue that caused pipeline uploads to fail.

    # Mitigation

    • Reverted the affected account to the last known-good version of the upload component, immediately restoring pipeline uploads.
    • Published a corrected, permanent version of the component to resolve the issue across all clusters.

    # Next steps

    • Fix the upload step to remove the underlying container-related defect that made this failure mode possible.
    • Update our release pipeline for this component so that publishing an image can never overwrite an existing version — every publish must create a new, distinct version going forward.

Keep exploring

More from Harness

Neighboring incidents on Harness's timeline and the rest of their record on OutageDeck.