Skip to content
OutageDeck

Incident detail

Deployment Degradation - Slowness in Prod3

Resolved incidentCritical3 affected services

Timeline window

to

Outage alerts

Get alerted the next time Harness breaks

Free email alerts for up to 5 providers — no card, live in about a minute. Paid plans add Slack, Discord, and webhook delivery across your whole stack, plus higher API quotas.

Set up free alertsHow Harness alerts work

Timeline

Incident updates

Updates are normalized from the official source chronology so timeline changes remain easy to scan.

  1. Investigating

    We are currently investigating this issue.

  2. Investigating

    We are continuing to investigate this issue.

  3. Identified

    We have identified the root cause as elevated queue depth and throughput saturation on the MongoDB layer, leading to processing latency. A fix is being implemented to optimize handling and restore database stability

  4. Identified

    Executions are running fine now. However, there is still some slowness in the execution graph generation so we might see some delay in the execution updates to show up in UI.

    But executions would work without any delay/slowness.

  5. Monitoring

    A fix has been implemented and we are monitoring the results.

  6. Monitoring

    A fix has been implemented and we are monitoring the system.

  7. Monitoring

    We are continuing to monitor for any further issues.

  8. Resolved

    This incident has been resolved.

  9. Resolved

    Dupe of Deployment Degradation – Failures in Prod1,2,3