{
  "meta": {
    "version": "v1",
    "pricing": {
      "public": {
        "label": "Public",
        "description": "Read-only API access for lightweight status checks and public integrations."
      },
      "premium": {
        "label": "Premium",
        "description": "API keys with higher hourly quotas, plus Slack, Discord, webhook, and email outage alerts across your vendor stack."
      }
    },
    "generatedAt": "2026-07-24T11:08:39.717Z"
  },
  "data": {
    "id": "incident_statuspage_harness_2rtzv97j78zg",
    "slug": "harness-custom-dashboards-are-failing-intermittently-in-prod3-2026-05-15",
    "title": "Custom Dashboards are failing intermittently in prod3",
    "summary": "Custom Dashboards are failing intermittently in prod3",
    "status": "resolved",
    "severity": "minor",
    "startedAt": "2026-05-15T20:35:15.095+00:00",
    "updatedAt": "2026-05-28T00:41:26.268+00:00",
    "resolvedAt": "2026-05-17T05:37:09.097+00:00",
    "provider": {
      "slug": "harness",
      "name": "Harness"
    },
    "affectedServices": [
      {
        "slug": "harness-platform",
        "name": "Platform & dashboards"
      }
    ],
    "links": {
      "html": "/incidents/harness-custom-dashboards-are-failing-intermittently-in-prod3-2026-05-15",
      "api": "/api/v1/incidents/harness-custom-dashboards-are-failing-intermittently-in-prod3-2026-05-15",
      "providerHtml": "/providers/harness"
    },
    "impactSummary": "Harness reported a minor event for the affected tracked services.",
    "source": {
      "id": "source_harness_status",
      "kind": "official_api",
      "name": "Harness Status",
      "checkedAt": "2026-07-24T11:00:50.68+00:00",
      "officialUrl": "https://status.harness.io",
      "statusPageUrl": "https://status.harness.io"
    },
    "updates": [
      {
        "id": "update_statuspage_harness_2rtzv97j78zg_lths7lvzpx71",
        "status": "investigating",
        "body": "We are currently investigating this issue.",
        "createdAt": "2026-05-15T20:35:15.221+00:00"
      },
      {
        "id": "update_statuspage_harness_2rtzv97j78zg_v0n182l1xc1z",
        "status": "monitoring",
        "body": "A fix has been implemented and we are monitoring the results.",
        "createdAt": "2026-05-15T20:59:21.102+00:00"
      },
      {
        "id": "update_statuspage_harness_2rtzv97j78zg_lq4dd110m5mv",
        "status": "monitoring",
        "body": "We are continuing to monitor for any further issues.",
        "createdAt": "2026-05-17T05:36:58.149+00:00"
      },
      {
        "id": "update_statuspage_harness_2rtzv97j78zg_p8b0vw9pwr6x",
        "status": "resolved",
        "body": "This incident has been resolved.",
        "createdAt": "2026-05-17T05:37:09.097+00:00"
      },
      {
        "id": "update_statuspage_harness_2rtzv97j78zg_0d4pxy1kppv3",
        "status": "resolved",
        "body": "# Summary\n\n Dashboards service in Prod3 experienced intermittent failures, causing dashboards to return errors and become unavailable to some customers. \n\n‌\n\n## Root Cause Analysis\n\nThe incident was caused by slow Looker `search_dashboards` API calls degrading from 7 seconds to over 30 seconds, which saturated the worker thread pool and prevented the health endpoint from responding to Kubernetes liveness and readiness probes.\n\n‌\n\n## Mitigation Steps Taken\n\n### Immediate Mitigations \n\n1. **Increased liveness probe timeout** from 15s to 30s \\(failure threshold kept at 3, allowing up to 90s tolerance\\)\n2. **Doubled  thread count** , increasing total concurrency\n3. **Increased pod replica count** to maintain higher minimum availability and reduce risk of all pods becoming simultaneously unavailable\n\n‌\n\n## Actions\n\nHarness will work on the following Action items to prevent recurrence.\n\n1. **Optimize Looker SDK timeout:** Tune timeout to Looker SDK `search_dashboards` calls to prevent indefinite thread occupation\n2. **Optimize Looker queries :** Investigate Looker-side query performance degradation to understand why `search_dashboards` latency increased",
        "createdAt": "2026-05-28T00:18:50.489+00:00"
      }
    ],
    "access": {
      "plan": "public",
      "keyed": false
    }
  }
}