{
  "meta": {
    "version": "v1",
    "pricing": {
      "public": {
        "label": "Public",
        "description": "Read-only API access for lightweight status checks and public integrations."
      },
      "premium": {
        "label": "Premium",
        "description": "API keys with higher hourly quotas, plus Slack, Teams, Discord, webhook, and email outage alerts across your vendor stack."
      }
    },
    "generatedAt": "2026-10-12T09:24:04.773Z"
  },
  "data": {
    "id": "incident_statuspage_upstash_ny2tc6h0mbgy",
    "slug": "upstash-qstash-us-region-service-disruption-2026-05-08",
    "title": "QStash US Region Service Disruption",
    "summary": "QStash US Region Service Disruption",
    "status": "resolved",
    "stale": false,
    "severity": "critical",
    "startedAt": "2026-05-08T09:46:14.126+00:00",
    "updatedAt": "2026-05-12T08:25:40.262+00:00",
    "resolvedAt": "2026-05-08T10:08:41.433+00:00",
    "provider": {
      "slug": "upstash",
      "name": "Upstash"
    },
    "affectedServices": [
      {
        "slug": "upstash-qstash-vector",
        "name": "QStash & Vector"
      }
    ],
    "links": {
      "html": "/incidents/upstash-qstash-us-region-service-disruption-2026-05-08",
      "api": "/api/v1/incidents/upstash-qstash-us-region-service-disruption-2026-05-08",
      "providerHtml": "/providers/upstash",
      "alerts": "https://outagedeck.com/account?stack=upstash&utm_source=api&utm_medium=response&utm_campaign=api_alerts&utm_content=incident"
    },
    "impactSummary": "Upstash reported a critical event for the affected tracked services.",
    "source": {
      "id": "source_upstash_status",
      "kind": "official_api",
      "name": "Upstash Status",
      "checkedAt": "2026-10-12T09:20:10.069+00:00",
      "stale": false,
      "officialUrl": "https://status.upstash.com",
      "statusPageUrl": "https://status.upstash.com"
    },
    "updates": [
      {
        "id": "update_statuspage_upstash_ny2tc6h0mbgy_8kh8bvzn36f6",
        "status": "investigating",
        "body": "We are currently investigating the issue.",
        "createdAt": "2026-05-08T09:46:14.186+00:00"
      },
      {
        "id": "update_statuspage_upstash_ny2tc6h0mbgy_prhw2mmngkgg",
        "status": "investigating",
        "body": "We are continuing to investigate this issue.",
        "createdAt": "2026-05-08T09:46:47.35+00:00"
      },
      {
        "id": "update_statuspage_upstash_ny2tc6h0mbgy_b7pt5w0jlwgp",
        "status": "monitoring",
        "body": "A fix has been implemented and we are monitoring the results.",
        "createdAt": "2026-05-08T10:06:38.01+00:00"
      },
      {
        "id": "update_statuspage_upstash_ny2tc6h0mbgy_bnp2kbxt1vtw",
        "status": "resolved",
        "body": "This incident has been resolved, we will publish RCA soon.",
        "createdAt": "2026-05-08T10:08:41.433+00:00"
      },
      {
        "id": "update_statuspage_upstash_ny2tc6h0mbgy_4vmx5q01fy4v",
        "status": "resolved",
        "body": "**Root Cause Analysis**\n\n  \nOn **April 24**, we deployed a more optimized scheduler implementation in the **US East \\(N. Virginia\\)** region.  \nOn **May 8**, a user who had active schedules deleted their account.  \nUnder normal behavior, scheduled tasks associated with a deleted account should wake up, detect that the account no longer exists, and exit after performing cleanup. Due to a bug introduced in the new scheduler implementation, this code path did not return early as intended. Execution continued and resulted in a nil pointer dereference.  \nA second issue then amplified the impact. When a panic occurs in the scheduler, it is designed to be recovered, logged, and isolated so that the process remains healthy. Because of another bug in the panic recovery path, the panic was not properly caught, which caused the worker process handling the scheduled job to terminate.  \nAfter that process exited, another worker picked up responsibility for delivering the same scheduled task. Since the same faulty execution path was still present, that worker also failed. This created a cascading failure pattern across workers attempting to process the affected schedules.  \n**Resolution**  \nWe deployed two fixes:\n\n* Added the missing early return in the deleted-account cleanup path, preventing the nil pointer dereference.\n* Corrected the panic recovery logic so that future panics are safely recovered, logged, and reported without causing worker processes to terminate.\n\nWith these changes in place, the affected execution path is now safe. Even if a future bug triggers a panic in this area, it will be isolated and reported rather than causing process-level failure.",
        "createdAt": "2026-05-12T08:25:06.043+00:00"
      }
    ],
    "access": {
      "plan": "public",
      "keyed": false
    }
  }
}