Skip to content
OutageDeck

Incident detail

KPO podspec changes for GPU-enabled nodes may cause KPO pod startup failures

Resolved incidentMinor

Timeline window

to

Outage alerts

Get alerted the next time Astronomer breaks

Free email alerts for up to 5 providers. No card, live in about a minute. Paid plans add Slack, Teams, Discord, and webhook delivery across your whole stack, plus higher API quotas.

Set up free alertsHow Astronomer alerts work

Timeline

Incident updates

Every update Astronomer posted, oldest to newest, exactly as it appeared on their official status page.

  1. Identified

    We've identified a change that enabled GPU support for KubernetesPodOperator pods had an unintended side effect when reading pod specs without proper limits set in the expected format.

    We're working on a fix, if you notice any unexpected failures for KPO tasks in your Astro deployments, please open a support ticket.

  2. Monitoring

    We have rolled back the changes that triggered this issue and are monitoring the issue for confirmation that this issue is resolved.

  3. Resolved

    Mitigations have been successful. A Platform release that corrects this errant behavior is being worked on now. Issue is resolved.

Keep exploring

More from Astronomer

Neighboring incidents on Astronomer's timeline and the rest of their record on OutageDeck.

← Older incident

Intermittent Delays Managing Airflow Deployments in Azure Central US

All Astronomer incidentsAstronomer live status