Skip to content

Incident detail

Elevated errors in AuthKit, Admin Portal, Directory Sync, and SSO

Resolved incidentCritical3 affected services

Timeline window

to

Get alerted the next time WorkOS breaks

Free email alerts for the handful of vendors you cannot afford to miss. No card, live in about a minute. Paid plans add Slack, Teams, Discord, and webhook delivery across your whole stack, plus higher API quotas.

Timeline

Incident updates

Every update on the official status source, oldest to newest, exactly as it appeared there.

  1. Investigating

    We are investigating an increase in errors with AuthKit and Admin Portal.

  2. Investigating

    We are continuing to investigate errors across AuthKit, Admin Portal, Directory Sync, and SSO.

  3. Investigating

    We are continuing to investigate this issue.

  4. Monitoring

    We've applied a fix that has mitigated upstream outages. We'll continue to monitor.

  5. Monitoring

    All services have returned to normal operation. We are continuing to monitor.

  6. Resolved

    This incident has been resolved.

  7. Resolved

    ## Summary

    On Thursday, June 12 from 18:02 UTC to 19:56 UTC AuthKit, Dashboard, Admin Portal, SSO, and their APIs were unavailable for some customers. The outage lasted 1 hour and 56 minutes.

    Customers can opt to serve WorkOS products from custom domains to maintain a consistent user experience. The custom domain routing infrastructure relies, in part, upon Cloudflare KV. This service was at the center of a widespread outage at Cloudflare.

    We identified the issue and deployed a mitigation as the upstream outage continued. We recognize that an outage this severe is unacceptable and we strive for full resilience in the face of inevitable outages in dependent services.

    ## Root Cause Analysis

    WorkOS offers the use of custom domains for a number of its products: AuthKit, SSO, Admin Portal, and its APIs. Cloudflare Workers are used to perform routing logic for these custom domains. For a subset of these custom domains, we persist a lookup table from custom domain to customer in Cloudflare KV.

    Due to the June 12 Cloudflare outage, these lookups failed. Because the Cloudflare dashboard was also impacted, we did not have immediate visibility into the exact cause of failures in Cloudflare’s platform. It wasn’t until Cloudflare posted details on affected services that we were able to identify the failing dependency and develop mitigations.

    ## Remediation

    Cloudflare Workers and Cloudflare KV are in the critical path for our services. We are developing redundancies for outages of these dependencies.

Keep exploring

More from WorkOS

Neighboring incidents on WorkOS's timeline and the rest of their record on OutageDeck.