Provider
WorkOSIncident detail
Elevated errors in AuthKit, Admin Portal, Directory Sync, and SSO
Timeline window
to
Get alerted the next time WorkOS breaks
Free email alerts for the handful of vendors you cannot afford to miss. No card, live in about a minute. Paid plans add Slack, Teams, Discord, and webhook delivery across your whole stack, plus higher API quotas.
Timeline
Incident updates
Every update on the official status source, oldest to newest, exactly as it appeared there.
Investigating
We are investigating an increase in errors with AuthKit and Admin Portal.
Investigating
We are continuing to investigate errors across AuthKit, Admin Portal, Directory Sync, and SSO.
Investigating
We are continuing to investigate this issue.
Monitoring
We've applied a fix that has mitigated upstream outages. We'll continue to monitor.
Monitoring
All services have returned to normal operation. We are continuing to monitor.
Resolved
This incident has been resolved.
Resolved
## Summary
On Thursday, June 12 from 18:02 UTC to 19:56 UTC AuthKit, Dashboard, Admin Portal, SSO, and their APIs were unavailable for some customers. The outage lasted 1 hour and 56 minutes.
Customers can opt to serve WorkOS products from custom domains to maintain a consistent user experience. The custom domain routing infrastructure relies, in part, upon Cloudflare KV. This service was at the center of a widespread outage at Cloudflare.
We identified the issue and deployed a mitigation as the upstream outage continued. We recognize that an outage this severe is unacceptable and we strive for full resilience in the face of inevitable outages in dependent services.
## Root Cause Analysis
WorkOS offers the use of custom domains for a number of its products: AuthKit, SSO, Admin Portal, and its APIs. Cloudflare Workers are used to perform routing logic for these custom domains. For a subset of these custom domains, we persist a lookup table from custom domain to customer in Cloudflare KV.
Due to the June 12 Cloudflare outage, these lookups failed. Because the Cloudflare dashboard was also impacted, we did not have immediate visibility into the exact cause of failures in Cloudflare’s platform. It wasn’t until Cloudflare posted details on affected services that we were able to identify the failing dependency and develop mitigations.
## Remediation
Cloudflare Workers and Cloudflare KV are in the critical path for our services. We are developing redundancies for outages of these dependencies.
Keep exploring
More from WorkOS
Neighboring incidents on WorkOS's timeline and the rest of their record on OutageDeck.