Skip to content

Incident detail

[After the Fact] Certificate Updates without SNIs lost SNI mappings for a period of time on November 6th

Resolved incidentMajor

Timeline window

to

Get alerted the next time Kong breaks

Free email alerts for the handful of vendors you cannot afford to miss. No card, live in about a minute. Paid plans add Slack, Teams, Discord, and webhook delivery across your whole stack, plus higher API quotas.

Timeline

Incident updates

Every update on the official status source, oldest to newest, exactly as it appeared there.

  1. Resolved

    This is an after-the-fact incident being reported as it was determined that a breaking API change was in production for some time on November 6th, 2025.

    On November 6 2025, for a period from 15:40 UTC to 19:20 UTC a breaking change to the Konnect API's certificate management endpoints was deployed, causing SNI’s to be disassociated with Certificates if Certificate updates did not include the full SNI list in each update.

    For users of the Kong Operator, which manages SNIs and Certificates separately and reconciles frequently, this resulted in SNI’s being deleted almost as soon as the API change was deployed to Konnect.

    Kong has reached out directly to impacted customers.

  2. Resolved

    # Summary

    On _November 6 2025_, a breaking change to the Konnect API's certificate management endpoints caused SNI’s to be disassociated with Certificates if Certificate updates did not include the full SNI list in each update.

    ‌

    For users of the Kong Operator, which manages SNIs and Certificates separately and reconciles frequently, this resulted in SNI’s being deleted almost as soon as the API change was deployed to Konnect.

    ## Additional Details

    The incident originated from an effort to align the Konnect API's behavior with the Kong Gateway API. While Kong Gateway allows Server Name Indicators (SNIs) to be managed directly through the Certificate endpoint, Konnect API historically managed certificates and SNIs as separate entities. To achieve consistency, the Konnect team updated the Create and Upsert endpoints to match Kong's behavior.

    ‌

    However, this update introduced an unintended breaking change: when no SNIs were provided in an upsert request, the new logic purged all existing SNIs associated with the certificate instead of leaving them unchanged as in the previous implementation.

    ‌

    This breaking change critically affected the Kong Operator, which manages certificates and SNIs through separate API calls using PUT requests. When the Kong Operator updated certificates without including SNI data, the new Koko logic deleted all associated SNIs. As a result, customer dataplanes fall back to using generic certificates, breaking customer traffic.

    ‌

    ## Kong’s Response

    The issue was discovered when the CRE team detected service degradation for a customer and alerted the Konnect engineering team. The incident was resolved by reverting the API to its previous version across all regions.

    ‌

    The Kong API does allow for PUT requests to not include every field for a resource. However, in some cases the API must respond with an HTTP400/Bad Request when the lack of some field could result in a destructive action.

    ‌

    Therefore, the Konnect engineering team is adding a mechanism to our API design and testing automation that will block any API that allows critical resource information to be excluded from a request. In this case, the API change would have been blocked, as SNIs would be marked critical to a Certificate request.

Keep exploring

More from Kong

Neighboring incidents on Kong's timeline and the rest of their record on OutageDeck.