Skip to content
OutageDeck

Incident detail

Prediction Errors

Prediction Errors

Resolved incidentMajor0 affected services

Timeline window

to

Timeline

Incident updates

Updates are normalized from the official source chronology so timeline changes remain easy to scan.

  1. Investigating

    We are aware of elevated errors for predictions being sent to one of our regions. This is impacting an subset of models targeting H100 and L40S Hardware types. We are investigating and will provide an update as soon as information is available.

  2. Investigating

    We have identified and remediated a data store related to queues that was wedged, preventing traffic from reaching it within one of our regions. Most inference and API usage as returned to normal with the exception of the Flux Schnell model. We are working on restoring service to the flux schnell model. The impact is large delays on predictions and elevated errors when submitting preditions.

  3. Investigating

    We have shifted traffic for the Flux Schnell model to a region with additional capacity. While we work through the backlog of requests, new requests should begin to be served within a normal timeframe. This continues to impact only the Flux Schnell official model.

  4. Investigating

    All models have returned to normal processing timeframes. Thank you for your patience.

  5. Resolved

    Incident is resolved.