Skip to content

Incident detail

Temporary Data Ingestion Delay for US Projects

Resolved incidentMinor

Timeline window

to

Get alerted the next time Mixpanel breaks

Free email alerts for the handful of vendors you cannot afford to miss. No card, live in about a minute. Paid plans add Slack, Teams, Discord, and webhook delivery across your whole stack, plus higher API quotas.

Timeline

Incident updates

Every update on the official status source, oldest to newest, exactly as it appeared there.

  1. Investigating

    We are currently experiencing delays in our data ingestion pipeline, which is affecting real-time data for projects enrolled in the US projects, resulting in delays for a subset of projects. While no data is being lost, our engineering team is actively investigating the matter and working to restore real-time functionality as quickly as possible. We appreciate your patience during this time.

    If you have any questions, please <a href="https://mixpanel.com/get-support">contact support.</a>

  2. Identified

    The issue has been identified and a fix is being implemented.

  3. Identified

    We've identified the cause of the ingestion delay as an upstream issue with our cloud storage provider, Google Cloud, impacting a subset of US projects. Google has confirmed this as a high-priority issue on their end but hasn't provided an ETA yet. We've deployed mitigations to ensure a smooth recovery once it clears.

    No events are being lost. This is strictly an ingestion delay, all affected data will continue to be reprocessed once the issue is resolved.

  4. Monitoring

    Our metrics show ingestion is caught up and no longer experiencing delay. Google has not yet confirmed resolution of the underlying upstream issue on their end, so we’re continuing to monitor closely in case of recurrence.

  5. Monitoring

    We are continuing to monitor for any further issues.

  6. Monitoring

    No change since our last update, we continue to observe no active ingestion delay on our end, and all systems remain healthy. We’re continuing to monitor closely.

  7. Monitoring

    We are continuing to monitor for any further issues.

  8. Monitoring

    Systems have remained stable with no recurrence of the ingestion delay. We’re keeping a close eye on things while we wait for final confirmation from our upstream provider.

  9. Resolved

    This incident has been resolved. The upstream issue with our cloud storage provider has been fixed, and our ingestion pipeline has been operating normally with no active delays. No events were lost throughout this incident, all affected data was fully reprocessed.

    Thank you for your patience throughout this incident.

Keep exploring

More from Mixpanel

Neighboring incidents on Mixpanel's timeline and the rest of their record on OutageDeck.