Provider
Grafana CloudIncident detail
Metrics Alert Rule Evaluation Failures in AWS Australia (prod-au-southeast-1)
Resolved incidentMinor
Timeline window
to
Get alerted the next time Grafana Cloud breaks
Free email alerts for the handful of vendors you cannot afford to miss. No card, live in about a minute. Paid plans add Slack, Teams, Discord, and webhook delivery across your whole stack, plus higher API quotas.
Timeline
Incident updates
Every update on the official status source, oldest to newest, exactly as it appeared there.
Investigating
We are investigating an elevated rate of failures when evaluating Metrics alert and recording rules in AWS Australia (prod-au-southeast-1). Some customers may see rules missed or delayed rule evaluations, which can result in missed or delayed alert notifications and gaps in recorded series. Next update by 10:20 UTC.
Investigating
We continue to investigate intermittent failures when evaluating alert and recording rules in AWS Australia (prod-au-southeast-1), first observed at approximately 08:00 UTC. Some customers may see rules fail to evaluate or be delayed, which can result in missed or delayed alert notifications and gaps in recorded series. We are continuing to investigate. Next update by 11:15 UTC.
Resolved
Between approximately 08:00 and 09:40 UTC (about 1 hour 40 minutes), some customers in AWS Australia (prod-au-southeast-1) experienced intermittent failures when evaluating alert and recording rules, which may have caused missed or delayed alert notifications and gaps in recorded series. This has been fully resolved and rule evaluation is operating normally.
Keep exploring
More from Grafana Cloud
Neighboring incidents on Grafana Cloud's timeline and the rest of their record on OutageDeck.