Provider
Neo4j AuraIncident detail
Cypher change in 5.25 release impacts cypher CASE statements within a WITH clause
Timeline window
to
Get alerted the next time Neo4j Aura breaks
Free email alerts for the handful of vendors you cannot afford to miss. No card, live in about a minute. Paid plans add Slack, Teams, Discord, and webhook delivery across your whole stack, plus higher API quotas.
Timeline
Incident updates
Every update on the official status source, oldest to newest, exactly as it appeared there.
Investigating
We have received some reports of queries failing after the roll-out of version 5.25. The affected queries have a CASE within a WITH clause. We are investigating.
Customers who had marked their instances as production are not affected, as the roll-out is delayed for those instances.
Investigating
We are continuing to investigate this issue.
Identified
We have identified the issue and are working on a fix. In the meantime, work around this issue by replacing Simple Case statements (https://neo4j.com/docs/cypher-manual/current/queries/case/#case-simple) with Generic Case (https://neo4j.com/docs/cypher-manual/current/queries/case/#case-generic) statements.
We will continue to update this incident as we progress with the fix.
Identified
We have identified a fix and are in the process of building and releasing it. Please utilize the workaround if you are impacted until the fix is released.
Identified
We have identified a fix and are in the process of building and releasing it.
In the meantime, work around this issue by replacing Simple Case statements (https://neo4j.com/docs/cypher-manual/current/queries/case/#case-simple) with Generic Case (https://neo4j.com/docs/cypher-manual/current/queries/case/#case-generic) statements.
Identified
We have identified a fix and are in the process of building and releasing it.
In the meantime, work around this issue by replacing Simple Case statements (https://neo4j.com/docs/cypher-manual/current/queries/case/#case-simple) with Generic Case (https://neo4j.com/docs/cypher-manual/current/queries/case/#case-generic) statements.
Identified
We have identified a fix and are in the process of building and releasing it. Please utilize the workaround if you are impacted until the fix is released.
Identified
We will be proceeding to the roll-out of the fix as soon as checks are completed. ETA 3 hours for this phase to be initiated
Identified
We have started the roll-out of the fix to the affected customers.
Resolved
We have verified the fix and confirmed it has resolved this issue as reported by customers.
Resolved
### What happened
We rolled out a new release (version 5.25) of the Neo4j database on 2024-10-25 starting with AuraDB Free tier. On 2024-10-28 at 10:30 UTC we progressed the roll-out to AuraDB Professional, and then next to AuraDB Business Critical and AuraDB Virtual Dedicated Cloud tiers. As we maintain a 48 hours delay before we roll-out to all instances marked as production, those instances marked as production were not affected and were still running the previous version (5.24.2) of the database.
The new release version 5.25 contained a change aimed at handling the newer GQL syntax for the CASE expression to support the “simple CASE” mode. The implementation affected the optimization phase of such queries. The recently modified handling of the WITH clause also leveraged an optimization, and the two clashed when both involved in the same query, causing the Cypher statements to fail.
The existing test suite for Cypher covered the individual keywords but not the combination of these two exact cypher keywords in this particular sequence that would trigger both optimization phases.
### How the service was affected
Customers running Cypher statements that comprise a CASE (simple) inside a WITH clause saw their queries fail and produce the error “`Neo.ClientError.Statement.SyntaxError”.`
Customers reported issues on some specific queries without any changes on their side.
We took the decision to halt further progress of the roll-out of the release until we had a confirmed fix for this issue.
As we identified both the pattern triggering the error and its root cause, we communicated a workaround of adjusting the queries that use a “simple CASE” inside a WITH clause and modify them to use a “generic CASE”.
A fix was prepared, but to prevent other issues, we ensured it completed the full Cypher test suite before being rolled out (2024-10-29 at 13:35 UTC) to all our customers.
### What we are doing now
We don’t believe that the issues our customers experienced are acceptable for a DBaaS. We have carried out a thorough analysis of what went wrong in this situation. The actions that we are carrying out to ensure that nothing like this can happen again fall into multiple areas:
- Test: We are enhancing the capability and coverage of our Cypher test suite. The syntax from any Cypher queries identified as bugs will automatically be included in the test suite.
- Detection: We are building a set of metrics and alarms to detect sudden spikes of errors on Cypher queries run in Aura whilst we roll any new release and get alerted of possible issues linked to a new release.
- Release roll-out: We are improving the agility and speed of rolling-out the releases to better handle problematic releases.
- Bug fixing: We are building a fast-track release pipeline to enable swifter delivery of hotfixes in safe conditions.
Communication: We will be reminding customers to have development instances and designate their production instance to better isolate them from possible issues during roll-out.
Keep exploring
More from Neo4j Aura
Neighboring incidents on Neo4j Aura's timeline and the rest of their record on OutageDeck.