Between 8:15 AM EDT and 11:00 AM EDT the platform experienced significantly elevated response time in both the EU and US. The root cause of this outage was determined to be a routine security upgrade of an external dependency, leading to high CPU on our application servers. Despite auto-scaling due to increased load, we did not see a satisfactory reduction in response time as expected. Reverting the dependency upgrade led to an immediate return to expected response times.
Posted Oct 24, 2024 - 11:57 EDT
Resolved
This incident has been resolved.
Posted Oct 24, 2024 - 10:56 EDT
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Oct 24, 2024 - 10:54 EDT
Update
We are continuing to work on a fix for this issue.
Posted Oct 24, 2024 - 10:44 EDT
Update
We are continuing to work on a fix for this issue.
Posted Oct 24, 2024 - 10:29 EDT
Update
We are continuing to work on a fix for this issue.
Posted Oct 24, 2024 - 09:50 EDT
Identified
The issue has been identified and a fix is being implemented.
Posted Oct 24, 2024 - 09:27 EDT
Investigating
We are investigating elevated load times in the US.
Posted Oct 24, 2024 - 09:16 EDT
This incident affected: US - Platform and EU - Platform.