Rustici SCORM Outage (US)
Incident Report for Thought Industries
Postmortem

Between 5:45 AM PDT and 7:45 AM PDT the SCORM Rustici service experienced a minor increase in error rates, followed by a more severe outage between 7:45 AM PDT and 8:45 AM PDT, after which point service was restored. The root cause of this outage was determined to be a misconfiguration in the internal load balancer, which resulted in general Rustici traffic routing to a single node and degraded performance when traffic exceeded a critical threshold. The infrastructure team has applied a fix as of the resolution of this outage and confirmed that traffic is correctly routing to all available nodes.

Posted Oct 21, 2024 - 17:07 EDT

Resolved
This incident has been resolved.
Posted Oct 21, 2024 - 12:12 EDT
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Oct 21, 2024 - 11:50 EDT
Update
We are continuing to investigate this issue.
Posted Oct 21, 2024 - 11:17 EDT
Investigating
We are investigating high loading times on Rustici SCORM launches in the US.
Posted Oct 21, 2024 - 10:26 EDT
This incident affected: US - Platform.