Intermittent Issue with Certificates Loading
Incident Report for Thought Industries
Resolved
Summary

There was an intermittent issue with certificates loading for a small subset of customers using Panorama V1 and/or using certificate functionality where ‘always regenerate certificates’ was not enabled.

Timeline

Effected customers started noticing intermittent issues with some learner certificates not loading from Friday 16th October. This only effected certificates which were connected to courses completed after October 15th.

Root Cause

The root cause of this intermittent issue was determined to be a regression to certificate functionality not using 'always regenerate certificates' enabled after the release of a small CSS block update which allowed further styling customization to Certificate Templates.

Action Taken / Response

On Monday morning we starting working a reported issue around some certificates not loading in a customers Panorama V1 functionality. As we began to investigate a small subset of our customers started to also report intermittent issues with some learner certificates loading. This was an inconsistent error where the specific issue could not be reproduced or identified immediately. By early Tuesday we decided to roll back the CSS block release update to resolve the intermittent loading issue as the exact cause could not be fully identified.

NOTE: The CSS block is still visible when you are on the initial certificate configuration window but this is currently disabled until we can identify what caused the load issue to certificates in the above circumstances.
Posted Oct 20, 2020 - 11:30 EDT