Increased errors and degraded performance in US-east-1
Postmortem
AWS experienced an outage in one Availability Zone in US-East-1 due to a power failure. Initially unaffected, Learnosity experienced some knock-on effects s few hours later. AWS has resolved the power issue and things quickly returned to normal.
Posted Dec 22, 2021 - 11:32 EST
Resolved
As of 16:00 UTC, we are still seeing no lingering effects from this mornings AWS outage and are resolving this issue.
Learnosity Support and Systems Engineering teams will follow up with a post mortem once we have completed root cause analysis and finalised any next steps or preventative measures required.
Please reach out if you have any questions or concerns.
Posted Dec 22, 2021 - 11:14 EST
Monitoring
Looking like AWS have resolved the issue on their end, our systems are starting to stabilise and looking to be in good health again. We will continue to monitor the situation and updated as needed. The error rates have dropped again to zero and all systems are looking green again.
Posted Dec 22, 2021 - 09:03 EST
Identified
AWS have experienced an outage in one Availability Zone and it's affecting the systems running in that AZ. Most of Learnosity systems are still operational but this AWS outage is having a knock-on effect that is impacting other AWS systems.
Posted Dec 22, 2021 - 08:14 EST
Investigating
As of 04:35 AM PST AWS has been experiencing problems that are trickling down to Learnosity. It appears that some areas of the U.S. are having trouble accessing some data and some assessments may be affected.
Learnosity systems are running impaired and we will update this status as more information becomes available.
Posted Dec 22, 2021 - 07:56 EST
This incident affected: AMER || Assessment (Loading and rendering of assessment UI, Loading and rendering of Items/Questions/Features, Saving of student responses, Branching/Adaptive services, Self-Hosted Adaptive services, Dynamic Content generation, Sending and receiving Live Activity Status by User report events, Uploading of rich student responses (audio/images/files), Processing and availability of rich student responses (audio/images/files), Loading, rendering and capture of student annotations), AMER || Analytics (Loading and rendering of reports, Availability of session information, Tag-based reports, Progress-based reports, Large-scale group reports, Live Progress (Live Activity by User) report, Learning Outcome report, Adaptive session information, Subscore calculation, LTI grade passback), and AMER || Data Centric (Creation of sessions, Creation of Item Pools, Updating session response scores, Scoring endpoint, Creation of report data sets, Firehose).