Intermittent latency affecting next-item selection during adaptive testing in us-east-1

Incident Report for Learnosity

Postmortem

As of 21:00 UTC, Learnosity identified and corrected an issue causing intermittent latency affecting next-item selection during adaptive testing in us-east-1.

Learnosity isolated and restricted database optimization tasks that were causing intermittent degraded performance affecting adaptive endpoints. Elevated connections to the DB were reduced to normal operating levels, eliminating the unexpected latencies.

This issue has been resolved and a fix identified to prevent this from occurring again. Please contact Support if you have any additional questions.

Posted Dec 15, 2020 - 16:10 EST

Resolved

As of 21:00 UTC, we are have resolved the intermittent latency issue affecting next-item selection during adaptive testing in us-east-1.

Adaptive services continue to operate normally.

Learnosity Support and Systems Engineering teams will follow up with a post mortem once we have completed root cause analysis and finalized any next steps or preventative measures required.

Please reach out if you have any questions or concerns.

Posted Dec 15, 2020 - 16:07 EST

Monitoring

As of 20:43 UTC, we have implemented optimizations to eliminate the previously observed latency when selecting items during adaptive testing in us-east-1.

Learnosity Support and Systems Engineering teams will continuing to actively monitor the issue, and will follow on with an update and resolution as soon as possible.

Posted Dec 15, 2020 - 15:45 EST

Update

As of 20:31 UTC, we are testing an optimization that is addressing today's intermittent latency experienced today during adaptive assessment. Adaptive item selection is currently operating at optimal levels.

Learnosity Support and Systems Engineering teams are continuing to actively investigate the issue, and will follow on with an update and resolution as soon as possible.

Posted Dec 15, 2020 - 15:34 EST

Investigating

As of 19:00 UTC, we are investigating Intermittent latency affecting next-item selection during adaptive testing in us-east-1.

At present, only select configurations are affected, but we are continuing to test widely.

Learnosity Support and Systems Engineering teams are actively investigating the issue, and will follow on with an update and resolution as soon as possible.

Posted Dec 15, 2020 - 14:33 EST

This incident affected: AMER || Assessment (Branching/Adaptive services).