Migrated Students Grade Batch Job Error

Incident Report for Xen Education

Postmortem

Why did this happen?

This happened because of course enrolment API pagination was not updated with the latest code from the performance optimisations.

What was the incident’s duration?
The incident lasted 8 hours from discovery to remediation.

What will prevent further similar incidents?

A fix for the pagination issue was deployed and all impacted batch jobs re-ran.

To prevent further occurrences, the monitoring of the batch export job was improved to pick up any issue earlier and automatically re-try.

Posted Jun 25, 2024 - 16:48 AEST

Resolved

Export of Progress & grades to external Student Information Systems was suspended for 8 hours due to a regression within the pagination schema of the Course Enrolment API endpoint.
Posted Feb 12, 2024 - 17:00 AEDT