The uLearn (Blackboard) application, a learning management system, was experiencing a service degradation.
Users of uLearn (School of Business) were affected by this outage.
The affected users were unable to see or access their courses from the current term once logged into uLearn. Users who only have enrollments in the current term could no longer log in to the system. During the restoration some students regained access to courses that they had dropped.
Start: 2015/02/05 12:15 Users of uLearn reported that course information was missing.
2015/02/05 12:45 IT support began investigating this issue.
2015/02/05 12:58 The issue was determined to be with the LDAP (Lightweight Directory Access Protocol) System.
2015/02/05 13:08 A data rollback in uLearn was started in order to restore missing data for the affected users. 2015/02/05 13:30 The initial data rollback was completed. This resolved issues for all but 122 users.
2015/02/05 13:35 An automated tool to resolve the remaining issues was built and tested successfully.
2015/02/05 13:45 The missing data for the remaining 122 users was successfully restored and a full verification was started to ensure the issue was resolved.
End: 2015/02/05 14:00 The issue was confirmed resolved.
The root cause of the issue was an import script failed when it was unable to connect to the LDAP system. The import operation caused all enrollment records for instructors and students to be dropped.
A backup of the records from the previous day was restored. This recovered the missing records. Any changes to student records on the day of the incident still existed in LDAP.
LDAP began a restore process. Once LDAP completed the data restore and updated, the records in uLearn were correct.