Incident Report Number: 2016-006

Bear Tracks and PeopleSoft Campus Solutions Applications

Ticket Number: INC0046676

Major Incident Number: MIR0001090
What happened?

Bear Tracks and PeopleSoft Campus Solutions Applications experienced an issue.

Who was affected?

All users of Bear Tracks and PeopleSoft Campus Solutions were affected by this outage.

What was the impact?

The affected users were not able to login or use the applications.

What was the timeline of the incident?

Start: 2016/03/15 07:12 – Monitoring systems discovered a problem with the www.beartracks.ualberta.ca website. IST support began investigating.
2016/03/15 08:15 – IST support analysts escalated issue to the third party vendor for further investigation and resolution.
2016/03/15 09:05 – Vendor support advised they had completed a restart of one application server at a time, but this did not restore service. Vendor support began to shut down all of the application servers and perform a full restart. This action also caused an outage for the PeopleSoft Campus Solutions application.
End: 2016/03/15 10:25 – After the full shutdown and restart service was still experiencing performance issues. Vendor support performed some configuration changes to the Bear Tracks environment which solved the performance issues and service was restored.

What was the root cause of the incident?

A significantly larger than normal volume of users logging into Bear Tracks. The volume of logins caused significant delays and queuing within Bear Tracks.

What was the work around and resolution for the incident?
Work Around

Vendor support made some temporary configuration changes to the Bear Track environment.



Resolution

Over the weekend of Mar 19, 2016 vendor support made permanent configuration changes to the Bear Tracks environment. Vendor continues to monitor the environment for performance.

What are any recommendations to prevent this incident from occurring again?

None, vendor support has made permanent configuration changes to the Bear Tracks environment.

Updates

None.