Incident Report Number: 2016-011

Internet Service in Lister Centre Residences

Ticket Number: INC0047751
Problem Number: PRB0010149
Major Incident Number: MIR0001095
What happened?

During planned maintenance some configuration errors occurred. University Wireless Service (UWS) and wired Local Area Network (LAN) connections providing internet access in Lister Centre Residences experienced an issue.

Who was affected?

Some UWS and LAN users in the Lister Centre Residences may have been affected by this outage.

What was the impact?

LAN users using wired connections were not able to access the internet. UWS users wireless devices were not able to connect to the UWS service to access the internet.

What was the timeline of the incident?

Start: 2016/03/29 22:00 – Planned maintenance began, Lister residence connections were disconnected.
2016/03/30 01:30 – Connections were moved to the new network devices and change was completed.
2016/03/30 01:50 – Monitoring tools showed some Lister residence network devices were unreachable.
2016/03/30 03:30 – Issue was related to a configuration issue for some of the network devices which was resolved by modifying the configurations, affected network devices became available.
2016/03/30 08:30 – UWS access issues in the Lister residences complex were reported.
2016/03/30 08:35 – IST analysts began working on the issue.
2016/03/30 09:05 – It was determined to be a configuration issue on Lister residences wireless access points. Configuration on the affected wireless access points was corrected.
End: 2016/03/30 09:12 – Service was confirmed restored.

What was the root cause of the incident?

During planned maintenance some configurations errors occurred when moving to new network devices. Configurations for network gateways for the LAN network were not updated when moving to new network devices. Configurations for Lister residences wireless access points were not copied when moving to new network devices.

What was the work around and resolution for the incident?
Work Around

None.



Resolution

Configurations were modified on new network devices to restore LAN internet connectivity and UWS internet service.

What are any recommendations to prevent this incident from occurring again?
  • Enhanced monitoring and testing procedures when performing future network changes.
Updates

None.