Incident Report Number: 2015-008

ESXi VMware Host

Ticket Number: ​INC0022909
Problem Number: PRB0010093
Major Incident Number: MIR0001053
What happened?

A ESXi VMware host, a server that hosts virtual machines, experienced an issue which caused several services/servers to become unavailable.

Who was affected?

Users trying to access services and servers located on the host. These included:

  • Arts database ­ Variety of arts services
  • Open­source Ticket Request System (OTRS)
  • Poker Research Application
  • Sitecore Content System
  • Canadian Vigour Center Terminal Server
  • Housing Management System (HMS)
  • Private Branch Exchange System(PBX)
What was the impact?

The affected users were not able to connect to the above services/servers.

What was the timeline of the incident?

Start: 2015/02/23 20:35 – Monitoring systems discovered a problem with multiple services/servers.
2015/02/23 21:00 – IT support analysts began working on the issue.
2015/02/23 21:30 – The affected host was rebooted to bring the virtual machines back online. The affected virtual machines were then transferred to other hosts.
End: 2015/02/23 22:35 – All services/servers were confirmed restored.

What was the root cause of the incident?

Further investigation determined that a setting on the Host Bus Adapter (HBA) is the cause.

What was the work around and resolution for the incident?
Work Around

Not Applicable



Resolution

The virtual host was rebooted to bring the affected virtual machines back on line. The affected virtual machines were migrated to other hosts.

What are any recommendations to prevent this incident from occurring again?

A change in the configuration of the affected hosts has been recommended by the vendor to resolve the issue.

Updates

Not Applicable