Incident Report Number: 2015-­004

Mail Delivery Delays

Ticket Number: ​INC0021063
Problem Number: PRB0010088
Major Incident Number: ​MIR0001048
What happened?

Mail delivery from University of Alberta email accounts experienced delays.

Who was affected?

Users and IT services that use University of Alberta email to send messages were affected.

What was the impact?

Email delivery was delayed for email users. Some IT services using the service may have encountered error messages.

What was the timeline of the incident?

Start: 2015/01/22 08:10 – Some users reported emails they had sent or received were delayed.
2015/01/22 08:15 – IST support began investigating this issue. 2015/01/22 08:35 – A problem with the database was discovered to be the cause of the email delays.
2015/01/22 08:40 – The issue with database was resolved and queued email was processed.
End: 2015/01/22 11:55 – All delayed messages were cleared from the queue.

What was the root cause of the incident?

A change to a database file permissions resulted in delayed email delivery

What was the work around and resolution for the incident?
Work Around

Not Appicable



Resolution

Correcting the file permissions error on the database file resolved the issue.

What are any recommendations to prevent this incident from occurring again?
  • Enhance test scripts which are run after any change to the mail cluster to confirm that there are no regressions.
  • As a preventative measure, increase monitoring on the mail systems by adding an alert to identify new delivery status messages.
Updates

Not Applicable