Connection issues

Incident Report for Help Scout

Postmortem

Connection latency between 11:51am and 12:08pm EDT, and again briefly at 2:23pm EDT, caused some slow page loads and roughly 5 minutes of downtime today. Both incidents were caused a problems with our upstream hosting provider, AWS. The first problem resulted in connection failures to session data stored in cache, and the second was a very brief failure to connect to our web servers.

We are capable of continuing service when a single data center is down, but in this case, the problems seem to have impacted the entire region. Service has returned to normal and all systems remain stable on our side of things.

Posted Jul 19, 2016 - 15:31 EDT

Resolved

We're back in business after experiencing slow page load times and roughly 5 minutes of downtime, between 11:51 AM EDT and 12:08 PM EDT. We’re still investigating the root cause and will add a post-mortem if any changes need to be made.

Posted Jul 19, 2016 - 12:41 EDT

Update

Page load times are returning to normal. We'll continue to dig in and monitor all services until we're in the clear.

Posted Jul 19, 2016 - 12:14 EDT

Update

We're still investigating long page load times and connection timeouts. Some customers might see an "Unable to load folder content" error when attempting to access a mailbox.

Posted Jul 19, 2016 - 12:07 EDT

Investigating

We're looking in to slow page load times across the web app. Some customers may also experience a time out when attempting to access Help Scout. We'll update as soon as we have more information.

Posted Jul 19, 2016 - 11:55 EDT