Connection Issues
Incident Report for Help Scout

What happened

Starting around 6:13pm EDT on Thursday, an unusually high spike of messages caused our message queueing services to fail. Help Scout services (web app, mobile API, and help desk API ) were down for roughly 18 minutes, and services returned to normal around 6:31pm EDT.

What we're doing about it

We're working to separate our message queue cluster to spread any unusual spikes like this across several clusters. Going forward, we're also adding new alerts, as well as additional error logging around our internal APIs.

Posted 3 months ago. Sep 01, 2017 - 21:34 EDT

Resolved
We're closing this out, as all services have returned to normal. We'll continue to monitor over the next few hours. We're still digging for a root cause, and will publish a post-mortem in the next 24 hours with a more complete description of what happened.
Posted 3 months ago. Aug 31, 2017 - 18:40 EDT
Update
The web application, help desk API, and mobile apps are back online, though page loads might be slow as we continue to investigate. Workflows, inbound and outbound email, folder counts, search, and reports will be delayed until we're in the clear.
Posted 3 months ago. Aug 31, 2017 - 18:31 EDT
Update
We're working as quickly as possible to restore services. We'll continue to update here as we make progress.
Posted 3 months ago. Aug 31, 2017 - 18:23 EDT
Investigating
Help Scout is currently unavailable. We're investigating.
Posted 3 months ago. Aug 31, 2017 - 18:16 EDT