Connection Issues
Incident Report for Help Scout

What happened

Starting around 6:13pm EDT on Thursday, an unusually high spike of messages caused our message queueing services to fail. Help Scout services (web app, mobile API, and help desk API ) were down for roughly 18 minutes, and services returned to normal around 6:31pm EDT.

What we're doing about it

We're working to separate our message queue cluster to spread any unusual spikes like this across several clusters. Going forward, we're also adding new alerts, as well as additional error logging around our internal APIs.

Posted 22 days ago. Sep 01, 2017 - 21:34 EDT

Resolved
We're closing this out, as all services have returned to normal. We'll continue to monitor over the next few hours. We're still digging for a root cause, and will publish a post-mortem in the next 24 hours with a more complete description of what happened.
Posted 23 days ago. Aug 31, 2017 - 18:40 EDT
Update
The web application, help desk API, and mobile apps are back online, though page loads might be slow as we continue to investigate. Workflows, inbound and outbound email, folder counts, search, and reports will be delayed until we're in the clear.
Posted 23 days ago. Aug 31, 2017 - 18:31 EDT
Update
We're working as quickly as possible to restore services. We'll continue to update here as we make progress.
Posted 23 days ago. Aug 31, 2017 - 18:23 EDT
Investigating
Help Scout is currently unavailable. We're investigating.
Posted 23 days ago. Aug 31, 2017 - 18:16 EDT