Reports data unavailable
Incident Report for Help Scout
Postmortem

The Help Scout reporting servers have a few large indexes. One of those indexes was added this week as we launched 2 new reports. The new index caused reporting servers to quickly run out of memory yesterday and reporting data was unavailable for about 15 minutes. Everything was back to normal after restarting the servers.

We have a good idea of what's causing the problem, but aren't 100% sure. To narrow it down, we're going to put this new index on a separate server. It should allow us to validate the issue and resolve it once and for all.

A small maintenance window is required in order to isolate the new index, which we have scheduled for Sunday morning at 9am EST.

Posted Oct 31, 2014 - 10:31 EDT

Resolved
Reports and any other application slowness has been resolved. We'll keep monitoring everything closely and do a post-mortem when we know more.
Posted Oct 30, 2014 - 17:02 EDT
Identified
Found the issue and are fixing now, should be back to normal shortly.
Posted Oct 30, 2014 - 16:55 EDT
Investigating
We're looking into some blank pages when fetching Reports data.
Posted Oct 30, 2014 - 16:52 EDT