Platform UI and API Outage
Incident Report for Subscribe Pro
Postmortem

We had an outage this morning that affected all Subscribe Pro customers. The root cause was a disk that filled up on our transactional database system. There were two other systems failures which caused this issue to lead to over an hour of downtime:

  • Our designated on-call personnel failed to respond as promptly as required.
  • We were operating with a mis-understanding of the auto-scaling settings around the disk that filled up.

We have taken action to address both of these system issues:

  • We have fine tuned our procedures for on-call personnel and increased our training frequency.
  • We have adjusted and tested the relevant auto-scaling settings for database disk.

We take any down time very seriously and have used this as opportunity to improve our systems and processes so that this type of issue can’t happen again.

Thank you.

Posted Jul 19, 2019 - 14:29 EDT

Resolved
This outage has been resolved. Please contact support@subscribepro.com if you experience any further issues accessing our platform web UI or API.
Posted Jul 19, 2019 - 09:24 EDT
Monitoring
A fix has been implemented and we are monitoring.
Posted Jul 19, 2019 - 07:20 EDT
Investigating
We are currently investigating an outage of our platform UI and API.
Posted Jul 19, 2019 - 06:50 EDT
This incident affected: Subscribe Pro Application (API | api.subscribepro.com, Management Portal | platform.subscribepro.com).