There are many reasons to choose Heroku Data services, but keeping the services you use secure and up-to-date rank near the top. This foundation of trust is the most important commitment we make to our customers, and frequent and timely maintenances are one way we deliver on this promise.
We do everything we can to minimize downtime, which is typically between 10 – 60 seconds per maintenance. There are ways for you to minimize disruption too (see the tips and tricks below). The rest of the post explains how we think about Heroku Data maintenances, how we perform them, and when we perform them.
An Ounce of Prevention...
Hackers exploit known but unpatched vulnerabilities or out-of-date software. Minimizing the time between when a patch or update becomes available and when it gets deployed is the most effective means of limiting damage. There’s nothing worse than seeing your company’s high-profile breach at the top of Hacker News and the Wall Street Journal.
This business and reputation risk is real. Like you, we’re faced with the same choice. We believe it’s best to budget some prevention time upfront for patching and updating data services. Otherwise, an incident may cost us (and you) a larger amount of remediation time and effort, to say nothing of the potential damage done to our (and your) brand, business, and customers.
That’s why we invest significant engineering, security, and operations effort into creating a proactive security posture that keeps your stack up-to-date through frequent, scheduled maintenances.
Security Starts with Sound Design Principles and Policies
All Heroku Data services are fully-managed by dedicated experts. We design and optimize our Postgres, Redis, and Apache Kafka services for uptime, performance, security, scaling, upgrades, support, and much more.
The initial building of services with these attributes is only half the work. We continually update them, both to counter new threats and deliver new capabilities. We do this for you—automatically and with minimal disruption—during maintenance windows.
We ask all Heroku Data customers to choose a weekly four-hour maintenance window at a time when your business and customer impact is low. When necessary, we use this window to update your data services. We bias toward the beginning of your window whenever possible. And again, we typically need no more than 10 – 60 seconds per maintenance.
Normal maintenances, when we apply non-critical patches and updates, happen approximately every 30 – 60 days. We schedule them in advance and aim to give you at least three days notice before your window. You can change your Postgres maintenance window or your Redis maintenance window at any time. You can also choose to manually trigger a maintenance at any time outside of your window. You can defer a normal maintenance, but it will eventually go through during the last available maintenance window.
Critical Vulnerabilities Require a Decisive Response
We are fortunate at Heroku to have a Security team that constantly triages vulnerabilities, assesses the proper response, and defines the timeline to act. Together, Data and Security form a rapid response team capable of removing critical vulnerabilities from our fleet of millions of customer databases in days, not weeks.
Given the size of our platform and the community relationships we’ve formed over many years, we often hear about CVEs (Common Vulnerabilities and Exposures) before public disclosure. In the rare event that we determine that the potential for damage is severe, we will mitigate the risk outside of your normal maintenance window. Due to the confidential and sensitive nature of embargoed CVEs, we may not always be able to tell you why we’re running the maintenance at the time. We will tell you after the embargo is lifted. Recent examples include a Postgres vulnerability that we detected and an embargoed Redis vulnerability, both patched before public notice.
Tips and Tricks
Proactive, regular maintenances require good hygiene around app development and database connectivity. Fact is, cloud data services are ephemeral, albeit on a much longer time scale than app containers. It’s good engineering practice to treat data services as transient and test against failure scenarios. You may never experience an adverse event in production, but at least you will be ready.
On that note, we've prepared the following updates to our documentation to help you understand and manage the impact of maintenances:
For Postgres customers:
For Redis customers:
For Kafka customers:
Please also ensure that your account email is correct so that you’ll receive maintenance notifications.
Feedback Welcome
Our mission is the same, whether you’re using a free Hobby database or a premium, highly-compliant Shield database; we see frequent and timely maintenances as a key feature of secure and up-to-date Heroku Data services. If you have any questions or concerns, please reach out to your account team or email data at heroku dot com. Your feedback helps us improve our products, processes, and policies.