Heroku Postgres Databases have been patched
Data is one of the most valuable assets of any company. As a database-as-a-service provider, one of our biggest responsibilities is ensuring your data is kept safe. A few weeks ago, one of the worst security vulnerabilities to date in PostgreSQL was discovered. To address this issue, Heroku deployed a point release upgrade across the entire Heroku Postgres service earlier this week. This resulted in a period of database unavailability, typically with a duration of less than one minute. Every database running on Heroku Postgres is now appropriately patched and is unaffected by the vulnerability.
PostgreSQL Vulnerability Details
The PostgreSQL project has provided official detail on CVE-2013-1899.
Several weeks ago there was a responsible disclosure of a serious security vulnerability within PostgreSQL by Mitsumasa Kondo and Kyotaro Horiguchi. The vulnerability allows unauthenticated remote users to use the ‘postmaster‘ process to write data to any accessible file, including critical internal database files.
The vulnerability was fixed and then committed to the PostgreSQL’s private git repository, but only after updates to anonymously accessible copies were disabled. Updated versions of PostgreSQL were released today to most large packaging repositories, as well as source code and installers.
Heroku Postgres Patching
The Heroku Postgres team worked with the PostgreSQL community to ensure we would be able to rapidly apply this patch. However, due to the nature of the issue, and aiming to mitigate risk for others, we were not able to discuss specifics until now. Our goal — in addition to ensuring your data was safe — was to continue monitoring this upgrade as it was deployed, providing early feedback to the community should bugs be found, and not jeopardizing in any way the coordinated public disclosure process stewarded by the PostgreSQL community. Most importantly, the PostgreSQL source code that included the patch was held in the utmost secrecy. In addition, the deployment plan was reviewed by PostgreSQL community members in advance.
Once the source code was released to the PostgreSQL packagers—of which a member of the Heroku Postgres staff is a part of—we began applying this patch to all Heroku Postgres databases, with the first updates starting on Monday. As of Wednesday at 6:30 PM PDT, all Heroku Postgres databases had been upgraded to their appropriate point release and were no longer vulnerable to CVE-2013-1899.
Conclusion
We realize that having no control over a maintenance window, however brief, is among the worst possible experiences. We are very sorry. Two reasons prevented us from working with you to schedule the security update. First, we prioritize ensuring your data is safe above all else, as a result making sure that every database was patched before this exploit was weaponized was paramount. Secondly, this was the first time we've had to deal with a security update of this scale, and have no machinery in place to schedule upgrades of this sort. Spending time to build such machinery would have prevented us from having every database patched in time. We will continue to work on improving our process around such maintenance to provide a better experience in the future.
As of late Wednesday all Heroku Postgres databases were upgraded and no longer at risk of CVE-2013-1899. No further action is required on your part to ensure your data remains safe.