On November 18th, a replication bug was found in Postgres that affected the most recent versions of every Postgres release. The corruption that this bug may introduce could go undetected, and it manifests itself as a follower potentially having an inconsistent view of the data. For example, data could be present in the primary and not on the follower, or data deleted or updated on the primary and not from the follower. The likelihood of triggering this bug is higher for write-heavy workloads, such as many OLTP applications seen at Heroku.

We always recommend placing applications in maintenance mode and scaling down workers when performing a follower based changeover, and following this...

When was the last time you rotated your database credentials? Is it possible that old colleague still has access to your data? Or perhaps they've been accidentally leaked in a screenshot. There are many reasons to rotate your credentials regularly.

We now support the ability to easily reset your database credentials, and it is as simple as running the following on your command line:

heroku pg:credentials:rotate HEROKU_POSTGRESQL_COLOR --app your-app 

When you issue the above command, new credentials will be created for your database, and we will update the related config vars on your heroku application. However, on production databases (crane and up) we don't remove the old...

At the Heroku Department of Data, we are always investigating ways to improve the reliability, security and performance of your database servers. We do this by monitoring the entire ecosystem around it; we monitor the reliability of the platform itself, as well as keeping a close eye on the hardware where your data is hosted on upstream servers. But this also includes listening to the community. We do that by staying involved with our users at developer meetups and hackfests, listening closely to support requests to find and resolve common patterns of pain, as well as any relevant mailing lists.

Whenever we spot a problem, we make it a priority to resolve it.

The last such occasion has a...