During the development of the recently released Heroku SSL feature, a lot of work was carried out to stabilize the system and improve its speed. In this post, I will explain how we managed to improve the speed of our TLS handshakes by 4-5x.

The initial reports of speed issues were sent our way by beta customers who were unhappy about the low level of performance. This was understandable since, after all, we were not greenfielding a solution for which nothing existed, but actively trying to provide an alternative to the SSL Endpoint add-on, which is provided by a dedicated team working on elastic load balancers at AWS. At the same time, another of the worries we had was to figure out how...

HTTP routing on Heroku is made up of three main logical layers:

1. The state synchronization layer ensures that all nodes in the routing stack are aware of the latest changes in domains, application, and dyno locations across the platform;
2. The routing layer chooses which dyno will handle an HTTP request (random or sticky), performs logging, error-reporting, and so on;
3. The HTTP proxying layer handles the validation, normalization, and forwarding of requests between clients and dynos.

This last part is the one the platform team is happy to open-source today with the Vegur library.

The Heroku Routing team does a lot of work with Erlang, both in terms of development and maintenance, to make sure the platform scales smoothly as it continues to grow.

Over time we've learned some hard-earned lessons about making systems that can scale with some amounts of reliability (or rather, we've definitely learned what doesn't work), and about what kind of operational work we may expect to have to do in anger.

This kind of knowledge usually remains embedded within the teams that develop it, and tends to die when individuals leave or change roles. When new members join the team, it gets transmitted informally, over incident simulations, code reviews, and other similar...