Flow is an important part of software development. The ability to achieve flow during daily work makes software development a uniquely enjoyable profession. Interruptions in your code/test loop make this state harder to achieve. Whether you are running unit tests locally, launching a local webserver, or deploying to Heroku there's always some waiting and some interruption. Every second saved helps you stay in your flow.
We’ve been working on reducing the time it takes to build your code on Heroku. Read through this post for details on the process we used to make builds fast, or check out the end result from the graph below:
Let's take a look at our process in delivering these improvements further.
It all starts with instrumentation
Every speed improvement effort starts with visiblity. We collected detailed metrics about every part of our build infrastructure and in each of our buildpacks. These metrics allowed us to see where time was being spent. We were also able to measure how each update we made impacted the build phase it was meant to improve.
Improvements to our build infrastructure
With detailed timing data in hand, we were able to make changes to how our build fleet and git service work. These included additional caching, improved mechanisms for file transfers and storage, and providing better user feedback to let you know what is happening during your builds.
One crucial step in the build process that had room for improvement was the creation of the slug archive. We found this was taking an appreciable amount of time and were able to improve it by using pigz which is a parallelized implementation of gzip.
Improvements to the efficiency of individual buildpacks
In addition to the changes to the build process, we had an opportunity for each of our language specialists to dig in and optimize their specific buildpacks. This included a range of improvements from being more efficient about how dependencies are downloaded, to making better use of the build cache, to pre-fetching common dependencies.
This process involved many changes across all buildpacks. Some of the most significant are highlighted in the Heroku changelog:
- Rails 4 asset pipeline speed improvements
- Play 2.x slugs exclude SBT compile-only artifacts
- New Node.js Buildpack
- Improved build caching for apps using multiple buildpacks
The results
We've achieved improvements in build time across all languages on Heroku. Although we looked at much more detailed metrics while changes were being made, our target was improving build times at all percentiles on a per language basis. We’ve showed you the 50th percentile or median because it proved to be a good proxy for the other metrics in this case.
Being able to deploy code and iterate on ideas quickly is a big part of developer happiness. We will continue to use the metrics we’ve created to look for more ways to improve build and deployment speed. As always, we welcome your feedback on how we can improve your experience and help you maintain your flow.