Last summer, Heroku became a polyglot platform, with official support for Ruby, Node.js, Clojure, Java, Python, and Scala. Building a platform that works equally well for such a wide variety of programming languages was a unique technical design challenge.
We knew from the outset that maintaining siloed, language-specific products – a Heroku for Ruby, a Heroku for Node.js, a Heroku for Clojure, and so on – wouldn't be scalable over the long-term.
Instead, we created Cedar: a single, general-purpose stack with no native support for any language. Adding support for any language is a matter of layering on a build-time adapter that can compile an app written in a particular language or framework into an executable that can run on the universal runtime provided by Cedar. We call this adapter a buildpack.
An Example: Ruby on Rails
If you've deployed any app to the Cedar stack, then you've already used at least one buildpack, since the buildpack is what executes during git push heroku master. Let's explore the Ruby buildpack by looking at the terminal output that results when deploying a Rails 3.2 app:
$ git push heroku master
Counting objects: 67, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (53/53), done.
Writing objects: 100% (67/67), 26.33 KiB, done.
Total 67 (delta 5), reused 0 (delta 0)
-----> Heroku receiving push
-----> Ruby/Rails app detected
-----> Installing dependencies using Bundler version 1.2.0.pre
Running: bundle install --without development:test --path vendor/bundle --binstubs bin/ --deployment
Fetching gem metadata from https://rubygems.org/.......
Installing rake (0.9.2.2)
...
Your bundle is complete! It was installed into ./vendor/bundle
-----> Writing config/database.yml to read from DATABASE_URL
-----> Preparing app for Rails asset pipeline
Running: rake assets:precompile
Asset precompilation completed (16.16s)
-----> Rails plugin injection
Injecting rails_log_stdout
Injecting rails3_serve_static_assets
-----> Discovering process types
Procfile declares types -> (none)
Default types for Ruby/Rails -> console, rake, web, worker
-----> Compiled slug size is 9.6MB
-----> Launching... done, v4
http://chutoriaru.herokuapp.com deployed to Heroku
To git@heroku.com:chutoriaru.git
* [new branch] master -> master
Everything that happens between Heroku receiving push and Compiled slug size is 9.6MB is part of the buildpack. In order:
- installing Ruby,
- installing and running Bundler to manage gem dependencies,
- injecting database configuration,
- compiling Rails assets,
- and installing Heroku-specific plugins for logging and serving static assets.
The slug that results from this Rails-specific build process can now be booted on our language-agnostic dyno manifold alongside Python, Java, and many other types of applications.
Using a Custom Buildpack
In the example above, the appropriate buildpack was automatically detected from our list of Heroku-maintained defaults.
However, you can also specify your desired buildpack using arguments to the heroku create command or by setting the BUILDPACK_URL config variable. This enables the use of custom buildpacks. If you want to run your Rails app on JRuby, for example, specify the buildpack created by the JRuby team at app creation time:
$ heroku create --buildpack https://github.com/jruby/heroku-buildpack-jruby
Arbitrary Language Support
Since language support can be completely contained inside a buildpack, it is possible to deploy an app written in nearly any language to Heroku. Indeed, there are a variety of third-party buildpacks already available:
- Perl by Lincoln Stoll
- Common Lisp by Mike Travers
- Go by Keith Rarick
- Dart by Ilya Grigorik
- Null by Ryan Smith
See the full list of third party buildpacks in the Dev Center.
Customizing the Build Process
In addition to enabling new language support, the ability to select a buildpack allows you to modify the previously closed Heroku build process for popular languages.
For example, consider a Ruby app that needs to generate static files using Jekyll. Before buildpacks, the only solutions would have been to 1) generate the files before deployment and commit them to the repository or 2) generate the files on-the-fly at runtime. Neither of these solutions are ideal as they violate the strict separation that should be maintained between the codebase, the build stage, and the run stage.
By forking the official Ruby buildpack, you could add a site generation step to your build process, putting file generation in the build stage where it belongs.
All of the default buildpacks are open source, available for you to inspect, and fork to modify for your own purposes. And if you make a change that you think would be useful to others, please submit an upstream pull request!
Adding Binary Support
Your app might depend on binaries such as language VMs or extensions that are not present in the default runtime. If this is the case, these dependencies can be packaged into the buildpack. A good example is this fork of the default Ruby buildpack which adds library support for the couchbase gem. Vulcan is a tool to help you build binaries compatible with the 64-bit Linux architecture which dynos run on.
Buildpacks Beyond Heroku
Buildpacks are potentially useful in any environment, and we'd love to see their usage spread beyond the Heroku platform. Minimizing lock-in and maximizing transparency is an ongoing goal for Heroku.
Using buildpacks can be a convenient way to leverage existing, open-source code to add new language and framework support to your own platform. Stackato, a platform-as-a-service by ActiveState, recently announced support for Heroku buildpacks.
You can also run buildpacks on your local workstation or in a traditional server-based environment with Mason.
Conclusion
Get started hacking buildpacks today by forking the Hello Buildpack! Read up on the implementation specifics laid out in the Buildpack API documentation, and join the public Buildpacks Google Group. If you make a buildpack you think would be useful and that you intend to maintain, send us an email at buildpacks@heroku.com and for potential inclusion on the third-party buildpacks page.