Polyglot Platform

Programming languages are silos. The libraries, development tools, deployment practices, and even naming schemes associated with one language — say, Ruby — rarely have much carry-over to another language — say, Python, Erlang, Java, or C++.

Professional programmers dedicate their careers to becoming experts in a particular language, making extensive personal investment in learning not only their chosen language's syntax and libraries, but also the ecosystem of tools and practices from that language's community. Similarly, companies build up codebases, deployment infrastructure, policies, and tacit knowledge in their engineering staff around a single language.

Deep investment in a single general-purpose programming language has a downside: the law of the instrument. This is completely rational: with so much investment, switching costs are high. But using the wrong tool for the job can heavily dampen developer productivity, and even leads to "religious debates" where programmers assert that their chosen language is the One True Programming Language and the best choice for every purpose.

There's a better way.

The Modern Age: Polyglot

In contrast to the one-language-per-career programmer, today's up-and-coming developers can often utilize many languages effectively. Borrowing a term from linguistics, we can call these versatile new developers "polyglot programmers."

The last decade has brought us a renaissance in programming language diversity, with languages like Clojure, Scala, and Go. But many of the languages now gaining popularity have existed for much longer, such as Objective-C, Javascript, and Lua. So what changed in the last few years to awaken the polyglot passions of modern developers?

A parable will illustrate:

Programming enthusiast Joe is browsing Hacker News on a Saturday afternoon, and sees a story about Clojure, a Lisp-like programming language which he keeps hearing about. The comments on the story are filled with positive experiences reported by his peers, so he decides to spend a few hours checking it out.

Within thirty minutes, he's installed the Clojure language runtime and tools; and downloaded the Peepcode screencast and Pragmatic Studio eBook. With a video describing the language's core principles running in a background window, Joe tinkers with the basic syntax in the REPL. By the end of the afternoon, Joe has written some simple programs and has a familiarity with both the basic usage of the language, and some of its underlying principles.

From here, it's a small step for Joe to use Clojure for his next hobby project. And that might lead to using it for a small tool or app at his programming day job, or at his next startup.

Breaking down Joe's story, we see three factors at play:

• Programming-centric news aggregators such as Hacker News and the programming subreddit give curious developers exposure to buzz about new languages.
• Wide availability of easy-to-consume learning materials from sources such as The Pragmatic Bookshelf and O'Reily along with countless individual blogs, as a way for developers to casually self-educate. Most of these materials can be purchased and downloaded in electronic form instantly; in fact, many programming books are now distributed primarily online, such as the Django book or the CouchDB book
• The ease of acquiring and installing most modern language runtimes (typically available as open source downloads) allows a developer to get a working toolchain with very low time investment. REPL shells offer an interactive environment for learning the language syntax; some REPLs are even available on the web (see Try Ruby, Try Clojure, Try Python, Scastie).

The polyglot tendencies of modern developers can also be seen in the book Seven Languages in Seven Weeks.

Parallels Between Language Communities

With more polyglot programmers than before, there is more cross-talk between language communities than in the past. And whether or not languages are actively sharing ideas, parallels tend to emerge in tools, simply because most languages need to solve similar problems.

For example, a dependency-management system and public repository of libraries are important milestones on the way to a language's suitability for production use. Tools like REPL shells, embedded webservers, and Rails-like or Sinatra-like web frameworks can now be found in almost every language.

Comparison of Tools and Libraries Across Languages

There's more. Most modern apps use the same types of backing services such as datastores (MySQL, PostgreSQL, CouchDB, Redis), caching systems (Memcached, Varnish), and queueing systems (RabbitMQ, Beanstalkd), rather than the language- or framework-specific services of yesteryear (e.g. Erlang's Mnesia or Zope's Object Database). There is even convergence on RPC protocols (e.g. REST, AMQP, ZeroMQ, Thrift, Protocol Buffers) and serialization protocols (e.g. XML, Yaml, JSON), making calls between apps written in different languages extremely easy.

Add all these factors together and we see striking parallels across the board between modern web apps, regardless of language used.

The Missing Piece: Deployment Infrastructure

But despite all the convergence between languages during the development cycle, deployment remains its own highly-siloed world. Every language has its own practices, its own tools, and its own terminology for deployment. A Rubyist might use Capistrano to manage deploys, while a Pythonista might use Fabric. Dependency management practices and tools vary wildly not only between languages, but between deploys: one shop might use Chef to install gems system-wide, while another might use Gem Bundler to install gems local to the app. JVM languages (Java, Clojure, Scala, etc) typically expect to be packaged as a WAR to run inside a servlet container such as Tomcat, while scripting languages like Ruby and Python typically use embedded webservers.

The result of all this diversity is that the safe bet, especially for large organizations, is to standardize on a single language and avoid (or even outright forbid) development in any others. Polyglot programming allows using the right tool for the job, offering substantial gains in speed and agility of development; but the switching cost of changing out the entire stack of deployment, scaling, management, and monitoring infrastructure necessary for an app written in a different language creates an intractable obstacle to reaping the benefits of polyglot.

Heroku began working on solving the polyglot deployment infrastructure problem at the start of 2010. Our team was energized by the opportunity we saw here, envisioning a polyglot platform with official support for many languages.

We made an explicit goal to avoid creating mostly separate, language-specific products — Heroku for Ruby, Heroku for Node.js, Heroku for Clojure, and so on — with little relation to each other. Our hypothesis was that it would be possible to create a general-purpose runtime substrate onto which we could layer language-specific workflows to provide a developer experience that felt natural and native for developers from each supported language.

At the same time, we saw the opportunity to bring more normalization and unification to deployment and app management practices across languages. And the developer experience for deploying an app written in any language should always feel like Heroku, with the simplicity, power, and instant-gratification that we are passionate about.

With the launch of Celadon Cedar's official support for Ruby, Node.js, and Clojure (and many more to come), we believe we've achieved that vision.

Heroku, the Polyglot Platform

Polyglot is as much about recognizing similarities across deployments of different languages as it is about enabling differences among them.

With a truly polyglot platform, choice of programming language is reduced to just syntax and libraries. Uniform deployment infrastructure means that switching languages doesn't mean fundamentally different deployment endpoints or models. Heroku's Cedar solves the switching-cost problem: developers can rewrite an app in another language if they choose without changing anything about how it runs, what services it consumes, or how it is monitored and scaled. With Cedar, language choice and deployment infrastructure are orthogonal.

General-purpose tools face the problem of "trying to please everyone pleases no one." Cedar's approach to solving this is to maintain language-level curation of developer tools, something we call language packs. The owners of each language pack are developers who use and love the language, and are part of its community. It's of crucial importance that we have real internal users driving the experience for their respective languages.

Implementing the polyglot platform has driven us to find and articulate a universal, language-agnostic interface between the platform and application interfaces. Language-neutral interfaces mean that benefits of the platform accrue to deployments in all languages. For example, logging on Heroku is accessible to and useful for every application, and works out of the box with most languages and web frameworks. Another example is add-ons: most add-ons in the Heroku add-ons catalog are usable from any language available on the Heroku platform, thanks to the language-agonistic configuration via environment variables.

Conclusion

We're on the cusp of a new golden era for application development: polyglot programming, which enables choosing a specialized language for the problem at hand. But development is only half the story, so it follows naturally that polyglot deployment is needed to complete the picture.

Switching costs on scaling, management, and monitoring infrastructure that historically accompanied a change of language an app was written in have been too high. Heroku Cedar, the polyglot platform, changes all of that. We look forward to announcing official support for more languages in the coming months.