RabbitMQ Add-on Now Available on Heroku

Today we're proud to announce the availability in beta of RabbitMQ add-on by VMWare. RabbitMQ is an open source implementation of the AMQP protocol that provides a robust, scalable and easy-to-use messaging system built for the needs of cloud application developers.

Getting Started

With the add-on, provisioning a fully managed RabbitMQ instance couldn't be easier to do:

$ cd rabbitdemo
$ heroku addons:add rabbitmq
-----> Adding rabbitmq to rabbitdemo... done, v2 (free)

$ heroku config
RABBITMQ_URL  => amqp://uname:pwd@host.heroku.srs.rabbitmq.com:13029/vhost

Your application's environment will now have the RABBITMQ_URL set pointing to your new instance. Most modern AMQP clients such as Bunny for Ruby will accept a connection string in URI format, making configuration a breeze. The following is a simple Sinatra app that demonstrates posting and getting messages from the default RabbitMQ exchange. You can grab the source from Github here. Let's have a look:

Entering a message and hitting "post" will send a message to a RabbitMQ queue, where it will sit until we tell the app to fetch the oldest message from the queue by clicking "get". Examining the application code, the first place to look is in "lib/sinatra_rabbitmq.rb" where we set up the connection from the environment variable, and declare a queue called "messages":

lib/sinatra_rabbitmq.rb

require 'sinatra/base'
require 'bunny'

module Sinatra
  module RabbitMQ
    def rabbitmq_client
      return @rabbitmq_client if @rabbitmq_client
      @rabbitmq_client = Bunny.new(ENV["RABBITMQ_URL"])
      @rabbitmq_client.start
      @rabbitmq_client
    end

    def rabbitmq_exchange
      @rabbitmq_exchange ||= rabbitmq_client.exchange("")
    end

    def rabbitmq_messages_queue
      @rabbitmq_messages_queue ||= rabbitmq_client.queue("messages")
    end
  end

  register RabbitMQ
end

Note that in RabbitMQ a message is never sent directly to a queue. Instead, it passes through an exchange, the type of which defines how messages are distributed to one or more client queues. In this case, we're side-stepping the concept of exchanges by using the default nameless exchange, which allows us to specify the target queue for our messages using a routing key. Once you get into more advanced usage of RabbitMQ such a broadcasting messages to set of known queues, you'll definitely want to learn more about exchanges.

With our connection established and queue defined, the main part of our app looks like this:

app.rb

require 'sinatra/base'
require "#{File.dirname(__FILE__)}/lib/sinatra_rabbitmq"

class RabbitmqDemo < Sinatra::Base
  register Sinatra::RabbitMQ

  get "/" do
    haml :index
  end

  post "/" do
    self.class.rabbitmq_exchange.publish params["message"], :key => "messages"
    @notice = "Message has been published."
    haml :index
  end

  get "/message" do
    msg = self.class.rabbitmq_messages_queue.pop
    if msg[:payload] == :queue_empty
      @notice = "No more messages."
    else
      @message = msg[:payload]
    end
    haml :index
  end
end

The first interesting part here is post to "/" where we send the message you typed in to the default exchange, using "messages" as the routing key. Then we pick up any outstanding at "/messages" by popping off the queue, which follows the FIFO principle. The rest, as they say, is just a bit of HTML.

Why would you want to use a messaging system?

The distribution of workloads across different process types is an essential aspect of modern web application architecture. For example, if your app handles file uploads, your web process might receive the upload and signal a background worker to do some processing on it via a queue. While you could use Delayed Job or another database-backed queueing library to do this, a true messaging system gives you much more flexibility, reliability and scalability in defining how your signals are distributed and received by your worker pool.

Finally, messaging is an important tool for the polyglot programmer. Having a language agnostic, data-centric message bus that can orchestrate communications between, say, a web app in Ruby, workers in Java and a chat server in Node.js is a key enabler in allowing application developers choose the right tool for any specific job.

Over the past few years RabbitMQ has emerged as one of the most popular, open source choices for messaging with clients in all major languages, and thousands of enterprises trusting it for mission-critical apps.