Heroku Postgres Launches

Heroku's Postgres database service, the origins of which date back to 2007, is one of the most battle-tested cloud database services around.

Over the last year, our growing data team has done an amazing job of dramatically increasing the scale, reliability, and durability of the service - now boasting 99.99% (measured) uptime and over 400 million write-transactions per day.

Until now, the service has only been available to Heroku customers, but today we are pleased to announce the launch of Heroku Postgres as a standalone database service. Included in this launch is a new web interface for managing databases, as well as rock-solid durability based on Continuous Protection technology.  Best of all, these improvements are effective immediately for all existing users of the Heroku Postgres add-on.

To learn more, check out the inaugural post of the new Heroku Postgres blog.  Future posts to this blog will contain product updates and articles about leveraging the advantages of PostgreSQL.

Announcing Heroku Postgres

Until now, Heroku's Postgres database service - originally launched in 2007 - has only been available to Heroku customers for use with Heroku platform apps. Today we're excited to announce the launch of Heroku Postgres as a standalone service.

With measured service uptime of four nines (99.99%), and designed data durability of eleven nines (99.999999999%), the service is trustworthy for mission-critical data. As of today, these production-quality Heroku Postgres databases are independently available for use from any cloud platform, provisioned instantly, metered by the second, and without contract.

Battle Tested

Heroku Postgres has successfully and safely written 19 billion customer transactions, and another 400 million write-transactions are processed every day. Leverage Heroku's scale and expertise while enabling your team to focus on building great apps rather than managing and configuring databases.

Continuous Protection

The safety and durability of your data is our number one priority. Continuous Protection is a set of technologies designed to prevent any kind of data loss even in the face of catastrophic failures. Taking advantage of PostgreSQL's WAL (write-ahead-log of each change to the database's data or schema), Heroku Postgres creates multiple, geographically distributed copies of all data changes as they are written. These copies are constantly checked for consistency and corruption. If a meteor were to wipe out the east coast, you won't lose your data. Continuous Protection come standard with every database running on Heroku Postgres.

100% Pure PostgreSQL

We believe deeply in the value of open-source software, and believe it is even more critical in the context of a database service. We are committed to running off-the-shelf, community PostgreSQL - unforked and unmodified. Rest assured that any standard libpq client will run flawlessly on our service. Should you ever decide that you want to go back to being your own DBA, you can; there is no technical lock-in.

Great Tools for Visibility and Productivity

Heroku Postgres has a rich API, accessible via command-line tools and a clean and simple web-based UI. Use it for provisioning, generating connection strings, viewing usage statistics, and capturing snapshots. These tools work for both standalone databases and those attached to Heroku apps.

The best way to learn more about the service is to give it a try, so check out postgres.heroku.com to signup and provision a database today.

Heroku Scheduler Add-on Now Available

Today we're happy to announce the availability of Heroku Scheduler. Scheduler is an add-on for running administrative or maintenance tasks, or jobs, at scheduled time intervals. It's the polyglot replacement of the Cron add-on, with more power and flexibility. And it's free; you just pay for the dyno time consumed by the one-off tasks.

A dashboard allows you to configure jobs to run every 10 minutes, every hour, or every day, and unlike the Cron add-on, you can control when. E.g. Every hour on the half-hour, or every day at 7:00am.

Scheduler Dashboard

Polyglot Tasks

Tasks are any command that can be run in your application or even the Unix shell.

For Rails, the convention is to set up rake tasks. To create your scheduled tasks in Rails, copy the code below into lib/tasks/scheduler.rake and customize it to fit your needs.

desc "This task is called by the Heroku scheduler add-on"
task :update_feed => :environment do
    puts "Updating feed..."
    NewsFeed.update
    puts "done."
end

task :send_reminders => :environment do
    User.send_reminders
end

If you're using Python with the popular Fabric automation tool, you can define a fab clean_sessions task:

from fabric.api import task

@task
def clean_sessions():
    url = urlparse(os.environ.get('REDISTOGO_URL'))
    db = redis.Redis(host=url.hostname, port=url.port, password=url.password)
    db.delete('myapp:sessions')
    print 'done.'

For apps built on other frameworks or languages, another convention is to add a script to bin/ that will perform the task. E.g. bin/updater.

Scheduling Jobs

To schedule a frequency and time for a job, open the scheduler dashboard by finding the app in My Apps, clicking "General Info", then selecting "Scheduler" from the Add-ons dropdown.

On the Scheduler Dashboard, click "Add Job...", enter a task, select a frequency and next run time.

Note that the next run time for daily jobs is in UTC. If you want to schedule the job at a certain local time, add the proper UTC offset.

For example, add rake update_feed, select "Hourly" and ":30" to update feeds every hour on the half-hour. Then add rake send_reminders, select "Daily" and "00:00" to send reminders every day at midnight.

Migrating From the Cron Add-on

Existing Cron add-on users should migrate to Heroku Scheduler as soon as possible. It has more functionality, is easier to use, and is free. Cron is restricted to running a single command, rake cron and does not provide control over when daily and hourly tasks are run. Scheduler can do everything the Cron add-on does, and more.

If you want your new jobs to be scheduled as close as possible to when your Cron jobs would run, go to the Cron dashboard and look at the "Scheduled for" information. Then in the Scheduler dashboard, create a new task, set it to be either hourly or daily, and then set the Next Run field to the selection closest to the previous scheduled time. Set the task to rake cron.

The Dodging Samurai Win the First Annual Octocat Dodgeball Invitational

So our good friends at GitHub put together the First Annual Dodgeball Invitational with the intention of pummeling various Bay Area tech companies into submission. Heroku has never shied away from a challenge, especially when there is a giant Octocat trophy at stake. The Dodging Samurai emerged victorious despite facing a pool of 21 teams and one crotchety old man wielding a wrench.



Through brute force, covert strategies and pink tutus, the Herokai were able to persevere through two teams from both GitHub and Engine Yard, a solid team from Code for America, as well a mysterious unnamed lucha libre fighter from Twilio. We commend both the spirit and the dodginess of all our competition and look forward to next year's event.

What's the best part? The bulk of the entry fees will now go to our charity of choice, Heifer International. $28,500 will now be donated to empower families around the world to become self-reliant through sustainable development projects.

The Octocat trophy has found a lovely home at our lunch table. We weren't sure if he was actually a cat in an octopus costume or a fully different species, but we bought catnip and various toys for him anyway. Check out this Quora thread about the Octocat's proud lineage.

Scala on Heroku

The sixth official language on the Heroku polyglot platform is Scala, available in public beta on the Cedar stack starting today.

Scala deftly blends object-oriented programming with functional programming. It offers an approachable syntax for Java and C developers, the power of a functional language like Erlang or Clojure, and the conciseness and programmer-friendliness normally found in scripting languages such as Ruby or Python. It has found traction with big-scale companies like Twitter and Foursquare, plus many others. Perhaps most notably, Scala offers a path forward for Java developers who seek a more modern programming language.

More on those points in a moment. But first, let's see it in action.

Scala on Heroku in Two Minutes

Create a directory. Start with this sourcefile:

src/main/scala/Web.scala

import org.jboss.netty.handler.codec.http.{HttpRequest, HttpResponse}
import com.twitter.finagle.builder.ServerBuilder
import com.twitter.finagle.http.{Http, Response}
import com.twitter.finagle.Service
import com.twitter.util.Future
import java.net.InetSocketAddress
import util.Properties

object Web {
  def main(args: Array[String]) {
    val port = Properties.envOrElse("PORT", "8080").toInt
    println("Starting on port:"+port)
    ServerBuilder()
      .codec(Http())
      .name("hello-server")
      .bindTo(new InetSocketAddress(port))
      .build(new Hello)
  }
}

class Hello extends Service[HttpRequest, HttpResponse] {
  def apply(req: HttpRequest): Future[HttpResponse] = {
    val response = Response()
    response.setStatusCode(200)
    response.setContentString("Hello from Scala!")
    Future(response)
  }
}

Add the following files to declare dependencies and build with sbt, the simple build tool for Scala:

project/build.properties

sbt.version=0.11.0

build.sbt

import com.typesafe.startscript.StartScriptPlugin

seq(StartScriptPlugin.startScriptForClassesSettings: _*)

name := "hello"

version := "1.0"

scalaVersion := "2.8.1"

resolvers += "twitter-repo" at "http://maven.twttr.com"

libraryDependencies ++= Seq("com.twitter" % "finagle-core" % "1.9.0", "com.twitter" % "finagle-http" % "1.9.0")

Declare how the app runs with a start script plugin and Procfile:

project/build.sbt

resolvers += Classpaths.typesafeResolver

addSbtPlugin("com.typesafe.startscript" % "xsbt-start-script-plugin" % "0.3.0")

Procfile

web: target/start Web

Commit to Git:

$ git init
$ git add .
$ git commit -m init

Create an app on the Cedar stack and deploy:

$ heroku create --stack cedar
Creating warm-frost-1289... done, stack is cedar
http://warm-frost-1289.herokuapp.com/ | git@heroku.com:warm-frost-1289.git
Git remote heroku added

$ git push heroku master
Counting objects: 14, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (9/9), done.
Writing objects: 100% (14/14), 1.51 KiB, done.
Total 14 (delta 1), reused 0 (delta 0)

-----> Heroku receiving push
-----> Scala app detected
-----> Building app with sbt v0.11.0
-----> Running: sbt clean compile stage
       Getting net.java.dev.jna jna 3.2.3 ...
       ...
       [success] Total time: 0 s, completed Sep 26, 2011 8:41:10 PM
-----> Discovering process types
       Procfile declares types -> web
-----> Compiled slug size is 43.1MB
-----> Launching... done, v3
       http://warm-frost-1289.herokuapp.com deployed to Heroku

Then view your app on the web!

$ curl http://warm-frost-1289.herokuapp.com
Hello from Scala!

Dev Center: Getting Started with Scala on Heroku/Cedar

Language and Community

Scala is designed as an evolution of Java that addresses the verbosity of Java syntax and adds many powerful language features such as type inference and functional orientation. Java developers who have made the switch to Scala often say that it brings fun back to developing on the JVM. Boilerplate and ceremony are replaced with elegant constructs, to express intent in fewer lines of code. Developers get all the benefits of the JVM — including the huge ecosystem of libraries and tools, and a robust and performant runtime — with a language tailored to developer happiness and productivity.

Scala is strongly- and statically-typed, like Java (and unlike Erlang and Clojure). Its type inference has much in common with Haskell.

Yet, Scala achieves much of the ease of use of a dynamically-typed language (such as Ruby or Python). Though there are many well-established options for dynamically-typed open source languages, Scala is one of the few with compile-time type safety which is also both practical and pleasant to use. The static vs dynamic typing debate rages on, but if you're in the type-safe camp, Scala is an obvious choice.

Language creator Martin Odersky's academic background shines through in the feel of the language and the community. But the language's design balances academic influence with approachability and pragmatism. The result is that Scala takes many of the best ideas from the computer science research world, and makes them practical in an applied setting.

Members of the Scala community tend to be forward-thinking, expert-level Java programmers; or developers from functional backgrounds (such as Haskell or ML) who see an opportunity to apply the patterns they love in a commercially viable environment.

There is some debate about whether Scala is too hard to learn or too complex. One answer is that the language is still young enough that learning resources aren't yet fully-baked, although Twitter's Scala School is one good resource for beginners. But perhaps Scala is simply a sharper tool than Java: in the hands of experts it's a powerful tool, but copy-paste developers may find themselves with self-inflicted wounds.

Scala Days is the primary Scala conference, although the language is well-represented at cross-community conferences like Strange Loop.

The language community has blossomed, and is now in the process of accumulating more and more mainstream adoption. Community members are enthusiastic about the language's potential, making for an environment that welcomes and encourages newcomers.

Open Source Projects

Open source is thriving in the Scala world. The Lift web framework is a well-known early mover, but the last two years have seen an explosion of new projects showcasing Scala's strengths.

Finagle is a networking library coming out of the Twitter engineering department. It's not a web framework in the sense of Rails or Django, but rather a toolkit for creating network clients and servers. The server builder is in some ways reminiscent of the Node.js stdlib for creating servers, but much more feature-full: fault-tolerance, backpressure (rate-limiting defense against attacks), and service discovery to name a few. The web is increasingly a world of connected services, and Finagle (and Scala) are a natural fit for that new order.

Spark runs on Mesos (a good example of hooking into the existing JVM ecosystem) to do in-memory dataset processing, such as this impressive demo of loading all of Wikipedia into memory for lightning-fast searches. Two other notable projects are Akka (concurrency middleware) and Play! (web framework), which we'll look at shortly.

The Path Forward for Java?

Some Java developers have been envious of modern, agile, web-friendly languages like Ruby or Python — but they don't want to give up type safety, the Java library ecosystem, or the JVM. Leaders in the Java community are aware of this stagnation problem and see alternate JVM languages as the path forward. Scala is the front-runner candidate on this, with support from influential people like Bruce Eckel, Dick Wall and Carl Quinn of the Java Posse, and Bill Venners.

Scala is a natural successor to Java for a few reasons. Its basic syntax is familiar, in contrast with Erlang and Clojure: two other functional, concurrency-focused languages which many developers find inscrutable. Another reason is that Scala's functional and object-oriented mix allows new developers to build programs in an OO model to start with. Over time, they can learn functional techniques and blend them in where appropriate.

Working with Java libraries from Scala is trivial and practical. You can not only call Java libraries from Scala, but go the other way — provide Scala libraries for Java developers to call. Akka is one example of this.

There's obvious overlap here between Scala as a reboot of the Java language and toolchain, and the Play! web framework as a reboot of Java web frameworks. Indeed, these trends are converging, with Play! 2.0 putting Scala front-and-center. The fact that Play! can be used in a natural way from both Java and Scala is another testament to JVM interoperability. Play 2.0 will even use sbt as the builder and have native Akka support.

Typesafe and Akka

Typesafe is a new company emerging as a leader in Scala, with language creator Martin Odersky and Akka framework creator Jonas Bonér as co-founders. Their open-source product is the Typesafe Stack, a commercially-supported distribution of Scala and Akka.

Akka is an event-driven middleware framework with emphasis on concurrency and scale-out. Akka uses the actor model with features such as supervision hierarchies and futures.

The Heroku team worked closely with Typesafe on bringing Scala to our platform. This collaboration produced items like the xsbt-start-script-plugin, and coordination around the release of sbt 0.11.

Havoc Pennington of Typesafe built WebWords, an excellent real-world demonstration of using Akka's concurrency capabilities to scrape and process web pages. Try it out, then dig in on the sourcecode and his epic Dev Center article explaining the app's architecture in detail. Havoc also gave an educational talk at Dreamforce about Akka, Scala, and Play!.

Typesafe: we enjoyed working with you, and look forward to more productive collaboration in the future. Thanks!

Conclusion

Scala's explosive growth over the past two years is great news for both Java developers and for functional programming. Scala on Heroku, combined with powerful toolsets like Finagle and Akka, are a great fit for the emerging future of connected web services.

Further reading:

Special thanks to Havoc Pennington, Jeff Smick, Steve Jenson, James Ward, Bruce Eckel, and Alex Payne for alpha-testing and help with this post.

Browse the blog archives or subscribe to the full-text feed.