The Heroku HTTP API Toolchain

Today we’re open sourcing the toolchain Heroku uses to design, document, and consume our HTTP APIs. We hope this shows how Heroku thinks about APIs and gives you new tools to create your own.

This toolchain includes:

An HTTP API design guide, describing how we structure both internal and public-facing APIs and document them using the JSON Schema standard.
A tool for working with JSON schemas and using them to generate API documentation.
Ruby and Go client code generators for APIs with JSON schemas.

Here’s some more information about these things, how we use them at Heroku, and an explanation of how you can try them yourself.

JSON Schema Foundation

We’ve developed the toolchain around the the JSON Schema standard for describing HTTP+JSON APIs. Having a consistent way to describe APIs gives us a powerful starting point for the toolchain described below.

HTTP API Design Guide

The HTTP API Design Guide shows how we design and document APIs at Heroku.

We use this guide to increase the design quality of the APIs we deliver, both for our user-facing products and for internal services. A concrete guide also minimizes time spent bikeshedding API design details and maximizes time spent on actual business logic of our apps.

This document includes guidance on versioning, resource structure, attribute naming, serialization, error handling, request ids, pagination, and caching support. It also describes how we use JSON Schema to describe our APIs in a machine-readable way, and generate API documentation from those schemas.

This document is a work in progress. We welcome discussion and contributions; feel free to open an issue on the GitHub repository.

Schema and Documentation Toolchain

The JSON Schema format provides a great machine-readable description of an API. As a complement to this, the JSON Schema management tool prmd (“pyramid”) helps you bootstrap a schema description, verify its completeness, and generate documentation from the specification.

For example, you can build up a schema with:

$ gem install prmd
$ mkdir schema/ schemata/
$ prmd init app  > schemata/app.json
$ prmd init user > schemata/user.json
$ vim schemata/{app,user}.json
$ cat <<EOF > meta.json
{
  "description": "Hello world prmd API",
  "id": "hello-prmd",
  "links": [{
    "href": "https://api.hello-prmd.com",
    "rel": "self"
  }],
  "title": "Hello Prmd"
}
EOF

Then combine into a final schema file and verify its correctness:

$ prmd combine -m meta.json schemata/ > schema.json
$ prmd verify schema.json

And finally build Markdown docs for your API:

$ prmd doc schema.json > schema.md

You end up with two key artifacts:

schema.json: machine-readable description of your API
schema.md: human-readable documentation for your API

Auto-generated Clients from API Schemas

One benefit of using JSON Schema is being able to automatically create API clients for your service. These clients help you quickly get started with an API in the language of your choice, and can also increase consistency of client usage across different services.

We’ve developed example auto-generating clients for both Ruby and Go.

Here’s an example of generating a Ruby client using Heroics for the Heroku Platform API:

$ curl -o heroku.json https://api.heroku.com/schema \
  -H "Accept: application/vnd.heroku+json; version=3"
$ gem install heroics
$ heroics-generate \
  -H "Accept: application/vnd.heroku+json; version=3" \
  Heroku heroku.json https://api.heroku.com \
  > heroku.rb

And here’s an example of generating a Go client with Schematic:

$ go get -u github.com/interagent/schematic
$ schematic heroku.json > heroku.go

Usage at Heroku

We’ve been using this toolchain at Heroku for both user-facing and internal APIs.

For the new Heroku Platform API we use JSON schema to describe the endpoints, prmd to generate API documentation, and Heroics for the Ruby client.

We also use this toolchain for internal services. We find that symmetry between our external and internal APIs increases our ability to reuse design practices and tooling, and also helps us raise the quality bar on our internal APIs.

At a higher level, JSON schema and associated generated docs are emerging within Heroku as a shared language that we can use to talk about API designs. To propose a new API or a change to an existing one, we present a JSON schema or a diff against an existing one, respectively. This practice sharpens our API design discussions and makes it easier to avoid miscommunications.

We’ve also seen interest in this toolchain from API developers outside of Heroku. We’d love to see more external adoption of this toolchain and welcome discussion and feedback about it. To facilitate this, we’ve created an independent GitHub organization at github.com/interagent as a home for this work and the discussion around it.

Summary

Well-designed and documented APIs are a great investment. The API toolchain described here can help you deliver them. We hope you’ll:

Check out the design guide
Try the prmd JSON schema tool
Experiment with auto-generated API clients using Heroics and Schematic

We’d love to hear your feedback; please feel free to open issues on any of the above repositories.

Video Transcript