Access to application logs on Heroku has historically been one of the least usable functions of the platform. The “heroku logs” command was nothing more than a broadcast fetch of the logfiles for every web and worker dyno in your app. This worked ok for small apps, but the user experience became very poor once you got past five or ten dynos.
I’m incredibly excited to announce that today we’re rolling out the public beta of our new logging add-on. This is a whole new way of capturing and collating logs, based on a syslog routing layer we’ve dubbed Logplex. Logplex routes syslog traffic the same way that our HTTP routing mesh routes HTTP traffic. Not coincidentally, Logplex was built by Orion Henry and Jacob Vorreuter, the same geniuses behind the Heroku HTTP routing mesh.
Using Logplex, we can route logs not only from all of your application processes (web, worker) into a single location. Even better, infrastructure components in the platform that interact with your app can also write to the same place. For example, here’s a snippet of logs from a single HTTP request to a Rails app:
$ heroku logs 2010-10-21T14:11:16-07:00 app[web.2]: Processing PostController#list (for 208.16.84.131 at 2010-10-21 14:11:16) [GET] 2010-10-21T14:11:16-07:00 app[web.2]: Rendering template within layouts/application 2010-10-21T14:11:16-07:00 app[web.2]: Rendering post/list 2010-10-21T14:11:16-07:00 app[web.2]: Rendered includes/_header (0.1ms) 2010-10-21T14:11:16-07:00 app[web.2]: Completed in 150ms (View: 27, DB: 121) | 200 OK [http://myapp.heroku.com/] 2010-10-21T14:11:16-07:00 heroku[router]: GET myapp.heroku.com/ dyno=web.2 queue=0 wait=0ms service=251ms bytes=24523 2010-10-21T14:11:16-07:00 heroku[nginx]: GET / HTTP/1.1 | 208.66.27.62 | 247 | http | 304
The logs show not only the Rails logs from the dyno which served the request (web.2), but also logs from the HTTP routing mesh and Nginx. We also log events like web dynos idling and un-idling, process restarts, and more.
The logging add-on also adds some seriously bad-ass client-side features. You can filter down to an individual source or process (for example, just your app logs, or just the logs from worker.1). And realtime tail (heroku logs —tail) is spectacular for any app that’s doing heavy background processing. The docs have all the details.
Keep in mind that this is a beta feature, so there will be a few rough edges. During the beta process we need to prove that Logplex can scale to handle the huge volume of logs being generated by all the apps running on Heroku. We’re also working on more options for long-term archival and search of app logs. Once these questions are answered we’ll be able to remove the beta label, at which time the logging:basic add-on will become the default for all new apps.