One of the many benefits of Rails is database independence. Migrations are particularly nice in this regard; and the easy-to-read / Rubyified display of your schema (via rake db:schema:dump) in schema.rb is icing on the cake.

But what about data? For import and export of the actual data, we’re stuck with mysqldump (or pg_dump, if you’re so inclined). Further, these dump formats are not terribly readable, contain lots of information you may or may not want to copy (like permissions, schema settings, views, triggers…you know, database features that Rails users are supposed to avoid).

Worst of all, ddata dumps are vendor specific, so you can’t move data between databases (e.g., SQLite, MySQL, PostgreSQL). Rails is database-independent, but you’re pretty much stuck with whatever you picked at the start of the project. That’s hardly agile.

Realizing this problem was going to affect us in a pretty direct way with the import/export features of Heroku, Orion and I threw together a plugin yesterday which we’ve given the (rather uninspired) name YamlDb. Once you’ve installed it:

script/plugin install git://github.com/adamwiggins/yaml_db.git


(use http://github.com/adamwiggins/yaml_db.git for Rails < 2.1)

...you’ll have the additional rake tasks db:data:dump and db:data:load, which access a file data.yml. This is a simple dump of your database’s tables, which along with schema.rb, describe everything you need to make a complete copy of your database. (db:dump and db:load are shortcuts to do both schema and data together.)

What’s awesome about this is that you can switch databases in the blink of an eye. Got a SQLite3 database that’s gotten too big for its britches? rake db:dump, create a MySQL database, update your config/database.yml, and rake db:load. Presto, your app is now running in MySQL. Want to give PostgreSQL a try? Same process. Run it for a week, and decided you miss mysqladmin too much? Reverse the process, bringing the data which was stored in Postgres during that week back into MySQL.

Caveats: it’s much slower than dumping and restoring with your database’s native tool, and probably will stop working altogether with a database that is larger than RAM. But for most apps it should work fine. In the future we will most likely write database-specific extensions to make dump/restore process happen in chunks, and thus be more scalable.

As for Heroku, it now uses yaml_db as its native import/export format. So when you export an app from Heroku, you’ll always see a db/data.yml with your current data. It also makes it easy to import data into your Heroku app, at the same time you’re importing the code, or later on. Just install yaml_db in your app locally and run db:dump. Include data.yml with the import (or run and upload it separately), and then run db:load at the rake console.

What about gems, plugins, and different Rails versions?

We are definitely going to support gems and plugins. We are almost finished with a slick gem and plugin installer you can use for each app. In the meantime, you can install plugins by importing or uploading the files directly into vendor/plugins.

Currently, we only support the latest stable version of Rails. You can use a different version by uploading a frozen vendor/rails, but this may not work because of the 10MB storage limit constraint (rake rails:freeze:edge won’t work, by the way, because there are no outgoing network connections yet – we’ll discuss this in more detail in another post). We hope to be expanding our features in this area shortly.

Are my application and data secure?
I can execute arbitrary shell commands!

This is a brand new, limited beta, so we are not making any promises about security. That being said, we have worked hard to provide a secure environment, and we believe your application and data are completely secure.

Because we are providing a platform for executing ruby code, you have full access to system commands. While you don’t have an actual shell, you do have the ability to run shell commands via ruby. This is a feature, not a bug. We don’t believe you should need to run shell commands for any purpose, but giving our users a full ruby environment is important to us. Being able to run arbitrary shell commands is not a security threat the way the system is architected.

Can I have multiple environments, staging and production?
Can I run an app with the environment set to production?

Yes, we plan to offer multiple instances of your app and allow you to set the environment type for each. For now, however, each application has only one instance, and is always in development mode.

What about testing?

Testing is very very important to us and we plan to fully support it (we haven’t yet decided if this will be the standard Test::Unit or RSpec – email us if you have thoughts on the topic). We have several great tools and features planned for testing. For now, however, a testing database is not provided, so tests can’t be run. This is coming soon.

Hosting, performance, paid accounts?

We plan to offer multiple paid account types, with varying levels of features and performance. We will also always offer a free account. For right now, we are only offering one-size-fits-all free accounts.

How long before I get in?

We are sending out tons of invites every day. We’d prefer not to have a waiting list, but doing it this way allows us to let people in only as we’re sure our infrastructure can handle the load. The number of people we let in each day keeps increasing, as our existing users give us feedback (thanks guys!) which helps us improve our product for the next batch of people coming in.

One of the areas we are most interested in is collaboration – this is, after all, a web application. We want to encourage existing users to invite their friends to become users and start collaborating with them, so we have given existing users the ability invite friends directly. So if you want to get an account faster, ask a friend for an invite.

If you are on the waiting list, we haven’t forgotten about you. We will get to you soon, and we really appreciate your patience.

I found a bug, what do I do?
What kind of feedback do you want?

Whether you actively found a bug, something just doesn’t seem to work or doesn’t work as expected, you can’t do something you want to do, or you have feature requests, please please please ! The whole purpose of this limited beta is to get your feedback and use it to make Heroku awesome and rock solid.

What about my existing tools and editor?
Do you support subversion?
Can you make your editor better?

We are hardcore vi, TextMate, and subversion fans. We will not consider Heroku completely usable until editing and version control either works as well as those tools, or there is a way to continue to use those tools with our service.

The existing Heroku editor absolutely works for making Rails applications. We know because we use it. So we also know that it kind of blows, as real developer editors go (so far we do think we’ve beat Notepad, though perhaps just slightly). It is getting better all the time, and we have some specific reasons we want to make a browser-based editor, which we’ll talk about in another post soon.

In the meantime, there are a couple of things you can do:

Import & Export

If you want to edit locally and deploy to Heroku, you can make liberal use of the import and export features. Setup your whole app locally, then just import it into Heroku. Your archive will replace the existing code base. If you make changes on Heroku and want to continue working locally, just export the app.

Snapshots

Snapshots allow you to take a named snapshot of your app’s code and data at any point. You can take as many snapshots as you want for version control purposes. Take a backup snapshot before you import, or take a snapshot before you try some crazy edits.

Can I import my huge complicated app?
What about MySQL versus PostgreSQL?

Yes, you can import your huge complicated app. It may or may not work. If it doesn’t work, please about it. Also, there is currently a 10MB limit for the storage footprint of each app. If your uploaded app is larger, you will see an overlimit message.

We are using PostgreSQL internally for several really important reasons, but we do plan to fully support MySQL database dumps, both in and out. We are almost done with this feature, so look for it soon.

A couple of weeks ago we quietly started accepting signups for our private beta. We knew people would be excited about Heroku; I mean, we’re pretty excited about it. Word seems to have gotten out, as over the last several days well over 1,000 people have signed up.

We are inviting users off the waiting list all day, every day, as fast as we can. So we’re also receiving a constant stream now of feedback email. Some quotes:

OH – MY – GOD, Heroku! It’s absolutely great. In my opinion, it’s really a revolution in Rails development. – Chris

Wow. This is impressive, I like what I see. – Ben

This is exactly what I was looking for right now. I’ve started teaching myself ruby on rails, and it’ll be fantastic to be able to have something online to mess around with. – Micah

I am really excited about this project. It may become my de facto rails development environment. – Tony

Boy this looks awesome! When you see it, you go… “Well, of course! Why didn’t I think of that?” – Dirk

Wow what an awesome idea. – Bill

There have been some nice blog posts too, including Peter Cooper’s Ruby Inside.

We are glad to hear so many of you are so excited. Thanks to all of our users for your enthusiasm and feedback. We can’t wait to see how you feel about all the features we have planned…