At EmberConf Terence Lee and I had a chance to sit down with Tom Dale and chat about the history of Ember.js and where it’s headed now, including some details on the newly extracted Glimmer.js rendering engine. This post details a lot of the history of Ember, including some of the motivation that led the framework to what it is today. Watch the blog for the second portion of this interview with all of the details on Glimmer.js. The next post will also include the full audio of the interview, with many questions we opted to omit from the transcription to save valuable bytes.
Jonan: So, we're at EmberConf speaking with Tom Dale, who gave a keynote today with some important announcements. We're going to dig into those in just a minute here, but I’d like you to introduce yourselves please.
Tom: Sure. Hey, I'm Tom. I just started working at LinkedIn as a senior staff software engineer, and I work on a really awesome team that works on Ember infrastructure. As you may have seen, LinkedIn’s website now is one big Ember application. So my job is to make the army of engineers at LinkedIn productive, and make sure that we're able to build a really awesome web software.
Terence: I'm Terence, I do language stuff and Rails on the languages team [at Heroku].
Jonan: There's a third-party Ember buildpack that you worked on, right?
Terence: Yes. That has no JavaScript in it.
Jonan: No JavaScript at all? But it ships Ember. I shipped my first Ember app on it.
Tom: That's not true.
Terence: It is true.
Tom: It's all Ruby?
Terence: Oh, yeah.
Tom: Awesome. See that's great. You know what, Ember is a big tent, as DHH would say. Not about Ember, he would say that about Rails and then I would copy that because that's basically what we do. We just take what DHH says, and we repeat them in the context of JavaScript, and it sounds very thought leadery.
Jonan: Would you describe Ember as Omakase?
Tom: I would describe it as being bespoke, artisanal, shade-grown Omakase.
Jonan: That's even better. So on the subject of Ember.It's been around for awhile now. How old is Ember? Five years plus?
Tom: It depends on what date you want to use. So if you're talking about Ember 1.0, I think it's been about five years.
Terence: Do you include SproutCore in that?
Tom: I mean I think we should. There is no Ember without SproutCore, and to me SproutCore was one of the first libraries or frameworks to adopt this idea of client-side architecture. So one thing that we talked about in the keynote yesterday was just how much the web has changed in five years, right? So five years ago, IE was the dominant browser but actually, SproutCore had it way worse. And we're talking about IE6 and IE7 and talking about ambitious things, what we do on the web.
Jonan: And you did it in an era where browsers were not even close to where they are today.
Tom: Not even close, not even close.
Jonan: That's interesting. So then, from SproutCore, Ember comes out five years ago and we're off to the races. A lot changed in that first year, you went 1.0 and you’ve said that there were a lot of things that went wrong along the way. In your talk, you had a slide where you mentioned a few of those things. From the 10,000-foot view, what kind of lessons did you learn in those first 5 years?
Tom: JavaScript apps felt broken and people didn’t know why but people always said, "JavaScript apps feel broken, you know, for whatever reason, please don’t use them" right? And people wanted to shame you for using JavaScript. The reason for that, I think, is URLs. URLs are kind of the linchpin that holds the web together. And so much of the value of the web over native is these URLs, and JavaScript apps just ignored them. SproutCore ignored them, and almost every JavaScript framework did. So, what Ember had to do was figure out how to build JavaScript apps that don’t feel broken on the web. That’s where all this work with the router started.
Nowadays, routers are taken for granted. Every framework, every library has a router that you can drop into it. But there was actually some novel computer science work that went into it, in how we tie the architecture of the app to the URL. That took a long time and it was a very organic process. I don’t think we fully understood the magnitude of the research project that was going on. There are a lot of examples of that where we tackled problems for the first time, so of course, there's gonna be kind of an organic exploration of that space.
Another example of this is that when we adopted the six-week release cycle, this train model with Canary Beta and the release, the only other people doing it were Chrome and, I think, Firefox. And when we adopted it, it paid dividends right away, and I'm so happy that we adopted it. One constraint that we have that Chrome and Firefox don’t have as much is that for us, we're always shipping the framework over the wire every time a user visits your webpage, right?
Jonan: Right.
Tom: So it's very easy to have feature flags and to keep all the APIs around when you're distributing a binary. It's much harder when every time you do that, your file size goes up, and up, and up. And so what we've had to figure out is okay, "Well, we really liked this release train model. People really like the fact that it's backwards compatible. People really don’t like ecosystem breaking changes like Python 2 to Python 3 or Angular 1 to Angular 2. That doesn’t work so what do we do?"
You know, you feel kind of stuck. So we've had to figure out a lot of things. Like one thing that we've been working on is something called Project Svelte, which is the ability to say, "You can opt out of deprecated features and we will strip those from the build".
Jonan: But that's the only way that you can really move forward there. I mean if you've got to make this smaller, you can't just deprecate things arbitrarily.
Tom: Right.
Jonan: You can't make those decisions for your user. Your file size is ever growing, which when you're shipping over the wire, is not a great thing.
This has already, historically, been an issue for Ember, the size of the framework.
So what you are providing people now is a way to opt out of some those deprecated features. So say that, "All right, I've stopped using this API in my codebase, we can strip this out."
That's known as Project Svelte?
Tom: Yeah, that's Project Svelte. It's really important to remember that when Ember started, there were no package managers. NPM wasn’t 1.0 or just hit 1.0, and was not at all designed for frontend packages. It didn’t do any kind of deduplication and distributing.
This is back in the day when the way that you got a library was you Googled for the website, you found it, they gave you a script tag to just drop in. I'm sure you all agree that's a horrible way to do dependency management.
So we felt compelled to say, "Well, if we wanna make something… If we want people to actually use something, we have to bake it in." Because when you're gathering all your dependencies by hand, you're only gonna have, you know, four or five of them. You're not gonna go get a million dependencies. Of course, that has changed dramatically and we have new technology like Yarn, which is more like a Cargo/Bundler style of dependency resolution for JavaScript.
What we found has not worked is trying to do big-design, upfront projects, because anything that we land in Ember gets that guarantee of stability and compatibility.
People feel a very strong sense of responsibility, that if we land this feature, this has to be something that we are ready to support for the foreseeable future, and that just takes longer. It's the same reason standards bodies move relatively slowly.
Jonan: Right. Now, this is something you brought up in your keynote. Rather than architecting or spending a huge amount of time and investment upfront architecting your system, you want to get it out in front of the customers as early as possible. But that conflicts with the idea that you're trying to present stable products, things that won't change, right?
Terence: Stability without stagnation is the tagline right?
Tom: Right. So that's the message but then we also know that you can't do a big design upfront, and you're not gonna get it perfect the first time. You ship an MVP and iterate.
So how do you balance this tension? If you look at the projects we've embarked on in the last couple of years, there have been some projects that were more big design upfront. And those have largely stagnated and failed because of the fact that we just couldn’t get consensus on them.
Then you have some other projects like Ember Engines and FastBoot. What we actually did was look at how web standards bodies work -- CC39, W3C, WHATWG.
There's something called the "Extensible Web Manifesto," which you may have seen, that says "Hey, standard bodies, open source libraries are able to iterate a lot faster than you are. So instead of focusing on building these big, beautiful, really-easy-to-use declarative APIs, give us the smallest primitive needed to experiment on top of that."
That’s something that we really take to heart in Ember 2. If you think of Ember as being this small stable core, what we can do is expose just the smallest primitive that you need, and then we can let the experimentation happen in the community.
So with FastBoot, for example, FastBoot is this entire suite of tools for deploying server-side rendered, client-side apps. You can easily push it to Heroku and, boom, it starts running, but that doesn’t need to live in Ember. We can do all the HTTP stuff, all of the concurrency stuff. All of that can live outside of Ember, all Ember needs to say is, "Give me a URL and I will give you the HTML for that back."
So that's what we did. There's this API called Visit, the ‘visit’ method. You call it, you give the URL, you get HTML back, and it's so simple and you can easily have discussion about it.
You can understand how it's gonna operate and that's the thing that we landed. Then that's given us a year to experiment in FastBoot and make a lot of really important changes.
Jonan: You were able to hide the complexity away behind this simple API.
Tom: Right.
Jonan: So some of the things that more recently you mentioned in your keynote as not having gone well, were Ember Pods, for example, and now we have Module Unification. So if I understand correctly, Ember Pods was a way to keep all of your component files related to a single component in one location?
Tom: Right. The Rails style where you have one directory that's all controllers and one directory that's all views or templates, which is how Ember started. It's still the standard way, the default way you get when you create a new Ember app.
People found it more productive to say, "I'm gonna have a feature directory", where you have your component and that component might have style. It might have JavaScript, it might have templates. I think it's just easier for people to reason about those when they're all grouped together, instead of bouncing around.
Jonan: I love this idea. When I first came into Rails, I distinctly remember going from file to file and thinking, "Where even is this thing. How do I find this?"
So you had said that Ember Pods, maybe, didn’t seem to take off? It wasn't a very popular solution to that problem, and now we have Module Unifications. How is that different?
Tom: I actually think that Pods was popular, it actually was very popular. So, there's something DHH says: "Beginners and pro users should climb the mountain together."
I think it's a bad sign, in your framework, if there's the documented happy path that beginners use, and then at some point, they fall off the cliff and see "Oh, actually there's this pro API. It's a little bit harder to use but now that you're in the club, now you get to use it". I think that leads to very bad experiences for both. You kind of wanna have both sets of people going up the same route.
So Pods is almost this secret opt-in handshake. And it was just one of those things where it started off as an experiment but then slowly became adopted to the point where, I think, we didn’t move fast enough.
Jonan: I see.
Tom: We didn’t move fast enough and now, there's almost this bifurcation between apps that are not using Pods and apps that are using Pods.
So with Module Unification what we did is we sat down and we said "OK, Pods was a really nice improvement but it didn’t have a ton of design applied to it. It was the kind of thing that evolved organically. So let's just sit down and try to design something."
For us, it was really important with Module Unification to say, "Not only does it need to be good but we need to have a way of being able to automatically migrate 99% of the Ember apps today. We should have a command that will just migrate them to the new file system."
So one thing that's really neat is that you can just have a component where all you have to do is drag it into another component's directory and now it's scoped. It's almost like a lexical scope in a programming language. We're using the file system to scope which components know about each other.
Jonan: So, forgive my simplification here but I'm not great at Ember. If I have a login component and it's a box to just log in, and inside of it I wanted to have a Google auth button and a Twitter auth button, each of those could be independent components.
Maybe I wanna reuse it somehow. I would drag them into my login directory and that makes them scoped, so we can't use them somewhere else.
Tom: Right. That ends up being pretty nice because often, what you'll do is you'll create a new component, give it a really nice and appropriate semantic name and, oops, it turns out your coworker used that for another page, a year ago. Now, you can't use it, because it’s completely different.
Jonan: So I've got my Ember app and I've been using Pods all this time, and now, we have Module Unification and there's a new way to do this. I can just move over to module unification right?
Tom: Yes.
Jonan: We run this script that you've written and it would migrate me over?
Tom: Yeah. So we have a migrator and because there's so many Ember apps using the classic system, so many Ember apps using the Pod system, it can handle both.
Terence: Could Module Unification have happened without Ember Pods happening first?
Tom: It's hard to say. I think it's something that people really wanted, and I think it's fantastic. This is something we touched on the keynote; one thing that we've always said about Ember, and I think this is true about Rails also, is that there's always a period of experimentation when something new comes along. You really want that experimentation to happen in the community. Then eventually, it seems like one idea has won out in a lot of ways. The things that we learned about with Pods fed directly into Module Unification design.
Jonan: So maybe, we could chat a little bit about deprecating controllers in Ember?
Tom: Sure, yeah.
Jonan: You announced that you were going to deprecate all of the top-level controllers by 2.0, and then pushed 2.1 and 2.2. That's still the plan to deprecate the controllers someday?
Tom: I think what we are always dedicated to is trying to slim down the programming model and always reevaluate what is the experience like for new people. I don’t want to say that we're going to deprecate controllers because that sounds like a very scary thing, right? There's a lot of people with a lot of controllers in their apps. But I do think what we will want to do is take a look at the Ember programming model from the perspective of a new user. And say, "Well, it seems like people already learned about components. And it seems like there's probably some overlap between what a controller does and what a component does."
So maybe there's some way we can unify these concepts so people don’t have to learn about this controller thing with its own set of personality quirks.
Jonan: Is this where routable components fit into the idea then?
Tom: So that's the idea of routable components and I think I don’t have a concrete plan for exactly how this is going to work. I think a lot of ways, the work that we want to do on that was blocked by the Glimmer component API.
I think what we'd like to do is add whatever low-level hooks in Ember are needed so that we can maybe do some experimentation around things like routable components outside. Let people get a feel for it and then once we have a design that we're really happy with, then we can land it back in mainland Ember.
That’s the end of our discussion on the history and direction of the Ember project. Stay tuned for part two and learn more about the Glimmer.js project.