Ember Blog

Ember Data Progress Update

2013-05-03T00:00:00Z

Just over a month ago, we told you about our plans for stabilizing Ember Data. I'd like to give you an update on the status of those efforts.

First, though, I'd like to thank everyone who has been contributing to Ember Data. As an open source project, a healthy, active community is our lifeblood. In particular, I want to call out Igor Terzic, Stefan Penner, Paul Chavard and Gordon Hempton for going above and beyond the call of duty.

Thanks to their efforts, we've been making steady progress towards triaging and reviewing pull requests and issues.

Additionally, Yehuda and I have been able to devote significant time to working on Ember Data thanks to the financial support of Addepar. They have my sincere thanks for being a company that understands the importance of investing in open source.

A New Focus

Imagine that you are starting a new job as a developer on an existing server-side web application. On the first day, you begin the process of familiarizing yourself with the codebase.

Since you have experience with the server-side MVC framework that this company uses, it's pretty easy going. You see that the templates are cleanly separated from the controller logic, which is itself cleanly separated from the models.

But, wait: something looks amiss in the model code. You take a closer look.

Instead of using an ORM (like ActiveRecord), which queries the database for you and automatically turns the results into objects, it looks like they're issuing queries manually.

"No problem," you think, "I can write SQL with my eyes closed."

But something doesn't look quite right. Upon closer examination, you see that database queries aren't written in SQL, but some query language you've never seen before.

You go to your manager's office and ask her about it.

"Oh yeah," she says, "We didn't have time to learn how to setup an existing database, and our needs were pretty simple at the time. We decided to roll our own, lightweight database with a simplified query language. I guess it sort of just grew over time as our needs grew."

Horrified, you tell her that you don't think it's going to be a good fit, turn in your badge, and head to your car.

Unfortunately, when you land your next job, the above scenario plays out all over again. Instead of using SQL, you discover that they too have rolled their own custom, ad hoc querying language! This repeats, over and over, until your descent into madness is complete and you retire to a cabin in the woods.

While this hopefully sounds like something out of a nightmare, the sad fact is that this is precisely the state of the art when it comes to writing web JSON APIs. Whether you're writing an Ember.js application, a mobile app for Android or iOS, or a desktop app, anything that consumes a web API typically has to deal with URL patterns and JSON structures that are ad hoc and informally-specified.

This is bad. Even though we have figured out a standard way of describing models and their relationships on the server, getting them into the browser requires every developer write custom, imperative code. It's a colossal waste of human time and energy.

The good news is that the solution should now appear obvious: if you have a conventional framework on the server and a conventional framework on the client, they should be able to communicate, automatically, using a standard interchange format.

The reality, however, is that Ember.js developers need to use existing APIs that aren't 100% consistent. While we were building towards the future, we were also trying to build something that could be used in the interim. In retrospect, trying to serve two masters simply muddled our message, confused users, and complicated the architecture.

Instead, Ember Data will now focus on being the best possible library for Ember.js apps to communicate with consistent, conventional APIs.

We will leave bridging the gap in the very capable hands of simpler libraries that require less conceptual overhead to work with custom JSON APIs, such as:

To be clear: Ember Data will not be limited to one particular convention. We are excited to see the number of third-party adapters, such as those for Parse, Django, localStorage, and Firebase, continue to grow. The only important feature is that the interchange between Ember Data and the backend be consistent across all models and their relationships.

What Does This Mean?

Most users of Ember Data will not have to make any changes to their application. If you are using the REST adapter with something like the ActiveModel::Serializers gem in Ruby on Rails, you will continue to not have to write any network code—defining your schema in JavaScript is enough for us to know how to load data from the server.

The only notable immediate change is that we will be removing the BasicAdapter that we described in last month's blog post. This adapter was designed to be a stopgap measure to help people use Ember Data with ad hoc JSON APIs, but we committed the cardinal sin of inventing an API instead of extracting it from a real application as we usually do.

Developers who had tried out the BasicAdapter appreciated our efforts, but found it lacking in several important areas. We believe focus is important, so we will be focusing our efforts on conventional APIs and will leave the cases previously covered by the BasicAdapter to other libraries.

The other change is that we will be updating the documentation on the Ember.js website to reflect this new focus, as well as providing documentation on how to connect Ember apps to less well-defined JSON APIs.

JSON API

In order for Ember Data apps to communicate with a server automatically, the interchange format must be semantically rich enough to capture all of the changes to models and their relationships that you may make on the client.

Over time, we have been building more functionality into the JSON format used by the RESTAdapter. We have decided to formalize this JSON format so that any backend, whether it be Ruby on Rails, Django, node.js, or your other favorite server technology, can interact with Ember.js apps without users having to write network code.

We have begun the process of documenting this format at JSONAPI.org with Brian Shirai. This is a living document that formally describes the JSON format we hope that the authors of server-side frameworks will use to get seamless compatibility with Ember.js applications. And, of course, other JavaScript framework are encouraged to adopt this format so that they can get the same benefit.

Because the best standards are driven by real implementations, Steve Klabnik and Santiago Pastorino (both major contributors to Ruby on Rails) have volunteered to integrate this specification into the rails-api and ActiveModel::Serializers projects. This proof-of-concept will allow us to verify the specification is solving real-world problems.

This work will make it extremely simple for Ember.js developers to integrate with Ruby on Rails, and we hope other server-side frameworks vendors follow suit.

Of course, the RESTAdapter will continue to be configurable, so if you want to use a JSON API that differs from the default specification, but is still consistent across all of your models, you don't have to write an adapter from scratch.

Regular Releases

Now that Ember Data is stabilizing rapidly, we will start doing regular beta releases to make it easier for users to communicate about different versions. The first release, which we'll be releasing in the next few days, will be version 0.13. In keeping with the SemVer standard, we reserve the right to change public APIs until we reach a 1.0.0 candidate. That being said, we will try our best to continue keeping the app-facing API stable.

We're excited about the future of Ember Data. In the same way we fought for strong application architecture conventions in the browser, we will continue to fight for better compatibility between the client and server. Hopefully soon, writing low-level XHR code will be an optional optimization, not a requirement.

Ember 1.0 RC3

2013-04-21T00:00:00Z

Today we are announcing the third Release Candidate of Ember 1.0. As we said when we released RC1, all releases until 1.0 is final will be about bugfixes and improvements, and should not have any breaking changes.

Performance

Since RC2, a series of performance improvements have landed.

When adjacent child views are scheduled for insertion they are are now batched together, and inserted together rather then one at a time.
Object deletion is now async, this prevents un-needed churn when destroying large interconnected object graphs.
The number of objects allocated during view rendering has been cut down.

The ongoing performance effort is being championed by Erik Bryn and Kris Selden.

Testing

There have been many fixes and improvements to App#reset, which should now correctly clear application state when invoked. Typically this is only used for integration tests.

  
  // before each integration test
App.reset();

Please Note: App.reset() now brings its own run loop, and no longer needs to be wrapped in one.

Initial work on a new ember-testing package has begun. The goal of this project is to make testing Ember applications easier, especially when it comes to testing asynchrony. As this effort matures, expect full documentation, examples and guides.

Props to Erik Bryn for researching integration testing approaches by many community members, including Jo Liss and Katie Gengler, and spearheading initial work on what will be our official library.

Ember Builds

Each successful CI run now publishes its build results to http://builds.emberjs.com/. This should make it much simpler to reference and use the the latest ember build.

Thanks to Stanley Stuart, Luke Melia, Erik Bryn and others for getting this set up.

New Input and TextArea helpers

TextField, TextArea and Checkbox views now have corresponding handlebars helpers.

  
    {{view Ember.TextField valueBinding="name"}}
  {{view Ember.Checkbox  checkedBinding="isActive"}}
  {{view Ember.TextArea  valueBinding="name"}}

can now be expressed as:

  
    {{input value=name}}
  {{input type=checkbox checked=isActive}}
  {{textarea value=name}}

We recommend using the "dynamic tag" forms rather than the {{view}} forms because they are equivalent to the static tags that we all know and love.

Note that when using dynamic tags, you do not need to use a Binding suffix and must leave out the quotation marks around the values. Ember will interpret quoted strings as static strings in this context.

Documentation

The API docs have been refined and filled in by more community members than we can mention here. Thank you everyone for helping out!

Changelog

The full CHANGELOG is available on Github, as always.

Announcing the Ember.js Security Policy

2013-04-05T00:00:00Z

We know that building your apps on top of a framework requires trust, and that trust is never put to the test more than when security vulnerabilities are discovered.

While we're very fortunate to work on an open source project that runs in a sandboxed environment, the browser, we realize that even JavaScript applications can be vulnerable to attacks from malicious third-parties.

Ember.js is designed to mitigate common forms of attack. For example, all values rendered using Handlebars are automatically escaped to prevent XSS attacks, and developers must explicitly opt in to outputting raw HTML.

To ensure that Ember applications stay safe, today we're announcing the Ember.js Security Policy, to help security researchers and developers responsibly disclose potential vulnerabilities in Ember and Ember Data.

We have also set up the Ember.js security announcements mailing list. This is an extremely low-traffic mailing list reserved solely for announcing security releases of the framework. If you're deploying Ember to production, you or your security team may wish to subscribe.

To be clear, there are no vulnerabilities we're aware of at this time and there is not a security release forthcoming. We just take security extremely seriously and believe that having a procedure in place ahead of time will allow us to respond most effectively should the worst happen.

If you have any questions or concerns that are not addressed by the new security policy, please email us at security@emberjs.com.

I'd like to thank the Ruby on Rails security team, from whom our security policy was lifted almost wholesale, for serving as a role model for open source projects everywhere.

Lastly, my personal thanks to Aaron "tenderlove" Patterson and Tony "bascule" Arcieri for reviewing our policy and answering many of my ignorant questions.

Let's stay safe out there.

Ember 1.0 RC2

2013-03-30T00:00:00Z

Today, we're releasing the second Release Candidate of Ember 1.0. As we said when we released RC1, the next few releases will be about bugfixes and improvements, and should not have any breaking changes.

Namespace Lookup for Controllers

It is now possible to look up controllers that are included in a namespace other than the main application namespace.

For example, when using the render helper:

  
  {{render 'posts'}}

This will render the posts template with the controller App.PostsController, where App is the main application namespace.

If you want to use an alternative namespace, you can use a /-separated path.

  
  {{render 'blog/posts'}}

This will render the blog/posts template with the controller Blog.PostsController.

Using Render With Multiple Models

In RC1, you could use {{render 'posts'}} to render the posts template using the app's instance of App.PostsController. Using render in this way will always render the template using the same singleton controller.

In RC2, we are adding the ability to render a template with a particular model: {{render 'post' post}}. When you supply a model, Ember.js will create a new instance of the App.PostController each time it is used.

This allows you to use {{render}} in a loop:

  
  {{#each post in posts}}
  {{render 'post' post}}
{{/each}}

Unwrapping an `ObjectController`'s Model

When passing an ObjectController as a parameter to the action or linkTo helpers, Ember now unwraps the underlying model and passes it through.

Most importantly, this allows you to add an itemController to {{#each}} without affecting action handlers in your controller or router.

If you have a template like this:

  
  <!-- posts.handlebars -->

<ul>
{{#each controller itemController='postItem'}}
  <li><button {{action selectPost this}}>{{name}}</button></li>
{{/each}}
</ul>

Your App.PostsController would look like this:

  
  App.PostsController = Ember.ArrayController.extend({
  needs: 'currentPost',

  selectPost: function(post) {
    // `post` here is an `App.Post`, not an
    // `App.PostItemController`
    this.set('controllers.currentPost.model', post);
  }
});

This should make using itemController less gotcha-prone.

Support for jQuery 2.0

Ember 1.0 RC2 supports jQuery 2.0.

This allows you to use a smaller, optimized build for newer browsers that is fully compatible with Ember.

If you're targeting both older IE and modern browsers, you can use the following snippet to get the best of both worlds:

  
  <!--[if lt IE 9]>
    <script src="jquery-1.9.1.js"></script>
<![endif]-->
<!--[if gte IE 9]><!-->
    <script src="jquery-2.0.0.js"></script>
<!--<![endif]-->

Because jQuery 1.9 and 2.0 are API-compatible, this strategy will allow you to target the widest range of browsers, but ship a smaller build for modern browsers.

The future is here!

Changelog

The full CHANGELOG is available on Github, as always.

Stabilizing Ember Data

2013-03-22T00:00:00Z

Yesterday, we gave you an update on our progress making Ember.js easier to use. One thing we didn't discuss was our plan for Ember Data.

It's no secret that, while many developers are building awesome apps with Ember.js, Ember Data still causes lots of frustration due to bugs and a changing, complex API. Documentation about it is also mixed in with Ember.js documentation, making it difficult for new developers to understand what is stable and what is not.

To be clear, Ember Data is not a dependency of Ember.js. Discourse, for example, uses its own, simple wrapper around $.ajax.

Even though Ember Data is not a dependency of Ember.js, loading data from the server is an extremely important part of most web applications, and it's a problem that every Ember.js application will have to deal with.

Our long-term goal is simple: we don't think most web developers should have to write any custom XHR code for loading data. Strong conventions on the client and strong conventions on the server should allow them to communicate automatically.

We know we're not there yet.

In order for this to work, there are many necessary features that must be rock-solid across all sorts of different persistence layers—local storage, relational databases, and key-value stores, to name a few. To top it off, the asynchronous environment of the browser (with an often unreliable internet connection) adds significant complexity, and means we can't simply port the solutions to these problems that have been pioneered on the server.

Getting all of these features working well together is a challenging problem, and we have not been able to deliver everything we thought we could in a reasonable amount of time.

Additionally, many developers are writing web applications that need to consume an existing JSON API that evolved organically and is not consistently named or structured. The API we have provided so far is suboptimal for this task.

To make the experience of writing an Ember Data application less frustrating, we're doing two things:

We're identifying a subset of features that already work reliably.
We're introducing a new, simpler API for working with remote data that makes fewer assumptions. You still get to use the model API in Ember Data, but can "bring your own $.ajax" to load and store records.

Right now, what gets documented is somewhat ad-hoc. Going forward, we will heavily document the stable features. Finding documentation about an Ember Data feature on the emberjs.com website will be your indication that we consider it stable and safe to use.

Over time, after we have put new features through their paces and written extensive documentation, the set of stable features will grow. In other words, we are refocusing on a small core of practical features, which we will slowly iterate towards our long-term, ambitious goal.

The Basic Adapter

The Ember Data adapter layer, which is responsible for finding and saving records, is currently designed to make it easy to build reusable adapters, like the ones people have written for Parse or CouchDB. It is not well-suited for delegating out to $.ajax to work with an API that is not 100% consistent.

To make it easier to use Ember Data with any kind of JSON data, we are introducing the Basic Adapter, which simply delegates to a sync object on your model.

Let's look at an example of using the Basic Adapter with the Twitter API.

First, note that the syntax for defining a model hasn't changed (historically, this has been one of the most stable parts of Ember Data):

  
  var attr = DS.attr, hasMany = DS.hasMany, belongsTo = DS.belongsTo;

App.User = DS.Model.extend({
  defaultProfileImage: attr('boolean'),
  description: attr('string'),
  screenName: attr('string'),
  isVerified: attr('boolean'),
  createdAt: attr('date'),

  tweets: hasMany('App.Tweet')
});

App.Tweet = DS.Model.extend({
  coordinates: attr('point'),
  createdAt: attr('date'),
  isFavorited: attr('boolean'),
  retweetCount: attr('number'),
  text: attr('string'),
  isTruncated: attr('boolean'),

  replyTo: belongsTo('App.User'),
  user: belongsTo('App.User')
});

Finding records also hasn't changed:

  
  // use the Promise API
App.User.find(userId).then(function(user) {
  return user.get('tweets');
}).then(function(tweets) {
  // do something with `tweets`
});

When you request the User and then its tweets, Ember Data will make calls to your models' sync object.

  
  App.User.sync = {
  find: function(id, process) {
    $.getJSON("/users/show", { screen_name: id }).then(function(user) {        
      process(user)                 
        .primaryKey('screen_name')
        .camelizeKeys()
        .applyTransforms('twitter')
        .load();
    });
  },

  findTweets: function(user, name, process) {
    var screenName = user.get('id');

    $.getJSON("/statuses/user_timeline", { screen_name: screenName }).then(function(timeline) {
      process(timeline)
        .primaryKey('id_str')
        .camelizeKeys()
        .applyTransforms('twitter')
        .munge(function(json) {
          json.isTruncated = json.truncated;
          json.replyTo = json.inReplyToScreenName;
        })
        .load();
    });
  }
};

As you can see, in each of the hooks on the sync object, the last argument passed in is a function called process. You use this function to load JSON data returned from the XHR into the store. It also includes several conveniences for common transformations, like camelizing property names and transforming values like dates.

Of course, you are not required to use these conveniences. You can write whatever imperative code you'd like to transform the JSON returned from the server into the form that Ember Data is expecting. Here is the above example re-written without using the chained conveniences.

  
  findTweets: function(user, name, process) {
  var screenName = user.get('id');

  $.getJSON("/statuses/user_timeline", { screen_name: screenName }).then(function(timeline) {
    var tweets = timeline.map(function(json) {
      // Map primary key
      json.id = json.id_str;

      // Camelize property names
      for (var prop in json) {
        var value = json[prop];
        delete json[prop];
        json[camelize(prop)] = value;
      }

      // Convert string-formatted date to object
      json.createdAt = Date.parse(json.createdAt);

      // Convert hash to JavaScript object
      json.coordinates = new Twitter.Point(json.coordinates);

      // Rename properties
      json.isTruncated = json.truncated;
      json.replyTo = json.inReplyToScreenName;

      return json;
    });

    process(tweets).load();
  });
}

Timeline

We have been working on this new API part-time for the past few weeks. You can see our progress on Ember Data's master branch, by looking at the tests or the implementation.

Over the next few weeks, we will be writing documentation that will be available in the Ember.js Guides. Once a few people have had the opportunity to use the Basic Adapter and sanity-check our work, we will start cutting beta releases of Ember Data. We think that this will be a lot easier for new developers than "make a build from master."

Our thanks go out to John McDowall, who has been tracking our progress on Basic Adapter and writing documentation to go with it.

Finally, we'd like to give a big thanks to Addepar for financially supporting us while we do this work. They are big users of and contributors to Ember Data, and we couldn't do it without them.

Making Ember.js Easier

2013-03-21T00:00:00Z

We frequently receive feedback from new developers about how frustrating it can be to get started with Ember. Yesterday, one of the most active comment threads on Hacker News was largely about just that.

We hear you loud and clear. Ember.js is not easy to get started with, and we take that very seriously. We are all working nights and weekends to make the framework as approachable as humanly possible.

One of the Hacker News commenters, kanja, nailed it. In response to another commenter who felt like the negative reaction was strong:

It's incredibly out of proportion - but people really want to use ember (because it promises all these great things!) and they're frustrated by the docs (because they're not really good for first time users) so this is kind of a flash point.

Absolutely right. Ember promises—and, we think, delivers—tremendous value. But ramping up to that point is not easy, and we received this feedback repeatedly and take it very seriously.

There was another common sentiment that we'd like to address:

I have seen this sort of attitude from quite a few domain experts - if you didn't get it, you aren't smart enough and "don't deserve to be in our group."

That does not reflect our feelings at all. If people find it difficult to use the tools we've created, that's a failure on our part. Period.

To keep things in perspective, we froze the Ember 1.0 API a mere month ago, when we released the first 1.0 RC. Before that, we were focused on iterating the API based on feedback we received from our early adopters. We believe that our willingness to change the API allowed us to build a better product than our competitors that locked in their first attempts. But that also means that much of the effort that people put into writing tutorials and documentation, including our own, quickly became obsolete.

Thankfully, that period is over. As we said in our EmberCamp keynote, our focus now is on stability and building the ecosystem.

Since EmberCamp, momentum has been incredible. We've seen the amount of material helping new users grow quickly.

To call out a few of the big ones:

Fire Up Ember.js PeepCode
Ember RailsCast (and Part 2)
RC1 Introduction Screencast by Toran Billups
NetTuts' Getting into Ember.js series by Rey Bango
The guides on the Ember.js website have been dramatically expanded

Additionally, great open source examples built on top of RC1 are out in the wild, like TodoMVC and the "Big Kahuna," Discourse.

We've also been working on tooling to make the process of developing Ember.js apps easier. For example, Ryan Florence's ember-tools significantly improves bootstrapping a new project, and the Ember Inspector plugin for Chrome (which we demoed at EmberCamp) should make debugging and understanding apps much easier.

But we know that these things aren't enough. Here are the specific steps we will be taking to improve the experience for developers getting started:

We're actively working on a Getting Started Guide with live examples that walks the user through building a new application from scratch
We're writing the script for a short screencast introducing the framework and illustrating how it works that will be prominently displayed on the homepage.
We're continuing to improve our guides to talk through areas, like the naming conventions we use in Ember, that might be confusing for new developers.
The Starter Kit will return, giving developers a one-click way to try out Ember. (We removed it temporarily to bring it up to date with the most recent idioms, but it will be coming back very soon. This was a hard decision, but leaving around an out of date starter kit seemed worse than removing it.)

We also understand that there are some features in Ember.js that consistently trip up developers new to the framework. The Handlebars {{bindAttr}} helper is a good example. We focus relentlessly on identifying and fixing these issues; you can see the start of an effort to improve this in the work that Yehuda has been doing on htmlbars, which will hopefully land in Ember 1.1.

We'd like to give a big thank you to all of the contributors who have helped make Ember so successful. We believe that a truly community-backed open source project is important. Although it takes time to build that community, versus being able to hire a full-time staff, we believe that in the long run it leads to a stronger, more robust project that can't be "Google Reader'd." Immunity to any one company going under or pulling support is baked right in.

Lastly, we'd love your help. If you think there's something we're missing, or if you'd like to volunteer to help, please let us know in this Discourse thread.

All the best,
Yehuda Katz & Tom Dale

Ember Blog

Ember Data Progress Update

A New Focus

What Does This Mean?

JSON API

Regular Releases

Ember 1.0 RC3

Performance

Testing

Ember Builds

New Input and TextArea helpers

Documentation

Changelog

Announcing the Ember.js Security Policy

Ember 1.0 RC2

Namespace Lookup for Controllers

Using Render With Multiple Models

Unwrapping an ObjectController's Model

Support for jQuery 2.0

Changelog

Stabilizing Ember Data

The Basic Adapter

Timeline

Making Ember.js Easier

Unwrapping an `ObjectController`'s Model