Wednesday, June 12, 2013

Our Backbone Stack

Backbone is a base to build on top of. It gives you a framework for separating your data and your presentation into models and views, but there is a lot that it doesn't give you. It's up to you to figure out what else your unique app needs, and how much of that you'll get from open-source libraries or decide to write yourselves. I see that as both a good and a bad thing. It's good because Backbone can lend itself to many different sorts of apps, with the right combination of add-ons, but it's bad because it takes longer to find those add-ons and get them working happily together.

Saying all that, I thought I'd share what our "backbone stack" is at Coursera. I'd love to hear about your own stacks in the comments.


AJAX calls

After finding ourselves making the same modifications repeatedly to our AJAX calls, we created custom wrapper libraries to take care of those commonalities:

api.js is an AJAX API wrapper that takes care of emulating patch requests, triggering events, showing AJAX loading/loaded messages via asyncMessages.js, and creating CSRF tokens in the client. We use this from both our Backbone and non-Backbone code.

backbone.api.js is a Backbone-specific wrapper for api.js that overrides the sync method and adds create/update/read/delete methods. We mix this into our Backbone models like so:

_.extend(model.prototype, BackboneModelAPI);

Relational Models

Out of the box, Backbone will take JSON from a RESTful API and automatically turn it into a Model or a Collection of Models. However, we have many APIs that return JSON that really represent multiple models (from multiple tables in our MySQL database), like courses with universities:

[{"name": "Game Theory",
 "id": 2,
 "universities": [{"name": "Stanford"}, {"name": "UBC"}
]

We quickly realized we needed a way to model that on the frontend, if we wanted to be able to use model-specific functionality on the nested models (which we often do).

Backbone-relational is an external library that makes it easier to deal with turning JSON into models/collections with sub collections inside of them, by specifying the relations like so:

var Course = Backbone.RelationalModel.extends({
   relations: [{
      type: Backbone.HasMany,
      key: 'universities',
      relatedModel: University,
      collectionType: Universities
    }],
});

We started use that for many of our Backbone apps, but we've had some performance and caching issues with it, so we've started stripping it out in our model-heavy-apps and manually doing the conversion into nested models.

You could also use: Backbone.nested.


Regions

Backbone lets you create views and render views into arbitrary parts of your DOM, but many developers soon run into the desire for "regions" or "layouts". We want to specify different parts of their page, and only swap out the view in those parts across routes - like the header, footer, and main area. That's a better user experience, since there's no unnecessary refreshing of unchanging DOM.

origami.js is a custom library that lets us create regions associated with views, and then in a route, we'll specify which region we want to replace with a particular view file, plus additional options to pass to that view. In the view, we can bind to region events like "view:merged" or "view:appended" and take appropriate actions.

Our syntax for specifying the view regions is admittedly a bit unwieldy, but you get the idea:

Coursera.region.open({
  "pages/home/template/page": {
    regions: {
      header: {
        "pages/home/template/header": {
          initialize: {
            universityPartnerType: university.get('partner_type')
          }
        }
      },
      body: {
        "pages/home/university/universityPage": {
          initialize: {
            university: university
          }
        }
      }
    }
  }
});

Our region library also keeps track of "dirty models" and is responsible for throwing up a modal alert when the user tries to leave the view that there's unsaved data (similar to how you'd do a window.unload for a traditional website).

You could also use: Marionette.js or Chaplin.


Templating

Backbone requires Underscore as a dependency, and since Underscore includes a basic templating library, that's the one you'll see in the Backbone docs. However, we wanted a bit more out of our templating library.

Jade is a whitespace-significant, bracket-less HTML templating library. It's clean to look at because of the lack of brackets and the enforced indenting (like Python and Stylus), but it's best feature (in my opinion) is that it auto-closes HTML tags. I've dealt with too many strange bugs from un-closed tags, and I like that it's one more thing I don't have to worry about when using Jade. Here's an example:

div
    h1 {#book.get('title')}
    p
    each author in book.get('authors')
        a(href=author.get('url')) {#author.get('name')}
    if book.get('published')
        a.btn.btn-large(href="/buy") Buy now!

You could also use: Handlebars, Mustache, or many other options.


Data Binding

Backbone makes it easy for you to find out when attributes on your Model have changed, via the "changed" event, and to query for all changed attributes since the last save via the changedAttributes method, but it does not officially offer any data ⟺ dom binding. If you are building an app where the user can change the data after it's been rendered, then you will find yourself wanting some sort of data binding to re-render that data when appropriate. We have many parts of Coursera where we need very little data-binding, like our course dashboard and course description pages, but we have other parts which are all data-binding, all-the-time, like our discussion forums and all of our admin editing interfaces.

Backbone.stickit is a lightweight data-binding library that we've started to use for a few of our admin interfaces. Here's a simple example from their docs:

Backbone.View.extend({bindings: {
    '#title': 'title',
    '#author': 'authorName'
  },render: function() {
    this.$el.html('<div id="title"/><input id="author">');
    this.stickit();
  }
});

We still do custom data-binding for many of our views (using the "changed" event, changedAttributes(), and partial re-rendering), and I like that because it gives me the most control to decide exactly how a view should change, and I don't have to fight against a binding library's assumptions.

You could also use: KnockBack


History

Backbone offers the Router class and Backbone.History for creating a single-page web app experience, where the URL changes completely and the history is managed via that URL. In some cases, however, I use Backbone to create "widgets" that I can place on existing URLs, and I want to maintain history and back button in those widgets without changing the main URL of the page.

jQuery BBQ is an external non-Backbone specific library for maintaining history in the hash, and as it turns out, it works pretty well with Backbone. You can read my blog post on it for a detailed explanation.

You could also use: Backbone.Widget.

Monday, June 10, 2013

Referencing DOM from JS: there must be a DRYer, safer way

In our JS apps at Coursera, here's what a typical Backbone view might look like:

var ReporterView = Backbone.View.extend({
  render: function() {
    this.$el.html(ReporterTemplate());
  },
  events: {
     'change .coursera-reporter-input': 'onInputChange'
     'click .coursera-reporter-submit': 'onSubmitClick'
  },
  onInputChange: function() {
    this.$('.coursera-reporter-submit').attr('disabled', null);
  },
  onSubmitClick: function() {
    this.model.set('title', this.$('.coursera-reporter-input').val());
    this.model.save();
  }
});

We render out a basic template, setup a few event listeners, and respond to them by manipulating the DOM or sending some data to the server. But there are a few things that irk me about this setup:

  • We are repeating those class names in multiple places.
  • We are using CSS class names for events and manipulation.

They've been irking for a while, but today I finally decided to try out a few approaches to those issues.


DRYing It Out

It worries me whenever I see class names repeated, because I know I have to either remember to update them if we change the class name, or I need full coverage tests. Ideally we'd have the latter for all our views, but hey, if there's something I can do to make my code generally safer, why not?

To avoid repeating those class names, I could store them in some sort of constants that are accessible anywhere in the view, and only access them via the constants. For example:

var ReporterView = Backbone.View.extend({
  dom: {
     SUBMIT_BUTTON: '.coursera-reporter-submit',
     INPUT_FIELD:   '.coursera-reporter-input'
  },
  render: function() {
    this.$el.html(ReporterTemplate());
  },
  events: function() {
    var events = {};
    events['change ' + this.dom.INPUT_FIELD]    = 'onInputChange';
    events['click ' +  this.dom.SUBMIT_BUTTON]  = 'onSubmitClick';
    return events;
  },
  onInputChange: function() {
    this.$(this.dom.SUBMIT_BUTTON).attr('disabled', null);
  },
  onSubmitClick: function() {
    this.model.set('title', this.$(this.dom.INPUT_FIELD).val());
    this.model.save();
  }
});

There are a few drawbacks to that approach: 1) we add more constants to our code, which may increase file size, and 2) our events definition gets a little bit more harder to read.

But it is certainly DRYer and there are also more benefits to it besides that, like easier-to-maintain testing code:

it('enables the submit button on change', function() {
  chai.expect(view.$(view.dom.SUBMIT_BUTTON).attr('disabled'))
 .to.be.equal('disabled');
  view.$(view.dom.INPUT_FIELD).trigger('change');
 chai.expect(view.$(view.dom.SUBMIT_BUTTON).attr('disabled'))
 .to.be.equal(undefined);
});

De-class-ification

It bothers me when I realize that I'm relying on CSS class names in my JS, because that means that:

  1. I need to create long, specific class names, since we compile the CSS used by our many views into just a few CSS bundles, and it would be a bad thing if two class names overlapped.
  2. I need to warn our designers not to touch those class names when they're doing a re-style of our HTML Jade templates, and/or I need to make sure that I have 100% coverage on the code using those class names.

I'd much rather have every CSS class name in our HTML templates be there purely for styling reasons, so that we only put effort into avoiding name collisions if it's actually necessary, and so that our designers can safely refactor the CSS independently of the JS.

An alternative to using CSS class names is to use data attributes instead, perhaps prefixing with js-* to indicate their use in JS. That would mean changing our selectors to something like:

var ReporterView = Backbone.View.extend({
  dom: {
     SUBMIT_BUTTON: '[data-js-submit-button]',
     INPUT_FIELD:   '[data-js-input-field]'
  },
...
});

There is a drawback to using data attributes, however: performance. According to local legend and this jsperf from Craig Patik, it is faster in all browsers to query by class instead of by data attributes, both via jQuery and the native document.querySelectorAll. Given that, a compromise would be to use CSS class names prefixed with js-* and to make it very clear to the engineering and design teams that they are not to be used in CSS class names. For example:

var ReporterView = Backbone.View.extend({
  dom: {
     SUBMIT_BUTTON: '.js-submit-button',
     INPUT_FIELD:   '.js-input-field'
  },
...
});

So, that's what I came up with. Now, your turn. What's your approach to referencing the DOM from your Backbone views or Javascript app? How does it do in terms of DRYness, future-proof-ness, testability, and all those fancy terms that may not actually exist?

Friday, June 7, 2013

Exporting a Google Spreadsheet as JSON

⚠️ Update (Apr 2023): I do not actively maintain this script, so it may not work. See my recent post on using Neptyne as another approach.

I often use Google Spreadsheets as a lightweight database, by setting up columns, encouraging my colleagues to update it, and subscribing to notifications of changes. Then I export the spreadsheet as JSON and update a .json file in our codebase. It's also possible to use the jsonp output of a published spreadsheet for dynamically updated JSON, but when I'm worried about performance or the information getting mis-updated, I use the export-and-update approach.

In order to export it as JSON, I use a Google Apps Script.

Here are the steps for using the script:

  1. Create a new spreadsheet, and put your data in columns. Give each column a name, and choose carefully. Since this name will be used for JSON keys, the best names are lowercase, whitespace-less, and descriptive. Freeze the first row with the column names. 
    Screenshot of spreadsheet with reading list
  2. Go to Tools → Script Editor, and it will open up a code editing interface in another window. 
    Screenshot of Google spreadsheets UI, Tools menu and Script editor option
  3. Paste the JavaScript from this gist into the code editor. 
    Screenshot of Google Apps Script editor with code inside it
  4. Reload the spreadsheet and notice a new menu shows up called "Export JSON". Click on that, and you'll see two options:
    1. Export JSON for this sheet (with default configuration)
    2. Export JSON for all sheets (with default configuration)
      Screenshot of Google spreadsheet with Export JSON menu
  5. Once you click one of the export options, it will process for a few seconds and popup a textbox with the exported JSON. Now you can do whatever you'd like with that!
    Screenshot of exported JSON for Google spreadsheet

So there you have it! If you make any improvements to the script to make it more flexible or easier to use or better in any way, please let me know in the comments.

Wednesday, June 5, 2013

Improving Backbone App Performance

At CSSConf and JSConf, one of the big themes was performance. I saw talks from Chrome engineers on Jank-busting, a talk from Adobe engineer Peter Flynn on performance tuning in Brackets, and had a lengthy lunch discussion about API performance and caching. Eventually, I couldn't stand all that talking with no action, and I broke down and spent my night in the bar investigating a part of Coursera that is noticeably slow: the course catalog. We looked into the backend API performance a few weeks ago, but I figured, hey, maybe there were frontend improvements that we could make too. Here's what I did and what I learnt.

Step 1. Identify Problem Areas in Timeline

In order to show our course catalog to users, the browser goes through many steps: loading the JS and CSS files, parsing them, finding the right Backbone route, creating the view, sending off an AJAX request to get courses, and rendering everything to the DOM. We could have performance issues in each of those steps (and there's ways to improve the performance of all of them), but I wanted to identify if there were big problem areas with obvious room for improvement.

To start off with, I loaded the catalog with the Chrome timeline view open and recording, and looked for big blocks. I quickly found two of them that looked ripe for the fixing:

  • The callback function for the courses AJAX request that turned the JSON into Backbone collections and models. This one was all "Scripting" blocks (JS execution).
  • The function that rendered the tags and courses to the catalog. This showed up as staggered "Rendering" blocks (DOM operations).

Here's what that looked like in the timeline:



Step 2. Break them down in Profiler

The timeline view will give you a good rough idea of where the browser is spending its time, but the profiler can give you exact statistics on what functions are taking the most amount of time. After reloading the catalog while recording a Javascript CPU profile, I identified a few functions taking a suspicious amount of time, both from external libraries:

  • Backbone.Relational took up the majority of the time with constructor and add operations
  • moment.js took up a significant amount of time with date processing functions.

Here's what the profiler looked like in tree view with relevant nodes expanded:



Step 3. Remove, reload, record

I immediately started thinking of ways to reduce our usage of those external libraries (as I wasn't interested in trying to understand why they were so time-intensive), but before I tried those, I wanted to make sure I was scientific about seeing what changes had the greatest effect on performance. To the measuring machine!

I inserted console.time("Backbonify") and console.timeEnd("Backbonify") statements at the start and end of our AJAX callback, so that I could immediately see that time duration in the console on every reload. I discovered that the time varied between loads (by 20-80ms) and determined I would need to record a range, not a single number. Before making any changes, the Backbonify time was about 750-850ms.

Now that I had a starting number to gauge my progress, I created a table of code-changes and time-ranges, and started changing code, reloading, and recording the new time spent. At first, I didn't worry about whether the code actually still worked - I just wanted to see if theoretically, I was able to remove some bit of code, how great of an improvement would I see? I didn't want to go through the effort of rewriting code until I understood how much I would gain.

Here's a screenshot of that table:



Step 4. Rewrite away the slowness

After my rapid-fire removal-and-reload session, I had a very good idea of what I could do to improve our performance while still retaining our functionality. Thankfully, we already had a number of tests for the code, so I had some confidence that I would know if I broke existing functionality.

Here's a run-down of the main changes that I made:

  • Removed Backbone.Relational
    Time saved: ~400ms
    We were using the library so that we could turn our JSON into nested collections of different models, and have a nice simple API for doing that, but I discovered that I could do the conversion myself and not take nearly as much time. Here's a basic conversion from the Topic.initialize() method:
    if (!this.get('courses') || !(this.get('courses') instanceof Courses)) {
      this.set('courses',
        new Courses(this.get('courses')), {silent: true});
    }
    

  • Deferred and cached moment.js functionality
    Time saved: 100ms
    We were using moment.js to set quite a few computed attributes on our Course models (related to start dates), and we were doing that upon initialization of the model, so that the computed attributes were always available just like normal attributes. So I turned them instead into explicit function calls (course.getStartStatus() vs. course.get('start_status')) and rewrote the code that called them, so that we we don't call the expensive moment functions until absolutely necessary. I also wrote a cache for moment.js so that I'd only call the .format() method if I'd never formatted that date before, for example. That improves performance both in calling the methods multiple time on the same model but also in calling them across different models with the same start dates. Here's a snippet of that cache:
      var MomentCache = {
        moments: {},
        formats: {},
        getMomentFor: function(arr) {
          var key = arr.join('/');
          if (!MomentCache.moments[key]) {
            var momentFor = moment(arr);
            if (momentFor.isValid()) {
              MomentCache.moments[key] = momentFor;
            }
          }
          return MomentCache.moments[key];
        },
        getFormatted: function(momentObj, formatString) {
          var key = momentObj.valueOf() + '|' + formatString;
          if (!MomentCache.formats[key]) {
            MomentCache.formats[key] = momentObj.format(formatString);
          }
          return MomentCache.formats[key];
        }
      };
    

  • Stopped listening to Collection "add" for renders
    Time saved: ~60ms
    For the second problem area that I identified, the one that was DOM-render-heavy, I realized the problem was that we were doing many separate calls to our templating functions and jQuery append, and it would obviously be more efficient to reduce them to one template and append call. This was actually due to the way we were using Backbone events - we were listening to "add" on a collection, and despite the fact that we were adding all of our models at once to the collection, the "add" event triggers *for each model*, passing in the added model. To speed it up, I triggered a custom event for that batch add, and rendered all the non-rendered models at once in the callback. The callback looked something like:
        addTopics: function() {
          var self = this;
          var topicsToAdd = [];
          this.model.get('collection').each(function(topic) {
            // If we already rendered it, we don't render it again
            if (self.$('.coursera-catalog-course-listing-box[data-topic-id=' + topic.get('id') + ']').length) {
              return;
            }
            topicsToAdd.push(topic);
          });
          this.$('.coursera-catalog-listings').append(
            listingTemplate({topicsInfo: topicsToAdd}));
        }
    

  • Removed "comparator" on Collections
    Time saved: 50ms
    We had a comparator function defined on some of our collections, which is a function that Backbone optionally looks for to create a default sort order for the models in a collection. As it turns out, we weren't ever relying on that default sort order, because we always use utility functions to sort our collections in different orders depending on the view and user interactions. I removed the comparators on the collections, and from now on, I will recommend that we never use them and always use explicit sorting functions.


Step 5. Profit!

After all those changes, my final Backbonify time went down to 90-110ms. If my measurements are as highly scientific as I hope they are, that means that I shaved off ~700ms in JS execution time. That was all locally measured times, but they're similar savings in production.

Here's what the profile view looks like now:

A part of me is tempted to put in an artificial delay back into the catalog, and do an A/B test to see if the performance changes actually do affect signups. But another part of me thinks that's a horrible thing to do to users. Maybe one day.

The Chrome profiling tools are a bit intimidating at first, but I encourage you to open them up, do some poking around, and see what your app's up to. I sure learnt a lot about ours — and I know I'll keep learning. There's still so much to improve!

Tuesday, June 4, 2013

Google I/O Talk: Online Learning Made Social

Over the last six months, we've been working with the Google+ Hangouts team to experiment with using hangouts to add another dimension of communication and social to the Coursera student experience. At I/O last month, I got the opportunity to share those experiments and talk about what we've learnt from them. It wasn't a very technical talk at all, as we've been learning more about humans than machines, but that made it a fun talk to give, too. I look forward to doing more experiments with hangouts and other forms of communication in the future. Stay tuned!

You can watch the talk on Youtube:

You can also follow along with my half of the slides below (I don't yet have the combined deck):

Friday, May 3, 2013

Managing history in Backbone widgets with jQuery BBQ

In my last blog post, I talked about the two different Backbone architectures we're experimenting with at Coursera: 1) single page web apps, where Backbone takes care of serving particular views for given URLs, and 2) JS widgets, where we write DIVs with particular data attributes into our HTML, and a JS module finds all of them on the page and turns them into Backbone views.


The Widget Approach

We are using this approach for our discussion forums. We have widgets for displaying lists of threads, rendering a single thread, displaying an entire forum with multiple thread lists inside it, and more. We can potentially mash up a few widgets on the same page, if we want, because they each encapsulate all of their functionality inside them.

For example, here's the bit of code that creates our forum widget:

$('[data-coursera-forum-widget]').each(function() {
  var forumId = $(this).attr('data-forum-id');

  var forum = new ForumModel({
    id: forumId
  });
  new ForumView({
    el: $(this)[0],
    model: forum
  });
});

The Problem

There was one problem with this approach, however: users kept losing their state in the widgets. For example, when a TA was paging through a long list of threads, and then they clicked away to visit one and came back, the widget would forget that it was on that page and they'd have to start from the beginning. When the forums were originally written, in the classical Web 1.0 architecture, the state was always stored in the query parameters of the URL, but now, since we are in JS-land and no longer need to change the URL to change the content of the page, we lost the URL-managed state.

That meant that we lost the ability for users to use the back button through the states of one widget, to go forward to a completely different page and back to the previous state, and to bookmark the state. Once I got enough reports from users who missed those abilities (mostly from our super users, who consider the forums to be their inbox), I realized it was time to take on this problem.


Possible Solutions

There were a few solutions that I considered and talked through with my colleagues:

  • Remember their previous state in cookies or localStorage, and always restore it from there. That wouldn't easily get me back button support, however, and it may have been odd for the user to open the forum in a new tab and see that same state.
  • Open all the links in a new tab, so that they never left the page and lost their state. Yes, I admit, this was a non-ideal solution, and I did try it for a few hours but I quickly remembered that I should not be the one deciding that the user wants those links in a new tab. Also, that wouldn't solve the problem of back button through widget state.
  • Move to the single page web app approach, and let the Backbone router manage the history. That would mean losing the mashability of my widgets, the ability to put any combination of widgets on the same page, and I'm not ready to give that up.
  • Use a JS library to store each widget's history in the URL hash, and use the hashchange event (and fallback implementations) to support the back button.

As you can guess from the title of the post, I went with the final approach. It's the only one that solved all of the users problems and let me keep my widget approach - plus it's a tried and true technique.


jQuery BBQ + Backbone

There are a few libraries out there that manage history via the hash, from very simple (js-hash) to a sophisticated polyfill approach (Hasher). The one that I was most familiar with was jQuery BBQ, and its also the one that did everything I wanted and not too much more. Plus, its docs described exactly our scenario:

<Widget> Yo, hash, update my state parameters.
<Hash> No prob, dude, done. And you didn’t even have to know about that other widget’s parameter, I just merged them in there for you.
<Widget> There’s another widget?
<Widget2> Huh? Did someone say my name?

jQuery BBQ has a straightforward API - you can pushState an object that is merged into the current hash values, you can getState on a certain key, and you can listen to the hashchanged event. To make it easy for multiple Backbone views to manage their state independently of each other, I wrote a WidgetView class with functions that can set and get state scoped to just the widget, and can trigger the view with an event whenever the widget's state changed in the hash. Here's what that class looks like:

Once my Backbone view extends that class, it can check the initial state when the view is loaded, it can set the state when the user clicks around (like on the sort or page controls), and it can listen to the state changed event to decide how to change the UI.

Here's a slimmed down version of the ThreadsView that demonstrates that:



We'll see how this works out once we build out more widgets, but so far, it seems to be working well. Let me know in the comments what approach you've taken.

Thursday, May 2, 2013

Server-side HTML vs. JS Widgets vs. Single-Page Web Apps

At the recent GOTO Chicago conference, I gave a talk on "Frontend Architectures: from the prehistoric to the Post-modern." In just my first 10 months at Coursera, I've experienced the joys and woes of many different frontend architectures, and I wanted to share what I learnt. I detail everything in the slides, but I'll summarize my thoughts here as well.

How do you pick an architecture?

We could make our decision by just checking Twitter and seeing what all the cool kids are talking about, but uh, let's pretend that we're more scientific about it than that, and figure out whats important to us as web developers.

To start off with, here are a few things that users care a lot about:

  • Usability: they can do what they want to do, quickly and intuitively
  • Linkability: they can bookmark and link to parts of the site
  • Searchability/Shareability: they can share what they're using on social networks, and find it on Google (this is important for user growth)
  • More features, less bugs: 'nuff said.

Of course, we also should think about what we as developers care about:

  • Developer Productivity: we want to be able to iterate fast, try new things, and implement features in a way that makes us feel code
  • Testability: we want confidence that our code won't break, and that we won't spend most of our time maintaining a decaying, fragile product
  • Performance: we want our servers to respond quickly and to serve minimum number of requests, and we don't want our users to wait unnecessarily.

And when developers are happier, we can more quickly and safely improve the product, and that makes users happy.

Let's compare a few architectures...

So that's what I looked at when I reviewed our different architectures, and as it turns out, there are definite differences between the architectures in those respects.


Server-side HTML ("Web 1.0")


In much of our our legacy codebase on class.coursera.org (where instructors create their courses with a combination of lectures, quizzes, assignments, wikis, and forums), we use PHP to output HTML, and it handles much of the interaction via page reloads. A bit of JS is sprinkled (in a not so pretty manner) throughout.

This architecture suffers the most in terms of usability - it's very hard for users to do many interactions in a small amount of time - but it does have definite benefits of easy linkability, shareability, and searchability.

As a developer though, I hate it. All of our data is entangled inside the HTML, and when we want to bring JS into the HTML, it quickly becomes an untestable mess. Plus, it ties us to a particular backend language, and if we want to change languages (which we're doing, to Scala/Play), we have to rewrite the presentation layer. Yes, it's possible to use this approach in a more elegant way, and try to use templates that are portable across languages, but it's just not likely.


JS widgets


For the class.coursera.org forums, we are starting to take a different approach: JS widgets. We write DIVs into the HTML with certain attributes (like data-coursera-forum-threads-widget) and then we include a widgets.js that turns those DIVs into Backbone views.

With this approach, we can create very dynamic interfaces with real-time updates, we can have decent linkability if we have server-side URLs and hash state, but we can't easily make the content in the widgets shareable and searchable, since bots don't understand the JS. However, that's not a big concern for us there, since much of the content is behind a login wall anyways.

From a developer perspective, we typically create APIs for the JS widgets to consume, and well-tested APIs mean that we can more easily create new frontends for the same data. After I ported our forums to the widgets approach, I was able to make new widgets for the same data in just a few hours, since I could re-use the same API. On the flip side, I have to spend more time writing tests for the frontend, since the user can change the state via sequences of interactions and many bugs may not surface until after a particular interaction.


Single-page web apps


On www.coursera.org, we serve the same HTML file for every URL, and that HTML file contains only require.js and a call to load a routes.js file. That routes file maps URLs to Backbone views, and we rely on the Backbone Router to figure out what view to load up and to manage history using the HTML5 history API (with the fallback hash technique in older browsers).

We get many of the same benefits and drawbacks as JS widgets with single page web apps, but there are a few key differences. We can have a potentially faster user experience because of the complete lack of true page reloads, but it's harder to do simple things like internal links (you'll end up with a double hash on older browsers), or listen to a window.onunload event. We would suffer from bad searchability here, but we decided that it's really important for users to be able to find and share the course descriptions, so we wrote a "Just in time renderer" that uses Selenium to render the HTML and serve that to bots instead.

On the developer side, it's much trickier to test, because our webapp now has state *across* routes, and our tests have to check different sequences of routes.


So which one's the best?

Trick question! None of them have it all, atleast not yet. I would argue that if you're doing anything highly interactive (like an admin interface), you really want to take a heavy-JS approach, and also I'll point out that we're still in the early days of JS-heavy architectures. We will get better at developers at tackling problems like testability, and bots and browsers will likely get better at problems like searchability and performance. So, yes, it's a bit of a risk to do go as far as the single page webapp approach today, but it will be less and less of a risk as time goes on.

In terms of JS widgets vs Single-page web apps, I think they each have their places. On class.coursera.org, I hope that we can give instructors increasing amounts of control and flexibility in how they put together their classes, and that's why I'm building out our functionality as widgets that can be combined together however they please. On www.coursera.org, where students browse classes, we are the only ones designing that experience, and we know what we want each page to be, so it makes more sense to go the single page web app approach. The only difference is in the routes vs. the widgets file (mapping URLs vs. transforming DIVs), so we can easily create Views and Models that we use in both apps and widgets.

For my full run-down of the good and bad bits about each approach, flip through the slides and click on the images in the slides to see more code, screenshots, or videos.

What's your approach? What do you like or dislike about it? Let me know in the comments!