At Coursera, we made the choice to use the Backbone MVC framework for our frontends, and over the past year, we've evolved a set of best practices for how we use Backbone.
I wrote a guide for internal use that documents those best practices (much of it based on shorter blog posts here), and I've snapshotted it here on my blog to benefit other engineering teams using Backbone and to give potential Coursera engineers an idea of the current stack. This was snapshotted on July 24th, 2013, so please keep in mind that the Coursera frontend stack may change over time as the team figures out new and better ways to do things.
If you're interested in joining Coursera, check out the many job listings here. The frontend team is a really smart and fun bunch, and there are a lot of interesting technical and usability challenges in the future.
The Architecture
There are many different frontend architectures to choose from, and at Coursera, we have made the deliberate decision to opt for a very JavaScript-heavy, JavaScript-dependent approach to our frontend architecture:
We build up the entire DOM in JavaScript, loading the data via calls to RESTful JSON APIs, and handle state changes in the URL via the hash or HTML5 history API.
This approach has several advantages, atleast as compared to a traditional data-rendered-into-HTML approach:
- Usabiity: Our interfaces can easily be dynamic and real-time, enabling users to perform many interactions in a small period of time. This is particularly important for our administrative interfaces, where users want to be able to drag-and-drop, tick things on and off, and generally manipulate many little things that are present on one screen.
- Developer Productivity: Since this architecture relies on the existence of APIs, that makes it easy for us to build new frontends for the same data, which encourages experimentation with new ways of viewing the same data. For example, after porting our forums to this architecture, I was able to create portable sidebar widgets based off the forums API in just a few hours.
- Testability: The APIs and the frontends can both be tested separately and rigorously using the best suite of tools for the job.
It also has a few disadvantages:
- Linkability: We have to go through a bit more work to make the JS-powered interfaces linkable, and previously simple things like internal anchors (page#section) are surprisingly difficult to implement.
- Search/shareability: Since Facebook bots and search bots do not handle JS-rendered webpages as well, we have to go through more work to make our public pages indexable by them, which we've done through our Just-in-time renderer.
- Testability: We have to write far more tests for our JS frontends since the user can change state via sequences of interactions, and some bugs may not surface until a particular sequence. We also now have state across URL routes when we use the HTML5 history API, and may have to test across multiple views.
- Performance: We must be constantly monitoring our JavaScript to make sure we are not pushing the browser to do too much, as JavaScript can still be surprisingly slow at processing data and turning it into DOM.
However, given the usability benefits of the JS-rendered approach, we have elected to stick with it, and we will need to become experts in overcoming the disadvantages of the approach. At the same time, we can hope that the browsers and tools make those disadvantages slowly disappear, as this is an increasingly popular approach.
The APIs
We have APIs coming from Python/Django, PHP, and Scala/Play. We try to be consistent in the API design, and when possible, we opt for a RESTful JSON API.
For example, if I want to retrieve information about a forum, we'd perform an HTTP GET to a RESTful URL and expect a JSON to come back with an "id" attribute and other useful attributes.
Request:
HTTP GET /api/forums/1
Response:
{
"id": 1,
"parent_id": -1,
"name": "Forums",
"deleted": false,
"created": "1369400797"
}
To create a new forum, we'd perform an HTTP POST to a RESTful URL with our JSON, and expect JSON to come back with the "id" filled in:
Request:
HTTP POST /api/forums/
{
"parent_id": -1,
"name": "Forums"
}Response:
{
"id": 1,
"parent_id": -1,
"name": "Forums",
"deleted": false,
"created": "1369400797"
}
To update an existing forum, we could do an HTTP PUT with the full JSON of the new properties, but when possible, we prefer to do an HTTP PATCH, only sending in the changed properties. That is a safer approach and means we are less likely to change attributes that we did not intend to change, and also makes our interfaces more usable by multiple people at once.
Request:
HTTP PATCH /api/forums/1
{
"name": "Master Forums"
}
Response:
{
"id": 1,
"parent_id": -1,
"name": "Master Forums",
"deleted": false,
"created": "1369400797"
}To delete a forum, we could do an HTTP DELETE, but we prefer instead to set a deleted flag on the object, and make sure that we respect the flag in all of our APIs. We often have users that accidentally delete things, and it is much easier to restore if the information is still in the database.
Request:
HTTP PATCH /api/forums/1
{
"deleted": true
}
Response:
{
"id": 1,
"parent_id": -1,
"name": "Master Forums",
"deleted": true,
"created": "1369400797"
}
If we are retrieving many resources, we may want a paginated API, to avoid sending too much information down to the user. Here's what that might look like:
Request:
HTTP GET /api/forum/search?start_page=1&page_size=20
Response:
{"start_page": 1,
"page_size": 20,
"total_pages": 40,
"total_results": 800,
"posts": [ ... ]
}
The JavaScript
JavaScript is a powerful language, but it can easily become a jumbled mess of global variables and files that are thousands of lines long. To keep our JavaScript sane, reusable, and modularized, we chose to use an MVC framework. There are approximately a million* MVC frameworks to choose from, but we chose Backbone.JS as it has a large community of knowledge built up around it and it is lightweight enough to be used in many different ways.
Backbone
Backbone provides developers with a means of separating presentation and data, by defining Models and Collections for the data, Views for the presentation, and triggering a rich set of events for communication between the models and views. It also provides an optional Router object, which can be used to create a single page web app that triggers particular views based on the current URL route. For a general introduction to Backbone, see these slides.
There is a lot that Backbone does not provide, however, and it's up to the app developer to figure out what else that app needs, and how much of that you'll get from open-source libraries or decide to write yourselves. That's good because Backbone can lend itself to many different sorts of apps, with the right combination of add-ons, but it's bad because it takes longer to find those add-ons and get them working happily together. As a company, it is in our best interest to converge on a recommended set of add-ons and best practices, so that our code is more consistent across the codebase. At the same time, it's also in our best interest to continually challenge our best practices and make sure that we are using the right tool for the job. If we discover a particular add-on is too buggy or slow, we should phase that out of the codebase and document the reasons why.
There is a larger question, of course: Is Backbone the right framework for us, given how many new frameworks have come out recently that may entice us with promises of speed and flexibility? That is not a question that I have an answer for, but I do think that one can spend forever trying out new frameworks to find the perfect one, and time might be better spent building up best practices around a single framework. However, there may be a time at which we become sufficiently convinced that Backbone is no longer working for our codebase and it is worth the cognitive effort and engineering resources to invest in a new framework.
Here's an exploration of the add-ons and best practices that we use in our Backbone stack.
Backbone Models
A basic model might look like this:
define([
'underscore',
'backbone',
"pages/forum/app",
"js/lib/backbone.api"
],function(_, Backbone, Coursera, BackboneModelAPI) {
var model = Backbone.Model.extend({
api: Coursera.api,
url: 'user/information'
});
_.extend(model.prototype, BackboneModelAPI);
return model;
});
We start off by declaring the JS dependencies for the model:
- underscore: This is a collection of generic utility functions for arrays, objects, and functions, and it's common to find yourself using them, so most models and views will include it.
- backbone: This is necessary for extending Backbone.Model
- pages/forum/app: Every model will depend on an "app.js", which defines a base URL for API calls and a few other details. It adds objects to the Coursera singleton variable, like Coursera.api, which is used by the Model.
- js/lib/backbone.api: This is a Backbone-specific wrapper for api.js that overrides the
syncmethod and addscreate/update/read/deletemethods. The api.js library is an AJAX API wrapper that takes care of emulating patch requests, triggering events, showing AJAX loading/loaded messages viaasyncMessages.js, and creating CSRF tokens in the client.
Then we define an extension of the Backbone.Model object with api and url options that help Backbone figure out where and how to pull the data for the model, and it mixes in the BackboneModelAPI prototype at the end of the file.
Backbone Models: Relational Models
Out of the box, Backbone will take JSON from a RESTful API and automatically turn it into a Model or a Collection of Models. However, we have many APIs that return JSON that really represent multiple models (from multiple tables in our MySQL database), like courses with universities:
[{"name": "Game Theory",
"id": 2,
"universities": [{"name": "Stanford"}, {"name": "UBC"}
]We quickly realized we needed a way to model that on the frontend, if we wanted to be able to use model-specific functionality on the nested models (which we often do).
Backbone-relational is an external library that makes it easier to deal with turning JSON into models/collections with sub collections inside of them, by specifying the relations like so:
var Course = Backbone.RelationalModel.extends({
relations: [{
type: Backbone.HasMany,
key: 'universities',
relatedModel: University,
collectionType: Universities
}],
});
We started use that for many of our Backbone apps, but we've had some performance and caching issues with it, so we've started stripping it out in our model-heavy-apps and manually doing the conversion into nested models.
For example, heres how the Topic model turns nested courses array into a Courses collection:
var Topic = Backbone.Model.extend({
defaults: {},
idAttribute: 'short_name',
initialize: function() {
this.bind('change', this.updateComputed, this);
this.updateComputed();
},
updateComputed: function() {
var self = this;
if (!this.get('courses') || !(this.get('courses') instanceof Courses)) {
this.set('courses', new Courses(this.get('courses')), {silent: true});
this.get('courses').each(function(course) {
if (!course.get('topic') || !(course.get('topic') instanceof Topic)) {
course.set('topic', self);
}
});
}
}
});
For a trickier example, here's how the Course model sets a nested Topic model. It has to require the Topic file dynamically, to avoid a cyclic dependency in the initial requires which will wreak all sorts of havoc:
var course = Backbone.Model.extend({
defaults: {},
initialize: function() {
this.bind('change', this.updateComputed, this);
this.updateComputed();
},
updateComputed: function () {
// We must require it here due to Topic requiring Courses
var Topic = require("js/models/topic");
if (this.get('topic') && !(this.get('topic') instanceof Topic)) {
this.set('topic', new Topic(this.get('topic')), {silent: true});
}
});
We could also look into using Backbone.nested, which seems like a more lightweight library than Backbone.Relational, and it may have less performance issues.
Backbone Views
Here's what a basic Backbone view might look like:
define([
"jquery",
"underscore",
"backbone",
"js/core/coursera",
"pages/site-admin/views/NoteView.html"
],
function($, _, Backbone, Coursera, template) {
var view = Backbone.View.extend({
render: function() {
var field = this.options.field;
this.$el.html(template({
config: Coursera.config
field: field,
}));
return this;
}
});
return view;
});
We start off by declaring the JS dependencies for the view:
- jquery: We often use jQuery in our views for DOM manipulation, so we almost always include it.
- underscore: Once again, underscore's utility functions are useful in views as well as models (in particular, debounce and throttle are great for improving performance of repeatedly called functions.)
- backbone: We must include Backbone so that we can extend Backbone.View.
- js/core/coursera: We include this so that we have a handle on the Coursera singleton variable, which contains useful information like "config" that includes the base URL of assets, which we often need in templates.
- pages/site-admin/views/NoteView.html: This is a particular Jade template that's been auto-compiled into an *.html.js file, and we include it so we can render the template to the DOM. We try to keep all of our HTML and text in templates, out of our view JS.
Then we create the view and define the render function, which passes in Coursera.config and a view configuration option into a template, and renders that template into the DOM.
Backbone Views: Templating
Backbone requires Underscore as a dependency, and since Underscore includes a basic templating library, that's the one you'll see in the Backbone docs. However, we wanted a bit more out of our templating library.
Jade is a whitespace-significant, bracket-less HTML templating library. It's clean to look at because of the lack of brackets and the enforced indenting (like Python and Stylus), but one of it's best features is that it auto-closes HTML tags. We've dealt with too many strange bugs from un-closed tags, and it's one more thing we don't have to worry about when using Jade. Here's an example:
div
h1 {#book.get('title')}
p
each author in book.get('authors')
a(href=author.get('url')) {#author.get('name')}
if book.get('published')
a.btn.btn-large(href="/buy") Buy now!
We could also consider using Handlebars, Mustache, or many other options.
Backbone Views: Referencing DOM
Inside a view, we find ourselves referencing the DOM from the templates repeated times, like to set up events, read off values, or do slight manipulations. For example, here's what a view might look like:
var ReporterView = Backbone.View.extend({
render: function() {
this.$el.html(ReporterTemplate());
},
events: {
'change .coursera-reporter-input': 'onInputChange'
'click .coursera-reporter-submit': 'onSubmitClick'
},
onInputChange: function() {
this.$('.coursera-reporter-submit').attr('disabled', null);
},
onSubmitClick: function() {
this.model.set('title', this.$('.coursera-reporter-input').val());
this.model.save();
}
});
There are a few non-optimal aspects of the way that we reference DOM there:
- We are repeating those class names in multiple places. That means that changing the class name means changing it in many places - not so DRY!
- We are using CSS class names for events and manipulation. That means our designers can't refactor CSS safely without affecting functionality, and it also means that we must come up with very long overly explicit class names to avoid clashing with other CSS names, since we bundle our CSS together..
To avoid repeating the class names, we can store them in a constant that is accessible anywhere in the view, and only access them via that constant. For example:
var ReporterView = Backbone.View.extend({
dom: {
SUBMIT_BUTTON: '.coursera-reporter-submit',
INPUT_FIELD: '.coursera-reporter-input'
},
render: function() {
this.$el.html(ReporterTemplate());
},
events: function() {
var events = {};
events['change ' + this.dom.INPUT_FIELD] = 'onInputChange';
events['click ' + this.dom.SUBMIT_BUTTON] = 'onSubmitClick';
return events;
},
onInputChange: function() {
this.$(this.dom.SUBMIT_BUTTON).attr('disabled', null);
},
onSubmitClick: function() {
this.model.set('title', this.$(this.dom.INPUT_FIELD).val());
this.model.save();
}
});
As a bonus, this technique gives us easier-to-maintain testing code:
it('enables the submit button on change', function() {
chai.expect(view.$(view.dom.SUBMIT_BUTTON).attr('disabled'))
.to.be.equal('disabled');
view.$(view.dom.INPUT_FIELD).trigger('change');
chai.expect(view.$(view.dom.SUBMIT_BUTTON).attr('disabled'))
.to.be.equal(undefined);
});
As for the use of class names entirely, we can avoid them by using data attributes instead, perhaps prefixing with js-* to indicate their use in JS.
We would still have CSS class names in the HTML templates, but only for styling reasons.
So then our DOM would look something like:
var ReporterView = Backbone.View.extend({
dom: {
SUBMIT_BUTTON: '[data-js-submit-button]',
INPUT_FIELD: '[data-js-input-field]'
},
...
});
Note that selecting via data attributes shown to be less performant but for the vast majority of our views, that performance difference is insignificant.
Backbone Views: Data Binding
Backbone makes it easy for you to find out when attributes on your Model have changed, via the "changed" event, and to query for all changed attributes since the last save via the changedAttributes method, but it does not officially offer any data ⟺ dom binding. If you are building an app where the user can change the data after it's been rendered, then you will find yourself wanting some sort of data binding to re-render that data when appropriate. We have many parts of Coursera where we need very little data-binding, like our course dashboard and course description pages, but we have other parts which are all data-binding, all-the-time, like our discussion forums and all of our admin editing interfaces.
Backbone.stickit is a lightweight data-binding library that we've started to use for a few of our admin interfaces. Here's a simple example from their docs:
Backbone.View.extend({bindings: {
'#title': 'title',
'#author': 'authorName'
},render: function() {
this.$el.html('<div id="title"/><input id="author">');
this.stickit();
}
});We still do custom data-binding for many of our views (using the "changed" event, changedAttributes(), and partial re-rendering), and I like that because it gives me the most control to decide exactly how a view should change, and I don't have to fight against a binding library's assumptions.
We could also consider using: KnockBack
Maintaining State: Single-Page-Apps vs. Widgets
After we've created a view for our frontend, we still have big decisions to make:
- How will users get to that view?
- What state of the view will be kept in the URL, i.e., what can the user press back on and what can they bookmark?
- Will our view be used in multiple parts of the site or just one?
In our codebase, we have two main approaches to those questions: "single page apps" and "widgets".
Single-Page-Apps
Besides being the buzz word du jour, a single-page-app ("SPA") is what Backbone was originally designed for, via its Backbone.Router object. A SPA defines a set of routes, and each route is mapped to a function that renders a particular view into a part of the page. Backbone.History then takes care of figuring out which route is referred to by the current URL, and calling that function. It also takes care of changing the URL using the HTML5 History API (which makes it appear like a normal URL change) or window.location.hash in older browsers.
For example, we could have this routes file:
define([
"jquery",
"backbone",
"pages/triage/app"
],
function($, Backbone, Coursera) {
var routes = {};
var triageurl = Coursera.config.dir.home.replace(/^\//, "triage");
routes[triageurl + '/items'] = function() {
new MainView({el: $('.coursera-body')});
};
Coursera.router.addRoutes(routes);
$(document).ready(function() {
Backbone.history.start({pushState: true});
});
});
After declaring its dependencies, it defines a mapping of routes, adds those to our global Coursera.router (an extension of Backbone.Router) and then kicks off Backbone.history.start() on page load.
SPAs: Syncing Users
More typically, for our logged in-apps, we will attempt to login the user before calling the routes, and our document.ready callback will look like this:
(new User())
.sync(function(err) {
Coursera.user = this;
if (!Backbone.history.start({
pushState: true
})) {
Coursera.router.trigger("error", 404);
}
});
SPAs: Regions
Backbone lets you create views and render views into arbitrary parts of your DOM, but many developers soon run into the desire for standard "regions" or "layouts". We want to specify different parts of their page, and only swap out the view in those parts across routes - like the header, footer, and main area. That's a better user experience, since there's no unnecessary refreshing of unchanging DOM.
For that, we use origami.js,
a custom library that lets us create regions associated with views, and then in a route, we'll specify which region we want to replace with a particular view file,
plus additional options to pass to that view. In the view, we can bind to region events like "view:merged" or "view:appended" and take appropriate actions.
In our SPAs, we always render into the regions instead, so our routes code looks more like this. It is a bit of an unwieldy syntax, but it gets the job done:
routes[triageurl + '/items/:id'] = function(id) {
Coursera.region.open({
"pages/home/template/page": {
regions: {
body: {
"pages/triage/views/MainView": {
id: "MainView",
initialize: {
openItemId: id
}
}
}
}
}
});
};
We could also consider using: Marionette.js or Chaplin.
SPAs: Dirty Models
In traditional web apps, it's common practice to warn a user before leaving a page that they have unsaved data, using the window.onunload event.
However, we no longer have that event in Backbone SPAs, since what looks like a window unload is actually just a region swap in JS.
So, we built a mechanism into origami.js that inspects a view for a "dirty model" before swapping a view,
and it throws up a modal alert if it detects that.
To utilize this, a view needs to specify a hasUnsavedModel function and return true or false from that:
var view = Backbone.View.extend({
// ...
hasUnsavedModel: function() {
return !this.$saveButton.is(':disabled');
}
});
SPAs: Internal Links
In traditional web apps, it is easy to link to a part of a page using an internal anchor, like /terms#privacy. However, in a SPA, the hash cannot be used for internal anchors, since it is used as the fallback technology for the main URL in some browsers, and the URL would actually be /#terms#privacy. We have experimented with various alternative approaches to internal links, and the current favorite approach is to use a URL like /terms/privacy, define a route that understands that URL, pass the "section" into the view, and use JS to jump to that part of the view, post-rendering. For example:
In the routes file:
routes[home + "about/terms/:section"] = function(section) {
Coursera.region.open({
"pages/home/template/page": {
regions: {
body: {
"pages/home/about/tosBody": {
initialize: {section: section}
}
}
}
}
});
};
In the view file:
var tosBody = body.extend({
initialize: function() {
var that = this;
document.title = "Terms of Service | Coursera";
that.bind("view:merged", function(options) {
if(options && options.section)
util.scrollToInternalLink(that.$el, options.section);
else
window.scrollTo(0,0);
});
},
// ...
});
In the Jade template:
h2(data-section="privacy") Privacy Policy
Widgets
In some cases, we do not necessarily want our Backbone view to take full control over the URL, like if we want to easily have arbitrary, multiple Backbone views on the same page. We take that approach in our class platform, because that will ultimately make it easier for professors who want to compose together views to their own liking (i.e. if they'd like to mix a forum thread and a wiki view on the same page, that should be easy for them.)
To create a widget, we use a declarative HTML syntax, specifying data attributes that define the widget type and additional attributes to customize that instance of the widget:
<div data-coursera-reporter-widget
data-coursera-reporter-title=""
data-coursera-reporter-url="">
Just one moment while we load up our reporter wizard...
</div>
Then, we create a widgets.js file that will be included on that page, and knows how to turn DOM elements into Backbone views.
Typically that file would know about multiple widgets, but we show one here to save space:
define([
"jquery",
"underscore",
"backbone",
'pages/forum/app',
'pages/forum/views/ReporterView'
],
function($, _, Backbone, Coursera, ReporterView) {
$(document).ready(function() {
$('[data-coursera-reporter-widget]').each(function() {
var title = $(this).attr('data-coursera-reporter-title');
var url = $(this).attr('data-coursera-reporter-url');
new ReporterView({el: $(this)[0],itemTitle: title, itemUrl: url}).render();
});
});
});
Widgets: Maintaining State
We still want to maintain state within those views and support the back button, however, without changing the main URL of the page.
jQuery BBQ is an external non-Backbone specific library for maintaining history in the hash, and as it turns out, it works pretty well with Backbone. You can read my blog post on it for a detailed explanation.
We could also considering using: Backbone.Widget.
Testing Architecture
First, let it be said: testing is important. We are building a complex product for many users that will pass through many engineer's hands, and the only way we can have a reasonable level of confidence in making changes to old code is if there are tests for it. We will still encounter bugs and users will still use the product in ways that we did not expect, but we can hope to avoid some of the more obvious bugs via our tests, and we can have a mechanism in place to test regressions. Traditionally, the frontend has been the least tested part of a webapp, since it was traditionally the "dumb" part of the stack, but now that we are putting so much logic and interactivity into our frontend, it needs to be just as thoroughly tested as the backend.
There are various levels of testing that we could do on our frontends: Unit testing, integration testing, visual regression testing, and QA (manual) testing. Of those, we currently only do unit testing and QA testing, but it's useful to keep the others in mind.
Unit Testing
When we call a function with particular parameters, does it do what we expect? When we instantiate a class with given options, do its methods do what we think they will? There are many popular JS unit testing frameworks now, like Jasmine, QUnit, and Mocha.
We do a form of unit testing on our Backbone models and views, using a suite of testing technologies:
- Mocha: An open-source test runner library that gives you a way to define suites of tests with setup and teardown functions, and then run them via the command-line or browser. It also gives you a way to asynchronously signal a test completion. For example:
describe('tests for the reporter library', function() { beforeEach(function() { // do some setup code } afterEach(function() { // do some cleanup code } it('renders the reporter template properly', function() { // test stuff } it('responds to the ajax request correctly', function(done) { // in some callback, call: done(); } }); - Chai: An open-source test assertion library that provides convenient functions for checking the state of a variable, using a surprisingly readable syntax. For example:
chai.expect(2+2).to.be.equal(4); chai.expect(2+2).to.be.greaterThan(3); - JSDom: An open-source library that creates a fake DOM, including fake events. This enables us to test our views without actually opening a browser, which means that we can run quite a few tests in a small amount of time. For example, we can check that clicking changes some DOM:
var view = new ReporterView().render(); view.$el.find('input[value=quiz-wronggrade]').click(); var $tips = view.$el.find('[data-problem=quiz-wronggrade]'); chai.expect($tips.is(':visible')) .to.be.equal(true); chai.expect($tips.find('h5').eq(0).text()) .to.be.equal('Tips'); - SinonJS: An open-source library for creating stubs, spies, and mocks. We use it the most often for mocking out our server calls with sample data that we store with the tests, like so:
var forumThreadsJSON = JSON.parse(fs.readFileSync(path.join(__filename, '../../data/forum.threads.firstposted.json'))); server = sinon.fakeServer.create(); server.respondWith("GET", getPath('/api/forum/forums/0/threads?sort=firstposted&page=1'), [200, {"Content-Type":"application/json"}, JSON.stringify(forumThreadsJSON)]); // We call this after we expect the AJAX request to have started server.respond();We can also use it for stubbing out functionality that does not work in JSDom, like functions involving window properties, or functionality that comes from 3rd party APIs:
var util = browser.require('js/lib/util'); sinon.stub(util, 'changeUrlParam', function(url, name, value) { return url + value;}); var BadgevilleUtil = browser.require('js/lib/badgeville'); sinon.stub(BadgevilleUtil, 'isEnabled', function() { return true;});Or we can use it to spy on methods, if we just want to check how often they're called. Sometimes this means making an anonymous function into a view method, for easier spy-ability:
sinon.spy(view, 'redirectToThread'); // do some stuff to call function to be called chai.expect(view.redirectToThread.calledOnce) .to.be.equal(true); view.redirectToThread.restore();
Besides those testing-specific libraries, we also use NodeJS to execute the tests, along with various Node modules:
- require: Similar to how we use this in our Backbone models and views to declare dependencies, we use require in the tests to bring in whatever libraries we're testing.
- path: A library that helps construct paths on the file system.
- fs: A library that helps us read our test files.
Let's see what all of that looks like together in one test suite. These are a subset of the tests for our various about pages. The first test is a very simple one, of a basically interaction-less, AJAX-less posts. The second test is for a page that does an AJAX call:
describe('about pages', function() {
var chai = require('chai');
var path = require('path');
var env = require(path.join(testDir, 'lib', 'environment'));
var fs = require('fs');
var Coursera;
var browser;
var sinon;
var server;
var _;
beforeEach(function() {
browser = env.browser(staticDir);
Coursera = browser.require('pages/home/app');
sinon = browser.require('js/lib/sinon');
_ = browser.require('underscore');
});
describe('aboutBody', function() {
it('about page content', function() {
var aboutBody = browser.require('pages/home/about/aboutBody');
var body = new aboutBody();
var view = body.render();
chai.expect(document.title).to.be.equal('About Us | Coursera');
chai.expect(view.$el.find('p').size()).to.be.equal(6);
chai.expect(view.$el.find('h2').size()).to.be.equal(3);
});
});
describe('jobsBody and jobBody', function(){
var jobs = fs.readFileSync(path.join(__filename, '../../data/about/jobs.json'), 'utf-8');
var jobsJSON = JSON.parse(jobs);
beforeEach(function() {
server = sinon.fakeServer.create();
server.respondWith("GET", Coursera.config.url.api + "common/jobvite.xml",
[200, {"Content-Type":"application/json"}, jobs]);
});
it('job page content', function(done) {
var jobBody = browser.require('pages/home/about/jobBody');
var view = new jobBody({jobId: jobsJSON[0].id});
var renderJob = sinon.stub(view, 'renderJob', function() {
renderJob.restore();
view.renderJob.apply(view, arguments);
chai.expect(view.$('.coursera-about-body h2').text())
.to.be.equal(jobsJSON[0].title);
done();
});
view.render();
chai.expect(document.title).to.be.equal('Jobs | Coursera');
server.respond();
});
});
Integration testing
Can a user go through the entire flow of sign up, enroll, watch a lecture, and take a quiz? This type of testing can be done via Selenium WebDriver, which opens up a remote controlled browser on a virtual machine, executes commands, and checks expected DOM state. The same test can be run on multiple browsers, to make sure no regressions are introduced cross-browser. They can be slow to run, since they do start up an entire browser, so it is common to use cloud services like SauceLabs to distribute tests across many servers and run them in parallel on multiple browsers.
There are client libraries for the Selenium WebDriver written in several languages, the most supported being Java and Python. For example, here is a test for our login flow that enters the user credentials and checks the expected DOM:
from selenium.webdriver.common.by import By
import BaseSitePage
class SigninPage(BaseSitePage.BaseSitePage):
def __init__(self, driver, waiter):
super(SigninPage, self).__init__(driver, waiter)
self._verify_page()
def valid_login(self, email, password):
self.enter_text('#signin-email', email)
self.enter_text('#signin-password', password)
self.click('.coursera-signin-button')
self.wait_for(lambda: \
self.is_title_equal('Your Courses | Coursera') or \
self.is_title_equal('Coursera'))
We do not currently run our Selenium tests, as they are slow and fragile, and we have not had the engineering resources to put time into making them more stable and easier to develop locally. We may out source the writing and maintenance of these tests to our QA team one day, or hire a Testing engineer that will improve them, or both.
Visual regression testing
If we took a screenshot of every part of the site before and after a change, do they line up? If there's a difference, is it on purpose, or should we be concerned? This would be most useful to check affects of CSS changes, which can range from subtle to fatal.
There are few apps doing this sort of testing, but there's a growing recognition of its utility and thus, we're seeing more libraries come out of the woodwork for it. Here's an example using Needle with Selenium:
from needle.cases import NeedleTestCase
class BBCNewsTest(NeedleTestCase):
def test_masthead(self):
self.driver.get('http://www.bbc.co.uk/news/')
self.assertScreenshot('#blq-mast', 'bbc-masthead')
There's also Perceptual Diffs, PhantomCSS, CasperJS, and SlimerJS. For a more manual approach, there's the Firefox screenshot command with Kaleidoscope. Finally, there's dpxdt (pronounced depicted).
We do not do visual regression testing at this time, due to lack of resources, but I do think it would be a good addition in our testing toolbelt, and would catch issues that no other testing layers would find.
QA (manual) testing
If we ask a QA team to try a series of steps in multiple browsers, will they see what we expect? This testing is the slowest and least automate-able, but it can be great for finding subtle usability bugs, accessibility issues, and cross-browser weirdness.
Typically, when we have a new feature and we've completed the frontend per whatever we've imagined, we'll create a worksheet in our QA testing spreadsheet that gives an overall description of the feature, a staging server to test it on, and then a series of pages or sequences of interactions to try. We'll also specify what browsers to test in (or "our usual" - Chrome, FF, IE, Safari, iPad), and anything in particular to look out for. QA takes about a night to complete most feature tests, and depending on the feedback, we can put a feature through multiple QA rounds.
Additional Reading
The following slides and talks may be useful as a supplement to this material (and some of it served as a basis for it):
No comments:
Post a Comment