Thursday, May 2, 2013

Server-side HTML vs. JS Widgets vs. Single-Page Web Apps

At the recent GOTO Chicago conference, I gave a talk on "Frontend Architectures: from the prehistoric to the Post-modern." In just my first 10 months at Coursera, I've experienced the joys and woes of many different frontend architectures, and I wanted to share what I learnt. I detail everything in the slides, but I'll summarize my thoughts here as well.

How do you pick an architecture?

We could make our decision by just checking Twitter and seeing what all the cool kids are talking about, but uh, let's pretend that we're more scientific about it than that, and figure out whats important to us as web developers.

To start off with, here are a few things that users care a lot about:

  • Usability: they can do what they want to do, quickly and intuitively
  • Linkability: they can bookmark and link to parts of the site
  • Searchability/Shareability: they can share what they're using on social networks, and find it on Google (this is important for user growth)
  • More features, less bugs: 'nuff said.

Of course, we also should think about what we as developers care about:

  • Developer Productivity: we want to be able to iterate fast, try new things, and implement features in a way that makes us feel code
  • Testability: we want confidence that our code won't break, and that we won't spend most of our time maintaining a decaying, fragile product
  • Performance: we want our servers to respond quickly and to serve minimum number of requests, and we don't want our users to wait unnecessarily.

And when developers are happier, we can more quickly and safely improve the product, and that makes users happy.

Let's compare a few architectures...

So that's what I looked at when I reviewed our different architectures, and as it turns out, there are definite differences between the architectures in those respects.


Server-side HTML ("Web 1.0")


In much of our our legacy codebase on class.coursera.org (where instructors create their courses with a combination of lectures, quizzes, assignments, wikis, and forums), we use PHP to output HTML, and it handles much of the interaction via page reloads. A bit of JS is sprinkled (in a not so pretty manner) throughout.

This architecture suffers the most in terms of usability - it's very hard for users to do many interactions in a small amount of time - but it does have definite benefits of easy linkability, shareability, and searchability.

As a developer though, I hate it. All of our data is entangled inside the HTML, and when we want to bring JS into the HTML, it quickly becomes an untestable mess. Plus, it ties us to a particular backend language, and if we want to change languages (which we're doing, to Scala/Play), we have to rewrite the presentation layer. Yes, it's possible to use this approach in a more elegant way, and try to use templates that are portable across languages, but it's just not likely.


JS widgets


For the class.coursera.org forums, we are starting to take a different approach: JS widgets. We write DIVs into the HTML with certain attributes (like data-coursera-forum-threads-widget) and then we include a widgets.js that turns those DIVs into Backbone views.

With this approach, we can create very dynamic interfaces with real-time updates, we can have decent linkability if we have server-side URLs and hash state, but we can't easily make the content in the widgets shareable and searchable, since bots don't understand the JS. However, that's not a big concern for us there, since much of the content is behind a login wall anyways.

From a developer perspective, we typically create APIs for the JS widgets to consume, and well-tested APIs mean that we can more easily create new frontends for the same data. After I ported our forums to the widgets approach, I was able to make new widgets for the same data in just a few hours, since I could re-use the same API. On the flip side, I have to spend more time writing tests for the frontend, since the user can change the state via sequences of interactions and many bugs may not surface until after a particular interaction.


Single-page web apps


On www.coursera.org, we serve the same HTML file for every URL, and that HTML file contains only require.js and a call to load a routes.js file. That routes file maps URLs to Backbone views, and we rely on the Backbone Router to figure out what view to load up and to manage history using the HTML5 history API (with the fallback hash technique in older browsers).

We get many of the same benefits and drawbacks as JS widgets with single page web apps, but there are a few key differences. We can have a potentially faster user experience because of the complete lack of true page reloads, but it's harder to do simple things like internal links (you'll end up with a double hash on older browsers), or listen to a window.onunload event. We would suffer from bad searchability here, but we decided that it's really important for users to be able to find and share the course descriptions, so we wrote a "Just in time renderer" that uses Selenium to render the HTML and serve that to bots instead.

On the developer side, it's much trickier to test, because our webapp now has state *across* routes, and our tests have to check different sequences of routes.


So which one's the best?

Trick question! None of them have it all, atleast not yet. I would argue that if you're doing anything highly interactive (like an admin interface), you really want to take a heavy-JS approach, and also I'll point out that we're still in the early days of JS-heavy architectures. We will get better at developers at tackling problems like testability, and bots and browsers will likely get better at problems like searchability and performance. So, yes, it's a bit of a risk to do go as far as the single page webapp approach today, but it will be less and less of a risk as time goes on.

In terms of JS widgets vs Single-page web apps, I think they each have their places. On class.coursera.org, I hope that we can give instructors increasing amounts of control and flexibility in how they put together their classes, and that's why I'm building out our functionality as widgets that can be combined together however they please. On www.coursera.org, where students browse classes, we are the only ones designing that experience, and we know what we want each page to be, so it makes more sense to go the single page web app approach. The only difference is in the routes vs. the widgets file (mapping URLs vs. transforming DIVs), so we can easily create Views and Models that we use in both apps and widgets.

For my full run-down of the good and bad bits about each approach, flip through the slides and click on the images in the slides to see more code, screenshots, or videos.

What's your approach? What do you like or dislike about it? Let me know in the comments!

No comments: