Reusing HTML/CSS/JS across Web & Mobile

At this point, there's only one of me working on EatDifferent. That's awesome because it means I get to learn everything about what it means to make a user-facing website, but it also means that I need to be careful about where I spend my time, since it is such a precious resource.

When I started it, EatDifferent was a web-only service, but I soon realized that when it comes to tracking your daily habits, users really want to be able to track on the go, from their mobile device. I realized I needed a real strategy for how I could offer it as a website, mobile-optimized website, and as multi-platform mobile app while keeping them all up-to-date and still having time to iterate on the core functionality. So I decided to find a way to reuse as much of my code across those offerings as possible. Here's how I do it:

Basic setup

• Web App: Flask microframework, Python App Engine. Designed for desktop web, but displays in mobile web browsers.
• Mobile App: PhoneGap, distributed on Android and iOS platforms.

Datastore

When a user logins in, I store their authentication in a session cookie. From then on, either my web app or mobile app can make XMLHttpRequests to fetch or save information for the user. In the mobile app, the calls are made over SSL (and in case you're wondering, the cross-domain restrictions aren't applied to files in Phonegap apps so that is not an issue).

HTML

The Flask microframework comes bundled with support for Jinja2 templates (like Django templates but better) so that's what I use for server-side templating in my web app. I mostly use the templates for includes/inheritance and not for variable rendering, as then I can keep my logic in JavaScript, making it easier to re-use that logic in the mobile app. For example, log.html extends from a base HTML and includes Jinja2 templates for the log sections.

I wanted to reuse much of the same HTML in my mobile app, so I use the Jinja2 template engine for it as well. My mobile app is actually one single HTML page, where each "page" is a DIV with a .mobile-page class, and many of the "pages" include Jinja2 templates (the same ones that are used by the web app). After I make changes to the base HTML or templates, I test them in a browser (using a Chrome extension to mimic a small screen), and then when I want to output them to a device, I render the templates using a script and copy them to the Android/iOS app folders.

CSS

I start off with Twitter Bootstrap for my CSS, because it makes for a slick but easy-to-customize foundation. I then use SASS for writing my own CSS rules, as I can write cleaner cross-browser CSS that way and can also do things like includes. I define my shared styles and variables in common.scss, and include that in web.scss (for the web app) and phonegap.scss (for the mobile app). I use CSS media queries in both common.scss and web.scss to define rules for smaller screens &emdash; some of those rules apply also to the mobile app, but some just to the web app.

JS

As I mentioned in an earlier post, I recently refactored my JavaScript to make it easier to share logic across the web and mobile app. I now have a shared.js for shared functionality, a web.js for web-specific functionality, and phonegap.js for mobile-specific functionality (makes sense, doesn't it? :). Since I also use different JS libraries across the web and mobile app versions, I use different rules in my Makefile to generate the final compressed JavaScript for the web and mobile apps.

Summary

Here's a Venn diagram that summarizes what's different and what's shared:

And here are screenshots comparing the log on the web versus in the mobile app:

I suspect that my HTML/CSS/JS for the mobile app version will diverge more as I try to make the app conform more to the expectations of mobile users (and iPhone users in particular), but I still like the idea of reusing as much of my code as I can. The less time I spend writing redundant code, the more time I can spend adding features and improving the EatDifferent service for all my users.

Using 3-Legged OAuth APIs with Flask

The Withings body scale is a nifty device — after you connect it to your wireless network and create an account, it wirelessly transmits your measurements to its site. It also tries to estimate your body fat ratio, so that even when your weight stays the same, you can see if your body fat ratio is changing. In the last week, I had several EatDifferent users start using a Withings scale to monitor their weight while they improve their eating habits, so I wanted to let them connect their accounts to their Withings accounts.

Fortunately, Withings offers an API for accessing user measurements. Their API uses OAuth for authentication and JSON for the responses. Since users could enter measurements at any time and they don't want developers polling their API constantly, they do the smart thing and let developers add subscriptions for each user, so that a URL on your server is pinged whenever there are updates for a user. The API is well-designed and standards-compliant, so I was looking forward to integrating with it.

The tricky part of using any OAuth API is setting up the flow — your site has to generate a request token from the OAuth provider, redirect the user to the OAuth provider to grant access, and then once the provider redirects back, your site has to exchange the request token for an access token and save the credentials. Finally, with those credentials, you can actually start using the API. Since the OAuth flow involves redirects and saving session information, the implementation varies depending on the server-side framework you're using.

For EatDifferent, I'm using the Flask Python microframework on Google App Engine, so I started my integration by looking for Flask OAuth examples. I started with the Flask OAuth extension, which wraps on top of the oauth2 library and provides various decorators for retrieving the session tokens and handling the redirect. The extension worked, but since I wanted to customize it more, I decided to go straight to the source and just write URL handlers and a Withings client library based on that oauth2 library. You can check out that gist to see the Withings client code, and read on for a description of my URL handlers.

To start off the authentication process, I have this authorize_withings() URL handler that creates a WithingsClient with my key and secret (provided by Withings on registration), gets a request token, saves them in the session as a tuple, and redirects to the authorization URL.

@app.route('/authorize-withings')
def authorize_withings():
    withings_client = withings.WithingsClient(WITHINGS_KEY, WITHINGS_SECRET)
    callback_url    = (util.get_host() + url_for('handle_withings_authorization') 
    request_token   = withings_client.get_request_token(callback=callback_url)
    session[SESSION_WITHINGS_TOKEN] = (
        request_token['oauth_token'],
        request_token['oauth_token_secret']
    )
    auth_url = withings_client.get_authorization_url(request_token) 
    return redirect(auth_url)

Withings should then redirect back to my handle_withings_authorization() URL handler that creates a WithingsClient (with the request token from the session), requests an access token, and saves the token as a property on the current User entity. By saving it in the datastore instead of session, I can access it at any time in the future too, not just for this session.

@app.route('/handle-withings-authorization')
def handle_withings_authorization():
    request_token   = session[SESSION_WITHINGS_TOKEN]
    withings_client = withings.WithingsClient(
        consumer_key       = WITHINGS_KEY,
        consumer_secret    = WITHINGS_SECRET,
        oauth_token        = request_token[0],
        oauth_token_secret = request_token[1])
    access_token       = withings_client.get_access_token(
        oauth_verifier = request.args['oauth_verifier'])
    withings_auth   = {
        'oauth_token':        access_token['oauth_token'],
        'oauth_token_secret': access_token['oauth_token_secret']
    }
    g.user.withings_auth   = util.serialize(withings_auth)
    g.user.withings_userid = access_token['userid']
    g.user.save()
    return redirect(url_for('device_settings'))

Whenever I want to use the Withings API for a user, I fetch their authentication information, create a WithingsClient, and fire off requests to the API. For example, I use this parse_withings() URL handler to respond to notifications from their API.

@app.route('/hook/parse-withings')
def parse_withings():
    user_id         = request.form.get('userid')
    user            = models.User.all().filter('withings_userid =', user_id).get()
    withings_auth   = util.deserialize(user.withings_auth)
    withings_client = withings.WithingsClient(
        consumer_key       = WITHINGS_KEY,
        consumer_secret    = WITHINGS_SECRET,
        oauth_token        = withings_auth['oauth_token'],
        oauth_token_secret = withings_auth['oauth_token_secret'],
        userid             = user.withings_userid)
    measurement_groups = withings_client.get_measurements(
        startdate=request.form.get('startdate'),
        enddate=request.form.get('enddate'))
    imports.import_withings(user, measurement_groups)
    return Response(status=200)

Now that I have the OAuth flow setup for Withings, I can easily support other APIs — whatever my users need most. ☺

Upgrading from jQuery Templates to jsRender

In making my recent port from jQuery to Zepto, I was the most worried about porting over from jQuery templates - I knew there were other templating engines out there (like mustache.js, which comes highly recommended) but I also know templating engines can be vastly different and I didn't want to spend a lot of time porting.

Fortunately, I found out that jQuery templates have actually been deprecated in favor of jsRender, a revision of the library that isn't dependent on jQuery. Perfect!

The templating language for jsRender is largely the same as jQuery templates, with a few purely aesthetic syntax changes and one logical difference — atleast from the perspective of the parts that I was using. The jsRender library is still in development, so the differences might change in the future, but in the meantime, I thought I'd write up a quick upgrade guide for anyone else porting over.

Syntax Changes

The operators are now prefixed with a "#". You can pretty much do a search and replace to port them over - {{if}} to {{#if}}, {{each}} to {{#each}}.

Previously, variables were outputted with ${var} - now, variables are outputted using the same double brackets as operators, like so: {{=var}} and {{=var.property}}. If you're using a server-side templating engine that also uses that bracket notation for templating, you will need to instruct the engine to ignore the jsRender templates. For Jinja2 templates, that means surrounding the script tags with {% raw %}{% endraw %}.

In addition, the syntax for specifying you don't want HTML escaping changed from {{html var}} to {{var!}} (notice the exclamation mark at the end). Finally, if you want to access the value of the currently referenced variable, the syntax changed from ${$value} to {{=$data}}.

To give you an idea of what an upgraded template would look like, here's some example data and templates:

var templateData = {
  label: 'Comments on post for Wednesday, Dec. 5th',
  comments: [
     type: 'freetext',
     creator: {fullName: 'Pamela Fox', profileUrl: 'http://everyday.io/user/1'},
     textHtml: 'Have you tried macademia nut oil instead? You can get it from <a href="http://www.amazon.com">.'
   ]
}

Using jQuery templates:

var dom = $('#stream-comment-tmpl').tmpl(templateData);

<script id="stream-comment-tmpl" type="text/x-jquery-tmpl">
  <span>${label}</span>
  {{each comments}}
  <div class="stream-comment">
    {{if type == 'highfive'}}
     <div>
      <span class="icon-highfive"></span>
      High five from <a href="${creator.profileUrl}">${creator.fullName}</a>!
     </div>
    {{else}}
     <a href="${creator.profileUrl}">${creator.fullName}</a>:
     {{html textHtml}}
    </div>
    {{/if}}
  </div>
  {{/each}}
</script>

Using jsRender templates:

var template = $.template('stream-comment-tmpl', document.getElementById('stream-comment-tmpl').innerHTML);
var html = $.render(templateData, template);

<script id="stream-comment-tmpl" type="text/jsrender-tmpl">
  <span>{{=label}}</span>
  {{#each comments}}
  <div class="stream-comment">
    {{#if type == 'highfive'}}
     <div>
      <span class="icon-highfive"></span>
      High five from <a href="{{=creator.profileUrl}}">{{=creator.fullName}}</a>!
     </div>
    {{else}}
     <a href="{{=creator.profileUrl}}">{{=creator.fullName}}"</a>:
     <div>{{=textHtml!}}</div>
    {{/if}}
  </div>
  {{/each}}
</script>

Array Handling

In jQuery templates, you could pass null as a value in an array and use the if operator to test whether it was defined. In jsRender, the template will not be called at all for a null value. Instead, you need to define objects in the array, set an object property to null, and check to see if that object property is defined. The reasoning behind the change is discussed more in this issue.

To give you an idea of the change, here's a before and after - notice I had to change the data format itself.

Using jQuery templates:

var templateData = ['Walked to work', null, null, 'Biked to work'];

<script id="notes-mini-tmpl" type="text/x-jquery-tmpl">
<div>
  {{each dates}}
    {{if $value}}
     ${value}
    {{else}}
     No notes for this date.
    {{/if}}
  {{/each}}
</div>
</script>

Using jsRender templates:

var templateData = [{notes: 'Walked to work.', {notes: null}, {notes: null}, {notes: 'Biked to work'}];

<script id="notes-mini-tmpl" type="text/jsrender-tmpl">
<div>
  {{#each dates}}
    {{#if notes}}
     {{=notes}}  
    {{else}}
     No notes for this date.
    {{/if}}
  {{/each}}
</div>
</script>

Porting from jQuery to Zepto

In my quest for better performance in my PhoneGap Android app, I finally decided to switch from using jQuery to Zepto, a framework that boasts a similar API with a much smaller footprint and additionally offers mobile specific touch events. Of course, since Zepto is designed to be lighter weight than jQuery, it does not offer all the same methods — and the methods that it does offer sometimes differ in their arguments. This makes sense but it means that porting from jQuery to Zepto requires learning how their differences affect your code, and how you can workaround them.

Since other developers might start going down the porting path too, I thought I'd write a post on the differences I encountered while porting. Because I reuse much of my application code between my Phonegap app and desktop web app and I'm not yet ready to eliminate jQuery in my desktop app, I wanted to port my code over in a way that would mean it would work with either jQuery or Zepto as a base framework. That effectively meant that I tried to keep my code the same, and simply add to the Zepto JS file where needed. I'll explain my ports below and then show my Zepto additions at the end.

This post got a bit long, so here are quick links to the sections:

• Porting core functionality
• Porting plugins
• Summary: What's different
• My Zepto modifications
• Performance comparison
• Conclusion: Was it worth it?

Porting core functionality

I spent most of my time porting 3rd party jQuery plugin code, as they tend to use the more esoteric jQuery features, but I also found a few places in my own application code where I was using jQuery functionality that's missing from Zepto:

• CORS:

  I use CORS (cross-domain XMLHttpRequests) to communicate from my mobile app to my server, and that means setting xhrFields in jQuery.ajax(), something that isn't yet supported in Zepto. Luckily, there's a patch for it, and I applied that. I also discovered a bug with sending empty strings in CORS requests, and I work around that by sending null instead.

• scrollTop:

  I use jQuery.scrollTop() to reset the scroll in my mobile app on page transition, so I took and Zepto-ified the jQuery implementation.

• position:

  I use jQuery.position() to calculate where to scroll to in some cases, and Zepto only supports offset(). I checked out the jQuery source code for calculating position and stuck that inside Zepto.

• non-standard selectors:

  jQuery supports several convenience selectors that aren't defined in the w3 spec and simply special cases them in their code, and these won't work as selectors in Zepto. There are some that correspond to methods - ":first" is .first() in Zepto, and ":last" is last(). There are others with no equivalents, like ":hidden", and ":visible", and I just removed my usage of them or checked display values manually. (I didn't feel like checking the jQuery source for them.)

• valueless attributes:

  There are times when I set and remove valueless attributes on form elements, like "checked", "readonly", and "disabled". In jQuery, I used .attr('disabled', false). That's apparently been replaced in jQuery 1.7 by .prop('disabled', false). Neither will work in Zepto, however, but removeAttr('disabled') will. Since I want my code working with both, I now have .attr('disabled', false).removeAttr('disabled'), just to be sure.

Porting plugins

I use five jQuery plugins in my mobile app, only one of which I wrote myself, and they took the longest to port. For all of these plugins, I started off by changing the final line in the file from '})(window.jQuery);' to '})(window.jQuery || window.Zepto);' and then hoped for the best.

• ColorSlider:

  I wrote this plugin to give users a colorful slider (red to green), and the only jQuery function it was missing was outerWidth(), so I wrote one based on the jQuery implementation. In doing so, I discovered that the Zepto width() function includes the padding and border, which jQuery.width() doesn't, so I submitted a patch clarifying that in the Zepto docs.

• jQuery Templates:

  I was pretty worried about this port, but then found out that the jQuery templates was being deprecated in favor of a non-jQuery dependent library, jsRender/jsViews. That library is in beta and the docs are minimal, but the templating syntax is largely the same, and the author responded quickly to my upgrade question. One thing to note, though: the jsViews library tries to take the global $ variable if it doesn't see jQuery defined, so you have to call $.noConflict() after loading it if you want to use $ with Zepto instead — but you don't want to call that if you are actually using jQuery. Here's what I have after I load my scripts: if (window.Zepto) $.noConflict();

• timeago:

  This handy library turns timestamps into pretty times like "1 day ago" and also auto refreshes elements with timestamps on an interval. It relies on jQuery.trim, which I just copied from their codebase. Along with the DateInput plugin, it relies on the ability to store objects as data attributes (instead of just strings), and thankfully, there's a data.js in the Zepto codebase that I copied into my Zepto JS to handle that.

• Twitter Bootstrap modal:

  This is a simple modal library that's designed to work with either jQuery or Ender(a lightweight package library), so it already relied on minimal jQuery features. It does attach functions to $.support, which Zepto doesn't define, so I added a one-liner to define it. It also uses $.proxy, which I copied from the jQuery codebase.

• jQuery Tools DateInput:

  This was by far the hardest plugin to port, and I did look around briefly to see if I should just switch to a completely different pure JS datepicker library, but I quite like this one so I stuck with with it. First, it uses $.expr, a jQuery object that isn't actually documented and (from what rumors tell me) might actually go away soon. I wasn't actually using the results of that function, so I simply defined the object so that the code would not error out. It also uses jQuery.clone and Event.isDefaultPrevented, which were simple enough to write my own versions of.

  Now we get to the tricky parts - methods that exist in both frameworks but don't have the same interface or behavior. I filed bugs on most of these, but the Zepto team could decide to keep their interface the same for simplicity: Zepto doesn't handle multiple self-closing tags in a creation string the same way as jQuery (Issue 322), Zepto does not assign this to the current iterated object in each() (Issue 295), and Zepto does not include an option for deep copy in the extend() function, and Zepto doesn't handle non-DOM objects in expressions/events (Issue 321, That last one unfortunately required a bit of dirty finagling in the DateInput code itself, as its non obvious how to extend Zepto to support binding and firing events on non-DOM objects.

Summary: What's different

To summarize, the following jQuery functionality is missing from Zepto (and much of it is in the gist below):

• scrollTop()
• position()
• ":first", ":last", ":hidden", ":visible"
• prop()
• outerWidth()
• trim()
• support()
• proxy()
• expr()
• isDefaultPrevented()

And this functionality differs in behavior/arguments:

• attr('disabled', false);
• ajax() CORS
• width()
• data()
• each()
• extend()

My Zepto modifications

Most of the above issues were resolved by defining functions or objects in Zepto, and this gist shows all of those:

I also made a few tweaks to the core Zepto code, and you can see those in this diff (but not all of them are necessarily successful tweaks, nor are they tested.)

Performance comparison

Since my main point in porting over to Zepto was to improve my loading performance, I celebrated the successful porting by measuring the loading times for my page. I used this timing code and compared the differences between including a minified jQuery 1.7 script tag and a minified Zepto script tag, both for a fresh app install and an app re-launch. On average, switching to Zepto shaved 22% off my total loading time. For the detailed results, check out this spreadsheet.

Was it worth it?

I'm happy that I ported, but it's not a decision to be taken lightly. jQuery alternatives are still in early stages, so if you use them, you have to think of yourself as a beta user, be prepared for issues, be prepared for lack of documentation, and do the responsible thing — report whatever issues or workarounds you find. And on that note, thanks to the Zepto authors for bearing with my barrage of issues over the last few days. :)

Logging JS Errors on iOS with PhoneGap

I've spent the last few days getting the EatDifferent PhoneGap app working on an iPhone (an app which previously worked on Android). The hardest part has been learning to debug in the iOS browser, so I thought I'd post on my findings:

• To view the output of console.log, you must open the XCode console. The iOS browser "Debug console" that most iOS debugging articles mention is only displayed in the standalone Safari browser, not in the WebView (where PhoneGap HTML lives).
• There seem to be times when console.log does not log the output (perhaps during loading?) - in that case, alert() always seems to work.
• If you log a JS object using console.log, it will just print "Object" by default. You must JSON stringify it to be useful.
• You can also use debug.phonegap.com (hosted weinre) to view the DOM and JS console logs as well.
• The WebView browser silently fails on JS errors - it stops running the JS code and does not report the error. To see the error, you must wrap the offending code in a try/catch block.

Given all of those learnings, here is my log() wrapper function that I use across my webapp:

    function log(something) { 
        if (window.console){ 
          if (something instanceof Date) { 
            something = something.toDateString(); 
          } 
          if (isIOS() || isAndroid()) { 
            if (typeof something == 'object') { 
              something = JSON.stringify(something); 
            } 
            console.log(something); 
          } else { 
            console.log(something); 
          } 
        } 
    } 

And I wrap various code blocks in try/catch, like the callback function for AJAX requests:

    try { 
      onSuccess(processJSON(responseJSON)); 
    } catch(e) { 
      log(e); 
    } 

I posted my observations in the PhoneGap group and the developers there made several recommendations: 1) use Ripple, a Chrome extension for mobile emulation 2) monkey-patch JS functions to always try-catch, as done in this library. I've taken a break from iOS debugging for a few days, but I'll probably revisit debugging soon and try out their ideas.

Code Quality Tools

Now that my EatDifferent application code is getting cleaner, I wanted to make it even cleaner by using automated code quality tools.

First, I ran jshint over all my *.js files and fixed a bunch of little issues (like using "===" instead of "==" in many places). Here's how I setup my Makefile to download jshint and run it:

Then I ran reindent.py script to fix the indenting on all my *.py files. I had been using 2-space indents, as that's we used at Google and also what I use in JavaScript, but I was convinced to go with the PEP8 standard, 4 spaces.

Then I downloaded SublimeLinter, a plugin for Sublime Text that automatically checks your code as you write it, using jshint for JS and pyflakes for Python. I configured that to ignore a few PEP8 warnings in the settings JSON ("pep8_ignore": [ "E501", "E221", "E203"]). I'm quite liking SublimeLinter - its lint tools actually catch a few things that could result in real bugs and its nice to be able to correct my code as soon as I type it.

PhoneGap Loading Performance in iOS

As I wrote about earlier, I've been working on the performance of my PhoneGap app, implementing many of the suggestions from this article.

One of the suggestions that article makes is to switch frameworks from something heavy like jQuery (which includes a lot of extra code that may not be necessary on mobile browsers) to something lighter like Zepto or XUI.

I currently use jQuery in my app -- not because I use jQuery a lot in my own code, but because I use a handful of third-party-written jQuery plugins. I could easily port over my own code to a new framework but I don't know how easy it would be to port over someone else's.

So, before I looked into porting from jQuery, I wanted to figure out exactly what effect jQuery had on performance. I'm most concerned with user-facing loading latency versus already-running performance as that's where I perceive the greatest latency in my app, so I decided to see how long it took for the browser to load my HTML, CSS, and JS.

First I wrote a basic function to record times of events and I used it to record events after the CSS, HTML, jQuery script tag, other script tag, document ready, and execution of my setup function This is a stripped down version of what it looked like:

Then I recorded the times in Chrome, iPhone Simulator, and actual iPhone (on the same WIFI network), and graphed them in a spreadsheet. Here's a chart of the results:

As you can see from the chart, the iPhone browser is noticeably slower than Chrome and the simulator, and the jQuery script tag (90 KB) takes 35% of the loading time - same as the custom script tag (220KB). But it's only 300ms, which isn't as slow as I expected and for now, it doesn't seem worth it for me to try to port away from jQuery.

Update (11/20/2011): I have since ported away from jQuery and written that up in this post..

Update (12/13/2011): It's been pointed out that when it comes to measuring the "loading times" for resources, this is not the most precise technique, as it it doesn't reveal the difference between the download time, the parsing time, and the execution time. (In the case of jQuery, I believe much of the time I recorded came from the execution.) For that breakdown, I've been recommended weinre.

JS & CSS Compiling, Compression & Cache-Busting

Everytime I deploy a new version of the CSS and JavaScript for EatDifferent to production, I run it through a series of steps to ensure code quality and performance:

• Code quality: I use JSHint to check for JavaScript code quality issues. Sometimes it's a matter of style, but other times it actually finds issues that can become runtime bugs.
• Concatenation: I use cat to combine my JS files and CSS files into one file each, so that the browser can issue less HTTP requests when loading the page.
• Compression: I use Closure Compiler to minify my JS and YUI Compressor to minify my CSS, so that those HTTP requests are smaller.
• Cache bust: I append the current timestamp as a query parameter to the JS and CSS in my base template HTML. I serve the files as static files off App Engine which would normally result in browsers caching them forever, but by appending new query parameters for each deploy, I force the browsers to re-download them only when they've changed.

I do all of this in a Makefile, including downloading the necessary tools. You can see the relevant bits in this gist:

Modularizing My JavaScript

I generally try not to get too distracted by the code quality in EatDifferent and focus on user-facing quality instead, but after a while, it hurts my head knowing that my code is messy -- and makes me not want to mess with the code further. So, I spent yesterday spring cleaning my JavaScript.

One of the big improvements was to take my smattering of global functions and put them in a namespace. There are a lot of ways to namespace in JS, but I opted for the module pattern described in this article (#3).

Here's the basic template for each module's JavaScript file -- notice how this technique lets me create functions in each module that are only used inside the module, and aren't exposed outside of it.

    var ED = ED || {};

    ED.util = (function() {

      function doSomethingPrivate() {
      }

      function doSomething() {
        doSomethingPrivate();
      }

      return {
        doSomething: doSomething
      }
    })();

I ended up with 6 JS modules used across the web and mobile (PhoneGap) version of the app.

ED.util	Utility functions, app-independent
ED.data	Global data constant definitions
ED.models	Classes representing data from the server DB
ED.shared	App functionality shared by web and mobile
ED.web	Web specific functionality
ED.mobile	Mobile specific functionality

I could have put the first 4 of these in "shared" but I like the conceptual division, and I'm concatenating them together before serving them to the user, so it doesn't hurt to have multiple files.

Now that my code is cleaner and more manageable, I can more confidently iterate on user-facing features!

Grammatical Personalization in JS

In EatDifferent, I have various places where I describe something about a user. For example:

• "Pamela Fox filled out her logs 3 days in a row.
• "In the last week, you haven't logged any measurements."

I wanted to be able to construct those sentences in JavaScript with some sort of mini templating language. I didn't find any existing libraries for it, so I wrote my own. Now to generate the above strings, I can write stuff like:

personalize('{{ They|name }}' filled out {{ their }} logs 3 days in a row.', 
   {gender: 'male', person: 'third', name: 'Pamela Fox'});
personalize('In the last week, {{ they|name }} {{ have|not }} logged any measurements.',
    {gender: 'male', person: 'first'});

You can see the code for the library in this gist and check out a live demo on jsfiddle.

Ada Lovelace Day: Women that Inspire Me

Today is Ada Lovelace Day, a day to celebrate the women of math, science, and technology, and the challenge today is to write a blog post highlighting a woman that inspires us. I blogged an interview with my mum a few years ago (the first woman in tech I ever knew!), so this year, I thought I'd highlight a few women that inspire me today.

• Sara Chipps:

  Sara started GirlDevelopIt in New York, with the goal of getting more women in web development by actually teaching them how to do it. She inspired me to run a GirlDevelopIt in Sydney, and I still remember those months as some of my favorite down under. In addition to being the force behind GDI, Sara still develops - just yesterday, she wrote an uber-useful Chrome extension that I installed instantly.

  Follow Sara online: Twitter | Google+ | sarajchipps.com

• Gina Trapani:

  I first knew of Gina for starting lifehacker, a popular blog of tips for using tech and optimizing your digital life (and the biggest source of traffic for my little Profane Game). I then knew her as the author of "The Complete Guide to Google Wave", an attempt to explain how to use Wave to the average joe. I was on the Wave team then, and I knew that was no small feat. And now, I know her as a developer, working on an open-source web app for social media analysis. Blogger, author, coder, speaker - Gina does it all.

  Follow Gina online: Twitter | Google+ | ginatrapani.org

• Leigh Fonseca:

  I met Leigh just this week, at the WITI Next-Gen Summit, where Leigh and I talked to high school girls about how we got into tech and why we love it. Leigh started off with a history degree, but when she realized she had a thing for analyzing customer data, she started data science departments at Rhapsody and then BookRenter, where she is now. Most companies don't have data scientists nor realize they need one, so Leigh is paving the way for others with her interest. When she's not analyzing data, she's preparing for triathlons. Impressive.

  Follow Leigh online: Twitter | Google+ | dataparty.blogspot.com

• Kendall Ronzano:

  I also met Kendall this week - she's actually one of the girls that Leigh and I talked to at the conference. She's a high school junior who's already started programming for her robotics team (C++!) and wants to learn more. She's also set a fun challenge for herself: build a house. I hope Kendall represents the next generation of women in tech!

So, who inspires you?

User Stats Visualization with HighCharts

For the first few months of EatDifferent's existence, it was only a small set of users -- family and friends -- and half of them were my buddies on the app. But then a food blogger tweeted it, and over the course of a day, there were 10x the number of users -- all strangers. Since I wasn't buddies with the new users and wouldn't see their updates in my stream, I needed a way of understanding who all these users were and how they were using the app

I wanted simple visualizations of my user demographics & usage data, and after a bit of research on what type of chart to use (i.e. not pie charts), I came up with this stacked horizontal bar chart visualization:

All the bars add up to 100%, the bar parts show their value and percentage on hover, and the bar parts are colored according to their value. For example, true is green and red is false, browsers and OSes are colored according to their debuggability (Ghrome is green since that's my browser of choice, guess which one is red?), and numeric values are colored in a monotone gradient. The charts are made using HighCharts, my favorite charting library.

It's nice being able to understand more about my users in a glance. :)

Client-Side Error Logging

After setting up my Python code to send server-side errors to my inbox, I wasn't satified. I wanted to know about JavaScript errors, too! So I set up my base template to send JavaScript errors to my server and log those as errors. Now I find out about all errors immediately in my inbox, and I can deploy a fix before it affects users for too long. So far I've found a couple errors this way that otherwise I wouldn't have known about (due to different user behaviors + browsers).

I use this JavaScript for sending errors to a handler on the server, which then logs them as an error using logging.error:

When I tweeted about my solution, I was pointed to a few related projects and techniques:

• An article with various tips on logging JS errors.
• jserrlog.appspot.com: A solution hosted on App Engine, where you just drop in the JS and view the logs on their server.
• Errorception: A startup devoted to "painless javascript error tracking" and will include more cross-browser logging, emails, and web hooks. Sounds promising!

Since deploying my solution, I've discovered that window.onerror doesn't work in all browsers (even some "modern" ones like Android), so if you are looking to catch every error, you should checkout those links to see the workarounds they employ.

Update (2/20/2012): I've updated my function to ignore errors that I know aren't mine, like those generated from 3rd party scripts. I could also update it to just only show errors from my server's JS, but for now I'm interested in what other errors folks encounter. You can see that updated code in this gist.

Pre-Deploy Git Check

Since I'm the only one developing EatDifferent, I've been content to store it in a Dropbox folder and use a local git repository for version control. But I got nervous thinking about what would happen if something went horribly wrong while I was unavailable (like on a plane), and decided I needed a backup collaborator -- someone who could quickly checkout the code and deploy a fix.

I probably could have shared the Dropbox folder with them, but I instead opted for a private Github plan for $7 a month, which lets me share my code repository with my backup collaborator. Plus, I now get to browse my code history and commits via Github's slick online interface.

I often forgot to git commit and push my changes, so I added this check to my Makefile which errors if there are uncommited changes. I run the check everytime I deploy my code to App Engine, to make sure the code always reflects what's deployed.

So far, the workflow seems to work...and now I can fly a plane in peace. ☺

Sending Errors to Email in App Engine

When my EatDifferent server-side code results in an error or exception, it gets logged in the App Engine dashboard. I was getting anxious wondering if there are new errors in the logs, refreshing the dashboard every few hours, so I decided to make sure all errors get immediately emailed to me and researched how best to do that.

First I checked out the App Engine ereporter module , but it was set to aggregate errors and email reports once a day -- not frequent enough for anxious me.

I then found a developer that wrote loggers to send errors over Channel API, PubNub, and XMPP, with a throttling mechanism built-in to prevent getting spammed when many errors happen at once.

I took his XMPP logger and converted it to send email instead, and set the throttling threshold to 5 minutes instead of 1 second. Now I feel much less anxious! You can grab the code from this gist:

Switching from jQuery Mobile to Twitter Bootstrap

I've been using jQuery mobile (jQM) to make a mobile-enhanced version of EatDifferent for the past few weeks and though I have much respect for the team behind it, I've grown increasingly frustrated with it. Why?

1. It typically enhances by adding additional DOM elements to the original elements - which can help in making them easier to use on a mobile device, but it means that whenever you want to tweak the styles of an enhanced element, you need to dig into the DOM and apply CSS overrides at various levels. If the DOM or inner CSS names change in later jQM versions, you may have to update those CSS overrides.
2. If you dynamically update an enhanced DOM element (like a form input, for example), you often have to tell jQM to force refresh that element, since it needs to redo the added DOM. That means you need jQM-specific JavaScript calls in your code.

I was fine with those issues when I used jQM for an earlier project, but for the EatDifferent app, they are becoming bigger issues because: 1) I'm doing more customization, and 2) I'm doing more dynamic form creation and updating.

And most importantly, I'm trying to re-use my HTML/CSS/JS across both my the desktop web version and mobile app, and I want minimal difference between them - both so users have a consistent user experience across them, and so that I can write the code just once.

So today I scrapped jQM and decided to use just HTML/CSS. And since I suck at making pretty CSS, I'm using Twitter's bootstrap library. It's pure CSS (no JS!), it's simple to use, and it's pretty. It's not specifically designed for mobile, but it works well enough on it and is easy to customize when it doesn't quite work.

Here's what the app looked like using jQuery mobile and the default black theme:

Here's what it looks like using Twitter bootstrap and some basic customization:

As you can see, jQuery mobile does create more mobile-optimized form input interfaces (larger areas for clicking, e.g.), but Twitter bootstrap creates clean CSS that I can easily add my own mobile-optimized CSS on top of when necessary. I'm happy I made the switch.

Spriting with Compass

I use little icons in various places on EatDifferent, like to show food bonuses in the stream:

The browser can take a while to load lots of images since it has to make a request for each one, so I decided to implement icon spriting - baking all the images into one image, and using background-position in CSS to make what appears like a standalone icon.

Thankfully, I was already using SASS and Compass for my CSS, and it comes with built in spriting.

After putting my icons in one folder (and sizing them all to be 16*16 pixels), I added this to the top of my .scss file:

    @import "icon/*.png";
    @include all-icon-sprites;

I also specified sizing and display properties for the special "icon-sprite" class in my .scss file:

    .icon-sprite {
      width: 16px;
      height: 16px;
      display: inline-block;
      margin-right: 2px;
    }

Compass then auto-generated CSS rules for my icons - one rule to specify the background for each of them, and a rule per icon to specify the background-position. It also applies any of the ".icon-sprite" rules it found to all of the generated icon classes. Here's a snippet of the auto-generated rules:

    .icon-sprite, .icon-activity, .icon-android, .icon-bodylog, .icon-buddies, .icon-camera, .icon-comment, .icon-edit, .icon-female, .icon-foodlog, .icon-grassfedmeat, .icon-highfive, .icon-home, .icon-homecooked, .icon-localfood, .icon-logs, .icon-organicveg, .icon-profile, .icon-reminder, .icon-settings, .icon-settings2, .icon-stats, .icon-stats2, .icon-sustseafood, .icon-tip {
      background: url('/img/icon-s97f5308db7.png') no-repeat;
    }

    .icon-activity {
      background-position: 0 0;
    }

    .icon-android {
      background-position: 0 -27px;
    }

    /* line 99, ../sass/_common.scss */
    .icon-sprite, .icon-activity, .icon-android, .icon-bodylog, .icon-buddies, .icon-camera, .icon-comment, .icon-edit, .icon-female, .icon-foodlog, .icon-grassfedmeat, .icon-highfive, .icon-home, .icon-homecooked, .icon-localfood, .icon-logs, .icon-organicveg, .icon-profile, .icon-reminder, .icon-settings, .icon-settings2, .icon-stats, .icon-stats2, .icon-sustseafood, .icon-tip {
      width: 16px;
      height: 16px;
      display: inline-block;
      margin-right: 2px;
    }

I measured the loading performance of my site before and after spriting, using the HAR Viewer, and these are the results:

Before:	28 requests	2.61s (onload: 1.92s, DOMContentLoaded: 1.64s)
After:	15 requests	1.09s (onload: 817ms, DOMContentLoaded: 600ms)

As you can see, spriting had a significant effect on performance. I definitely recommend spriting (and Compass) for sites that display multiple images on page load.

WDCNZ: The Developer Experience

Last month, I had the honor of giving the first talk of the day at WDCNZ, a new conference in Wellington, New Zealand. WDCNZ was the brainchild of Owen Evans, one of the lead developers at Xero, and his goal in putting it together was to provide a full day of technical talks for web developers and to bring the local web development community together. I wanted to give a talk that would be interesting to web developers of all different sorts, and also would be something I'm passionate about myself - so I decided to give a talk around a topic I wrote a handbook about and discussed in recent blog posts - "The Developer Experience."

For my talk, I wanted to explain what I think of as "developer experience", motivate people to really care about it, and give concrete examples of what makes for a great experience. It was an interesting talk to prepare and give, and hopefully gets other people thinking about the topic and giving similar talks in the future. I've embedded the slides and video below so you can watch for yourself what I said. Thank you to WDCNZ for giving me the opportunity to give the talk and for an awesome conference!

The Developer Experience (Slides)

The Developer Experience from WDCNZ on Vimeo.

So, what am I, exactly?

Lately I have struggled to find a good answer to the question "So, what are you?". When I was a kid, nobody asked me that - I was just a kid. At the most, they'd ask me what grade I was in, and that was easy. Then, in my first (and only) job at Google, when people asked me that, I could answer with my job title. But now that I'm not a kid or a Google employee, I'm not quite sure what I am.

My standard response has been "web developer." I do web development, I go to all the web development events, and all the people I helped as a Googler were web developers. But I'm not convinced, and I become less convinced the more I am approached by people looking for "web developers" thinking that I fit the bill. I know this doesn't bode well for my job opportunities in the future, but I don't think I'm actually a "web developer", at least not the kind they are looking for.

See, when I do web development, I do it because it is the best way that I know to turn an idea into something tangible; I don't do it purely for the joy of programming. I don't dream of spending all day coding, I dream instead of making things that I enjoy, and maybe a few others too. In fact, I don't think I would be happy spending all day programming for months on end; I like too many other aspects of making things. I want to write about them, I want to brainstorm them, I want to tell the world about them. I want to spend equal parts of my life using the different parts of my brain, and I want to be able to share creations with the world as a result of that synergy.

Why does it matter what I am, anyway? Well, first, I need an answer for all the people that I meet now, who want to be able to fit me into a nice bucket in their head. But, second, I need to figure out what I'm doing next in life. If I am a web developer, then I should probably join a new company, learn more web skills, and increase my knowledge there. But if I am not - and I suspect as much - maybe I should be picking a different path. I have done many side projects, and now it might be time for me to pick an idea and make it my main project, to see how far I can take it and how many people I can share it with. Or, put in the more usual terms of today's web world, perhaps I should become a (co-)founder.

I am not 100% sure of this - part of me is worried that I don't have the focus to pick one idea and stay with it, part of me is worried that I'll pick the wrong idea, and part of me is worried that I'm not business-oriented enough to make something profitable. But the other part of me thinks that I should find out or I risk spending the rest of my life wondering.

Reading My Way to Better Design

I realized recently that I am completely capable of coding up a website frontend, but I struggle to make that website frontend look really good. My webdesign tends to be either too simple (“austere” is how one observer described it) or too tacky (my love for everything 80s seeps into all parts of my life). I was okay with this before when I was just making developer-facing demos, but now I’m at a stage where I want to make user-facing webapps, and I want the users to experience them and think “Wow, slick.”

So, I’ve set out to acquire a better taste in design, and I decided one way to do that is to surround myself on a daily basis with examples of great design. After Google’ing around and reading this HackerNews thread, I set up a folder in my Google Reader of 10 design-related blogs, and I now spend every morning reading through the posts (after drooling over my recipes feed, of course). It’s a great mix of topics, ranging from web design how-tos to well-designed physical products. It's hard for me to quantify if my design skills are improving, but I am at least thinking much more about it than before and acknowledging its importance.

If you want to do the same as me, you can subscribe to my folder feed or subscribe to the individual blogs listed below:

• A List Apart (RSS)
• Brain Pickings (RSS)
• I love typography (RSS)
• Logo Design Love (RSS)
• Noisy Decent Graphics (RSS)
• Smashing Magazine (RSS)
• swissmiss (RSS)
• UI-patterns.com (RSS)
• UX Movement (RSS)

Please let me know in the comments if you have other suggestions for great blogs to read (and books too, while you’re at it).

No, Really, I'm Shy: My Ignite I/O Talk

Update: I've now created a site with all my tips for event attendees and organizers that grapple with shyness. Check it out at ProjectIcebreak.com

As many of you know, I'm kind of an Ignite junkie. I love the format - 5 minutes, 20 slides, 15 seconds auto-advance - and I love that I can use that format to talk about topics that I can't fill a whole hour with. At this year's Google I/O 2011 Ignite show, I got to give a talk on something that I think about a lot: my shyness. Most people don't believe me when I tell I am actually quite shy at my core, and so I dedicated my five minutes to explaining what shyness is to me and how I workaround it. The talk went well, and multiple folks approached me afterwards about their own shyness and hacks around it. Mission accomplished!
You can watch the video on Youtube - I'm at 24:09:

And/or you can read the slides on Slideshare:

No, Really, I'm Shy

Roundup: Developer Documentation Generators

While researching what developers love about their favorite API documentation sets for my last post, I also investigated what tools the API teams use to generate the documentation. Some teams just hand-write HTML, but most use code-based generators for the references and/or some sort of markdown for writing the guides. A good tool or toolset should make it easy to update the reference for every API release (outdated documentation should be avoided at all costs) and painless to write documentation about how to use the API (writing is one of those tasks that people put off - the less obstacles there are, the better). Based on my experience, here are the questions you should be asking when picking a tool:

• What is the input for the tool?
  • Does it run over code and/or supplementary files? It's great to use a tool to automate your reference generation, but even better if you can use the same tool to write your developer's guide as well.
  • What languages will it process? If your API is written in multiple languages (like Flurry, the company I did this research for), you probably want to pick a tool that will work over all of them so you can have a consistent process and output.
  • What style of comments does it parse? If you're already using comments to instruct your compiler, you won't want to use a different format just for the docs.
• What is the output for the tool? HTML is a must, even better if it can do multi-page and single-page. PDF and other formats can also be quite useful.
• What is it written in? It needs to work in your development environment, and ideally hook into your build system.
• How easy is it to customize for your needs? The ideal tool includes easy customization options that don't require digging into the code, but is also open-source in case they don't do it for you.

To give you an idea of (just a fraction of) what's out there, here is a breakdown of the documentation tools used by the APIs from the last post.

 

Multi-Language Generators

Tool	Input files	Input language	Input comments	Output	Written in?	Examples
Natural Docs	In code or other files	C#, Perl, AS2, partial support for other languages.	JavaDoc, or custom "natural syntax"	HTML	Perl	jqPlot, Users list
Doxygen	In code or other files	C++, C, Java, Objective-C, Python, IDL, Fortran, VHDL, PHP, C#	JavaDoc or QT style, with some support for other styles	HTML, TEX, RTF, PS, PDF, ...	C++	KDE, Users list
docco	In code	CoffeeScript, JS, Ruby, Python	Markdown	HTML	CoffeeScript	data.js and drops.coffee
Sphinx	In code or other files	Python, C/C++	RST	HTML, TEX, PDF	Python	Python, Django
DocBook	Generated from XML files	XML		HTML, PDF, CHM, RTF, ...	PHP	PHP.net

 

Single Language Generators

Tool	Input files	Input language	Input comments	Output	Written in?	Examples
JavaDoc	In code	Java	JavaDoc	HTML	Java	Android
joDoc	In code or other files	JS	Markdown	HTML	Perl	PhoneGap
JSDoc 3	In code	JS	JSDoc	HTML	Java/JS	None yet, in beta.
JSDoc 2 (JsDoc-toolkit)	In code	JS	JSDoc	HTML	Java/JS	Google Maps API v3
JSDuck	In code	JS	Markdown, HTML, some JSDoc tags	HTML	Ruby	Ext-JS 4
ErlDocs	In code	Erlang		HTML	Erlang	ErlDocs.com

 

If you're interested in exploring tools further, check out the Documentation Generators section of my Developer Support handbook and this nearly comprehensive comparison of documentation generators from Wikipedia. There are always new tools being built out there by developers not quite satisfied by the current offerings and going at it their own way - and that's a great thing for all of us.

Let me know in the comments what tools you use (or are building yourself!) and your experience with them.

Roundup: The Best Developer Docs

As I mentioned in my previous post on "What makes a good Developer Experience?", I'm currently consulting with Flurry to advise them on improving their developer experience. They're starting with a revamp of their documentation and API reference. Per what I wrote in the Documentation chapter from the Developer Support Handbook, good documentation should include tutorial style content, an API reference, and sample code, and should be fully linkable and searchable.

To confirm that my criteria for good docs aligns with what developers in the wild think, I conducted an informal survey on Twitter asking developers for their favorite and least favorite docs. I reviewed their picks, and as I guessed, here's what was common amongst the favorites:

• Easy navigation: everything linked, clear table of contents, every page just takes 1-2 clicks to get to. Example: Yii.
• Easy search:. prominent search box with autocomplete for method/class names. Example: Sencha.
• Reference *and* guide: most offer both a technical breakdown of every part of the API (the reference) as well as a step-by-step narrative guide for using the API. Example: Twilio.
• Inline code samples: in the reference, every class/method comes with example code below the definition, and sometimes that code can even be run in the browser. (Or for HTTP APIs, sample request and responses in every data format are offered). Example: CoffeeScript.
• Comments: a threaded comment system on every page, for developers to discuss issues or usage. Example: jQuery.

And here are some cool bonus features from the favorites (that may not work for everyone):

• Community edits: The Mozilla Developer Center lets anyone login and edit the information themselves.
• Feedback loop: The PhoneGap docs prompt the developers to give them feedback on every page, and they use that to improve the docs.

Many of the favorites are written by the engineers themselves, which likely keeps them more up to date and accurate (and proves that engineers can write!).

The least favorite documentation sets typically suffer from basic usability issues, like lack of search or deep links. When you're writing documentation, you need to remember that your developers won't benefit from it unless they can find it - searchability and linkability should be a top priority.

I've rounded up the documentation sets in the tables below, with links, short description (including how they were generated, if known), and a thumbnail. I've also embedded a slideshow of the screenshots. Feel free to comment with your own opinions on what makes for a great set of developer docs!

Screenshots

Rave Reviews

Ordered by popularity, these documentation sets received all positive reviews. Great work!

jQuery	One page for each method. Includes code examples and discussion. Manually written.
Django	One page for each topic, narrative tutorial-like structure. Generated with RST and Sphinx, More info here.
PHP	One page for each method, includes examples and comments. Generated with DocBook (XML), More info here.
CoffeeScript	One page developer's guide with interactive code examples in each section.
Twilio	One page for each topic, includes example requests and responses for REST API and example XML for XML API. Manually written by the engineers.
Mozilla Developer Center	One page for each topic, includes example code, encourages community edits. Uses wiki/social CMS system, MindTouch
PhoneGap	One page for each class, includes both short and long example code, and a popup feedback widget. Uses joDoc for automated generation.
Backbone.js	One page for each method (and example code), with framed (but still linkable) navigation. Manually written.
Yii	One page for each class, lots of internal linking, prominent search with autocomplete. Includes code samples and links to original source code. Generated with custom, open-source PHP tool.
NodeJS	One page for each topic, or can be viewed all on one single page. Includes code examples for each method. Generated using doctool and Markdown.
Play! Framework	Includes step-by-step tutorial and developer's guide (no reference).
Sencha ExtJS	One page for each class, highly linked, inline code examples, and search with auto-complete. Generated using JSDuck and Markdown.
SimpleGeo	Organized by topic with one page for each. Written in a narrative form and includes code examples. Generated using Django and templates, More info here.
Twitter	One page for each API endpoint ("resource"), with example request and responses in XML/ATOM/JSON. Partially generated from code, manually curated by engineers.
Android	Javadoc style, with ability to change language and API level, plus search with auto-complete. Incudes tutorial style articles as well. Generated from code using JavaDoc.

Mixed Reviews

These documentation sets had both positive and negative reviews. (What works for one developer doesn't always work for another).

Python	One page for each topic, with embedded references and code examples. Complaints: no search/comments. Generated using Sphinx and RST.
Facebook	A mix of reference style docs, articles, and github-hosted readmes. Includes search and (Facebook) comments. Complaints: too loosely organized.

Not-So-Great Reviews

These documentation sets had all negative reviews. (Well, they can always improve!)

Ruby	Javadoc style with framesets. Complaints: can't be searched or linked, no per-topic organization. Generated from code using RDoc.
Amazon Products API	Frameset docs, both tutorial and reference style. Complaints: takes many clicks to find the reference, provided sample code does not work.
Paypal	Includes articles and reference docs. Complaints: instructions are dumbed down, deprecated information isn't marked as such, there are too many places to find a piece of information, not enough example code.
OpenLayers	Wiki and reference. Complaints: No code examples, hard to find docs. Generated using NaturalDocs.
CouchDB	Wiki with articles by topic. Complaints: Hard to navigate. Uses the MoinMoin wiki engine.

What makes a good "developer experience"?

(Short version: To help me answer this question, fill out this survey. Thanks!)

We are all quite familiar with the term "user experience" (UX) because as developers, we are typically spending our time developing products that we want people to use, and we want the UX to be positive so that they continue using them. However, we are now in the age of web APIs and developer platforms, and increasingly more of us are spending our time developing products for developers themselves to use.

Just like with user products, we want the "developer experience" (DX) to be positive so they continue developing on top of our service, and so they feel good relying on us to be part of their app's technology stack. When we talk about user experience, we focus on the usability of the user interface and the various user actions (like sign-in, search, share). A developer experience is very different, though, and when we talk about it, we should be focusing on the usability of the API and the developing-and-debugging experience. How does a developer first learn about your service? Where do they go when they need help? How easy is it for them to develop arbitrary apps on top of it? Can they use it successfully in their preferred environment? Those are the sort of questions that we need to ask to figure out what the developer experience is like.

Developer experience is important and non-trivial, but unfortunately, it's been nowhere near as discussed as user experience - a quick Google search yielded mostly this one article from UXMag. When I worked in developer relations at Google, we were basically tasked with designing the DX for our APIs and tools, and when I wrote up the Developer Support Handbook, I was attempting to document some of what I learnt from my years on the Maps and Wave APIs. That handbook only talks about some aspects of the experience - it doesn't talk about API usability at all - and it's only based on my own limited experience. I know that there is much more to be documented and discussed, and that there are many different types of developers and developer services out there, and just as with UX, there may not be just one set of DX principles that fits everything.

Why do I bring this up? A few weeks ago, I met with a mobile analytics company called Flurry to discuss their developer experience and how they can improve it to better support their current developers and encourage future developers. I gave them my own opinions, but I really would love to hear the opinions of more developers to give me a broader perspective. To find out what you guys think about what makes a positive developer experience, I've put together a short survey. Please fill it out if you have the time, or feel free to leave comments on this post. Thank you for your thoughts and for caring about what it means to have a good developer experience.

SF jQuery Conference 2011: My Learnings

For the last 5 years, I've really only attended conferences that I was speaking at or somehow representing Google at, and I was always paid by Google to attend. This past weekend, I convinced myself to pay out of my own pocket to attend a conference for the first time, and it was completely worth it.

jQCon was a 2-day event in the Microsoft Mountain View campus with a wide range of well-known speakers, like Nicholas Zakas, Steve Souders, Paul Irish, and of course, John Resig, plus equally skilled but lesser known contributors to the world of jQuery and JavaScript programming. It was a great way to learn new techniques and tools for client-side web app development from the people that are doing it 24/7 for some of the biggest sites on the web.

I personally gravitated to the more mobile-oriented talks, as I've recently become interested in mobile webapp development (I finally use my phone here!) and have been using jQuery mobile for the next version of SpeakerMeter. I learnt a lot about how people are currently writing mobile webapps, and more importantly, I realized that this is a rapidly changing area, particularly as the line between desktop and mobile becomes increasingly blurry. I was hoping to find the "one true way" of making a HTML5 mobile webapp, but I've realized instead that I need to try the current ways out there (like PhoneGap + jQuery Mobile) and be ready for that to change.

I took notes throughout the talks on tips and resources, and am sharing them here both for my own reference and to give you an idea of what was discussed. I look forward to learning more at the next jQCon, whenever that may be. :)

Day 1:

jQuery Mobile (Todd Parker & Scott Jehl)

• Many developers are using jQuery Mobile in combination with PhoneGap to deploy mobile-optimized webapps.
• jQuery Mobile now supports more mobile browsers, including Kindle Webkit.
• The jQuery Mobile team put a lot of work into making all the controls accessible.
• The team is working on an improvement that will include better URLs (hash-less) for browsers that support it (HTML5 pushState).
• The team made sure that it was easy for developers to make custom themes/colors, and they want more developers to take advantage of that.
• Browse the jQuery Mobile gallery to see what other developers have done.
• The M Project is an open source project to make it easier to make mobile webapps with jQuery mobile.
• The team recommends specifically optimizing the UI for iPad/tablets and not just using the standard narrow mobile layout. They hope in the future that the framework can do more of that for developers.
• CSS3 media queries can be used to change the UI for different device sizes.
• jQuery Mobile can be used with grids frameworks (can be useful for tablet layout).

Mobile Performance (Steve Souders)

• Its better to have your site be fast when users first experience it - study showed slow sites lost users (and it took a while for them to come back).
• Use Blaze.io for seeing the mobile performance of your site (like webpagetest.org).
• Use PCAP Web Performance Analyzer to analyze performance of websites (can be used locally).
• Use JDRop with mobile bookmarklets to store performance data in the cloud for later analysis.
• Use data URIs and localStorage for better mobile performance.

Progressive Enhancement 2.0 - Because the Web isn't Print (Nicholas Zakas)

• Read: "Understanding Progressive Enhancement"
• "The relationship between a web browser and a webpage is like a television and a TV show."
• Examples of progressive enhancement in the real world: escalators and electric toothbrush.
• Websites don't need to look exactly the same in every browser.
• Don't try so hard to make old browsers do what new browsers can (consider effort versus value) - like rounded corners, drop shadows, and gradients.
• Developers use multiple browsers, but most users only use one browser - they won't know if the experience is different.
• Progressively enhance for browsers that support newer technology - Facebook chat is an example of a progressive enhancement.

Integrating Code Quality tools into your jQuery Development Workflow (Anton Kovalyov )

• JSHint is the alternative to JSLint - does not enforce so many "crocklamations" by default.
• Recommends using git precommit and postcommit hooks for running JSHint, to make sure all team members are using it. Break the build if it errors.
• Many community-written addons for integrating JSHint with IDEs. He likes Sublime.
• Read: Perfection Kills, A minute with Brendan

Prototyping and Unit Testing with Mockjax & mockJSON (Elijah Manor)

• MockJAX is a jQuery plugin for simulating AJAX requests and responses.
• MockJSON is a library for generating random JSON responses based on JSON templates. Works well with MockJAX.

Day 2:

Filling the HTML5 & CSS3 Gaps with Polyfills and Shims (Rey Bango)

• Rey is both a Microsoft employee and a jQuery team member - MS is a big supporter of jQuery.
• A "polyfill" mimics a standard/future API and makes it work in older browsers, whereas a "shim" provides an arbitrary API for achieving some functionality. "Polyfill" was named by Remy Sharp after the British term for spackling - it's like filling holes in old browser cracks.
• When you are using a polyfill or shim, vet them carefully: see how many developers are working on them, how frequently they update them, etc.
• Long list of polyfills and shims.

How jQuery is helping Pixar to make their movies (Philip Floetotto)

• Pixar is a big Python user. For their project management system, they used Django for the backend and jQuery for the frontend.
• They built their UI on top of the TableSorter plugin.

Presentational jQuery (Doug Neiner)

• Be concious of when you use linked stylesheets vs. STYLE tags vs. inline style setting. Never use jQuery to set initial state. Reduce use of .css() as much as possible.
• "Use selectors like you run errands" - try to do as much in one call as you can.
• Experiment with easing (on different properties) for better animations.

Breakout Session: jQuery Mobile

Much discussion on using jQuery Mobile to build static sites that are deployed as WebViews inside of native apps.

• Advantage of PhoneGap over Titanium is that it deploys to more mobile platforms.
• Phone Gap Build lets you deploy mobile apps without needing to download anything.
• Use something like JSON Template or Handlebars for client-side templating of the jQuery mobile headers and footers.
• Use HT Track or Hyde to use Django templates locally and still output a static site.
• Recommended reading: Responsive Web Design.

The Middle Way: Developing Hybrid Mobile Apps (Alex Kessinger)

• Use localStorage to store data locally, and use LRU to stay under the 5MB size limit.
• If using AppCache, inline everything (CSS, JS, HTML) so everything stays in sync in the cache.
• For a better user experience, render *anything* to the user first (versus nothing), even fake DIVs.
• For easier mobile debugging, try JSConsole.
• OpenAppMkt is a marketplace specifically for mobile-optimized webapps.

Breakout Session: CouchDB + jQuery

Wrote a small JQuery & CouchDB app (with this view).

Creating mind-blowing UX - "Building great user experiences with the help of jQuery" (Darcy Clarke)

• User experience is about emotions - you want your users to have positive emotions when using your site. Surprise is an emotion that can be either positive or negative.
• Inspirational design agencies: F-I, Zurb, Blitz