User Stats Visualization with HighCharts

For the first few months of EatDifferent's existence, it was only a small set of users -- family and friends -- and half of them were my buddies on the app. But then a food blogger tweeted it, and over the course of a day, there were 10x the number of users -- all strangers. Since I wasn't buddies with the new users and wouldn't see their updates in my stream, I needed a way of understanding who all these users were and how they were using the app

I wanted simple visualizations of my user demographics & usage data, and after a bit of research on what type of chart to use (i.e. not pie charts), I came up with this stacked horizontal bar chart visualization:

All the bars add up to 100%, the bar parts show their value and percentage on hover, and the bar parts are colored according to their value. For example, true is green and red is false, browsers and OSes are colored according to their debuggability (Ghrome is green since that's my browser of choice, guess which one is red?), and numeric values are colored in a monotone gradient. The charts are made using HighCharts, my favorite charting library.

It's nice being able to understand more about my users in a glance. :)

Client-Side Error Logging

After setting up my Python code to send server-side errors to my inbox, I wasn't satified. I wanted to know about JavaScript errors, too! So I set up my base template to send JavaScript errors to my server and log those as errors. Now I find out about all errors immediately in my inbox, and I can deploy a fix before it affects users for too long. So far I've found a couple errors this way that otherwise I wouldn't have known about (due to different user behaviors + browsers).

I use this JavaScript for sending errors to a handler on the server, which then logs them as an error using logging.error:

When I tweeted about my solution, I was pointed to a few related projects and techniques:

• An article with various tips on logging JS errors.
• jserrlog.appspot.com: A solution hosted on App Engine, where you just drop in the JS and view the logs on their server.
• Errorception: A startup devoted to "painless javascript error tracking" and will include more cross-browser logging, emails, and web hooks. Sounds promising!

Since deploying my solution, I've discovered that window.onerror doesn't work in all browsers (even some "modern" ones like Android), so if you are looking to catch every error, you should checkout those links to see the workarounds they employ.

Update (2/20/2012): I've updated my function to ignore errors that I know aren't mine, like those generated from 3rd party scripts. I could also update it to just only show errors from my server's JS, but for now I'm interested in what other errors folks encounter. You can see that updated code in this gist.

Pre-Deploy Git Check

Since I'm the only one developing EatDifferent, I've been content to store it in a Dropbox folder and use a local git repository for version control. But I got nervous thinking about what would happen if something went horribly wrong while I was unavailable (like on a plane), and decided I needed a backup collaborator -- someone who could quickly checkout the code and deploy a fix.

I probably could have shared the Dropbox folder with them, but I instead opted for a private Github plan for $7 a month, which lets me share my code repository with my backup collaborator. Plus, I now get to browse my code history and commits via Github's slick online interface.

I often forgot to git commit and push my changes, so I added this check to my Makefile which errors if there are uncommited changes. I run the check everytime I deploy my code to App Engine, to make sure the code always reflects what's deployed.

So far, the workflow seems to work...and now I can fly a plane in peace. ☺

Sending Errors to Email in App Engine

When my EatDifferent server-side code results in an error or exception, it gets logged in the App Engine dashboard. I was getting anxious wondering if there are new errors in the logs, refreshing the dashboard every few hours, so I decided to make sure all errors get immediately emailed to me and researched how best to do that.

First I checked out the App Engine ereporter module , but it was set to aggregate errors and email reports once a day -- not frequent enough for anxious me.

I then found a developer that wrote loggers to send errors over Channel API, PubNub, and XMPP, with a throttling mechanism built-in to prevent getting spammed when many errors happen at once.

I took his XMPP logger and converted it to send email instead, and set the throttling threshold to 5 minutes instead of 1 second. Now I feel much less anxious! You can grab the code from this gist:

Switching from jQuery Mobile to Twitter Bootstrap

I've been using jQuery mobile (jQM) to make a mobile-enhanced version of EatDifferent for the past few weeks and though I have much respect for the team behind it, I've grown increasingly frustrated with it. Why?

1. It typically enhances by adding additional DOM elements to the original elements - which can help in making them easier to use on a mobile device, but it means that whenever you want to tweak the styles of an enhanced element, you need to dig into the DOM and apply CSS overrides at various levels. If the DOM or inner CSS names change in later jQM versions, you may have to update those CSS overrides.
2. If you dynamically update an enhanced DOM element (like a form input, for example), you often have to tell jQM to force refresh that element, since it needs to redo the added DOM. That means you need jQM-specific JavaScript calls in your code.

I was fine with those issues when I used jQM for an earlier project, but for the EatDifferent app, they are becoming bigger issues because: 1) I'm doing more customization, and 2) I'm doing more dynamic form creation and updating.

And most importantly, I'm trying to re-use my HTML/CSS/JS across both my the desktop web version and mobile app, and I want minimal difference between them - both so users have a consistent user experience across them, and so that I can write the code just once.

So today I scrapped jQM and decided to use just HTML/CSS. And since I suck at making pretty CSS, I'm using Twitter's bootstrap library. It's pure CSS (no JS!), it's simple to use, and it's pretty. It's not specifically designed for mobile, but it works well enough on it and is easy to customize when it doesn't quite work.

Here's what the app looked like using jQuery mobile and the default black theme:

Here's what it looks like using Twitter bootstrap and some basic customization:

As you can see, jQuery mobile does create more mobile-optimized form input interfaces (larger areas for clicking, e.g.), but Twitter bootstrap creates clean CSS that I can easily add my own mobile-optimized CSS on top of when necessary. I'm happy I made the switch.

Spriting with Compass

I use little icons in various places on EatDifferent, like to show food bonuses in the stream:

The browser can take a while to load lots of images since it has to make a request for each one, so I decided to implement icon spriting - baking all the images into one image, and using background-position in CSS to make what appears like a standalone icon.

Thankfully, I was already using SASS and Compass for my CSS, and it comes with built in spriting.

After putting my icons in one folder (and sizing them all to be 16*16 pixels), I added this to the top of my .scss file:

    @import "icon/*.png";
    @include all-icon-sprites;

I also specified sizing and display properties for the special "icon-sprite" class in my .scss file:

    .icon-sprite {
      width: 16px;
      height: 16px;
      display: inline-block;
      margin-right: 2px;
    }

Compass then auto-generated CSS rules for my icons - one rule to specify the background for each of them, and a rule per icon to specify the background-position. It also applies any of the ".icon-sprite" rules it found to all of the generated icon classes. Here's a snippet of the auto-generated rules:

    .icon-sprite, .icon-activity, .icon-android, .icon-bodylog, .icon-buddies, .icon-camera, .icon-comment, .icon-edit, .icon-female, .icon-foodlog, .icon-grassfedmeat, .icon-highfive, .icon-home, .icon-homecooked, .icon-localfood, .icon-logs, .icon-organicveg, .icon-profile, .icon-reminder, .icon-settings, .icon-settings2, .icon-stats, .icon-stats2, .icon-sustseafood, .icon-tip {
      background: url('/img/icon-s97f5308db7.png') no-repeat;
    }

    .icon-activity {
      background-position: 0 0;
    }

    .icon-android {
      background-position: 0 -27px;
    }

    /* line 99, ../sass/_common.scss */
    .icon-sprite, .icon-activity, .icon-android, .icon-bodylog, .icon-buddies, .icon-camera, .icon-comment, .icon-edit, .icon-female, .icon-foodlog, .icon-grassfedmeat, .icon-highfive, .icon-home, .icon-homecooked, .icon-localfood, .icon-logs, .icon-organicveg, .icon-profile, .icon-reminder, .icon-settings, .icon-settings2, .icon-stats, .icon-stats2, .icon-sustseafood, .icon-tip {
      width: 16px;
      height: 16px;
      display: inline-block;
      margin-right: 2px;
    }

I measured the loading performance of my site before and after spriting, using the HAR Viewer, and these are the results:

Before:	28 requests	2.61s (onload: 1.92s, DOMContentLoaded: 1.64s)
After:	15 requests	1.09s (onload: 817ms, DOMContentLoaded: 600ms)

As you can see, spriting had a significant effect on performance. I definitely recommend spriting (and Compass) for sites that display multiple images on page load.

WDCNZ: The Developer Experience

Last month, I had the honor of giving the first talk of the day at WDCNZ, a new conference in Wellington, New Zealand. WDCNZ was the brainchild of Owen Evans, one of the lead developers at Xero, and his goal in putting it together was to provide a full day of technical talks for web developers and to bring the local web development community together. I wanted to give a talk that would be interesting to web developers of all different sorts, and also would be something I'm passionate about myself - so I decided to give a talk around a topic I wrote a handbook about and discussed in recent blog posts - "The Developer Experience."

For my talk, I wanted to explain what I think of as "developer experience", motivate people to really care about it, and give concrete examples of what makes for a great experience. It was an interesting talk to prepare and give, and hopefully gets other people thinking about the topic and giving similar talks in the future. I've embedded the slides and video below so you can watch for yourself what I said. Thank you to WDCNZ for giving me the opportunity to give the talk and for an awesome conference!

The Developer Experience (Slides)

The Developer Experience from WDCNZ on Vimeo.