Wednesday, October 26, 2011

Logging JS Errors on iOS with PhoneGap

I've spent the last few days getting the EatDifferent PhoneGap app working on an iPhone (an app which previously worked on Android). The hardest part has been learning to debug in the iOS browser, so I thought I'd post on my findings:

  • To view the output of console.log, you must open the XCode console. The iOS browser "Debug console" that most iOS debugging articles mention is only displayed in the standalone Safari browser, not in the WebView (where PhoneGap HTML lives).
  • There seem to be times when console.log does not log the output (perhaps during loading?) - in that case, alert() always seems to work.
  • If you log a JS object using console.log, it will just print "Object" by default. You must JSON stringify it to be useful.
  • You can also use debug.phonegap.com (hosted weinre) to view the DOM and JS console logs as well.
  • The WebView browser silently fails on JS errors - it stops running the JS code and does not report the error. To see the error, you must wrap the offending code in a try/catch block.

Given all of those learnings, here is my log() wrapper function that I use across my webapp:

    function log(something) { 
        if (window.console){ 
          if (something instanceof Date) { 
            something = something.toDateString(); 
          } 
          if (isIOS() || isAndroid()) { 
            if (typeof something == 'object') { 
              something = JSON.stringify(something); 
            } 
            console.log(something); 
          } else { 
            console.log(something); 
          } 
        } 
    } 

And I wrap various code blocks in try/catch, like the callback function for AJAX requests:

    try { 
      onSuccess(processJSON(responseJSON)); 
    } catch(e) { 
      log(e); 
    } 

I posted my observations in the PhoneGap group and the developers there made several recommendations: 1) use Ripple, a Chrome extension for mobile emulation 2) monkey-patch JS functions to always try-catch, as done in this library. I've taken a break from iOS debugging for a few days, but I'll probably revisit debugging soon and try out their ideas.

Friday, October 21, 2011

Code Quality Tools

Now that my EatDifferent application code is getting cleaner, I wanted to make it even cleaner by using automated code quality tools.

First, I ran jshint over all my *.js files and fixed a bunch of little issues (like using "===" instead of "==" in many places). Here's how I setup my Makefile to download jshint and run it:

Then I ran reindent.py script to fix the indenting on all my *.py files. I had been using 2-space indents, as that's we used at Google and also what I use in JavaScript, but I was convinced to go with the PEP8 standard, 4 spaces.

Then I downloaded SublimeLinter, a plugin for Sublime Text that automatically checks your code as you write it, using jshint for JS and pyflakes for Python. I configured that to ignore a few PEP8 warnings in the settings JSON ("pep8_ignore": [ "E501", "E221", "E203"]). I'm quite liking SublimeLinter - its lint tools actually catch a few things that could result in real bugs and its nice to be able to correct my code as soon as I type it.

PhoneGap Loading Performance in iOS

As I wrote about earlier, I've been working on the performance of my PhoneGap app, implementing many of the suggestions from this article.

One of the suggestions that article makes is to switch frameworks from something heavy like jQuery (which includes a lot of extra code that may not be necessary on mobile browsers) to something lighter like Zepto or XUI.

I currently use jQuery in my app -- not because I use jQuery a lot in my own code, but because I use a handful of third-party-written jQuery plugins. I could easily port over my own code to a new framework but I don't know how easy it would be to port over someone else's.

So, before I looked into porting from jQuery, I wanted to figure out exactly what effect jQuery had on performance. I'm most concerned with user-facing loading latency versus already-running performance as that's where I perceive the greatest latency in my app, so I decided to see how long it took for the browser to load my HTML, CSS, and JS.

First I wrote a basic function to record times of events and I used it to record events after the CSS, HTML, jQuery script tag, other script tag, document ready, and execution of my setup function This is a stripped down version of what it looked like:

Then I recorded the times in Chrome, iPhone Simulator, and actual iPhone (on the same WIFI network), and graphed them in a spreadsheet. Here's a chart of the results:

As you can see from the chart, the iPhone browser is noticeably slower than Chrome and the simulator, and the jQuery script tag (90 KB) takes 35% of the loading time - same as the custom script tag (220KB). But it's only 300ms, which isn't as slow as I expected and for now, it doesn't seem worth it for me to try to port away from jQuery.

Update (11/20/2011): I have since ported away from jQuery and written that up in this post..

Update (12/13/2011): It's been pointed out that when it comes to measuring the "loading times" for resources, this is not the most precise technique, as it it doesn't reveal the difference between the download time, the parsing time, and the execution time. (In the case of jQuery, I believe much of the time I recorded came from the execution.) For that breakdown, I've been recommended weinre.

Saturday, October 15, 2011

JS & CSS Compiling, Compression & Cache-Busting

Everytime I deploy a new version of the CSS and JavaScript for EatDifferent to production, I run it through a series of steps to ensure code quality and performance:

  • Code quality: I use JSHint to check for JavaScript code quality issues. Sometimes it's a matter of style, but other times it actually finds issues that can become runtime bugs.
  • Concatenation: I use cat to combine my JS files and CSS files into one file each, so that the browser can issue less HTTP requests when loading the page.
  • Compression: I use Closure Compiler to minify my JS and YUI Compressor to minify my CSS, so that those HTTP requests are smaller.
  • Cache bust: I append the current timestamp as a query parameter to the JS and CSS in my base template HTML. I serve the files as static files off App Engine which would normally result in browsers caching them forever, but by appending new query parameters for each deploy, I force the browsers to re-download them only when they've changed.

I do all of this in a Makefile, including downloading the necessary tools. You can see the relevant bits in this gist:

Thursday, October 13, 2011

Modularizing My JavaScript

I generally try not to get too distracted by the code quality in EatDifferent and focus on user-facing quality instead, but after a while, it hurts my head knowing that my code is messy -- and makes me not want to mess with the code further. So, I spent yesterday spring cleaning my JavaScript.

One of the big improvements was to take my smattering of global functions and put them in a namespace. There are a lot of ways to namespace in JS, but I opted for the module pattern described in this article (#3).

Here's the basic template for each module's JavaScript file -- notice how this technique lets me create functions in each module that are only used inside the module, and aren't exposed outside of it.

    var ED = ED || {};

    ED.util = (function() {

      function doSomethingPrivate() {
      }

      function doSomething() {
        doSomethingPrivate();
      }

      return {
        doSomething: doSomething
      }
    })();

I ended up with 6 JS modules used across the web and mobile (PhoneGap) version of the app.

ED.util Utility functions, app-independent
ED.data Global data constant definitions
ED.models Classes representing data from the server DB
ED.shared App functionality shared by web and mobile
ED.web Web specific functionality
ED.mobile Mobile specific functionality

I could have put the first 4 of these in "shared" but I like the conceptual division, and I'm concatenating them together before serving them to the user, so it doesn't hurt to have multiple files.

Now that my code is cleaner and more manageable, I can more confidently iterate on user-facing features!

Tuesday, October 11, 2011

Grammatical Personalization in JS

In EatDifferent, I have various places where I describe something about a user. For example:

  • "Pamela Fox filled out her logs 3 days in a row.
  • "In the last week, you haven't logged any measurements."

I wanted to be able to construct those sentences in JavaScript with some sort of mini templating language. I didn't find any existing libraries for it, so I wrote my own. Now to generate the above strings, I can write stuff like:

personalize('{{ They|name }}' filled out {{ their }} logs 3 days in a row.', 
   {gender: 'male', person: 'third', name: 'Pamela Fox'});
personalize('In the last week, {{ they|name }} {{ have|not }} logged any measurements.',
    {gender: 'male', person: 'first'});

You can see the code for the library in this gist and check out a live demo on jsfiddle.

Friday, October 7, 2011

Ada Lovelace Day: Women that Inspire Me

Today is Ada Lovelace Day, a day to celebrate the women of math, science, and technology, and the challenge today is to write a blog post highlighting a woman that inspires us. I blogged an interview with my mum a few years ago (the first woman in tech I ever knew!), so this year, I thought I'd highlight a few women that inspire me today.

  • Sara Chipps:

    Sara started GirlDevelopIt in New York, with the goal of getting more women in web development by actually teaching them how to do it. She inspired me to run a GirlDevelopIt in Sydney, and I still remember those months as some of my favorite down under. In addition to being the force behind GDI, Sara still develops - just yesterday, she wrote an uber-useful Chrome extension that I installed instantly.

    Follow Sara online: Twitter | Google+ | sarajchipps.com

  • Gina Trapani:

    I first knew of Gina for starting lifehacker, a popular blog of tips for using tech and optimizing your digital life (and the biggest source of traffic for my little Profane Game). I then knew her as the author of "The Complete Guide to Google Wave", an attempt to explain how to use Wave to the average joe. I was on the Wave team then, and I knew that was no small feat. And now, I know her as a developer, working on an open-source web app for social media analysis. Blogger, author, coder, speaker - Gina does it all.

    Follow Gina online: Twitter | Google+ | ginatrapani.org

  • Leigh Fonseca:

    I met Leigh just this week, at the WITI Next-Gen Summit, where Leigh and I talked to high school girls about how we got into tech and why we love it. Leigh started off with a history degree, but when she realized she had a thing for analyzing customer data, she started data science departments at Rhapsody and then BookRenter, where she is now. Most companies don't have data scientists nor realize they need one, so Leigh is paving the way for others with her interest. When she's not analyzing data, she's preparing for triathlons. Impressive.

    Follow Leigh online: Twitter | Google+ | dataparty.blogspot.com

  • Kendall Ronzano:

    I also met Kendall this week - she's actually one of the girls that Leigh and I talked to at the conference. She's a high school junior who's already started programming for her robotics team (C++!) and wants to learn more. She's also set a fun challenge for herself: build a house. I hope Kendall represents the next generation of women in tech!

So, who inspires you?

Wednesday, October 5, 2011

User Stats Visualization with HighCharts

For the first few months of EatDifferent's existence, it was only a small set of users -- family and friends -- and half of them were my buddies on the app. But then a food blogger tweeted it, and over the course of a day, there were 10x the number of users -- all strangers. Since I wasn't buddies with the new users and wouldn't see their updates in my stream, I needed a way of understanding who all these users were and how they were using the app

I wanted simple visualizations of my user demographics & usage data, and after a bit of research on what type of chart to use (i.e. not pie charts), I came up with this stacked horizontal bar chart visualization:

All the bars add up to 100%, the bar parts show their value and percentage on hover, and the bar parts are colored according to their value. For example, true is green and red is false, browsers and OSes are colored according to their debuggability (Ghrome is green since that's my browser of choice, guess which one is red?), and numeric values are colored in a monotone gradient. The charts are made using HighCharts, my favorite charting library.

It's nice being able to understand more about my users in a glance. :)

Tuesday, October 4, 2011

Client-Side Error Logging

After setting up my Python code to send server-side errors to my inbox, I wasn't satified. I wanted to know about JavaScript errors, too! So I set up my base template to send JavaScript errors to my server and log those as errors. Now I find out about all errors immediately in my inbox, and I can deploy a fix before it affects users for too long. So far I've found a couple errors this way that otherwise I wouldn't have known about (due to different user behaviors + browsers).

I use this JavaScript for sending errors to a handler on the server, which then logs them as an error using logging.error:

When I tweeted about my solution, I was pointed to a few related projects and techniques:

  • An article with various tips on logging JS errors.
  • jserrlog.appspot.com: A solution hosted on App Engine, where you just drop in the JS and view the logs on their server.
  • Errorception: A startup devoted to "painless javascript error tracking" and will include more cross-browser logging, emails, and web hooks. Sounds promising!

Since deploying my solution, I've discovered that window.onerror doesn't work in all browsers (even some "modern" ones like Android), so if you are looking to catch every error, you should checkout those links to see the workarounds they employ.

Update (2/20/2012): I've updated my function to ignore errors that I know aren't mine, like those generated from 3rd party scripts. I could also update it to just only show errors from my server's JS, but for now I'm interested in what other errors folks encounter. You can see that updated code in this gist.