An Unofficial Guide to Geeking It Up Around Sydney

Before I came to live in Sydney, I actually spent a day googling for information on the Sydney developer scene, learning about the local startups, user groups, and mailing lists. I didn't know anyone in Sydney, and I wanted to make sure I both had a way to get to know locals (make new friends!) and also figure out what sort of Google developer events I should organize. In just my first weekend in Sydney, I attended BarCamp Sydney 4 and met many of the people that I'm friends with today, and since then, I've attended something like 50 local meetups, "drink-ups", hackathons, and conferences.

I think it's really fun to attend events from a get-to-know-others perspective and really useful to attend events from a learn-what-others-are-doing perspective, and I want to make sure every Sydney developer is aware of the events going on around them. So, for last night's Girl Geeks Dinner lightning talks, I put together a short preso on the local developer scene, and I've embedded it below. Hope to see you at an upcoming event!

Sydney International Food Festival Maps

Every year, Sydney has this awesome International Food Festival filled with food events and deals on meals at local restaurants. In particular, they have this "Lets do lunch" deal where you can get nice lunchtime meals at fancy restaurants for $35, and I usually like to get some colleagues together to hit up some of the restaurants. Unfortunately, they never have a map visualizing the locations of all the restaurants, and it's hard to find the places near my work.

So, every year, I make a map of the restaurants. Per request of @MorselsMusings, I've also made maps of the Cocktails, High Tea and Sugar Hit deals. The deals last until the end of October, so check them out now while you have time!

If you're a developer and wondering how I whipped these together, here's the short version: 1) I used Dapper to get a CSV of the name, description, and link from the main page, 2) I converted those into a Google spreadsheet and used the importXML function to get the address from the individual restaurant page, 3) I published the sheets and got the latitude/longitude coordinates using my Spreadsheets Geocoding tool, 4) I made the maps using my Spreadsheets Maps API wizard. It is a bit of a process, but as I spend half my life doing this sort of thing, it doesn't actually take very long (< hour).

Happy fooding!

lscache: A localStorage-based, memcache-inspired library

Over the past few years, I've developed a fair few Python App Engine apps, and I've come to have a huge admiration for memcache. The memcache API is incredibly simple, but at the same time, it's a very powerful way of making my apps scale better with increased user demand. The first time I wrote an App Engine app, it went over quota in the first 6 hours. After adding memcache support in, it never went over 1% of quota ever.

When I'm writing client-side apps, I find myself yearning for something like memcache to reduce my number of asynchronous server requests. HTML5 does offer the localStorage API for setting and getting key/value strings, but that API has no notion of an expiration date. It's great to be able to store copies of my data locally, but most of the time, I also want to be able to expire that data after some amount of time (5mins, 1hour, 1 day). So, I wrote a simple library called "lscache" that wraps on top of localStorage, but adds on the notion of expiration. Here's what it looks like to set and get some data:

lscache.set('somedata', {'name': 'Pamela'}, 60);
if (lscache.get('somedata')) {
  console.log(lscache.get('somedata').name);
}

The library lets you store pretty much anything, like a string, number, or object. The localStorage API only stores strings itself (though according to the spec, that should change soon), but the library uses JSON.parse and JSON.stringify to try to store non-string objects. There are some objects that can't be stringified, like the Document object in an XMLHttpRequest, in which case you need to convert them to a simple JSON object yourself first.

The library stores the expiration time in a separate key, and when you try to retrieve a key, it will only return it if the current time is before the expiration time (and will remove it otherwise). It calculates the time using the JavaScript Date object, so it is subject to error if the user futzes with their clock - but if they do, it just means the objects will be cached for a bit less or a bit longer than expected, and the world probably won't fall over. And those silly people should stop futzing with their clock. :)

If the user's browser doesn't support localStorage, the library just won't store anything and will return null when trying to retrieve objects. This works wonderfully with the memcache style of coding, where you never assume that anything is stored, and always fall back to re-retrieving that data if it's not stored. People with "older" browsers will simply get a not-as-speedy performance.

I've pushed the lscache code to a github repo (my first!), and also published a demo of the functionality. I originally wrote lscache to speed up the performance of the XMLHttpRequests in a Chrome extension popup, but as that extension isn't published yet, I've also incorporated it into my public RageTube mashup to cache the JSON results of the Dapper and Youtube APIs.

Check out the library, and if you have any suggestions for improvements, feel free to fork. :)

SydJS: JavaScript Libraries Panel Roundup

At last night's monthly meeting of Sydney JavaScript developers, the organizers tried a different format: the panel. I will admit I was a little wary of the format choice, as I've heard a lot of commentary about panels not being so great (and I've attended SXSW, which seemed to re-inforce that) -- but I think it turned out quite well. The panel had a range of JS developers that represented different libraries, and articulated their differences in opinion in constructive ways.

The panelists were:

• James McGill, Google, Maps API
• Dan Nadasi, Google, Closure (@DanielNadasi)
• Matthew Sain, Yahoo!7, YUI 2
• Tom Hughes Croucher, Yahoo!, YUI 3 (@sh1mmer)
• Dmitry Baranovskiy, Sencha, Raphael (@DmitryBaranovsk)
• Evan Trimboli, Sencha, Sencha
• Julio Ody Cesar, Awesome By Design, jQuery (@julio_ody)
• Jared Wyles, Atlassian, AUI (@rioter)

(Photo courtesy of halans - Check out his other SydJS pics.)

I typed up notes during the panel on the questions and answers, and am sharing a paraphrased version of those notes here. I didn't type anything word-for-word, and I missed some of the answers, so don't consider this to be 100% faithful. But perhaps it's useful for folks who want to remember what was discussed last night, or see the kind of questions that were asked at the panel.

Q: Can your libraries be loaded asynchronously, and if not, how do you sleep at night?
Tom: YUI was designed with that in mind from the beginning. It's the only way of loading it. James: The Maps API does it automatically - developer only includes the bootstrap and the API pulls the full library in after. All APIs should do it automatically, otherwise developers won't usually bother.

Q: How do you test the Maps API offline?
James: You must mock, and trust that we've designed our API in a way that makes for reliable mocks (we have). Check out the Closure mocking framework, based on EasyMock.

Q: Why doesn't Google use Raphael for SVG?
James: The Maps API only uses paths, and it isn't that hard to get paths working cross-browser. When you use a library, you're putting performance in someone else's hands, and the Maps API can't afford to do that.

Q: What do you think of CoffeeScript?
Julio: CoffeeScript is not equal to JavaScript. Different languages appeal to different people.
Tom: I talked with Douglas Crockford, and he actually likes CoffeeScript, and thinks that maybe we should have done JS that way (if we could have). But now, even if we could reinvent JS, even if we could make it better, it would be a separate language. JS is winning right now, not because it's the best language ever, but because it's the language of the web. So we should stick with JS.
Dmitry: There is real life, and there is play. I like to play- I like canvas- but I don't use canvas in real projects because it doesn't work in IE. It's not real life. It'd be sad if there was a CoffeeScript programmer that did not also know JavaScript.

Q: What's the purpose of the Closure inspector?
Daniel: It is easier to debug your Closure code before it has been obfuscated, and that's what you normally do. Some bugs, however, only occur after obfuscation, and the Closure inspector is useful for debugging the post-obfuscation bugs. Sometimes those happen when using the advanced compiler options with symbol exporting.

Q: How will we develope JS in the future?
Julio: We will see more modularization, where every page has its own miniframework, its own collection of modules used by the page.
Jared: We will see JS in the server more.
Evan: We will also see JS in the desktop.
Dmitry: We will have better IDEs for developing JS.
Tom: We will have better code, and new features of the language (like strict mode) that encourage developers to write better code.

Q: Do you support touch events in your frameworks?
All: Yes.
James: Currently, it's not too difficult to support touch events since the mobile browsers all followed Apple's lead in implementing them and the API is the same. But, in 8 days, Microsoft is going to release Windows Phone 7, which bundles a ported version of the IE7 browser, and we don't have any idea what touch events will look like. Mobile development is going to get significantly more interesting; we're going from a 1 browser field to a 2 browser field.

Q: What place do UI frameworks like Yuki and ExtJS have - should we be using them, and are they accessible?
Tom: ( Re accessibility) Frameworks should use ARIA for accessibility, which marks up the page to give screen readers additional info. The fact that most screen readers don't support JS is a technology failure. There's nothing inherent about JS which means it can't be supported. Current accessibility guidelines put the onus on the screen readers. Modern readers like Jaws 7 do support JS. Pick a framework that supports ARIA.
Tom: (Re frameworks) Rebecca Murphy has written recently about the problems with using jQuery for enterprise sites. With libraries lik jQuery and Prototype, people tend to write DOM-focused code. With libraries YUI and Closure, people write component-based code. jQuery has done a good job of getting people started with JS and making it easy to get your site going, but when you're building full apps, the component-based architecture with its inheritance and modules works better. DOM-style architecture gets ugly fast.
Nadasi: Part of the reason we use component-based approach in Closure is because it works better at scale. In base.js, goog.inherits lets classes inherit from other classes easier. The ability to inherit is better for distributed development, for having many developers working off the same codebase.

Q: I come from a Java/C++ background, where tooling support is important. In moving to JS, I'm missing the tools. A lot of the times, I feel like I'm guessing-and-checking. Are you guys keeping secrets from us, or is that the way it is?

Julio: I work with Ruby as much as JS, and it has really nice testing tools - the best of any language I've seen. I hope JS will have similar tools somedays.
James: Maybe you should check out Closure compiler. It lets you enforce static typing, which is like testing for a dynamic language - it will catch half the errors. But, in defense of JS, what other development environment lets you inspect your UI, change your UI on the fly, and reflect the changes back?
Dan: Keep in mind that the cost of implementing static typing is that you lose dynamic typing. Static typing can be great for development at scale, but it deprives you of some flexibility.
Tom: I recommend SpeedTracer from Google for performance checking, Firebug for Firefox, and Visual Studio Debugger for IE.
Buchanan (audience): We would love more tools for JS, but on the other hand, I've never lost a day of development due to my IDE refusing to work.

Q: Do we need libraries now? Browsers are better, JS is easier cross-browser.
Julio: We're not at the stage yet where cross-browser isn't an issue.
James: When you're getting started with building an app, you can use libraries to take care of the hard parts for you. But once you start getting significant users, like we did with the Maps API, you'll understand their specific usage patterns, and you will find it better to write your own code that optimizes for their use case.

Q: What was the design problem you were trying to fix when building your library?
Matthew: We designed YUI for the frontpage of yahoo.com, and then decided to open-source it for others to use.
Dmitry: IE has a problem with speed, it's very slow, and it has a problem with DOM, it's just fucked up.

Q: Should we use jslint?
All: Yes.

Q: What would you do better if you were starting over?
Dan: With Closure, we would design API from the start instead of adding bits and pieces. The API is surprisingly consistent, likely due to having the same 2 code reviewers throughout, but it could be better. The documentation is shit. Michael Bolin's book fills in the gaps, but you shouldn't need to read a book to understand how to use it.
James: I would design the code to make it testable first, because it's hard to go back and make something testable. If I really wanted to hate my life, I would code everything in IE6 and get it working reasonable there, and *then* see how awesome it performed in other browsers.
Evan: Augmenting the prototype of basic objects (Array, String, etc) was not a good idea.

Generating Slides from a Spreadsheet

Lately, I've been playing around with the HTML5 slides deck from slides.html5rocks.com. A few months ago at the GTUG campout, I hacked together an App Engine app for generating HTML5 slide decks. Last week, in preparation for my GDD Tokyo talk on Google's JavaScript APIs, I wrote a client-side mashup that generates an HTML5 slide deck based on data in a published Google spreadsheet, and used it both as my actual slides and as a demo for the talk.

There are some definite benefits to writing your slide content in a spreadsheet:

• You can look at the revision history of just your content (instead of a confusing mix of code and content)
• You can share the spreadsheet with other people, and collaborate on the content with them.
• You can print the content for easy studying.
• You can create alternate views of the same content, like differently themed slide viewers or slide viewers that show additional columns of information.

Of course, there are disadvantages as well - primarily the fact that Google Spreadsheets wasn't designed as a content management system, and it isn't terribly easy to author multi-line content in the cells.

In case any of you do want to try using a spreadsheet to build your slides, I've made a generalized version of the viewer that will work with any public spreadsheet.

To get started, create a spreadsheet with three columns, 'type', 'title', and 'content'. The type can either be 'intro', 'normal', or 'section', and the title and content can be any text (inc. HTML tags). For example, check out this sample spreadsheet.

Next, publish that spreadsheet using the "Share -> Publish" menu in Google Spreadsheets, grab the key from the URL in the browser, and pass that as the key parameter to the generic slides viewer. For example, here's the URL for the sample spreadsheet:
http://imagine-it.org/sslides/slideshow.html?key=tp1JIiBR7gyKgOKUMDk4d_g

As a bonus feature, you can also include a 'narration' column in your spreadsheet, and display that narration below the slides by passing 'narration=on' to the viewer. For example, here's the Japan talk with narration on:
http://imagine-it.org/sslides/slideshow.html?key=0Ah0xU81penP1dDZWcmxjUjdhOTMyOGxxWXAxMnpKUWc&narration=on

As usual, I will keep experimenting with HTML-based slide viewers until I find the holy grail, so stay tuned! :)

History of Client-side Web API Technology

As part of GDD Tokyo last week, I gave a talk on Google's "client-side Web APIs". That phrase is a bit of a mouthful (and not a common one), so here's what I mean by it: An "API" is a way for one piece of software to interact with another piece of software, a "Web API" is a way for one website to interact with another website, and a "client-side Web API" allows a website to interact with another website purely using client-side technology. For example, Google's client-side Web APIs include the Charts API (image-based), the AJAX Search APIs (JavaScript-based) and the Gadgets API (iframe-based).

Both because I needed a way to explain the underlying technology behind our APIs and because I found it interesting, I started that talk with an intro to the history of client-side Web API technology -- the series of milestones in the evolution of the web that brought us to where we are today.

I was just a wee lass when the web was young, so I've cobbled this history together by reading old blog posts, mailing lists, tutorials, and press releases. (And yes, it is a bit Google-biased, since I have more access/knowledge of our history than others). I'm posting it here in hopes that others will learn from it and others will teach me more about the early web. Please let me know in the comments if you have corrections or additions.

It all started with...

1990: HTML

In 1990, Tim Berners-Lee created the first prototype web browser and HTML page, and you can actually still view that page today.

At this point, HTML was capable only of text and links, so we could link to other server's data, but we had no way of including it in our own page.

1993: IMG

3 years later, Marc Andreesen was working on the NCSA Mosaic Browser and realized they wanted a way to include images on webpages - so he proposed the IMG tag, implemented it, shipped the browser, and it's stayed to this day. That is a typical story for how HTML gains a new tag - someone needs it and implements it, others copy it, and eventually its considered part of the standard.

The IMG tag could point to image resources on external servers anywhere on the web, so it was actually the first way you could bring data from other servers onto your page, though the data had to be in image form.

Perhaps the first commonplace use of the IMG tag as an API of sorts was for "hit counters." People would put hit counters on their sites to track visitors, and each counter was actually just an IMG tag pointing at a server, passing in an ID parameter.

1995: JavaScript

In 1995, Netscape and Sun teamed together to introduce JavaScript, a language they predicted would transform the web. At the time they introduced it, JavaScript could only really programmatically do the things you could already do in HTML - like programmatically creating IMG tags -- but it was an important step towards making client-side APIs more possible.

1996: IFRAME

In 1996, Microsoft introduced the IFRAME element. The IFRAME element could embed another webpage on your page, which is one way of bringing in another server in a simplistic manner. It could also be hacked to bring data asynchronously into a page from the same server, similar to what XMLHR would make possible later.

1996: Flash

In 1996, Macromedia launched the Flash Player plugin. The EMBED or OBJECT tags could now be used to embed a SWF file from anywhere on the web. Flash embeds meant we could embed something more interactive than just an image, like a game or animation.

1999: XMLHttpRequest

In 1999, Microsoft innovated again by introducing the XMLHttpRequest object in JavaScript. XMLHttpRequest could make a request to your server, get data back from it, process the data, and render it into the page however it liked. By default, you could only bring data in from your own server using XMLHttpRequest. With particular security settings in IE only, you could actually bring data in from other servers.

XMLHttpRequest was important in getting people to think about getting data into webpages, and making them consider the possibility of getting data from servers other than their own.

2004-5: GMail, Google Maps

In 2004, Google launched GMail, the first popular web application that relied upon XMLHttpRequest/IFRAME for asynchronous data retrieval from the server, and really showed off what was possible with those technologies. In 2005, Google launched Google Maps, which used the same technology to transform online maps into an interactive experience.

Feb. 2005: "AJAX"

Around the same time, Jesse James Garrett coined the term "AJAX" to describe the new GMail-style of applications which fetch data asynchronously using XMLHR and reduce the time users spend waiting on page loads. After he coined the term and popular JS libraries built in support for XMLHR, it quickly rose in popularity amongst web developers as the new, right way to build web applications.

We were still limited to using AJAX to just getting data from our own domain, however.

June 2005: Google Maps API

A few months later, Google launched version 1 of the Google Maps API, the first JavaScript API by Google and also one of the first JS APIs by a big company. This first API was really a JavaScript library that would dynamically create IMGs for each map tile, position them with CSS, and make it easy to add control DIVs and marker IMGs. Developers would typically use the Maps API in conjunction with AJAX requests to get map data from databases on their server.

Dec. 2005: "JSONP"

In December 2005, Bob Ippolitto wrote a blog post describing a technique he named "JSONP", which used ("hacked") the SCRIPT tag to asynchronously bring data in from other servers.

Finally, with JSONP, we had a way to bring data in from another server without using a server ourselves - as long as that server provides JSONP-compatible output.

May 2006: Google AJAX Search API

Next year, Google launched the Google AJAX Search API, which let you embed a Google search box on your site and display the results there, without ever having to leave the site. Since this API needed to get data from Google servers while being used on any domain, it used JSONP behind the scenes and wrapped it up in some JavaScript functions.

This was one of the first APIs that used JSONP to let developers get data from other servers and include it on their site, and paved the way for others.

And that bring us to...

Present-day: Client-side Web API Technology

Thanks to the contributions to HTML/JS over the years, we now have these technologies to power client-side Web APIs:

• Images
• Iframes
• JavaScript
• JSONP
• Flash

Note: It has been pointed out in comments that other plugin technologies are also important in this history, and used as or inside APIs, like Java Applets and Silverlight. ActiveX scripting also has a part. We do not use these particular technologies in Google client-side Web APIs, but I will look into their history and update this when I have the chance.