Thursday, November 12, 2009

Developer Relations Explained (The Nerdy Way)

I often find it hard to answer the question, "So, what do you do at Google?" I find it hardest when that question is asked by someone completely outside the tech field, like a face-painter or a taxi driver, but it is hard even to explain to techies. It's not as simple as saying that I'm a tester, a software engineer, a product manager, or a support staff, because it's a crazy awesome combination of all the above. I'm in developer relations, and there are an awful lot of ways to relate to developers. We write articles and blog posts, create sample code and one-off projects, maintain open-source projects, respond to forum posts, triage bugs, speak at conferences, meet with partners, and generally do what needs to get done.

One way of explaining our various roles in developer relations is to break it down into two parts: first, the type of developers that we support, and second, the particular developer product(s) that we support. For example, I generally support long-tail developers, and I supported the Maps APIs for the first 3 years, and now I'm supporting the Wave APIs. Some of my colleagues support our higher-touch partners or even our paying customers, and their role involves more 1-on-1 meetings and emails. You can play with the little app below to experiment with more configurations of products and developer types:

Currently, in developer relations, we have one of two specific titles: "Developer Programs Engineer" and "Developer Advocate". Basically, the DPE supports the long-tail and the DA supports the high-touch partners. It doesn't really split as even as that in reality, but that's the basic idea.

So, that brings me to the real reason for this post. We are now hiring DPEs and DAs. The jobs site only lists positions for Mountain View and New York, but there are also scattered positions around the world, in places like Australia, China, and India. It's a fun challenge to figure out how best to relate to developers using our growing number of developer products (including the just-released Go language), and we can always benefit from having a broad range of opinions and skills represented on our team. If you're interested, apply! :)

Wednesday, October 28, 2009

Being a Girl in CS Doesn't Suck

When I posted last week about an experience with gender bias in "Should I Defend My Cred?", some of the re-tweets basically implied that post was proof that it sucks being a girl in CS. Well, I want to clarify that it doesn't suck. It's actually pretty freaking awesome.

What rocks:

  • Working with mostly guys.
    I'm sorry, girls, but I've always gotten along better with guys, ever since I was in elementary school. It's probably because I didn't quite fit into the girl mold (or any mold) back then, and thus girls were hesitant to accept me, while guys didn't really care as long as I was willing to shoot cannons with them. I know a lot of other girls in CS who say the same thing - that they're mostly friends with guys - and actually, those are the girls that I get along with best. So, yes, I should learn how to get along better with girls. But in the meantime, I'll always feel more comfortable working with guys.
  • "Working" with mostly guys.
    I once famously said "I only date guys who program in derivatives of ECMAScript". Well, it's kind of true. I prefer dating guys in tech because I have more stuff to talk about with them, so it's pretty handy that the tech industry is mostly guys. And yes, there is a bit of truth in the phrase often applied to guys in CS: "The goods are odd, but the odds are good"...but I'm pretty sure that the ratio of good guys is still in my favour. It's nice to be in an industry where I constantly encounter so many potential mates, and I don't have to rely on meeting random guys in a bar (which appears to be the dating preference of choice amongst adults, ick).
  • Being recognizable.
    As a girl amidst a sea of guys, I'm pretty easy to find. At a conference, all someone has to say is "find the girl with the red|blue|pink hair", and they can make a positive match pretty quickly. This easy recognition means that I can connect with more of the developers that want to talk to me. It can sometimes cause problems when people remember me better than I remember them (there's a limit to the number of white guys I can store in my head at once), but they are usually quite forgiving.
  • Getting more opportunities.
    Okay, I'm still not 100% sure how I feel about affirmative action.. except that I bet it's worked for me, and that I've probably benefited greatly from it. I remember asking my high school English teacher to review my college essay (which revolved around toilet flushing), and he glanced it over and said, "Well, you're applying as a girl in CS with a high SAT score, right? This shouldn't matter much."
    When an employer or university is comparing 2 equally skilled people, it makes sense that they select the one that increases the diversity of their staff/student body/conference/etc. Diversity is a good thing for teams, and it just so happens that the easiest way to measure diversity is by looking at aspects like race or gender. Theoretically, affirmative action could also applied be to hobbies and interests, and I could be selected due to my intense passion for 80s music (which actually does come up surprisingly often in my talks and demos). Until that happens, I will take advantage of the opportunities that I get by being a girl, and trust that I am not being given opportunities that I don't deserve.

What sucks:

  • Not enough chick flicks.
    I'm pretty sure I would have more excuses to watch romantic comedies if I hung out with more girls. But oh well, I'll just schedule a few more 12 hour trans-pacific flights.
  • Too little dancing.
    Guys don't generally go out dancing as their night activity of choice, at least not guys in western cultures. They tend to prefer a night of tipsy talk at a pub. I sometimes wish that I was a rock star or celebrity, and got to spend every night clubbing in exotic places... but until I acquire any sort of stage talent whatsoever, I think I'll settle with tricking guys into going to pubs with good music.

What really sucks:

  • Being afraid to admit all that.
    As a girl in CS, I am meant to have a very particular opinion about girls in CS. I feel that I risk being viewed as a traitor if I express other opinions. So, I generally try not to express any at all, and just go on my merry way. But, I now worry that other girls may feel the same fear, and that it is cowardly of me to keep my opinions to myself. Now, it's up to you to express your opinions, whatever they may be.

Wednesday, October 21, 2009

Joomla & Google Wave: The Possibilities

Last weekend, I presented a talk on Google Wave to Joomla Day Sydney, and then proposed various ways that Joomla and and the Wave APIs could be used together. I've summarized them below, and would love to hear ideas from Joomla developers on other types of integration.

Using Wave to Produce Joomla Content

Embedded Waves

You might decide that instead of static HTML for a particular article, you want to embed a Wave and enable real-time collaboration. An enterprising developer has already written a Joomla plugin that uses the Embed API and makes it easy to embed Waves in articles, so you can use that.

For example, I can create a Wave with the content of the Joomla Day home page:

Then, in my Joomla instance, I can install the plugin and insert this code in an article:

{googlewave srv=|!w+FspA8QgYA|width=100%|height=600px|fsize=12px|bg=#FFF|ffamily=Georgia}

And then it would produce a page like this:

Note: Only people with accounts for the embedded server will be able to view the Wave, and only if they are a participant or the Wave is public to that whole server. In the future, we hope to allow for another type of public waves that can be viewed anonymously.

You may be creeped out by the idea of anyone being able to edit the Wave, and maybe you don't really want a full conversation on the page. That brings us to the second option.

Exported Waves

If you want to use the collaborative nature of Wave to create your articles, but then still use static HTML for your articles, then you could write a robot to export from Wave into Joomla or a Joomla-compatible format.

For example, the Exporty bot listens for changes in a blip, extracts the contents of the root blip, and stores it in its App Engine datastore. Then, it renders that as HTML at a URL, and when someone requests that HTML, it displays the blip if its Wave was public, and otherwise it asks the user to sign in and checks if they are a participant of the Wave.

So, I can create a Wave with the content of the article, add the co-authors, and then add Exporty-bot:

Then I can create an iframe pointing at the HTML in my article content:

<iframe src="!w%252B7w-1O4JSA" width="100%" height="700" frameborder="0"></iframe>

And then it would produce a page like this:

A more elegant solution would involve eliminating the iframe and baking the content into the page. But I will leave it to someone more experienced with Joomla to figure out best way to do that.

Using Wave to Extract Joomla Content

Using robots and gadgets, you can find ways to bring in content from your Joomla site.

For example, a user of your site could use Wave to subscribe to changes in the site. The RSSy bot lets you specify an RSS feed and updates the Wave with blips for each new item in the feed.

So, you can create a new wave, add RSSybot, and specify the RSS from a Joomla site:

The resulting Wave looks like this:

This bot doesn't do the best job at handling the HTML in the feed, but you could improve that to insert iframes with the HTML, or convert the HTML into a Wave-compatible block of text with annotations for the styles.

Should I Defend My Cred? (...Yes)

Tonight, I gave another Ignite talk on "Google Wave & Collaborative Mapping"/. The talk went well, and it was a great opportunity to hear what other people are thinking of doing with Wave. But, something interesting happened after...

I was asked by a speaker to basically defend my cred- as he looked at my appearance (a short green skirt & t-shirt), saw that I gave a presentation that glossed over the technical details, and assumed that I wasn't that technical. When I explained to him that I actually do write code, he was fairly taken aback. He then recommended that I start off each presentation by clarifying my level of knowledge, and getting "respect" from the audience. His basic theory is that girls are not respected as technical peers until they sufficiently prove themselves, and apparently, particularly not if the girl is decent looking. Now, I want to explore that theory further (outside of the noisiness and distractions of the crowded pub).

When I was in high school, I participated in MUN (Model United Nations), where high schoolers would be delegates for a particular country and argue position papers. At the conferences, I remember that I myself mentally discounted the ability of the female delegates when they went up to speak. I was willing to believe in them, but only after they really showed their stuff. I didn't have this same feeling with the guys, and I came to the conclusion that there are some areas where one gender garners more of an immediate respect than others. I decided then that I would have to come off as incredibly confident (but not bitchily so) in order to win the respect of the MUN people, as I assumed that they would have that same accidental bias. The bias made sense to me in the area of speaking - men naturally have deep, confident voices, and so you just want to believe in that voice. I don't know how to describe women's voices, but it's certainly not like that.

I think the respect bias may extend beyond debating in the tech world, however. When I look at the Twitter account for a self-professed "girl geek", I grow immediately suspicious of their geeky claims. When I see a girl go up to present on the stage, I usually assume they will talk about something less technical. Maybe this is because my suspicions are usually confirmed -- because we live in a world where we try to extend a geek label as far wide as possible, to try and sneakily get more girls "in CS." Maybe it's because girls naturally hate girls (a well documented phenomenom in women's magazines), and this is an extension of the phenomenon.

So, I'm biased, he's biased, and potentially others are as well. I don't know that we'll be able to eliminate our subconcious tendencies, but we can help people squash their own.

When you give a talk, always start off with an introduction slide that describes your background and experience. If you're an expert on the topic, admit it (humbly). If you're just learning and wanted to share your learnings, admit it.

I think part of the reason that we try to rely on other (possibly incorrect) clues to help us form opinions is that people don't give us enough information about their credentials. And I don't think that we mind if someone is or isn't technical - we just want to know, one way or the other, and not feel like we're being misled. We respect people for who they are, but we don't respect people if we suspect that they're trying to be something they're not.

Thoughts welcome, of course. :)

Tuesday, October 13, 2009

Why I Sent You to a Group

One aspect of my role as an API support engineer is making sure that developer's questions are answered.

So, when a developer personally emails me a question, you would think that I would rush to answer it.

No. In fact, I usually ask them to post their question in the relevant group, and even if I know the answer to the question, I do not reveal it in my response.

This is because another aspect of my role is making sure that developer's questions are answered in a scalable manner.

If a developers emails me a question and I then answer it, that question and answer exist only inside our inboxes. They are absolutely undiscoverable by anyone else in the world, and basically useless to 99.9999999999999% of developers. However, if a developer posts a question in a forum and then receives an answer, that thread exists in the public group, it exists in the cached Google pages, it exists in the mail group archives - and most importantly, it exists in Google search results. That means that when another developer has the same question and search for it, they will find an answer - without even having to post their question. So, potentially 100% of developers can benefit from the question and answer. Now, *that* is scalable.

Now, a developer might think "But, I'm the only one with this question." This is nearly almost not the case, and it is not something that a developer can predict. Better safe than sorry, and share the question with the world.

Another aspect of my role is making sure that questions are answered in a timely manner.

If a developer emails me a question, and I happen to be quite busy then - with conferences, travel, launches, etc. - then I may not be able to respond for a few weeks. However, if a developer posts that question in the group, all of the group members may see it, and the chances are that atleast one of them know the answer and happen to have the time to respond. If they post in the group and don't get a response after a few days, it is fine protocol to ping the thread nicely, or hey, I don't even mind when a developer pings me personally and asks me to respond to the group thread.

So, oftentimes, when I ask a developer to post in a group, it's because I want them to get an answer faster than I can provide, and I happen to know that the Group is the best way to guarantee that.

Basically, the only times that I *happily* respond to a one-on-one email (or wave) is when the matter concerns personal account information and must be worked out on a case-by-case basis - for example, requests for higher geocoding limits, or the recent problems with some sandbox users not getting wave preview invites.

If you are a developer that I have sent to a group, please understand it is for your own good and the good of the world. Thanks for understanding!

Thursday, October 8, 2009

A World of Words.. And Me

Tonight, I gave a talk titled "A world of words" at Ignite Sydney, and I want to give a bit more information about why I gave that talk.

As I said at the beginning of the talk, I have always had a thing for words. When I was a wee lass, I had a favorite song called "Great Big Words" and loved to finish crossword puzzles. In high school, I realized that I needed to study words in order to do well on the SATs, and I could get away with more word geekery that way. I downloaded multiple flashcard programs on my PC, and I forced friends in my carpool to bring in words to discuss during each morning ride. Needless to say, I aced my test... and I was also no longer part of that carpool. Then, in university, I joined an honor society and suggested that we educate our peers by plastering the kiosks with "Word of the Week" posters. On each poster, I took great pleasure in coming up with a ridiculous sentence that used the word, and an overlay thorough description of its etymology. (See: poster for "diaphanous")

So, I realized I had an obsession, and I needed to make it legit. I enrolled in a linguistics minor, and I was hooked. Everything about language was fascinating - why we make the sounds we do, why our sentences can become ambiguous, even stuff like why we can say "abso-fucking-lotely" and not "ab-fucking-solutely" (seriously, that was covered in two classes). Inspired by my newfound knowledge, I applied for and was accepted into the John Hopkins Computational Linguistics summer workshop. There, I spent the summer working on a tool for visualizating the parse trees for sentences translated into multiple languages (See: screenshot).

That project got me thinking about the various ways that I could combine my computer science skills with my love for linguistics. I eventually concocted a project that would use both those, and my other minor, 3d animation, and convinced my advisors it was worthy of credit. It was entitled "A Computational Framework for Simulating Cross-Linguistic Acquisition of Spatial Prepositions", and it was basically an attempt to get a computer script to learn prepositions (above/under/on) by analyzing a 3d-scene and given sentences. And, hey, it kind of worked. (See: e-poster). I was on a roll!

So, I was at the point where I was trying to figure out what was next in life. I knew what I loved, but couldn't decide what to do with what I loved. It was suggested that I apply for a Fulbright grant - a grant that would let me do research somewhere in the world - and I thought that was a mighty fine idea. I wrote up my proposal for an app that I had always fantasized about. It was called "A World Wide Web of Comparative Linguistics", and it would let user visualize and search through etymological history on a map-based interface. (See: proposal). In my opinion, the app would bring immediate world peace. How could you possibly fight with someone after you realized how connected your words were?

Unfortunately, the Fulbright committee didn't see it that way, and I didn't get a grant. I instead went on to get a Masters degree and get hired by Google, and that's where I am today. I've been pretty busy supporting the Maps API for the past 3 years, but lately, I've started craving a bit of linguistics in my life. I started listing to word-of-the-day podcasts on my iPod during my morning commute, and it just gets my brain all tingly.

So, I put together this talk as one way of satisfying that craving (or, more likely, making it much bigger), and also to share this bit of knowledge with others. It's a nice break from my typical web talks, and I think I surprised a few folks.

You can see the talk embedded below or on slideshare, complete with captions of what I actually said.

You can also watch the recorded talk below:

For more resources on the topic, you can check out the sites that I bookmarked while researching the talk, subscribe to the podictionary podcast, or start looking up words in the online etymology dictionary.

And, of course, stay tuned to the blog to find out if I do ever create that etymology visualization app.. It would be a damn nifty thing indeed, even if it didn't bring world peace. :)

Thursday, September 10, 2009

How Google Wave Could Improve Education: Group Work

Last night, at the Google Wave Sydney User Group, we brainstormed a lot of different ways that Google Wave could be used in a vast spectrum of fields. A recurrent theme was education, and we touched upon various ideas, and basically just agreed that Wave (and collaborative technology, generally) can have huge implications for education. One of those ideas we discussed was group projects, and I wanted to discuss that area in more detail here.

As a teacher, you often want to encourage group projects, since they can help students learn cooperation and teamwork, and since they can often have a synergistic effect and produce amazing results. But, there's always that same concern with group projects - someone will do all the work, someone will do none, and it's impossible to know who deserves the good grade. Well, Wave could potentially solve this, both in terms of knowing who to give credit to, and encouraging a better balance of work across group members.

Here's an example proposal based on my own experience:

You're a teacher running a game projects class, and you've divided the class into project teams for the semester. The first aspect of creating a game is writing a design doc for the game (describing gameplay, objective, characters, etc), and you make that the first assignment for each team. You create a Wave for each team that has the design doc template in it (headers), and you give them a week to work on it. You recommend that they use reply blips and inline blips to divy up the work, and tell them that you will be monitoring the project and will do a review of the first draft in a week, and grading of the final a week after that.

When you review, you reply in the Wave and comment on what parts need work or clarification, and which parts look good. When it comes time for grading, you use the playback functionality to see how everyone contributed. You see who took the lead and made decisions, who wrote up large sections, who revised sections for clarity, and who just sit back and watched. Now, hopefully, since every team member knows that you can see all the edits and attribute them to each of them, they will actually feel more motivation to contribute, and you won't have many students that just do nothing. But if you do, well, you can penalize them accordingly, and perhaps next time, they will learn that their actions (or lackthereof) are being watched.

You could even automate some aspects of this with robots. There could be a robot that counts the word count of each team member, and outputs that in a stats gadget at the end. This could feel a bit too strict for the students, but it can also serve as a nice "hey, you're not quite doing as much" reminder. That robot could also just monitor progress as a whole, making sure the team stays on progress. The gadget could list stats like "5 sections completed, 5 days left until final due". Then, those per-team-member stats just become another indicator of progress on the whole, and don't feel as accusatory.

One issue I see with this proposal is the tracking of "peer waving." Some students may sincerely find it more productive to sit at one computer and write up sections together - perhaps one giving the ideas, the other putting them into words. These words would only be attributed to one student, and the idea giver could be unfairly penalized. Possible solutions for this would be to annotate the text [Peer waved w/] or to recommend that students alternate between accounts when doing this.

Another issue, of course, is that not all group projects are writing up a blob of text, and in fact, many of them are not. In computer science, many of them are actually writing code. You can already ask students to use SVN and monitor commits from each team member for less fine-grained monitoring, but you could potentially even have students write their code in a Wave, and use a robot to compile it out, for real-time monitoring. In other fields, many group projects involve the the creation of a physical (non-digital) object. Perhaps you could have students write a design doc for that physical object, or even use a collaborative gadget to sketch it out.

So, yes, this solution isn't perfect, and for non-text projects, involves a bit more creativity. But, I still think it's pretty damn good. If you're a teacher, try it out and report back!

Tuesday, September 8, 2009

The Quest for Karaoke-able Slidesets

I'm a massive fan of karaoke, and I've often expressed my opinion that some songs are far more "karaoke-able" than others, and that someone/me/an entepreneur should create an app capitalizing on that. The app could let you rate the karaoke-ness of songs, upload video evidence, create karaoke playlists, and find lyrics for practicing. It'd be an awesome app to improve the karaoke experience, but somehow, people are surviving and managing to enjoy singing drunkenly without it.

Well, perhaps traditional karaoke doesn't need an app, but "Powerpoint Karaoke" definitely does. For the unfamiliar, Powerpoint Karaoke is yet another new style of presentation, and it basically involves having the speaker speak to a set of slides that he's never seen before, and that were originally delivered by someone else. It serves partially to make fun of the sometimes incomprehensible nature of slides, partially to just be damn funny as the surprised speaker deals with the surprise slides, and partially to improve the speaker's presentation skills. If you can present an arbitrary slideset and woo the audience, then you can present nearly anything.

For our most recent set of internal lightning talks, I gave speakers the option of Powerpoint Karaoke instead of their own slideset, and one brave speaker (Jez) took on the challenge. And I, being the dumb-ass who suggested a format I have never done nor seen, was tasked with picking the slideset for him to present. Yes, I could have gone the route of randomly finding a slideset on the web as soon as Jez got on the stage, but considering that this was the first and only Powerpoint Karaoke that the office would witness that day, I wanted to make sure it went well. These were the approximate requirements I had in my head:

  • The slideset should fit well into our lightning talk 5 minute format, with about 15-30 slides. Ideally, I could have the slides auto-advance, to challenge the speaker's reflexes further.
  • The slideset should have a title and picture(s) on each slide, but not much else. If there was too much text, the speaker could just read that off and know the original intention of the slide. If there wasn't a title on any slide, then the speaker would have too much freedom in what he said.
  • The slideset shouldn't offend anyone in terms of politics/religions/beliefs - I didn't want to pick a slideset with the clear intention to make fun of some group. It might offend, and that's too easy, anyway.

So, using Slideshare's search and browse features, my search began. I looked at the top slides in various categories for all time, I did searches for random topics, I looked for slides from speakers that I knew did the kind of slides I was looking for. After an hour of searching, I had nothing. Most slidesets were the wrong length, the wrong amount of textual content, or just plain spam (slidesets with 1 image advertising a product). I then asked the office for suggestions of topics*, and used those random suggestions to seed my searches. After another half hour, I found 2 possibilities: "How to Defend Against Zombies" and "How I learned to love ballet". I picked the second one, as it had exactly 20 slides (Ignite format!), referenced the speaker's wife (Jez's wife works in the office too), and I thought it would be more humorous if Jez could turn ballet funny than if he could turn zombie invasions funny (since, well, they already are).

How did it go? Jez took that slideset and freaking ran with it - he did more with it than I could have imagined. His best line was when he got to the slide about "Barre" (a position) and explained that was the Japanese pronunciation for "Ballet".

But, I've learned now that it is much harder to find karaoke-able slidesets than karaoke-able songs, and it makes sense when you realize that the majority of songs are there are similar length & lyrics density, whereas the slidesets in the world are a highly diverse species. So, I would like an app for finding suitable slidesets, or even better, I would like Slideshare to add the following features to their search: restrict by # of slides (a range?), restrict by text density (avg words per slide?). Then, everyone in the world can get on the Powerpoint Karaoke bandwagon, and it can be the next great (geek) thing.

*And for your amusement pleasure, here are the list of topics that my colleagues proposed. I asked for these when I was contemplating actually creating the slideset myself with random images, but realized that isn't really the point of Karaoke, and used the ideas as keywords for slideshare searches. That could become an entirely new style of presenting though, especially if someone creates an app that can generate slidesets from just the titles - complete with images, graphs, and venn diagrams, of course. :)

  • "Lightning bugs"
  • "Japanese pop culture from the 1980's"
  • "Sesame Street and Alcoholism"
  • "100 Things to do with Dwarves and Leather"
  • "How to prepare for a zombie invasion"
  • "Analytic and algebraic topology of locally Euclidean parametrization of infinitely differentiable Riemannian manifolds"
  • "How Google could make billions through the power of hypnosis"
  • "Early 20th century ballet and swimming pool repair"
  • "Properly Disambiguated Java Generics"
  • "How to bluff and give an insightful talk when you have to make it up on the spot with no prior warning"
  • "Latin American Hedge Funds and You"
  • "The Social and Economic History of the Grapefruit"
  • "Care and Feeding of the Dugong"
  • "Unicorns"
  • "England Won the War of Independence - a counterfactual world history"
  • "Keelhauling, Walking the plank, and other piratical forms of corporal punishment"
  • "Why Robot Ninja Monkey Zombies are AWESOME"
  • "A complete etymology of My Little Pony"
  • "Zonkies: discuss"
  • "The renovations that Start City *should* be making"
  • "A 5-minute tutorial in interpretive dance"

Friday, June 5, 2009

The Poem Store: "Write a Poem About Her Hair"

During the crazy week and a half that I was back in the states (after 8 months of being in Australia), I took a break from launching products and speaking at conferences to attend the Makers Faire with my old uni buddies from Los Angeles. The Faire was awesome - arts and crafts, fire-breathing objects, corn dogs, rockets, bicycle bands, sno cones, massive mouse traps, everything you could ever want in an amusement park for geeks. One of the really random things we encountered was a guy sitting under a tree, with a sign that said "POEM STORE: Your topic, Your price." That seemed like the coolest promise ever, so my friend Diana approached him and commissioned a poem on my behalf - about my hair. After I gave him a 2-minute recent history of my hair, he quickly typed up this on an old-school typewriter...

sky blue
but nights
falling down
the plummets
spectrum doesn't
cover everything
but it does make
it darker than it
was last once weekly
dying now we survived
 our out last dead protein
             goes DEEP

..which I think is awesome, even if I don't get all of it. We chatted with him more, and found out that he had offered the same service outside Google I/O, and that "web people gobble it up." Well, fine, we do. :) His name is Zach Houston, and he's always looking for more places to show up. Shoot him an email if you've got a poem-hungry crowd at your disposal.

Sunday, May 31, 2009

Ignite I/O - Growing Up Geek: My Dad, The Computer Scientist

"Ignite" is a style of presentation: 5 minutes, 20 slides, 15 seconds each, auto-advance. It's probably the most challenging style of presentation out there (lightning talks are a breeze in comparison), which is why it intrigues me. I did my first Ignite in Sydney a few months ago, with a talk entitled "HTML 5 versus Flex for RIAs". It went well, as I was able to remember all of my lines, and I managed to squeeze quite a bit of content into those 5 minutes. But I discovered that the best Ignite talks there weren't the informational ones, like mine - they were the narratives ones, the ones that told stories and left the audience feeling moved in some way.

So when I was asked by Brady Forrest to give an Ignite talk at Google I/O, I decided that I would find a story to tell, and hopefully one with a geeky twist. Thinking of how much my JAOO Brisbane audience loved the slide in my App Engine talk about my dad not giving me allowance unless I'd programmed Java that week, I formulated my talk: "Growing up Geek: My Dad, The Computer Scientist". I didn't have an exact outline for the talk when I proposed it, but I figured I had enough entertaining anecdotes about life with geeky parents (my mum ultimately became a computer scientist as well) that I could fill up 20 slides.

Creating the slides took ages, since my search for authentic pictures for the slides meant digging through our old archive of 5000 digitized family photos to find evidence of our myriad PCs, and using the Wayback Machine to capture screenshots of my original websites and Perl programs — and man, nolstalgia is a bitch of a distractor. Soon, a real story began to emerge amongst the slides: the story of me wanting to do many things — including programming, my dad disapproving of these non-programming activities, and me perservering on with my renaissance approach to scholarism and finally gaining his approval. The point I want to make with the talk is this: you should always encourage your kids to explore everything that interests them. Yes, this may mean that they don't become a specialist, or that they don't become the particular profession you've set out in your heart for them — but it means that they're doing what makes them happy, and people always excel the most at what makes them happiest.

The final slides are embedded below, with my pre-scripted lines pasted in the slides (it's impossible not to script them, trust me). I've also entered the slides in a "Tell a Story" contest that Slideshare is running now, since they happen to tell a story; you can vote it up on Slideshare if you like. The video is now up on Youtube. Enjoy!

Sunday, May 17, 2009

Google Spreadsheets + APIs = Interactive Flash Cards

Update: Google is deprecating Google Spreadsheets gadgets, as they announced in this post, so I am no longer updating or supporting them. If you're a developer, you can try Apps Script or the Spreadsheets Data API to see if you can accomplish the same thing. If you're a user, sorry, sometimes Google shuts down little used features.

For about 6 months in the Mountain View Google office, I went to a once-weekly hour-long German language class. Since I didn't have much time to really do homework outside of class, I looked for excuses to use Google APIs to give me different ways of learning German. My first attempt was a gadget that combined image search with a spreadsheet-stored German vocabulary wordlist. I liked the idea of using computer-generated hints for the words and wanted to make a more general solution for any topic/language.

So, I made the interactive flash cards gadget. Each flash card has a hint and a form of guessing, with different options for hint generation and guessing strategy. The hints can come from the spreadsheet (e.g. user-entered definitions), Google translate (any supported language pair), Google image search, or Wikipedia (*currently down). The guesses can be entered in a simple text input, selected from multiple choices, or solved in a word jumble. Hopefully, these options are diverse enough for all types of learners.

Here are steps for using the gadget:

  1. Create a new spreadsheet, put a list of words in the first column, and put hints in the second column if you'd like to use that option. My sample spreadsheet has an animals wordlist:

  2. Click on the "Insert" menu and then select "Gadget..." This presents you with various categories of gadgets to choose from (similar to the iGoogle directory). My gadget isn't yet in the gallery, so you'll need to select "Custom" and then type in the URL to the gadget:

  3. The gadget will appear embedded in the current worksheet, and it will prompt you to select a range of data to send to the gadget. Select whatever columns you've created (either just words, or words and hints), and you should see the Range text field update with the range. If it doesn't work, you can manually type it in. Click "Apply".

  4. The flash cards gadget is designed to take up a bit more space, so it's best to move it into its own sheet. Click the menu in the upper left of the gadget and select "Move to own sheet".

  5. You can now play with the different hint/guess options to find your favorite. Several combinations are shown below.

    Image Search + Word Jumble

    Second Column + Multiple Choice

    Translation + Type-in

You can play with my sample flash cards here. One day, I hope to add the ability to track your progress in learning the flash cards content... but if you're a developer, feel free to beat me to it!

If you're a developer and want to tweak this gadget further, you can grab the code from here. It's licensed under Apache 2.0, so feel free to tweak it and use it however you'd like. (And if you've got fixes, I'm happy to patch them back in).

Creating a Wordsearch using Google Spreadsheets

Update: Google is deprecating Google Spreadsheets gadgets, as they announced in this post, so I am no longer updating or supporting them. If you're a developer, you can try Apps Script or the Spreadsheets Data API to see if you can accomplish the same thing. If you're a user, sorry, sometimes Google shuts down little used features.

I'm a fan of alternative learning and testing techniques. Back when I was the teaching assistant for the "History of Video Games" class (yes, that's a real class), I gave the final exam as an illustrated crossword puzzle. It was surprisingly hard to find software for creating that crossword, so I hoped to make a Spreadsheets gadget to make it easier. Unfortunately, crossword-solving algorithms that run entirely in JavaScript are hard to find, and I gave up and went for second best: a wordsearch gadget. (A big thanks to Robert Klein for the wordsearch JavaScript library.)

Here are steps for using the gadget:

  1. Create a new spreadsheet, and put a list of words in the first column. (Or, alternatively, use an existing spreadsheet that has a column of words you're interested in). My sample spreadsheet has a simple animals wordlist:

  2. Click on the "Insert" menu and then select "Gadget..." This presents you with various categories of gadgets to choose from (similar to the iGoogle directory). My gadget isn't yet in the gallery, so you'll need to select "Custom" and then type in the URL to the gadget:

  3. The gadget will appear embedded in the current worksheet, and it will prompt you to select a range of data to send to the gadget. Select all the cells that contain the desired words, and you should see the Range text field update with the range. If it doesn't work, you can always manually type it in.
  4. You can now customize the number of rows and columns. The default is 10 by 10, but if you have more words, you likely want a larger wordsearch. Click "Apply", and see the generated output.
  5. You have a few options for how you use the wordsearch. You can play with it immediately, inside that gadget, or you can use the option on the gadget menu to move the gadget to its own sheet and use it there. Note that each time you reload the spreadsheet, the wordsearch will be randomly generated with a new layout - so don't rely on it staying the same.

  6. You can also publish your spreadsheet so that other people can play the wordsearch. Click "Share > Publish" and you'll see the published URL, which you can share with friends. You can also use options on the gadget menu to embed the gadget in your iGoogle page or inside a normal webpage.

You can play with my sample wordsearch here. Enjoy, and hopefully one day, I'll have a gadget that makes crosswords too.

How to Convert a Google Spreadsheet into JSON, XML, and MySQL

Update: Google is deprecating Google Spreadsheets gadgets, as they announced in this post, so I am no longer updating or supporting them. I've written a post on a new technique here.

As some people know, I have a huge fetish for Google Spreadsheets - maybe because I'm always dealing with small datasets, and Spreadsheets is perfect for tinkering with them. Often, I start with my data in Spreadsheets, and later want to move it into a different static format - like a JSON file, MySQL data, or XML file. I originally did migration by concatenating column values together (e.g. ="" & A1 & ""), but I decided to make the process easier by creating a generic converter gadget. Using my gadget, you can easily convert any spreadsheet into those formats.

Note: I could get JSON and XML using the Spreadsheets data API, but then I would have to deal with alot of ATOM cruft when I'm only interested in the bare data.

Here are the steps for using the gadget:

  1. Create a new spreadsheet, and put your data in columns. Give each column a name, and choose carefully. Since this name will be used for MySQL attribute names, JSON keys, and XML tag names, the best names are lowercase, whitespace-less, and descriptive. The image below shows a spreadsheet of geocoded pizza locations:

  2. Click on the "Insert" menu and then select "Gadget...":

  3. This presents you with various categories of gadgets to choose from (similar to the iGoogle directory). My gadget isn't yet in the gallery, so you'll need to select "Custom" and then type in the URL to the gadget:

  4. The gadget will appear embedded in the current worksheet, and it will prompt you to select a range of data to send to the gadget. Select all the rows for the column with your address data, and you should see the Range text field update with the range. If it doesn't work, you can always manually type it in.

  5. You can now select either "JSON", "XML", or "SQL" from the dropdown, click "Apply", and see the generated output. To get the output in your clipboard, just double-click inside the frame, select all and copy.

You can see the published example spreadsheet, and see the XML output of the gadget in the second sheet. I generally move gadgets to their own sheet to make my workspace cleaner.

Tuesday, May 5, 2009

How do you document an options Object?

In JavaScript APIs and libraries, you can make a constructor or function more flexible and easy to use by having developers pass in an object of additional options.

Here's an example of a badly designed constructor that doesn't take advantage of this feature - the final 5 arguments are all optional, but yet, location must be passed in if the developer wants to specify hobbies.

EvilPerson(name:String, location?:String, age?:Number, job?:String, hobbies?:String);

When used, it's hard to understand what this constructor is doing:

var evilPerson = new EvilPerson("Taz the Haz", null, null, "Annihilator", "Blowing stuff up");

Here's an example of a well designed constructor that makes developers specify the absolute core information, and then pass everything else via an options object:

GoodPerson(name:String, options:Object);

When used, it's very obvious what options we're setting:

var goodPerson = new GoodPerson("Snuggly Muggly", {job: "Cuddler", hobbies: "Spooning"});

So, most people are in agreement that this is a good technique, but there's one problem: there's no standard on how these options objects should be documented.

In the JavaScript Maps API, we give the objects a class name, link to the class reference from the constructor, and then list the keys there. The advantage is that we can refer to these objects in the developer's guide or forum by name, the disadvantage is that we list a class in our reference that developers cannot actually instantiate, and that can be confusing.

The screenshots below are from the GMarker and GMarkerOptions reference.

In the Maps API for Flash, we turned those name objects into actual classes that developers must instantiate. This is good for documentation, but does perhaps bloat our API and make constructors more verbose.

The screenshots below are from the Marker and MarkerOptions reference.

In jQuery UI, every widget gets its own single-page documentation, with "Options" being a section along with "Events" and "Methods". There is no actual reference for the widget constructor itself, presumably because every widget's constructor takes only an options object. This likely works for jQuery UI developers as they're accustomed to this convention, but I personally found it confusing that there was no constructor reference.

The screenshots below are from the Dialog reference and options section.

In Dojo API, the reference for many of the Dijits specify that they take an Object for additional parameters, but they don't actually make it clear what can be passed into that object. My best guess is that the fields listed in the "Properties" section can all be passed in. This is probably once again a convention that seasoned Dojo developers are accustomed to, but arguably pretty confusing for a new Dojo developer.

The screenshots below are from the Dialog reference and the properties section.

In the Gears API, none of the constructors take a options argument, but there is something somewhat analagous: objects passed into event callbacks. In this case, Gears documents the object entirely inline with the reference. This technique has the advantage that everything about the function and the object are in one place - and the disadvantage that there's not a great way of referring to that specific event callback object elsewhere (besides saying "the object that the onerror function passes back").

The screenshot below is from the WorkerPool reference.

All of these styles have their pros and cons - and there's no one technique that satisfy our goals, like being easy for newbies to understand easy to refer to elsewhere in the documentation. What do you think? Have you seen a documentation that does it better, or do you think one of these techniques is clearly superior?

Saturday, April 25, 2009

What is an API?

A game designer asked me the other day to explain the meaning of "API" - and I thought, wow, that is a good question, considering our massive (over?)use of the term on the web. Of course, the obvious meaning is "Application Programming Interface", but that's pretty damn non-helpful, particularly for a non-programmer. So here's what I told him:

There's the traditional meaning of "API" -

Some programmer writes a library with lots of functionality in it, and exposes the functionality as different functions, so that other programmers can just call the functions.

For instance, I could write a function that goes through all the complex steps of making out:

function makeOut(passionLevel, partsOfBody) {
 for (each partOfBody in partsOfBody) {

So if you want to have your code make out, then you could just call:

makeOut(10, ["neck", "ear", "mouth"]);

You don't have to worry about the "implementation details" - all that code I wrote - you just know that it works.

The "library" is the actual code, the "API" is what programmers need to know in order to use it.

So my API documentation would provide the minimal info needed to use the library, and it could look like this:

makeOut(passionLevel: Number, partsOfBody:Array)

For an example of actual API docs, Java has the online documentation for all of its common libraries here:

And then there are "Web APIs" -

A Web API hosts all of its code on one server, and documents how other servers can call that code.

There are JavaScript Web APIs, like the Google Maps API, where the Javascript is publicly viewable but quite "obfuscated" (compressed, hard to read).

This is our actual code:

This is our API documentation:

Developers only need to know what the API looks like, they don't have to figure out what all that crazy code means.

Then there are HTTP APIs, which means that all of the code is hidden on the server, and the code is executed when a programmer hits a particular URL. Often times, these APIs will create new information on the server for a user. All of our consumer products with user data have HTTP APIs - like Google Spreadsheets, Google Calendar, Blogger, etc. A programmer can hit a particular URL with an authentication header, and either get or create more user data.

One of the most popular HTTP APIs is the Flickr API, which can be used to retrieve information, upload photos, create albums, etc. Their API is documented here:

You can use the API without authenticating if you just want to do a simple search:

Notice the query parameters after the "?" like method, api_key, and tags. These are like the parameters to a function (like passionLevel and partsOfBody from above). You could imagine that flickr has code on their server that looks like:

function searchPhotos(api_key, tags) {

But once again, a developer never has to see the Flickr server code, they can just use the API and get the results they want.

Above all, an API is a promise.

It's a promise that no matter how the author changes the code (the implementation), the interface will remain the same. You can always call that makeOut function, and expect it to make out, even if it stops complaining about people watching.

Monday, April 20, 2009

Web09: Design, Code, Community.. and Maps!

Earlier in the year, I was invited to speak at Webstock, an annual conference in Wellington, New Zealand. It was a fantastic conference filled with inspirational speakers from around the world who discussed the future of the web, ways in which it sucks now, what we could do to change it.

This week, I had the opportunity to speak at Web09, a similarly named conference in Auckland, New Zealand - just an hour flight north. I was a bit surprised that New Zealanders felt the need to hold 2 web conferences in a year, and wondered if this conference would be different from Webstock. Well, it was completely different - in a good way. Web09 was a cornocupia of real-world advice from both local developers - like Karl Von Randow's talk about gaming the iPhone App store, and global evangelists - like Ryan Stewart's talk about new Flex features. Some of my favorite talks were:

  • Dan Rubin gave us a variety of ideas for creating better designs, including adding noise to a background, using your own photographs as the focal piece of a website, and scanning in every day objects to get web textures. In trying to define good design, he referenced Don Norman as the cognitive scientist who originally articulated the problem of doors that are so un-intuitively designed that they require signs - a problem that we have in our new Google building. Norman authored The Design of Everyday Things, which I've now put on my reading list.
  • Paul Burnett, Adobe Flash Evangelist, showed off various new features in Flash CS4 and its much better integration with AIR (now better than Flex Builder!). I was most impressed by the fact that Flash now supports adding bones (inverse kinematics) to shapes and graphics, and being able to set weights and properties and tweak the RTS graphs for every animation. It takes me way back (2 years!) to futzing around with character animation in Maya and Max, and makes me want to get CS4 and try animating a few simple characters - zombie, whale, maybe even a streetwalker.
  • David Karp, creator of Tumblr, described 5 areas of community building: engagement, use, negativity, change, feedback. Karp suggested that the reason that people participate in communities is for the "promise" of something happening - like the people who upload to Youtube thinking they have a small hope that one day their video will shoot to fame and they'll be a Youtube rockstar. Karp also described how Tumblr went from 1 support email a day to hundreds, and how they dealt with it. They categorized every support email as both "type of user" plus the actual problem - with user type ranging from paying customers and new customers to evangelist users. Visualizing the requests that way enabled them to decide what features were important to growth.

I myself gave a talk called "Avoiding Red Dot Fever: Tips for Improving Usability of Maps Mashups". It was a series of tips for ways web developers could improve the maps on their sites, based on my experiences seeing many badly-designed maps over the past 2 years. The talk went over really well (so well I was asked to give it twice), and gave me an excuse to meet the web developers of many New Zealand mapping sites - Zoodle, Yellow Pages NZ, vodafoneNZ, EventFinder, etc. The slides are embedded below, and there should be a video up in a few weeks.

Overall, it was a great conference, and I hope to see both Webstock and Web09 happen next year.

Thursday, April 9, 2009

Google Image Search + Color Restrict: Just look what you can find!

This week, Google officially announced support for filtering by colors in Google Image search results.I don't actually know if this is actually a useful tool yet, but even if it's not - it's freaking fun. Here are some cool results from myself and the twitterverse.

You can find beautiful landscapes: sky, red

You can find your school logos: usc, red
(via @oGLOWo)

You can find a world covered in fail stickers: fail, red
(via @niick)

You can find a phrase that's lived up to its colors: state, blue
(via @ManoMarks)

And one more of those: journalism, yellow
(via @KevinMarks)

You can find two colors at once: red, black

And finally, you can find creatures you never thought existed: dolphins, pink
(via @niick)

Wednesday, April 1, 2009

API Design: Beautiful Interfaces

This is part of a series on Web API design. The intro post is here.

When some developers think about the Google Maps API, they may envision a sea of GMarkers, GPolygons, GScaleControls, GOverviewMapControls, and the like. But really, it boils down to 2 things: GOverlay and GControl.

These two classes are interfaces in the API, and every class that is a visual item on the map extends from one of those two. Conceptually, a GOverlay is something that moves when the map moves and is somehow related to latitude/longitude coordinates, and a GControl is something that always stays in the same place, regardless of map movement. The screenshot below points to various examples in a standard maps mashup:

To add one of these to the map, a developer just calls addOverlay or addControl:

map.addControl(new GScaleControl());
map.addOverlay(new GMarker(new LatLng(37, -122));

To remove them, they call removeOverlay or removeControl.

There are some mapping APIs out there with calls like addMarker, addPolyline. This is is a bad idea for several reasons. It means you need an additional method each time you add a new visual class (bloating your API), and it likely also makes it hard for developers to extend the API. With generic methods like addOverlay or addControl, you can add a custom extension the *same way* you add a native class - so newbie developers don't have to learn a new syntax to add one to their map.

Along with using generic methods for adding stuff to the map, we can also take advantage of the interface to make our events more generic. A developer can listen to "addoverlay" or "removeoverlay" to find out when any overlay has been added to the map, and when they listen to a "click" event on the map, their callback gets passed in whatever GOverlay was clicked on. This means that whenever we add a new type of overlay to our API, the developer can continue using the same events they were before. We do often add specific events for each overlay later, but it's nice that the generic ones come for free.

And something for the advanced developers...

These interfaces make our API infinitely extensible in a standard way, and they're the reason I used the API for my personal and research projects even before working for Google. If I have a fundamental aesthetic issue with our GLargeMapControl3D, then I can simply create my own control (ExtLargeMapControl). If I want to visualize 5,000 markers, and GMarker is simply too full-featured and heavyweight for the job, then I can create my own lightweight marker (MarkerLight). If I want to visualize my points as <div>-based bar graphs instead of <img>-based markers, then I can create my own overlay (Bar).

To extend the interfaces requires defining only a few minimal methods:

GOverlay: initialize, redraw, remove

The initialize method, called when addOverlay is invoked, should create DOM objects for the overlay and append them to the desired map pane (there are several). The redraw method is called whenever the map moves, and it should re-position the overlay depending on the new latitude/longitude -> pixel mapping. The remove method, called when removeOverlay is invoked, should clean up any DOM objects created by the overlay. More info and an example is in the docs.

GControl: initialize, getDefaultPosition

The initialize method, called when addControl is invoked, should create and return the div for the control. The getDefaultPosition method should returns the recommended position for the control. There is no remove method, since the initialize method returned a <div> that the map can remove whenever removeControl is invoked. More info and an example is in the docs.

Using just these methods and some JavaScript/CSS skills, a developer can create anything their hacker heart desires. Custom controls and overlays account for 15/20 of the libraries in our official utility library, another 15 of the libraries listed in Mike William's list of 3rd party extensions, and based on my experience, many more are likely floating around in developer's code on the web.

Don't forget the actual API engineers...

Along the same lines, these interfaces also make API development easier for internal developers. If they want to add a visual class to the API, they just need to figure out whether it's a control or an overlay, define the methods, document that the new class exists, and external developers will use their existing familiarity with addControl or addOverlay to add it to their map.

It can't all be fluffy rainbows and unicorns...

We've certainly made a few mistakes along the way with the interfaces - but this post is getting long, so I'll save those for later.

To sum it all up...

The interfaces are easy for newbie developers to understand, easy for advanced developers to extend, and easy for internal engineers to implement. Beautiful.

Monday, March 30, 2009

Web API Design: The Good, The Bad, and the Ugly

Two recent events have prompted me to spend a lot more time thinking about API design: 1) A series of "Good, Bad, and Ugly" talks at the Google Sydney office which examine internal architectures and coding decisions, 2) An internal reading group starting with "Beautiful Architecture", a book of essays. Though the design of internal software (largely the focus of the talks and book) is important and should be done well, it's my opinion that the design of external-facing Web APIs is even more important, and yet, a much less discussed topic.

Why? Web APIs have some unique challenges:

Flexibility: Web APIs are intended to by whatever random developer finds them useful, not by a restricted set of internal developers. And, I can tell you from experience, random developers do things that you never even realize was possible with your API - and then they beg for feature requests to make it easier for them - and you'll want to fulfill their requests because you'll think "hot damn, that was cool!" So, the easier it is to implement new and different features, the better.

Backwards-Compatibility: Web APIs have to stay around for a long, long time. Even if you threaten that you'll change your API every 2 years and you warn users 1 year ahead of time, you'll still have some developers use the old version of the API and write angry posts in your forum about their site breaking. Or, developers that have written code for client sites, moved on to bigger and better things, and left it with webmasters that have no idea how to upgrade it. Or, developers using wrapper libraries that abstract on top of the older version of the API, and have no notion of the underlying API. So, it's safe to assume that you can never really deprecate an API without breaking old sites, and you should design the API with that in mind (future-proof it).

.. and other challenges, like needing to be equally usable by both newbie and advanced developers, and needing to be a reasonable number of kilobytes (for client-side APIs).

In the land of Web APIs, the Google Maps API makes for an interesting case study. It is one of the most used Web APIs in the worlds (150,000 sites), the first interactive web mapping API, likely the first hosted JavaScript API, and it has grown from a simple API with just markers and zooming to a complex API with 90+ classes. It's also gone from version 1 to version 2, but the changes were primarily in terms of latency; the actual interface has remained largely the same. All these facts make it interesting to look at the Maps API from an architecture and design point of view, and point out the places where it went right - and where it went wrong. Of course, as an advocate of the Maps API and someone who uses it daily, I should point out that I still think we have the best Maps API in the world. But even gods can have their flaws (I hear Zeus had a weakness for mortal women).

So, in an upcoming series of blog posts, I'll single out design decisions in our API that I think were good, bad, or ugly. To start it off on a good note, I'll start with something beautiful... stay tuned!

APUGS Round-Up: Bzoo, AIR, & Eclipse

Tonight, despite half of Sydney CBD being in a blackout (and zombies attacking the city), I attended the monthly APUGS (Adobe Platform Users Group Sydney) meeting. 3 developers gave talks, and I learnt a great deal of random information. I realize that all of my Flex experience currently comes from supporting the Maps API for Flash - so, if a developer hasn't asked it, I've never learnt it - and apparently, developers haven't asked me all the questions in the world yet. Here's a roundup of the topics covered:

  1. Developer Bachir El Khoury spoke about his AS3 library for creating a client-side database with full CRUD operations, called Bzoo. It can create the database from JSON, YAML, or ArrayCollections, and optionally, it can store the database persistently using Flash Shared Objects. Shared Objects are basically super cookies for Flash that are persistent across all browser sessions on a desktop, and can take up to 100 KB per domain. Cookies can be useful in Flash apps for remembering game configurations, backing up data before a save, or user sign-in state.
  2. Developer Andrew Muller showed off Bowernest, an AIR app he's creating for a client (BookTagger). It's a desktop app with a slick interface for managing books and other media. Andrew used the Mate (Mah-tay) RIA framework for creating the app, and the award-winning skin from ScaleNine's contest. Cleverly, the app can import files from Shelfari and LibraryThing, the two big competitors to BookTagger, so its easier for users to switch over. It then stores all the data locally in an AIR SQLite DB, and offers the user the option to export back out to the Booktagger file format. As a heavy user of web-apps, I find it a bit shocking that the data isn't constantly backed up to a BookTagger user-account, but hey, apparently some users are into the whole desktop-only app thing. When I asked Andrew about other apps out there that were desktop-only, he showed us Balsamiq Mockups, an awesome app for mocking up web prototypes in a sketchy way.
  3. APUGS organizer Chris Velevitch showed some tips for working in Eclipse. I only work in FlexBuilder, but it was useful to see the similarities and differences to Eclipse development. Eclipse has a cool feature called local history, which lets you see every version of the files in your project - for up to "9,999 days"! Also, in both Eclipse and FlexBuilder, you can right-click on a file tab and select "New Editor" to get the same file opened in two tabs. In FlexBuilder, you can then put one tab in Design mode and the other in Source mode, and drag one so it's below the other. Highly useful.

Friday, March 27, 2009

BarCamp Canberra 2: Mapping the Fires

I just spoke a few minutes ago at BarCamp Canberra 2 about our experience making a map of the Victoria BushFires. Here's a quick run-through of the presentation, with links:

  • We woke up, saw there were crazy fires going on, and looked at how we could help. The websites with the information were DSE (Public Lands) and CFA (Private Lands). The problem with the DSE site was the lack of an interactive map and the lack of an RSS feed. The lack of RSS feed meant that people were constantly reloading the page to see what had changed, and their server was crumbling under the load. The problem with the CFA site was the lack of a map -- and given all the number of small towns in Australia, it's very hard to visualize where the fires are happening in respect to where you or your loved. The problems with the RSS feed was that it lacked latitude/longitude coordinates, and the CFA website was struggling and producing corrupt feeds some of the time.
  • First we tried using Yahoo! Pipes to turn the RSS into a GeoRSS feed, but it didn't quite work correctly, and we were worried about relying on the service during an emergency.
  • So then I set up a database on my server, ran a script geocoding any new locations every 2 hours, output the database as an ActionScript object, copied and pasted it into my Flex app, and re-uploaded the SWF. It was ghetto and painful, but it worked.. provided I woke up every 2 hours. Sometimes, we had to manually find locations - like "DSE - 6KM ESE OF WARBURTON" - by using the distance measurer mapplet in Google Maps and eyeballing it. It sure would have been easier if CFA had done the geocoding for us!
  • So then we created the map. We sent the RSS feed through Google App Engine, caching it every 5 minutes to avoid hitting the CFA web server (and to ignore corrupt feeds), parsed that feed in the Flash map, and displayed the fires as markers. We later added a legend, search box, and colored/sized markers, but we basically had our map up 4 hours after we started coding. Not bad!
  • One important thing we did was make a gadget version of the map. This meant news agencies, blogs (like Mashable, and random concerned folks could embed the map in their sites. A lot of people discovered the map through the gadget, and the news agencies benefited since they didn't have to create their own visual to complement their articles.
  • The map got around 60 QPS at its peak - and App Engine handled it with no problems. It probably helped that we used our inside influence to raise all of our quotas to the absolute max (14 petabytes of bandwidth!). :)
  • After we had the map up, we wanted to make the geocoding less ghetto. We changed from doing the geocoding on my server to doing the geocoding on App Engine, and outputting the feed as a GeoRSS feed instead. The great thing about GeoRSS is that it's a pseudo-standard, and there are multiple websites that can parse and display them (including Google Maps). The cool thing about outputting the data in this portable format was that one concerned guy was able to import the GeoRSS into a Google My Map, plot markers where his family members were located, and figure out if they were in trouble.
  • At this point, we were doing good things with the CFA data, but we still had no data for the public lands. DSE was unwilling to give us structured XML data, as they had various concerns about copyrights, attribution, and data distortion. We still wanted to represent public lands data though, and that's when we discovered that NASA takes satellite imagery of South Australia every 12 hours and makes them available in various standard formats.
  • Using MapTiler, an open-source GUI for the command line GDAL utilities, we converted the NASA GeoTIFF into map tiles. Those map tiles can be used in most APIs - Google, Microsoft, OpenLayers, etc. We, of course, overlaid them on our Google Flash Map, and let people toggle the imagery on. NASA does an automatic analysis of the imagery to look for fires, so you could actually see red outlines around the center of big lines of smoke. Since we're still serving the imagery from 25 days ago, you can still see that on the website.
  • The aftermath of all of this is that we learned lessons about the issues with getting emergency data in a format that can be re-used, cached, syndicated, and visualized in useful ways. We wrote a white paper of sorts with recommendations for Emergency Data portability and we're consulting with various agencies to help them get their data up to snuff.

Tuesday, March 24, 2009

Ada Lovelace Tribute: A Q&A with My Mum

A few months ago, I pledged to write a blog post for Ada Lovelace day. The hope is that atleast 1,000 bloggers will write about a woman in technology on today, March 24th, and that this will draw (good) attention to woman in technology. I don't necessarily know if we'll succeed in the objective, but hey, it's different and could work, so why not?

Since I don't actually know very many women in technology (I'm too distracted by all the tech boys?), I've done a Q&A with my mum, Rosemary Kennett. Mum studied Physics at Caltech, met and married my dad there (a professor! scandal!), had 3 kids, worked for NASA JPL, did physics research at SU and Cornell, taught Astronomy for a few years, and then moved on to her current job as a scientific programmer in Spectral Sciences. Here goes:

Why did you choose to study Physics?
I actually became fascinated in high school where I was taught by a teacher who had just returned from a sabbatical at Oxford and was intrigued by the new physics e.g. nuclear physics. I remember that I used to borrow her collection of popular scientific paperbooks.
What was the first line of code that you wrote?
I remember writing code to model an oscillating violin string when I took an early math course involving programming as an undergraduate (we did not have CS courses back then). Most of the early coding was in Algol 60 (which most people agree was a pretty good language but somehow it died out)
What language have you programmed the most in?
Fortran, then C/C++, then Java
(Hmm. Is Fortran kind of like 4chan? - PF)
What app are you proudest of creating?
For the general public - there's a simple Java applet that often gets referred to: Epicycle and Deferent Demo.
You taught astronomy and ran the Syracuse Astronomical Club for a few years. What's the most fascinating aspect of astronomy for you?
How much we are learning about the distant past (likewise through high energy physics experiments, which now are trying to mimic conditions soon after the start of the universe!). Also the number, variation and beauty of objects in space, especially as observed using the Great Observatories (Hubble, Chandra for X-Rays, Spitzer for IR etc)
I always thought it was pretty cool to have a mum that worked at JPL. What were you doing there (besides rocket science, of course)?
Actually the technical work of which I'm most proud was that done at JPL. One project was to check the performance of a new design of a radar instrument called a scatterometer which measures oceanic winds (strength and direction) by bouncing radar off the waves - the Bragg effect means that the radar return depends on the roughness of the waves, which is in turn influenced by the wind speed. You do this from different directions and you can tell the direction of the wind also. Anyway that style of scatterometer is now the standard, although launches of these instruments are few and far between
(That's right, my mum's invention is orbiting space above your heads right now. Boo-yah! - PF)
You have 3 kids, and two of them ended up in Computer Science. Do you credit/blame nature or nurture?
(Hmm. I'm pretty sure it was the fact that we had a T1 line before any other kids had the internet, and that we moved to cold, miserable upstate NY and I was forced to hibernate inside our well-supplied computer room. - PF

Bonus Factoid: My mum knows all the cool kids in CS: Steven Wolfram, because he was a student working with my dad on the Mathematica forerunner, Richard Feynman, because he was a Physics professor at Caltech, and Larry Page, because he attended a Hertz event and asked her a question about her research... I'm jealous!

Tuesday, March 17, 2009

Adelaide: More than OK, Actually

My mum's visiting Australia for the first time in her life, spending the majority of her three weeks wandering around Sydney (it's a big place!). She traveled with me last weekend to only one city outside of Sydney: Adelaide, in South Australia. When I told Aussies where we were going, the question was always "why?" - Adelaide doesn't exactly have the greatest reputation. The Melbourne mayor went so far as to effectively recommend that Adelaide be shut down, due to its uselessness. So I went to Adelaide prepared for a city of churches and dull-dom, but I actually encountered amazing natural sights and a vast array of entertainment options. Check out my description of the highlights below, or if you're a visual sort, just go through my Adelaide Flickr collection.

Faulty Towers Interactive Dinner

My mum is British, so when I saw the advert for a dinner in the style of this classic British comedy, I knew it'd be perfect for our first night in Adelaide. Our hosts were Basil, Sybil, and Manuel, and the dinner was a series of well orchestrated haphazards and improvised arguments with us, the guests. And as much as they tried to make the dinner service disruptive, the food was still delicious - my first stone grilled steak, but definitely not my last.

Glenelg Town Hall Museum

Yes, yes, a town's museum. But, this museum contained more than the usual "bla bla discovered X and settled Y and then war happened and stuff". It had *2* special exhibits dedicated to retro arcades - machines that you'd either wind or put a coin in, and you'd see some puppet animation - like a monkey's eyes moving, or 2 sharks boxing. Though we've moved way beyond such simple mechanics these days, it's still very satisfying to think "hmm, what's going to happen?", do some winding, and then, "oo! the arms moved!". One of the coin-operated arcades was a skeleton fortune teller - I asked it if I should ever get a PhD, and it responded back with exactly my thoughts (below). Oh, well...

Kangaroo Island

Most people know this as the "only good reason" for visiting Adelaide, and it is indeed a good one. During the guided bus tour of the island, you visit Seal beach, Remarkable Rocks (aptly named, trust me), Birds of Prey park, and Admiral's Arch. The most memorable moment was at the seal beach, when the guide was telling us we had to keep atleast 10 meters away from any seals at all times, and then suddenly realized that a seal was walking in front of us. We patiently followed 10 meters behind the seal as he led us to the beach, and at that point, I think we were all sufficiently in love with seals. Seeing 2 baby seals scampering on the beach "sealed" the deal. :)

Coorong River

In this tour, you spend much of the day on a boat with about 30 other people, and stop to take walks among the sand dunes. It's a lesser known tour, and that's a good thing - it's still got an intimacy/charm that the commercialized Kangaroo Island tour is lacking. Everything was a highlight: hunting for "cockles" (clams) in the ocean and swallowing them raw, eating deliciously fried cockles, rolling down a sand dune, accidentally stumbling upon an aboriginal skeleton, digging for (and drinking) freshwater from the ground, and spotting a koala on the ride back.

Botanical Gardens

Adelaide has an impressive set of gardens - cacti, roses, giant freaking Amazon water lilys, and more. It was a great place to wander around with a camera and a nose, and I was thrilled that a butterfly finally rested on a flower long enough for me to take a close-up.