Monday, April 2, 2012

Converting Addresses to Timezones in Python

In a perfect world, we'd all live in the same timezone and it would be the same time everywhere. Unfortunately, we have a sun, and the earth goes around the sun, and the earth is round, and someone invented the concept of time, and now we programmers have to deal with it.

For EatDifferent, I give users the option of getting email reminders — one in the morning, and one at night — so that means I need to know the user's timezone. I could just ask them for their timezone by prompting them with a giant drop-down, but I don't want to make my sign-up form longer, and I have yet to find a user-friendly timezone selection widget. So, instead, I ask for their location, and try to programmatically figure out their timezone from the location string.

When I originally implemented location to timezone conversion, I used an API from SimpleGeo that I could send an address to and get a timezone back. Unfortunately, SimpleGeo was acquired by Urban Airship, and their APIs were shut down this week — so I needed to find a new solution, stat. After searching around for a while, I found a few APIs that convert latitude, longitude coordinates into timezones, but no APIs that convert addresses into timezones — which meant I needed to first use a geocoding API to convert the address into coordinates, and then feed that into one of those timezone APIs. Since I only do this conversion once per user and I needed to use two APIs for one conversion, I wanted to use APIs that were either free or low-cost, transaction-priced.

Addresses -> Coordinates

When I asked on Twitter for geocoding suggestions, I got a few good tips: CloudMade, Yahoo! PlaceFinder, Geocoda, DeCarta, and of course, Google. CloudMade and Yahoo both looked like promising candidates, but since I'm already so familiar with the Google API from my years of actually working on the Google Maps API team, I decided to go with what I know. The Google terms of service requires that all apps using the geocoder eventually display the geocoded coordinates on a map, so I'll add mini maps to user profiles to appease the terms.

To use the Google geocoder from my app, I used this Python wrapper for the API. Here's what my code looks like for geocoding the address string and saving the results to the User entity:

try:
    geocoder_client = geocoder.Geocoder()
    geocoder_result = geocoder_client.geocode(user.location.encode('utf-8'))
    user.country    = geocoder_result.country__short_name
    user.city       = geocoder_result.locality
    user.latlng     = db.GeoPt(*geocoder_result.coordinates)
except geocoder.GeocoderError, err:
    logging.error('Error geocoding location for user %s: %s' % (user.get_id(), err))      

Coordinates -> Timezones

I had a few options for calculating the timezone now: Geonames, AskGeo, EarthTools, World Time Engine. I went with Geonames because it returns an Olson timezone string (which is what Python pytz uses) instead of an offset or other timezone identifier, and because its free for my needs.

I looked around for a Geonames python wrapper, but when I found only old ones, I wrote a really simple wrapper based on the Geocoder API client, so that I could call it in a similar way.

try:
    geonames_client = geonames.GeonamesClient('myusername')
    geonames_result = geonames_client.find_timezone({'lat': user.latlng.lat, 'lng': user.latlng.lon})
    user.timezone = geonames_result['timezoneId']
except geonames.GeonamesError, err:
    logging.error('Error getting timezone for user %s: %s' % (user.get_id(), err))

My solution requires chaining two APIs together, which means double the requests and double the chance of failure, but I calculate the timezone in a deferred task, so that the user isn't waiting for it to happen, and I can set it up so that the task is retried in case of error. So far, the APIs have both been responsive and the solution is working as well as the original SimpleGeo single API call.

After I calculate the timezone, I let the user edit it by creating a dropdown with all the possible timezones (and there are a lot, atleast according to pytz) and selecting the calculated timezone (or if none was found, a default of America/Los_Angeles). Here's the code that creates the timezone dropdown using WTForms:

import pytz
import datetime
timezones = []
for tz in pytz.common_timezones:
    now = datetime.datetime.now(pytz.timezone(tz))
    timezones.append([tz, '%s - GMT%s' % (tz, now.strftime("%z"))])
timezone = wtf.SelectField('Timezone', choices=tuple(timezones))

In the future, I would like to actually present the user with a slick way of choosing their timezone, once I figure out what that looks like. So, how do you deal with timezones in your apps?

No comments: