Wednesday, October 13, 2010

lscache: A localStorage-based, memcache-inspired library

Over the past few years, I've developed a fair few Python App Engine apps, and I've come to have a huge admiration for memcache. The memcache API is incredibly simple, but at the same time, it's a very powerful way of making my apps scale better with increased user demand. The first time I wrote an App Engine app, it went over quota in the first 6 hours. After adding memcache support in, it never went over 1% of quota ever.

When I'm writing client-side apps, I find myself yearning for something like memcache to reduce my number of asynchronous server requests. HTML5 does offer the localStorage API for setting and getting key/value strings, but that API has no notion of an expiration date. It's great to be able to store copies of my data locally, but most of the time, I also want to be able to expire that data after some amount of time (5mins, 1hour, 1 day). So, I wrote a simple library called "lscache" that wraps on top of localStorage, but adds on the notion of expiration. Here's what it looks like to set and get some data:

lscache.set('somedata', {'name': 'Pamela'}, 60);
if (lscache.get('somedata')) {
  console.log(lscache.get('somedata').name);
}

The library lets you store pretty much anything, like a string, number, or object. The localStorage API only stores strings itself (though according to the spec, that should change soon), but the library uses JSON.parse and JSON.stringify to try to store non-string objects. There are some objects that can't be stringified, like the Document object in an XMLHttpRequest, in which case you need to convert them to a simple JSON object yourself first.

The library stores the expiration time in a separate key, and when you try to retrieve a key, it will only return it if the current time is before the expiration time (and will remove it otherwise). It calculates the time using the JavaScript Date object, so it is subject to error if the user futzes with their clock - but if they do, it just means the objects will be cached for a bit less or a bit longer than expected, and the world probably won't fall over. And those silly people should stop futzing with their clock. :)

If the user's browser doesn't support localStorage, the library just won't store anything and will return null when trying to retrieve objects. This works wonderfully with the memcache style of coding, where you never assume that anything is stored, and always fall back to re-retrieving that data if it's not stored. People with "older" browsers will simply get a not-as-speedy performance.

I've pushed the lscache code to a github repo (my first!), and also published a demo of the functionality. I originally wrote lscache to speed up the performance of the XMLHttpRequests in a Chrome extension popup, but as that extension isn't published yet, I've also incorporated it into my public RageTube mashup to cache the JSON results of the Dapper and Youtube APIs.

Check out the library, and if you have any suggestions for improvements, feel free to fork. :)

No comments: