Website Monitoring Services

Update: Here's an article that reviews a bunch of services.

After posting the iPhone app for EatDifferent in the App store last week, I've seen a noticeable increase in new user signups — despite the fact that there's no signup through the app itself, only through the website. That means more traffic to the App Engine hosted webapp, and unfortunately last night, it meant more traffic than my webapp was configured to handle. My webapp went over quota at around 10:30pm and served 503s until midnight, at which point the App Engine quotas reset and it started fresh. I was sleeping at the time, blissfully unaware that it was over quota, and I didn't find out until the next morning, thanks to a personal email from a friend and early user.

Why didn't I find out? Well, I have a lot of error logging in my app, but 503s don't actually trigger the error logging, since that logging is in the app code itself, so I didn't see any errors come through my inbox. Most webapps are like that — when their host goes down, their logging goes down with it — so that's why many website monitoring services exist. These services live outside of your webapp (hopefully on a different hosting system altogether), ping it every so often to make sure its returning HTTP 200s as expected, and alert you when it's not. I knew about such services, but of course, I didn't think to set myself up with one until after I needed it. You live, you learn.

After beating myself up for a few hours in the morning, I started investigating website monitoring services — and by investigating, I mean that I asked my Twitter followers for their recommendations. Here's what they suggested:

• Monitor.us
• Pingdom
• UptimeRobot
• BlameStella
• NewRelic
• CloudKick

All of those services offer standard downtime monitoring, but some of them also offer performance monitoring and offer their service as an API or SaaS. For my purposes, I wanted something easy and free, and since it doesn't hurt to have multiple monitoring services, I went with UptimeRobot and a Pingdom free plan. They both offer alerts over email, SMS, and Twitter, plus Pingdom offers an Android app with push notifications. It was easy to set them up, and (its funny saying this about a service) hopefully, I will never have to hear from them... But just in case, I'm glad they're around!

Testing Facebook Login with Selenium

On EatDifferent, I give users two options for signing up: Facebook, for people that like the convenience of using FB to sign in everywhere, and email+password, for people that hate Facebook. I see about 50/50 usage for these options, so they are both equally important — which means testing both of them are equally important.

Until today, however, I only had integration tests for email signup and login, because, well, I was afraid. All I have to do for email login testing is enter a few text fields, but for Facebook testing, I would need to navigate to the Facebook popup window and manipulate *their* DOM, and I would need a test user account that was really and a truly a test user (and not my cat, who actually is surprisingly active on Facebook). As I was making some changes that affected Facebook users today, I decided it was time to face my fears. Good thing I did, because testing Facebook login turned out to be pretty easy. Read on to see how I did it...

Create a Test User

In the land of old, creating a test Facebook user meant digging up an old email address and signing up with your cat's details. That kind of sucked, since you only have so many old email addresses, and it wasn't exactly Facebook TOU-friendly. Thankfully, Facebook introduced a way earlier this year to create multiple test users, both programmatically and from the developer dashboard. For the purpose of my tests, I only need one test user, so I created him semi-manually with these steps:

• Go to http://developers.facebook.com/apps, and select your app. Click 'Edit app' and then 'Roles' on the left-hand side.
• Under 'Test Users', click 'Add'. Select '1' and don't select 'Authorize this app' (if your purpose is to test authorization).
• Once created, click 'Modify' in the 'Test Users' section. You'll see a table with information about the test user, like their name (which you can change to something snappier, like "Sharky Shark" :) and ID.
• Now you need to find out the user's password, so you can enter that with Selenium. First, get an access token for your app by putting this URL in the browser and replacing the client_id and client_secret with your app key and app secret:

  https://graph.facebook.com/oauth/access_token?client_id=APP_KEY&client_secret=APP_SECRET&grant_type=client_credentials

• Now paste this URL in the browser, replacing the ID with your test user ID, access_token with the string you just got, and password with your desired test password:

  https://graph.facebook.com/TEST_USER_ID?password=TEST_PASSWORD&method=post&access_token=ACCESS_TOKEN

• Click 'Login' in the users table so that you're logged into Facebook as them. Visit their profile, take note of their email address, and change any desired fields (depending on what your app uses - I gave mine a profile pic and location).

Presto, now you have a test user.

Test the Facebook Popup

For my integration tests, I use Selenium with its Python API and Python's unittest module.

For my FacebookTests test case, I extend my BaseTests class and add two test methods, one for testing authorization and the other for testing login.

Those test methods use the FacebookDom helper class (an extension of my DomHelper class) for finding and manipulating the Facebook window and DOM. Note that after the user has granted the authorization, Facebook will no longer ask for authorization unless the user specifically revokes it - so my do_authorize_flow function checks to see if the auth button is actually there before trying to click it.

Ta-da, now you have a tested Facebook authorization and login flow! Some may argue that you shouldn't try to test 3rd party APIs because they may go down or be flaky or such and that you should mock them out instead, but I'm of the philosophy that integration tests should try to mimic the production environment as much as possible. And hey, I want to know if Facebook goes down, changes their response, or becomes flaky.