The ultimate guide to testing your website

At Clearleft we prototype in HTML, CSS and some jQuery, creating 'mid-fi' prototypes – good enough to be usable but rough enough to be quick and easy to create.

Quantitative or qualitative?

Quantitative tests measure numerical things such as task completion percentages. For these tests you need quite a few users, but the resultant statistics can help put a financial value on usability improvements.

Qualitative testing is more concerned with watching people use the system and learning how well they understand it. As such, it's suited to the guerrilla approach and formative testing, and it's what we're focusing on here.

Of course, these two extremes aren't mutually exclusive, and a well-designed test can have both quantitative and qualitative elements.

Once you've chosen what to test, you should write some scenarios for the test. Examples for, say, a car classified site might be:

"You're looking to buy a used estate car for the family and have £2,000 to spend."

"You want to read some user reviews of the Peugeot 307." "You want to know how much your J-reg Mercedes 190E is worth."

Make sure the scenarios reflect the user's overarching goals, not how they should do it. Although you know the 'right' way to answer these scenarios on the site, you want to see whether it's obvious to users. Your scenarios should also involve the most important functionality you have: there's no point testing the subtleties of a photo cropping interface if the user can't log in.

The complexity of your scenarios will dictate how many you can fit in a single test, but five or so is common. To check you're covering the right number, estimate how long your scenarios will take to walk through. As a rule of thumb, double the time it takes you to walk through them all, then double it again to cover admin and briefing time. If this comes to between 30 minutes and an hour, you've got it about right.

How many users should you test your site on? Even one user is better than none, but it's worth getting a few people in to eliminate freak results and catch all of the common problems. For a single round of testing, five users will find the majority of errors.

For a quantitative test, you may need 20. Pace yourself – usability testing is surprisingly hard work and you'll struggle to cope with more than four or five tests in a day. For guerrilla tests, try three in an afternoon or even just one over a lunch break.

It's important that you find test subjects similar to your intended user base. If you've created personas for your site, these should be your guide. If not, give some thought to your target audience, but don't just focus on demographics; it's actually more important to find people who have similar needs. For example, a taxi driver and a high-flying stockbroker might both need to check their bank balances on the move.

Finding the right people

You can find participants through friends, family, Twitter, Facebook and so on. Get the word out and include some basic screener questions to filter out those who don't meet your criteria.

Recruitment can take a while: you'll need to find people, assess their suitability, schedule mutually acceptable times and agree payment, so allow a couple of hours per user. Alternatively, you can use a specialist recruiter – prices vary but you can expect to pay £30-£50 per participant.

If you do lots of testing, it might be useful to set up your own pool of users you can pick from. However, try to cast your net widely. It's not ideal to have one person testing the same site twice; they may remember things from the first test and skew your results.