worthwhile user testing?

I recently read the results of some user testing we’d done on our website (part of a regular program of usability testing) and was shocked to realise that there were only 4 participants. That’s 4 out of the hundreds of thousands of users we have, and on the basis of their comments we are going to refine our website. Now whilst I agree with the principle of user testing I just don’t think that 4 is anywhere near enough to get a represeantative view of how the website works. It’s nowhere near a mathematically representative sample of our users and makes me wonder if it’s worth doing at all. I do realise that the more people you involve the more expensive it is and the longer it takes etc, but surely if you’re going to do it at all you want to make sure it’s worthwhile doing and will give you meaningful results?


  1. You’re right. That doesn’t sound like a lot of input to base a decision on. Is there a way that you could have a special event in your library or special day where you set up a computer and have people just spend a few minutes reviewing the site and then making their comments? We did this when we were starting a new online system. It made people feel like they were involved, plus it gave us a chance to see what people were looking for and how they were searching and using the site. Just an idea.

  2. That would be a great idea, except that we’re a distance learning organisation with users all over the world and so the users who would be able to easily get to the Library would still not necessarily be representative. We do have a feedback form on the website where anyone can comment, and we do listen to comments that come in through that method too.
    The testing that was done was fairly structured and well thought-out, and did try to include a range of different user types.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s