Usability Testing 101: Two Ways to Test

Editor’s Note: This article is excerpted from How to Get Useful Website Feedback, written by Christopher Butler and originally posted on The article recaps Chris’s presentation at the 2012 HOW Interactive Design Conference. To learn more about how to user-test your website designs, catch Chris’s web design tutorial, How Simple Usability Testing Benefits Your Web Design.

Simple Usability Testing

Now, a bit of a caveat is in order: I’m using the word ‘simple’ intentionally. Most people think of usability testing as something that requires facilities, equipment, people, expertise, and time that they just don’t have. So, in the absence of all of that, they conclude that they can’t do any usability testing.

But these days, while you can hire usability testing experts that may fit right into that expensive, high-tech picture, you can also do a lot of it yourself. That, of course, means simplification. But simplifying the process doesn’t reduce the amount of actionable insight you can derive from it. So here’s a simple usability testing procedure that I know you can (and will) put to use.

Website usability testing examples

What You Need for Easy Website Testing

  1. A Quiet Space: Not a soundproof chamber in a lab somewhere, just some place quiet where you and the volunteers you’re going to recruit can focus for a few minutes without distraction or interruption.
  2. A Computer: Nothing fancy. The one you have will probably work, as long as it has…
  3. A Webcam: Most laptops have these built-in these days.
  4. Screen-Capture Software: You’ll also need to run some kind of screen-recording software. This will allow you to gather the information collected by your webcam, the computer’s microphone, and the screen into one consolidated interface.
  5. A Volunteer: Most people assume that a volunteer for usability testing needs to correspond perfectly with a website’s target audience. Really, the only qualification needed for a usability testing volunteer is basic digital literacy. Can they use a computer, surf the web, and talk about it? Since what we’re doing here is testing whether there are any barriers to proper use of the website, this is really all we need from our volunteers.
  6. A Moderator: That’s you! You’re there to provide the environment, the technology, and the process, but most importantly, you’re there to observe. Besides the information that the screen-capture software is designed to collect, you’ll be able to observe lots of other stuff and ask questions of your volunteer to draw out more information when you need it.
  7. A Test Plan: What we’re not doing here is taking a volunteer, sitting them down in front of a website, and asking them to click around a bit while we watch. Instead, we’re crafting specific test plans that address the core purpose and goals of a website with directly worded procedures for the volunteer to follow. There are two specific plans I’d like to review with you that I’m sure you can begin to put to use on your own sites right away.

Website Test Plan 1: Goal-Focused Testing

This test plan is crafted around the core goals of a website. If the purpose of your website is to generate leads by converting passive visitors into subscribers or customers, then the means by which you do that are what we’ll test here. For example, if one of your website’s goals is to get visitors to subscribe to your newsletter, one task might be to find the latest newsletter, let you know what it’s about, and then sign up to receive it by email. Iif the website presents any difficulties for the user, they are made clear by the test experience.

Next, you want to take the volunteer through 2 or 3 goal-focused tasks. Again, these should correspond to specific conversion points — content subscriptions, event registrations, purchases. Try to keep these brief — and the whole test experience to under 10 minutes — as the longer it goes, the more fatigued your volunteer will get.

Biases in Usability Testing

Any kind of usability testing will have some inherent biases — there’s no way of avoiding that. What’s more important than avoiding biases is understanding them so that we can properly assess the results of the testing given the effects of bias. One fundamental bias is called the Hawethorne Effect, which is simply the effect that awareness of being observed has on a volunteer’s actions.

Usability testing is meant to approximate the experience of using a website, but because we’ve invited someone in to help us do that, they already know that what they’re doing is synthetic. Everything you ask them to do will probably be done better and with more patience and effort than we could ever expect of a typical user.

Biases don’t make usability testing worthless, of course. They just make it a bit handicapped and before we congratulate ourselves when a volunteer sails through each task, we need to remember that these results are skewed. We need to pay close attention to the details of the volunteer’s experience being mindful of these biases so we can assess whether any specifics would make things more or less difficult for an un-observed user out there on the internet somewhere.

As moderators, we need to make sure our questioning doesn’t exacerbate the biases, too. We don’t want to ask questions like, “Didn’t you see that “submit” button down there on the right?” Instead, we want to ask general questions that draw out more from our volunteers, like, “It looks like you’re having some difficulty. Could you tell me what’s going on right now?”

An Example of Goal-Focused Usability Testing

This video clip of a usability testing session shows one task from a series of tests I ran on my firm’s website last year. I’ve asked my volunteer to find our upcoming webinar and register for it.

Test Plan 2: 10-Second Testing

I developed the “10-Second Testing” idea after reading that the average web user decides whether they will stay on a page or go within 10 seconds of opening it. What a tiny window of opportunity for us to give the right first impression. But then I realized that we humans can take in a ton of information, make judgements on it, and act on it in far less time than 10 seconds. (If we couldn’t, we wouldn’t be able to drive a car.) I realized that this would make a perfect format for usability testing.

The procedure is simple: You pick a page you want to test — ideally a content page or landing page — and give your volunteer 10 seconds to view it. Again, we’re going to ask them not to click any links during that time. After 10 seconds, we hide the page. Then, we want to ask a very simple question: What was this page about?

Ask some follow-up questions, like, “What do you remember about the page?” or “Did anything stand out to you, like colors, or text, or images?” or “Based upon what you remember about this page, would you continue reading it?” or “Would you consider this a trustworthy source of information?”

An Example of 10-Second Usability Testing

This video clip of a 10-second usability testing session shows two different sessions with the same volunteer. In the first, he doesn’t remember much about the page I showed him; in the second, he’s able to recall lots of detail, including the page’s title. What does this information tell us about the design of the two test pages?

Recap: Getting Feedback on Your Web Designs

Getting feedback isn’t a checklist item. It’s not even a job, really, or a “hat” you wear some of the time. To make an impact, feedback needs to be a cultural shift for everyone on a team. It’s a posture, a way of working, not a task.

Read Chris’s full article, How to Get Useful Website Feedback, for more information on usability testing and on how Google Analytics also provides valuable insights into the success of a website design.

More resources on website testing and feedback