Making Sense of the Data: Google Analytics + User Testing

I opened the envelope and took out the single sheet of paper it contained. Leaning against the doorway to my kitchen, I stared at it for several long minutes. It might as well have been in another language entirely. Eventually, I gave up and called the number at the top of the sheet.

“I just wanted to know if everything was normal—the test results, I mean.”

“Yes, everything is fine. Your blood work showed nothing out of the ordinary.”

Just days before, I’d had my first physical in years, which, in addition to the usual poking and prodding, included a routine blood test. Receiving the results in the mail made me feel good, like my doctor was looking out for me. But then I tried to read the report, assuming it would tell me a simple story: whether I was healthy or not. Instead, it contained rows of numbers I wasn’t able to interpret on my own. If the data is incomprehensible to a layman, why send it to me?

how to use google analytics to determine website effectiveness

We’re handed documents like this all the time, especially on the job. Instead of telling you the story you need to hear, disorienting data delivers a confusing mystery to sort out. For designers, especially, leaving data unexplained and unexplored is a big no-no. The last thing you want to say when asked to explain your design decisions is, “I don’t know, I just thought it would work.”

More analytics advice

Christopher Butler will be speaking about using feedback for design projects at the HOW Interactive Design Conferences in Washington, DC, and San Francisco this fall.

After all, an important part of design is telling a compelling story of why something doesn’t work, how it could be done better and why your solution could help. Without grounding that story empirically, all we have is a hunch. That’s where measurement comes in: It connects action to defensible outcomes.

We designers need to be proactively involved in measurement. Knowing how to gather and interpret interactive data will better position us in the job market, not to mention prevent us from being held accountable to someone else’s vague interpretation of huge reports. Data will ground our vision in reality and build credibility with clients. We want clients to trust our judgment, but our measurement process should also lead them to the same conclusions if they were to do it themselves.

Unfortunately, measurement is a discipline that tends not to be included under the umbrella of design. It’s time to change that.

Google Analytics for Designers

See Google Analytics in action in Christopher Butler’s live design tutorial on Sept. 13. Sign up today!

Focus on Questions, Not Numbers

If you’ve ever examined a report with rows and rows of quantitative data, you know that numbers easily lead to conclusions. If only. Numbers can be helpful answers to questions, but if you don’t ask the right questions, you won’t get the right answers, and you won’t be led to the right actions. If no questions are being asked, no action will take place at all. This is why reporting tends to be a repetitive task with no results.

In a typical website report, each number represents some metric: the number of visitors, the number of pageviews, the length of a user’s visit and so on. All of these things are of interest, of course, but if they’re not evaluated in a qualitative way, they’re meaningless. After all, these metrics will vary greatly depending upon the purpose of the website and its unique audience. So just pulling the numbers isn’t enough to evaluate success or failure of a web project. We need to be asking different kinds of questions, though, rather than just “How much?” and “How many?”

Truly meaningful measurement can be reduced to five key questions. They are:

  1. Who is using my site?
  2. Where are they coming from?
  3. What content are they consuming?
  4. How are they engaging with that content?
  5. What can I do to make their experience better?

Numeric visitor data will help us answer these questions, of course, but it won’t suffice on its own. Measurement isn’t just about the numbers; numbers are a means to an end.

Getting Analytics

If you haven’t already, adding Google Analytics to your website is a snap. You can do it yourself in three simple steps.

Step 1

Get a free Google account at www.accounts.google.com. If you already have a Gmail account, head straight for www.google.com/analytics, click the “Sign Up Now” link and log in with your Gmail credentials.

Step 2

Once you’re in, you’ll need to provide Google with some information. Add your URL, name your account (e.g., “My Blog”), provide some contact information and agree to their terms of use.

Step 3

This is the most complicated part. Google will provide you with code you’ll need to add to your site. If you have access to your site’s master template file, you’ll need to paste this code into the footer of your pages, right before the </body> tag at the end of the template. If you’re using a plug-in, you’ll probably just need the account number, which looks like “UA-1234567-1.” That’s it. Give it a day to connect and start pulling in data then log in to make sure it’s working.

Question one is about the identity of a website’s users. Practically, there’s only so much we can know about them (and out of respect for their privacy, there’s a limit to what we should try to find out). For our purposes, simply observing their behavior is going to be of much greater value than knowing specific details like their names and addresses. We’ll be able to learn quite a lot about their behavior and answer the question generally after we’ve answered questions two through five. So, while it’s the first question we have in mind, it will be the last one we answer.

Questions two and three are best answered by gathering data from an analytics tool like Google Analytics, where we’ll begin. Then, we’ll employ a simple usability testing procedure to answer questions four and five.

Using Google Analytics

We now know that numbers aren’t really the answers we’re looking for. But it bears repeating this principle while using Google Analytics. Because it’s structured around traffic data—specifically the number of visitors within a particular date range, parsed in various ways—it can be easy to fall victim to Analytics Myopia: the tendency to focus in too closely on a particular metric while ignoring the big picture. These metrics, however, are all interrelated, and it’s in the connections that the important information resides. So before spending any time in Google Analytics, it’s worth committing this motto to memory: There are no independently meaningful metrics. Let me also introduce one caveat: Google Analytics is an enormous and powerful tool, far too large to cover in comprehensive detail in an article like this one. Rather than attempt to describe all of its functionality, I’m going to offer a simple, two-step routine from which you can derive plenty of insight. Each step will involve looking at a qualitative metric and then evaluating it on the basis of a quantitative one.

google analytics dashboard, web statistics

Upon logging in to Google Analytics, you’ll see a dashboard view that will give you a big-picture glance at your website’s traffic. (Click the image above to enlarge it.) By default, it shows you the data for the last month, but you can reconfigure the report for any date range you like. There’s much to see here, of course. But, once I’ve glanced at it, assuming there have been no major traffic dips or spikes, I usually get right into my two-step routine. The first step is to check in on my website’s traffic sources.

Step 1: Traffic

google analytics, traffic report

The Traffic Sources report provides a list of all the other websites that link to yours, and helps you easily answer question two: Where are my users coming from? For this particular website, I can see that most visitors are coming in from Google searches—almost four times the number of visitors who came in directly. That’s a good thing. When the number of people discovering a website by searching for the information it contains is larger than any other group of visitors, it means that the website’s reputation is growing on the basis of the value of its content.

It’s nice to also see that visitors are coming in from a variety of sources, like e-mail marketing campaigns, social networks and other websites with large and active audiences. These different sources indicate that awareness of this website is spreading through referrals across a wide range of familiarity—from more “objective” sources like search engines to much more “subjective” and intimate ones like someone’s Facebook wall.

Now that we’re digging in to these sources, we should evaluate: Which sources are bringing in users that engage more deeply?

google analytics, setting up goals

One simple way to answer this is to set up goals. A goal can be any opportunity for a user to engage, whether it’s filling out a contact form, making a purchase, registering for an event or even just downloading a file. You can configure your Google Analytics account to track specific goals, and then filter traffic by visitors who have completed goals. By applying an advanced segment, which is just a sub-routine telling the program to show you only the traffic that corresponds to any set of rules you determine, you can now see a new graph running in parallel with the basic Traffic Sources report. This particular segment has parsed the traffic sources data by those visitors who have completed at least one goal. The resulting orange line traces goals that have been hit from day to day—obviously, less than the overall traffic. In the list below the graph, each source is now also broken out by a goal conversion rating, shown in green.

While traffic is exciting, it’s only meaningful once it’s valued.

Step 2: Content

google analytics, top content reportThe Top Content Report shows us which pages on our websites are getting the most views, how long users stay on them and which pages have the highest bounce rate.

The bounce rate, by the way, is one of the myopia-inducing metrics I mentioned earlier. A very simple definition for bounce rate is the percentage of users who landed on a page and then left the website without looking at anything else. Google Analytics displays the site-wide average bounce rate on the first dashboard screen you see when you log in, but it makes more sense to look at bounce rates for individual pages instead. A page with a high bounce rate might not be meeting the expectations of visitors who found it in a search. But a page with a low bounce rate is successfully engaging visitors, either by leading them to a goal or to more information on other pages. Like goals did for traffic sources, bounce rate helps us to better understand how users engage (or don’t) with our website’s content.

google analytics, top landing pages reportBe aware, though, that the way bounce rate is displayed can be misleading. On the Top Content report, Google Analytics will display the bounce rate for a page along with the number of views it has received. If the report displays a page with 870 views and a bounce rate of 67%, you would think that 583 of those people bounced. But that’s not correct. Remember, bounce rate measures the percentage of traffic that landed on a specific page and went no further. To see how many actual bounces occurred, you need to look at the same page in Google Analytics’ Top Landing Pages report, which displays only the traffic that entered the website by landing on that page. If the report lists 726 entrances through that page that had 870 views and a bounce rate of 67%, it means that only 488 users really bounced. That’s almost 100 fewer visitors than we would have thought had we not looked at both reports.

Digging in to a website’s content and measuring it on the basis of goals and bounce rate reveals plenty of examples of how there are no independently meaningful metrics. Analytics data can mislead us just as easily as it can empower us, which means we’ve got to keep our big-picture questions in mind at all times when interpreting these reports.

Gathering Data from Users

Our last two questions—about how visitors are engaging with content and what we can do to make that better—are best answered by observing users.

But another caveat is in order: The type of user testing that I’m about to describe isn’t the most scientific process you could pursue. The amount of work and the high cost to conduct truly scientific usability studies—using heat mapping technology and the like—is a very real barrier to entry for most of us. But a much simpler version can offer very legitimate results. In fact, my experience has been that the simpler the process, the greater the actionable insights.

The setup you’ll need is minimal: a volunteer, a quiet space that’s free from distraction, a laptop running some kind of screen-capture software, a webcam to capture your volunteer’s face while working, a test plan to follow and a moderator. Your volunteer need not be an expert in your field or have any deep familiarity with the information or concepts contained in the website you’re testing. The fresher the better. You are there to provide context, technology and guidance, and, most important, to observe.

In either testing scenario, feel free to ask follow-up questions that draw out more information from the volunteer, but be careful not to lead them to conclusions. Five or more participants is going to be the most useful, but even one is better than none.

Test Plan 1: Goal-Focused Site Test

This test will help you evaluate the clarity of your website’s purpose to an outside user. Once you’ve started recording the session, the first step is to have the volunteer orient herself on the homepage. Give her one to two minutes to explore—scrolling only, no clicking—before asking her to explain what the website is about.

Next, have her do a few more tasks that match the goals of the website—such as submitting a form, making a purchase, filling out a registration or downloading an item that you might track in Google Analytics. For example, one task might be to have the user find a particular article on your website, leave a comment and subscribe to receive website content by e-mail. Observing this flow will demonstrate all kinds of things you need to know, like whether your website’s navigation and search tools are working, content is searchable and calls to action are clear.

This kind of test can be customized in any way that makes sense for the goals of your website, but should be kept to fewer than 10 minutes. As soon as fatigue sets in, the quality of the results will drop considerably.

Test Plan 2: 10-Second Test

User experience expert Jakob Nielsen recently published a study showing that most users decide whether to stay or leave a webpage within 10 seconds of opening it. That may seem like a tiny window of opportunity, but humans are able to perceive and make judgments in even less time. Rather than have a volunteer evaluate an entire website, in a 10-second test he will focus on just one page and help you observe the factors that contribute to the bounce rate.

The setup here is the same as before, but the procedure is slightly different. Once you’re rolling, give your volunteer 10 seconds to view a page (again, scrolling only). After 10 seconds, close the browser and ask him to explain what the page was about. Just like before, his answers will help you evaluate the clarity of your page, especially how design decisions can either help or hinder a user’s ability to get an accurate first impression of a page. Design is often the difference between a bounce and a conversion (which means the user following through on one of your goals).

Questions: Answers: Action

Reducing complicated measurements to meaningful questions and pursuing answers to them through simplified analytics procedures and usability testing is all about one thing: producing the right action. Don’t wait to be handed someone else’s number-heavy and insight-light report. Making measurement a part of your repertoire today will help you find the answers you need to take action.

This article appeared in the July 2012 issue of HOW

More interactive design resources

COMMENT