Blog Posts About: Online Usability

I’ve been through a whirlwind of usability testing and focus groups for the past 3 weeks. We tested 3 concepts across 15 users and 2 focus groups. It’s a good thing we’re using paper prototypes or we couldn’t do it- we can barely keep up on paper.

Our very last test yesterday was fascinating to me. We tested 7 people on the same concept and all but the final person “got�? the concept – they could describe the core functionality and how they would use it. What made the last user interesting was not that he didn’t “get it�?; it was the gap between his words and actions.

If we were to show you a transcript of what he said about the product, you would think he got it and was really impressed. He had great things to say about the “potential�? and said things were “great�? many times. Never did he utter one word of confusion or dismay.

Yet, his actions told a different story. In observing him use the tool, you could see evidence of apprehension and confusion. Even as he spoke positive words, you could tell he wasn’t in tune with the core concepts that we were testing. It didn’t speak to him in the way it did to others.

As a tester, this was a great experience. Having gone through 6 others that got it, this one user made us look more closely at our assumptions about who else will get it. He showed us that some people will not get the current design and we have a more work to do. Also, it was a perfect example of why testers need to rely on actions and not words.

Here are my lessons from this session:

• Users sometimes want to please the testers and this desire can mask their actual feelings.
• Depend on what you observe, not what the participant says.
• Always be prepared for a user to come along open your eyes to new perspective that diverge from what you consider the “normâ€??.
• When someone doesn’t get it, don’t make it personal- focus on why the product is not communicating value or context to this type of user.

Happy Thanksgiving to my fellow Americans out there!

Blogging has been light lately and it always makes me feel a little bad. I feel this sense of neglect, like I’m not doing my job or something. In actuality nothing could be further from the truth.

A project for my biggest client has recently been thrown into high gear. I’ve been traveling for the last two weeks doing usability studies using paper prototypes. We’re trying to turn around a new version of the same process every week and it’s about to kill me. Yesterday was an 18 hour day, I got up this morning after 3.5 hours of sleep and now I’m on a plane back to Seattle, where I will go directly into meetings- and in one of which I’m am presenting. Last week was the same schedule, and I’m really hoping next week will change.

The usability testing has been awesome. I am continually amazed at how valuable paper prototyping can be. People easily suspend reality and use sheets of paper like a computer. Clicks are done with a pen and the participant writes instead of typing. It takes a lot of prep work and discipline (something I’m starting to learn). We have identified many, many problems that we can solve without touching one piece of code. They are just drawings.

I feel like we could never test enough, having gone though a couple of weeks of this process. It’s fascinating to see multiple people stumble on the same problem and then make changes to fix it – then see new people totally “get it�? the next round. However, there are always new problems that crop up in the early stages.

My biggest challenge right now is experience and practice. I’m learning that becoming a disciplined usability tester is something that can only come with practice. I’ve read many books, but nothing is a replacement for the real world experience. I’m really trying to focus on paying attention to the user’s actions and trying to pick up on what is REALLY happening.

Yesterday, a user came to a page and read the description. The first words out of his mouth were “excellent�?, said in a normal tone. I noted this and asked him about it in the debrief. As it turned out, he was being sarcastic. He read the description and thought “yeah right- too good to be true�?. To date, this was my greatest testing moment- probing for deeper meaning and finding that reality was very different from my perception.

Soon we’ll be moving to Flash implementations of the product, which will take things to a new level. I’m hoping that the level it comes with sleep and less travel.

Reach Customers Online pointed me to George Olsen's article on Boxes and Arrows: Making Personas More Powerful: Details to Drive Strategic and Tactical Design. It's a comprehensive look at developing and using personas- and fills in gaps left by the father of personas: Alan Cooper.

I've posted before about a project I'm working on that is using Contextual Inquiry (ethnography) as a way to build a usable distributed learning resource. As part of this process we've created personas that represent our archetypal users.

Just a few days ago, we got our personas laminated -- a final step to give them a more permanent and formal feel. The team is growing and we're working through feature specifications, so the personas are on our mind a lot. The article above was a great refresher for this phase of the project.

See also by George Olsen: Persona Creation and Usage Toolkit (18 page .pdf)

First Rule of Usability? Don't Listen to Users (Alertbox)

Tom Coates points me to an older (2001) Alertbox from Jacob Nielsen. These are lessons I've learned and experienced recently. In fact I just said it in a recent comment- what people say they do vs. what they really do are often two different things. If you design solely-based on what they say- you may be missing important clues.

Just yesterday, I got my first taste of creating personas based on workers we've been observing recently. So far, the creation of these personas has been the most fun and interesting part of the project.

Suddenly, we see all the disparate data from Contextual Inquiry come together to create fictional characters that represent the various types of folks we've observed.

The more we get to know these fictional characters and match them with our targets, the more likely our future products are to match with the real worlds of the folks we're studying. It won't be long until we can say "Would Jeffery like this feature?" and If the answer is no, then we need to rethink it.

Jerry Michalski recently pointed me to this blog post that links to an article about how Best Buy is using personas.

Ergonomics Today(TM) - What is Cognitive Ergonomics?

As I've been learning and doing contextual inquiry (observing users at work), the whole idea of making products easier to use through observing the user experience has completely absorbed me.

I had never see the term "cognitive ergonomics" before, but I think it captures something that online tool builders should strive for- designs that fit with the way people think.

Ergonomics is sometimes described as "fitting the system to the human," meaning that through informed decisions; equipment, tools, environments and tasks can be selected and designed to fit unique human abilities and limitations. Typical examples in the "physical ergonomics" arena include designing a lifting job to occur at or near waist height, selecting a tool shape that reduces awkward postures, and reducing unnecessary tasks and movements to increase production or reduce errors and waste. "Cognitive ergonomics," on the other hand, focuses on the fit between human cognitive abilities and limitations and the machine, task, environment, etc. Example cognitive ergonomics applications include designing a software interface to be "easy to use," designing a sign so that the majority of people will understand and act in the intended manner, designing an airplane cockpit or nuclear power plant control system so that the operators will not make catastrophic errors.

Unfortunately, that's not an easy task, but I believe that watching people use and talk about a product in their environment is about the closest you can get to hearing them think.

Via: Robin Good