Interview with Tom Brinck

In which Tim catches up with Tom Brinck to discuss reasons for doing both focus groups and usability testing in Diamond Bullet’s projects, as well as usability anecdotes from shopping experiences around Ann Arbor...

Interviewer: Timothy Keirnan
Guest: Tom Brinck

Tim: Tom, thanks for speaking with me some months after your presentation to the Michigan Chapter of UPA.

Tom: No problem, I’m glad to do it.

Tim: You have both focus groups and usability testing in your project schedule for designing and developing a website. For those new to the usability profession who read this interview, would you explain the difference between those two things?

Tom: Focus groups help answer what the site should do, while user testing helps answer how.

Focus groups are usually done very early in a project to bring a group of potential users together and find out their needs and preferences, which guide feature selection and early brand direction. From our project we learned in the focus groups, for instance, that people really wanted to know what time commitment any given clinical trial would require, and that's a piece of information that we wouldn't have guessed and wasn't even available to us at the outset.

User testing involves bringing in individuals and asking them to perform specific tasks on our prototypes while we observe what kinds of errors and confusions they have. It identifies design problems that would be too subtle or subconscious to learn about in a group setting--or even in interviews--because people have to actually try to do things on the site.

Tim: What do you do if a focus group generates what I call a "users' wish list" but one of the items on it conflicts with what human factors research has proven is superior in regard to readability?

For example, I sometimes pass by a coworker's computer and they have their preferences set for a really garish color combination that, in my mind, should prevent them from being able to read text as well as if they had black text on a very light background. My hunch is that if I put them through some usability tests, I could prove that they cannot read the garish screens as efficiently as the classic color combinations, but emotionally they might still prefer their garish colors. When designing a website for a client, what do you do about clients wanting an aesthetic that clashes with human factors principles?

Tom: The user is not a designer. The user is an expert at doing what they do. Ask about what they do, not about how to design.

There are rarely times that a wish list should conflict with good design. Mostly in a focus group you need to focus on what problems the users want to solve, not how those problems get solved.

However, you also want to know their preferences, and sometimes things like color preferences can be in conflict with good design. However, if these preferences are consistent for your audience, then they aren't "wrong". They are, however, a tradeoff, where colors they like may result in worse performance. It's important to recognize it as a tradeoff, because it's not valid to just ignore them and give them colors they hate. At this point, a designer goes into problem-solving mode, and it's often, but not always, possible to find a design that fulfills both goals. For example, you can incorporate their color preferences in a limited fashion as a highlight color or logo color such that it doesn't interfere with the main task.

Tim: I really liked how you showed us consecutive usability test data as the project progressed, so you could measure UI success against benchmarks. Do you sometimes get clients who think one usability test is enough and more would be overkill? If so, how do you respond to that?

Tom: Sometimes, the more common challenge is to persuade people to do a single test. Once you've done one test, there are perfectly valid financial reasons you may stop there for small projects. However, the numbers often speak for themselves. If you've done one test, you'll see that you've reached, for instance, an 80% success rate. Well, that's usually obvious it's not enough. Sometimes the reaction is that, gosh, why didn't we do better than that, after all we're experts. That's got to be handled by setting expectations up-front, and showing them data from projects making it clear that even though you can do a good job as experts, nobody does a great job without iterative user testing and refinement.

However, once you get to the 90% range for success rate, people feel like they've reached the point of diminishing returns, and I think, while 90% is a great success, it's misleading at that point to have numbers, because I'm rarely in a project where I feel like I've reached the point of diminishing returns in design refinement.

Think along the lines of the biggest online bookseller, a site that has been through a great deal of design refinement, and I'm sure they can pull out stats showing they've had great success in doing design refinement. And yet, the site obviously could use some design improvements, they're practically glaring. They still have the opportunity that if they can fix a feature on the site that's not working great, then they can increase revenue by millions. Sites should budget for steady long-term design refinement rooted in user testing and user feedback. This enables steady improvement and adaptation to changing customer needs and competitive environments.

Tim: Now let’s switch to another usability topic that’s been on my mind. You said in our meeting that you shop for electronics by the remote controls, not the main units themselves.

Tom: (Laughs)

Tim: Bear with me on this. I shop the same way, and I was shopping for a DVD player recently. I went into two national chain stores and in both these places, Tom, the remote controls are nowhere to be seen.

Tom: Right.

Tim: Back 10 or 15 years ago when I’d go into those stores, the remote control for a unit was usually there to try, usually glued to an old phone cord to discourage theft.

Tom: Well what happened is they found out that that just makes people like the units less.

Tim: (Laughs)

Tom: …so why have all the hassles of letting them be stolen and yet all the remote controls do is undermine the product?

Tim: (Still laughing) Are you being sarcastic or do you think that’s calculated on their part.

Tom: I think that that’s a part of it, I really believe it. Just imagine working in the stores: all you ever deal with, when people do look at the remote controls, is trouble. They want explanations and you’re there like “Oh, I don’t know. I’m just a salesman. I don’t know how it works.” Each company’s remotes are different. Remote controls are just hell for salespeople, so why would they want to show them at all? There’s just no merit whatsoever, as far as the company making a sale, to having them out.

Tim: Well, they didn’t get my money, they didn’t get a sale, and they’ve earned my eternal contempt because I explained very politely to these people that I’m ready to buy something today but I need to see the remotes so I can make my decision between X and Y.

Tom: Well, the remote is the product, it’s the whole product, that’s how it works!

Tim: Exactly.

Tom: If you don’t show me the remote, I don’t know what I’m buying!

Tim: Right!

Tom: They all play DVDs, but so what?!

Tim: (Laughing) It was infuriating because there was this nice guy at the first place and he said he’d look for them because he heard they were in a drawer someplace. He came back and said if they were there, he couldn’t find them now. So I went over to the second place, and they seemed almost kind of mocking, like “why would anybody want to look at the remote control? You can just bring it back if you don’t like how it works.” And I live quite a distance from town, for crying out loud.

Tom: I know. The problem is there’s a massive culture problem where the people who do buy, who think “I’m a smart DVD buyer”, the things that they base their decision on are just crazy #@^*! You know? Like, I don’t even know what they’re thinking. What the hell?! Because if it’s not the remote, what the hell is it?!?!

Tim: (Laughing even harder) I think it’s the color on the LEDs on the front where the clock is.

Tom: Or it’s some technology thing, which is irrelevant, you know?

Tim: Something they can’t usually see or hear in double blind testing, but it’s promoted in the marketing literature.

Tom: Right.

Tim: I hope we can get shoppers to think “usability” more in those stores and reward the manufacturers that use better design—and stores that provide what we might call a “usable shopping experience”. There was a happy ending to this story, though, because I got fed up with these chain stores and thought a private shop might have more on the ball, maybe I’ll find someone who knows what the heck they’re talking about.

Tom: Oh, yeah?

Tim: So I drove over to one on the west side—

Tom: —I’ve never been there but I saw an ad for them once—

Tim: I walked in there and they had all their DVD players, VCRs and such with the remote controls right on top of the units. And when I asked about deciding what to buy based on how it works, they were pulling them down and hooking them up to some televisions so I could use ‘em.

Tom: (In a reverently hushed tone) They’re amazing.

Tim: And I thought well that’s it, I’m sure gonna buy from them because they had time for me and they also had the complete product—the remotes—out on display. It was no big deal to them to haul out the machines, hook them up, and there I was seeing exactly how the models worked. And I’ll tell you what: the amount one saves buying from a chain store cannot possibly make up for buying the wrong product for you because you don’t know how it works or it is plain annoying to use.

Tom: Yeah, I’m with ya. Tell the masses!

Tim: Well we need to tell them, you know, if there’s such a thing as a usable shopping experience, we need to tell them about it.

Tom: I don’t know that you want to name names, but I don’t think there’s any shame in informing people—well no, I think you ought to name names, by God.

Tim: Any of our members can talk to me at the next meeting and I’ll tell ‘em. How’s that for a teaser to attract new members to the chapter?

Anyway, it really got me thinking about the whole retailing concept and websites like you were talking about at your presentation. I once saw a website—I don’t recall whose it was—but they had individual pop-up pictures of the remote controls for the units.

Tom: Uh-huh.

Tim: And I thought that was great because even though I’m not actually able to use the device over the Web, they had a clearly labeled link that said “remote control close-up” and you click it and a big picture shows you the layout and colors on the remote, so at least you could see how they mapped the controls on this thing.

Tom: At the August meeting, did I tell you my full story about shopping and remote controls?

Tim: No, you casually mentioned it but not in detail.

Tom: I was at someplace looking for a VCR, and I found the remote I wanted, but the VCR it belonged to was too expensive. More than I wanted to pay, anyway. So I’m looking at the lower-end ones and they all had really awful remotes. But I see this salesperson and I say “Do you have any of these lower end models with this kind of remote?” And at first he was just baffled and he tried to convince me that there was something else I should be paying attention to, but I convinced him that I really wanted that remote but I couldn’t afford the higher-end VCR it belonged to. And he was like “Well, um, how about I sell you this higher-end one at the cost of the lower one and then you can have your remote?”

Tim: Wow!

Tom: And I said OK, and he sold it, you know? I’m like, ah-ha! By taking a stand for usability, I got the better unit at the lower price, which wasn’t my intent, but it turned out to be my unwitting negotiating tool!

(Sighs) Oh man, I just think too many people look at the specs rather than look at the interface, which in my opinion is the product. More and more so. Like with MP3 players: I have a friend who’s showing me his and he’s into it because it’s cheap, that’s all. He didn’t want the best one because he wanted a cheap one. And that’s cool, I mean hey if you need cheap, you need cheap, regardless. But I look at it and it’s just covered with indicators for all sorts of things; for instance, it tells you the rate at which the song was encoded. So 128kbps or something. And I thought who would ever on their MP3 player want to know the rate at which songs are encoded? You’re wasting screen space! Put a clock there or something, you know? Because “128kbps” is completely a waste of space. And no wonder the best MP3 player is doing so well when all these other manufactures are totally not getting it all even years later. It just shocks me. I mean, just pay attention.

Tim: Well, that requires a certain mindfulness that usually is not on the priority lists of management at those companies, even if they have designers who are good and say “I really think it should be this way” and their bosses say “I think it’s good enough as is” or “I don’t see the benefit of refining this” and that’s the point, they don’t see, they’re not in close touch with their customers and they don’t trust or verify the vision of the designers.

Tom: I mean, even the best one could use some really massive improvements and it would be so easy to improve upon it, but competitors don’t pay attention to interface, or even try to imitate the best. I like the generally acknowledged best one because it really is the best interface right now. Bill Gates was quoted as looking at it and saying “I don’t see anything here that we can’t do.” And that’s his way of saying they’re going to be able to compete, you know? And I think Bill, you’re just such an idiot in this area, that’s exactly what your problem is: all you want to do is copy and, by the time you copy, the original manufacturer will be on the next step. Why don’t you come up with your own creative ideas? (Laughs) And it’s so easy to come up with something better than the best one so far and nobody tries. It’s just so frustrating.

Tim: It’s very sheep-like, isn’t it, as far as design in products.

Tom: (Archly) And I don’t want to give away what my secret for a better player is…

Tim: Oh, of course not! I just had to talk with you about this remote controls in retailing issue, because it’s really been burning me up.

Tom: OK.

Tim: I feel better now.

Tom: You’ll get my bill next week.