All projects: Gel, Jobs, Gootodo, Games, Uncle Mark, Goovite, Blog, Bit Literacy
The top 8 mistakes in usability (and companies investing in it)
I recently gave a talk to a company that is beginning to invest more in the customer experience of its website. They wanted to know: how do we avoid the errors of other teams making this investment? There are lots of gurus, blogs, and trade groups, all promoting their own tools and methods - usability, user experience, interaction design, information architecture, and so on. The team knows that they want "better usability" but aren't sure about the next step.
And this company is growing fast, so a lot is at stake in them getting it right. If they build the right processes in-house (or hire consultants that offer them), they'll reap the rewards.
I told them that when committing to customer-centered development (of a product, service, website, or whatever), it's important to stay strategic, always try to improve the business, and listen to customers (as human beings, not as users of a tool).
But in doing so, avoid the following:
1. Not conducting any customer research.
Some companies still don't conduct customer research, but instead rely on their best internal guesses as to what their customers want. Except in organizations where ESP is a common employee skill, this tends not to lead to healthy, customer-centered operations.
2. Conducting "pretend" research.
Let's pretend our user's name is Jane. Let's pretend she is 38 years old, drives a purple Prius, reads mystery novels, loves bulldogs, and likes to go sailing. Let's pretend she comes to our website and likes feature A but not feature B. Therefore, we should develop more things like feature A. See? We're very customer-centered.
This is the fun of creating a persona, which allows teams to make decisions based on fictional people, rather than doing the hard work of listening to real customers. (Yes, I'm being provocative; yes, personas can be useful in some cases - see more in this post.)
3. Conducting research, but the wrong type.
One of the most popular research methods in business today is the focus group: an individual moderator, typically a high-energy person, encourages a live panel of many respondents to give feedback on a product or service. This can be useful in some situations. But where customers interact individually with a company - say, on a website or in some other customer experience - the one-to-many method of focus groups doesn't yield very appropriate findings.
4. Conducting one-on-one research, but with tasks defined beforehand.
Traditional usability dictates that the moderator should write the test questions beforehand. But how can you know the right questions to ask before you've even met the customer? Task definition comes from the age of software, when the tool - a piece of software - was being optimized (thus the term "usability" refers to - and focuses on - a tool, not a human). Customer experience is concerned with the customer; their individual, real-life experience is what we're supposed to be observing. It's beyond presumptuous to think you can predict the appropriate tasks before the session starts. (Read more in this column.)
5. Not inviting stakeholders to attend research.
I've often heard the complaint from UX professionals that "we don't have enough impact in the organization." Maybe that's because too many practitioners write reports about their work, and lob them over the cubicle walls, rather than getting stakeholders involved in the research. Writing reports may work in the publish-or-perish academic world, but in the business world, it's infinitely better to have stakeholders physically sit and watch customers as they interact with the website (or product or service or whatever).
6. Not prioritizing findings.
My favorite, love-to-hate conclusion of a usability report goes something like this: "We uncovered 52 usability errors on the site, and here's a list of all of them." Oops: an unending list of tactics that no one will want to wade through. Instead, whenever discussing results (presumably in-person, to stakeholders who attended labs), focus on the most important two or three strategic findings - the ones that will really move the needle on key business metrics. (You DO focus on the business, don't you? See the next point.)
7. Not relating to business objectives.
Some usability researchers seem to see their work as an extension of their master's thesis in human factors - a scholarly exercise that demonstrates their mastery of various research and analysis methods. This may work in academic research labs, but in the business world, the point of this work is to improve the business. If you want to have an impact, then conduct the work in the light of business objectives: increasing revenue, or cutting costs, or improving usage or conversion rates or pageviews or something that helps pay the bills.
8. Missing the larger picture.
Tactical disciplines like usability and information architecture are useful, valuable, and have their place in the development process. But what's much more important is to understand the people, the human beings, who make the company possible. The customers, the visitors, the patients, the readers, the guests, whatever you call them - their experience is what determines the company's success or failure. So focus first on the overall experience. It's strategic, not tactical. It's about the people, not the tool. Focusing on the larger picture first will set a better context in which to work - later - on usability tactics.
- - -
See also: You DO talk to customers, don't you?
Mark,I have a few quibbles with the list.
At #1, there is clearly a place for involving customers, but there is conflicting (and we could go on forever about it) evidence on whether product innovation/development is best done by involving customers or by deploying that ESP you mention. Think Apple. And if you have a great product already, it can be tough finding people that want anything more than they already get.
Perhaps more important in the context of your list, is how #6 applies. Not everyone can afford to undertake all this stuff. That's why they have focus groups, give people questions and direction etc. In an ideal world we'd all live with people that use our products for weeks on end just observing them (well maybe not ideal for them!). The fact is we don't have limitless funds, so what are the rules of thumb for determining which bits we should save up for? What gives the best return if we only have limited resources?
Mike -
Re (1) - if a company decides not to conduct any customer research so that "we can be like Apple," good luck to them ;)
Re (6) - everyone has limited funds. That's a given. Even more reason to prioritize findings - and look for strategic, not tactical, wins.
Mark: Not sure how I would recommend a 'fix', but this powerful piece is made less powerful by the title. The problem, at least for me is that the prevailing perspective of usability is that it is something to 'lead' with, which you clearly do not embrace. But I clearly missed that while reading the piece initially. Perhaps you were trying to avoid being confrontational. The problem is that some of the 'leaps' in understanding that we'd hope for with good advice such as this, just aren't happening. We have to be more direct in our advice, without being accusatory (something I'm sure you can readily correct me of at any time -- a fatal, but unintentional flaw).
The disconnect with 'usability' as a hammer is clearly made worse by the fact that there is certification in the practice, and currently there are more usability professionals than there are true usability jobs (effectively lab-focused work) -- so many of them are applying their highly-focused skills in more 'generalist' roles (e.g. design). That's not to undermine or devalue their skills, when applied appropriately and within the context in which the skills are most appropriate. The problem is that their training does not help them balance how they apply their skills in design settings. Some, fortunately naturally make that leap -- typically, others do not.
This, and limited understanding as new roles/practices evolve, result in the list you've so wonderfully provided in this piece.
Paula - thanks for the comment. Thing is that most companies, when starting out, understand this to be a "usability" issue. Hence the wording -
Mark -- So many individuals seek to "learn by example" regardless of the specific topic of interest. For this reason in particular, I encourage anyone to examine and "put through the wringer" for Web Usability and Web Accessibility the Sonoff Consulting Service, Inc.'s (Scsi's) Productivity and Knowledge Transfer (P&KT) Web site at http://sonoffconsulting.com/ -- and the Mobile Web counterpart site at URL address http://sonoffconsulting.mobi/.
By doing so, they will find out for themselves that adherence to Web Best Practices is the surest way to accomplish this objective -- and will provide for high Search Engine Optimization (SEO) rankings as a consequence of doing so.
I've spent more than six years evolving Scsi's "Perfect 10" Web Site Standard, and I encourage others to adopt and implement Scsi's TOTAL ACCESS philosophy as a first-pass solution for achieving Ubiquitous Web Access.
Further details are presented on the Web site for anyone who is interested in diving below the surface. Good luck!
Mark - Your article is spot on. I have had the 'pleasure' of working with a number of consultants who made many of these mistakes. No wonder I have such a low opinion of them.
Raymond, you made my day, thank you!
ROTFL.
Mark, interesting piece and follow up comments. As I see it a lot of the issues you raise are valid but refer to bad usability practice. I don’t agree with all the points however – for example proper personas are derived from good research not just dreamt up in the office and you seem to miss the point of having them entirely, but I am not going to pick them all up here.
One example you use, ‘not prioritizing recommendations’ is typical bad practice from a usability consultant that lacks the fundamental business grounding to compliment their qualifications. There are lots of these about; in fact I would argue the majority of usability consultants are more academic than commercial and it is a big issue for the sector. Accreditation is a possible solution although I am not convinced that accreditation would include the right balance of technical and commercial skills. I think Paula makes a similar issue regarding design and this relates to the same point about holistic training and development.
Foviance championed the business focused usability cause in the UK and as a result have developed very strong working relationships with clients and agency partners but this is because we are not seen as academics we are seen as business people. Not everyone can afford decent consultancy or anything at all as Mike says so they have to do the best they can with the resources available. When they do pay they should expect to have decent work done for them and regrettably for the industry this is absolutely not the case.
The issues you raise are not solely about companies investing in it, they are about bad practice and that is something end customers have little control over other than through voting with their feet.
Mark, one more:
People doing validative research, but then put no time in the product life cycle to use the results.
Also about personas. I think most people use them incorrectly so your point is right, but I think the way you portray personas doesn't allow for their correct usage, i.e. as done quite rigorously by Cooper as a tool for modeling robust research data and as an analysis tool for interaction design requirements.
-- dave
6. Not prioritizing findings.
We've been advocating something like this at MAYA for a while; you might be interested to read about our method, which helps prioritize findings not just by the importance to the user experience, but considers the implementation difficulty as well (which includes design challenges, organizational-political challenges, and so on).
This allows clients to focus on the highest-ROI issues, and allows them to distinguish between strategic issues and "the other 48 things that won't move the needle."
It's an old paper, but I believe still relevant: (skip to the part where it talks about the cost-benefit chart)
http://citeseer.ist.psu.edu/mcquaid01integrated.html
http://tinyurl.com/5ot58h
David Bishop
MAYA Design, Inc.
Very interresting, the same in france. but we have to merge from tactical to strategical. good design is the way ?
my post in french (sorry for that)
I've been involved in building customer support programs and teams for years and it doesn't take long to see that few companies manage their company and product the way you describe above. I appreciate your past link as well. I saw an interview with the president of Mindshare (a customer survey company) on Fox Business and he has a similar viewpoint.