What's wrong with this user research?

This spatula, it flies in the front door, down the stairs of the split-level home, through the hallway, around the living room where everyone is watching TV, and into the kitchen where it stops, pauses, looks around, SEES the frying pan cooking the pancakes and FLIPS the pancakes UP into the air, they ROTATE and FLOP back down into the pan, showing the beautifully browned FLIP SIDE. The spatula which is wearing a BOW TIE, smiles and WINKS, and Fade to the logo and URL of the customer, www.spatula-forever.com

I think a flythrough would be essential.

"And what would you like to see the 3-D flythrough of?"

Probably something along the lines of the 36-second mark of this: http://www.youtube.com/watch?v=2XbCWmY0eqY

As for your first question... "I gave them a list of everything they could ask for..." D'oh.

Did you overhear my most recent client or something? That's hilarious, but not so much since it's so stinkin' true!

How about the flythrough is of a spatula beating Stan?

a 3D spatula thwacking stan on the head over and over. and i'll use it as my avatar.

I work in higher ed and therefore see nothing wrong with this exchange of ideas.

Oh, I can definitely see this, you're right the flythrough is essential. But I'm not a fan of teal I much prefer chartruese. I wouldn't go to the website if they added too much teal, even if they did have the great flythrough!

I wish I thought that this scenario would never happen in real life but I'm betting a variation of this happens all the time. In fact, I'm sure I've seen some of the resulting website. Lots of them as a matter of fact.

Truly the customers have spoken via George G's comment. With a flythrough like that, who wouldn't want a spatula immediately? :) Nice one, George -

Just build it then I'll tell you what I don't like about it

But ONLY as long as the flying spatula is teal. Teal is also essential, remember?

But honestly, limiting users' wishes to a list as your only "user research" seems even sillier than a bow tie wearing, winking, flying, teal spatula.

What goal are they trying to get that spatula to accomplish? Is there an even better way to achive that goal that users never even thought of but the spatula company could create? What issues are they running into with the current model of spatulas? How could the _experience_ be improved, not just the cosmetics of the current item?

Before we vote on pyramids or Boston City Hall, I think these questions may merit consideration.

Balls to the flythrough, guys-- the teal is the THING. Don't you know the subtle spiritual message that is conveyed through teal? Think "Sedona, Arizona," where everything not made of stone is painted teal, even the McDonalds on the main drag. Dig? Throw in a little teal and you send a message that your customer's kitchen--in fact the very skillet that they are cooking on right now--could become One with the great Cosmic Vortex, producing uber-radiant enlightenment-inducing transcendental pancakes. Teal out, dudes!

Whoa, duuude... count me as the first member of the Teal Zeniquarium of Master Chris Jordan!

P.S. Nicely put, Erin - thanks

I'd like to know what was on the list that the customers DIDN'T ask for. Probably a lot more useful

This sounds like a perfect example of EFR (Executive Friendly Research).

EFR consists of many of the following techniques:
1. Ask users what they think of Executive Pet Projects (EPPs). Dismiss naysayers as outliers.

2. Put together survey with leading questions friendly to EPPs.

3. Conduct usability testing on prototype. Focus executive summary presentation on what everyone loved. Include one slide in 9pt font of 'Concerns'.

4. Substitute ethnographic research with 1 hour executive interview.

Results guaranteed to please (management)!

Disclaimer: EFR is not intended for project support. Side effects may include invalid data, missed opportunities, loss of value, and may cause headaches in actual users, not to mention UX practitioners.

This is a classic case of listening to your customers and doing what they say vs. interpreting what they say. User experience designers have a ball with these exact situations every day at work.

LOL. I laughed myself hoarse. Then I cried myself dry. I've seen this more than I'd like. This is the kind of research that yields spatulas w/ no handles.

Jack: Where's the handle?

Stan: The users didn't say they wanted a handle. They wanted teal. And a bow tie.

Brilliant, Jeremy, thanks. E.F.R. on an E.P.P. looks F.U.N.! (Not supplying an expansion to that last acronym, don't ask)

Listen to customers, sure. Interpret what they say, how? By watching what they do and then having an interpretive conversation about it.

Love the facetious comments but I have to be a little serious.

Users don't ever see their problem space. They jump immediately to solutions and express their problem as a solution.

I remember some colleagues who were managing a large user doco library receiving a customer contact that said the index was terrible and needed to be improved. In fact the real problem was that the team's attempt to break the doco up and ensure everything was written only once, meant that the user had to visit multiple pages in multiple volumes to get a "whole" answer. NO index could have solved that problem. People use indexes to FIND an answer not to FIND the TRAIL to an answer.

Somehow, we designers have to keep pushing back to the problem space. I've build tools that enable the users and stakeholders to analyse the problem space in terms of user and business agenda, and in that situation, users come up with brilliant "solution suggestions"... but the source of inspiration is a good long look at the problem space.

What is wrong with the picture? Obviously the list was incomplete. Stan forgot to ask the users to pick between pyramids or Boston City Hall.

But brilliant insight. Nothing says spatula like award-winning architecture.

Users needs to be observed so we can see if their perceived problems are the same as their actual problems with the usage.

I come across this kind of thing every day at work. I actually had a conversation this week about a beta site (with no usability testing whatsoever) that's about to launch.

When I asked how we'd know it if it worked better or was more usable than the current site - without user testing, I was told "if people use it, we know it's working." Arrghh!

Great posting Mark - an excellent classroom/real life example for crappy research. Yet, we must not disregard the findings with ease. I know the rule says not to listen to what they say but to look at what they do. Still, the current website could be gray font on pale gray background, so teal could be the color that makes sense as it adds contrast, and people are usually smart enough to collectively pick the most appropriate solution.

In a similar way, people may want to see a flythrough just to get an idea of what the site offers. Yes, a flythrough could be a killer for regular customers or experienced web users but could make a difference for new ones. Most people are visual beings and prefer watching to reading. Some people cannot read, or can read but with difficulties or in a different language. I have tested people who, strangely enough, enjoyed banner adds. I sometimes visit kid sites together with my children - they scold me when I click Skip Intro.

Somehow, the flythrough concept brings to mind the old flying toasters screen saver. But a flying, animated, anthropomorphic, teal spatula; now that would be something that would stick in the ol’ forebrain.

LOL, that's pretty funny. However, anybody doing research should remember that what people SAY they want to see and what they will actually pay money for are two completely different things.

There have been quite a few marketers who did these "tests" and then given the customers exactly what they said they wanted only to find out they wouldn't pay for it. What you really need to do with your marketing is to find a way to meet an actual NEED. That, people will pay for.

Biggest problem was that it wasn't usability research. If the goal is to sell spatulas, you need to find out whether people can buy spatulas. Going to the point of this site, it wasn't about the experience. It's still useful to find out what people want (if you have a good list - colors and decorations are usually not the right list) but the #1 job is to figure out whether they can and will do what you want them to do - buy spatulas.

On the other hand, George Girton's flythrough will a priori be a net benefit.

Who was giving this guy feedback? People who get on the computer 3 in the morning? This usability is totally useless. throw it out the window.

The bowtie must be teal, definitely. Make it a costume as recognizable as Superman's cape. Launch a comic book series of Spatula as a new superhero rescuing meals and romances, and resolving ethnic conflicts between the slotted and non-slotted utensils and negotiating contact contracts between steel and teflon and plastic and wood. Background music from Verdi's Rigoletto recommended for the theater version.

LOL, that's pretty funny. However, anybody doing research should remember that what people SAY they want to see and what they will actually pay money for are two completely different things.
Thats a great one. Enjoyed reading it thanks.