All projects: Gel, Jobs, Gootodo, Games, Uncle Mark, Goovite, Blog, Bit Literacy
A lesson from trivia games
Feb 20, 2008
An eternity ago, in Internet years, I started my career at an early online startup called Yoyodyne Entertainment, founded by Seth Godin. Yoyodyne created online multiplayer games paid for by sponsors, which would get advertising space throughout the game.
This being the mid-90s - like I said, this was a long time ago - the most popular Internet technology was still that "killer app," plain old text e-mail. So most of the games we created were the kind perhaps best suited to the medium of e-mail: trivia games.
A sample question might go like this:
He's no hack, having won Best Supporting Actor for his role in "Unforgiven." What's his first name? (Just reply to this note.)
If you replied "Gene", as in Gene Hackman, you'd get the points and move on to the next level.
By today's standards these were low-tech games, to put it lightly - but the experience of creating Yoyodyne games taught me something that has paid dividends in my career ever since, even to the present day of AJAX, iPhones, widgets, social networking, and - dare I utter the blasted phrase again - Web 2.0. It's a truth about customer experience (or user experience, usability, whatever) that I think will continue to hold true no matter what happens online.
I have to tell you how it all started, because it wasn't obvious at first, like it is now.
When we began creating online trivia games, we looked to a comparable model of success, maybe the most successful trivia game of all time: Jeopardy.
And what distinguished Jeopardy over the also-ran shows that littered game show history? (Other than the inverted question-answer format, that is.) It was, we concluded, the bulletproof writing and sourcing of the questions. There were supposedly two rounds of fact checks on every Jeopardy question, and an army of writers getting the wording just so, eliminating ambiguity while maintaining a sense of style.
And so we got to work, writing questions, imbuing them with style and personality, fact-checking carefully, eliminating any ambiguous wording, and so on.
Our games began to roll out, and people played. Five thousand here, ten thousand there - these were real numbers in the mid-90s, and sponsors paid up.
Still, there was a problem. After players signed up for a game, not enough of them would play through the whole game. In fact, a fair percentage wouldn't even start by answering the Round 1 question. We needed more participation.
Then came the week when a sponsor bought a new game at the last minute, and we had to deliver a game on a very short deadline. There was no time to write smart Jeopardy-style questions: we just had to get some fact-checked questions out the door.
And so the game came out looking something like this:
What city is both the capital of Colorado and the hometown of the NFL's Broncos?
The whole game went like this. Easy but bland.
When the game launched, we were more than a little surprised to see higher participation rates than any other game we had running. The players loved it. But... what about Jeopardy? we thought. What about working carefully on the questions? We had put less time and energy into this game, so why would we be getting rewarded for it?
It's pretty obvious now what was going on, but it took a few weeks, and another game or two, to piece together what was at work. But we finally got it: the vast majority of e-mail trivia players out there weren't looking for Jeopardy mind-stretchers. Just the opposite, in fact. They wanted to open their e-mail, read a quick question, and know the answer right away, without having to think, do research, or indeed make any physical movement but a few keystrokes. Furthermore, they wanted quick feedback that they had gotten the question right and were doing great.
In other words, they wanted some perceived value - "you got the right answer!" - without having to work for it.
The easier the game, the more people played.
Stated more precisely, the easier we made it for users to achieve their goal, the more users would try it in the first place.
It was around this time that I began noticing that sites were springing up everywhere on the new World Wide Web, and most of them made it incredibly difficult for average, non-techie users to achieve the stated goal on the site. Most search engines were almost as hard as a computer science quiz, e-commerce sites were buggy or poorly featured, and media sites were practically nonexistent.
Thus came my epiphany, circa 1996. Could it be that the young Web industry was heading in the wrong direction? Didn't these companies realize the importance of allowing users to achieve their goals? More pertinently for me, would at least some companies be willing to invest in learning what drove online success?
Luckily, having started Creative Good in early 1997, I guessed the correct answer on all three: yes, not yet, and yes. That's one trivia quiz I'm glad I got right!
I used to laugh at how easy the questions on TV shows were until I realised that the easier the question, the more people would phone oin with the answer, the more people who phoned i with the answer the more money the TV company earned through telephone charges. Doooaaaa.
Was your next company Bigboote Enterprises?
I agree that this is this case and that it is important that we create the experience to support the user's needs. But, isn't it kind of sad that people only want questions they can get right? I know we're only talking trivia here, so maybe it's not such a big deal, but I'm afraid the trend extends to most content. [there's a good article in the NYT from yesterday "Dumb and Dumber: Are Americans Hostile to Knowledge?"]
This phenomenon further solidifies the concept that the experience is important and that if we create a good experience, maybe we content creators can buy ourselves a little room to challenge people.
Sigivald: "wherever you go, there you are" :)
Gretchen: well put. I may dive into this in a future column, since this was just a quick pass on the topic. It's not always the best strategy to simply give people exactly what they (think they) want.
I have some experiences of how simplicity can enhance customers’ positive experience of products. As a product manager, I found that the simpler a product becomes, the more likely the user will try and use it. However, I found it very interesting that in lots of cases developers enjoy developing complicated features thought.
Absoultely agree. I learned this lesson the hard way, when we designed a game for schoolboys that got an awfully low participation (much worse than you can imagine) just because it took too much effort to complete.
I think there's a bit of narcissism, if you dig deeper in this: designers want to show how clever, cool and sophisticated they are at creating entertainment on line. They often design for themselves, not for the audience.
Just as a "rebuttal" of sorts: I was an early Yoyodyne player, and I distinctly remember when your tests stopped being personal, clever, and occasionally inscrutable. Since that was the only thing appealing about them to me, I dropped out shortly thereafter.
Here's another data point on the topic. I was on the show "Jeopardy" (and won the Tournament of Champions in 1995, in fact), and so have talked to a lot of viewers about the show.
I've often heard a variation on one particular idea: "I usually know the $400 answers" or "I'm good down to the $600 level, but not beyond that." In other words, the format of the show lets people self-calibrate their own expectations. A random selection of questions -- some very difficult and some very easy -- can be irritating. You feel alternately stupid or patronized.
But in the Jeopardy system, people know which questions they "ought" to get and then still (as you say) feel pleased when they do. It also give them an extra thrill when they get one right that's beyond their normal range. "Hey, I got a $1000 question!"