 |
A project to make businesses more aware of their customer experience, and how to fix it. By Mark Hurst. |
| About Mark Hurst | Mark's Gel Conference | New York Times Story on This Is Broken | Newsletter: Subscribe | RSS Feed |
Search this site:
Categories:
- Advertising
- Current Affairs
- Customer Service
- Fixed
- Food and Drink
- Just for Fun
- Misc
- Not broken
- Place
- Product Design
- Signs
- Travel
- Web/Tech
Previous: Sugar/sweetener packets | Main | Next: Travel game package
December 3, 2004 12:44 AM
Broken: SAT practice question
An anonymous reader writes in:
One of the Princeton Review practice tests for the SAT is broken! Bob must have felt really ill after drinking all those cookies!
C'mon, Bob put the cookies in the blender and poured in some milk, and made a cookie shake and drank it.....
I had a Princeton Review program a while ago where one of the practice tests had a problem involving determining the length of one of the sides of the triangle shown in the accompanying figure.
Only one problem: They forgot to include the accompanying figure. Good luck trying to figure out the answer, since none of the angles or sides were specified in the text...
You know how lots of things do their best to diversify their pictures, questions, etc? I think it'd be interesting if that were the case with SATs. Imagine having this same problem, but instead of Bob, Mary, Dolly, and Max, we had Pierre, Muhammad, Sir Charles Rielly, and Pocahontas. It'd make the SATs a lot more interesting, for one thign.
Even ignoring the drink/eat error, it's still a horrible question. Depending on how you read the last part of it, the answer is either 15 percent (if each eat half of what was left in succession), 30 percent (if they shared half of what was left), or zero (if they each ate half of what was left, as the question says).
(LONG) The question is even more broken than that: you can solve it without really understanding it, and this is true of a lot of SAT questions.
SATs in general are broken, as are IQ tests. What both measure, primarily, is test-taking ability, something that is eminently coachable and largely independent of intelligence or any meaningful skill or ability.
This question is a good example of how broken SATs are: you can get a good shot at being right without ever doing the math! Let me show you...
A well-coached SATist would reason like this: well, the answer can't be 50% or 60% because Max ate as much as Bob, so choosing one of those two immediately puts the answer over 100%. So now we're down to just 3 responses. Even if we can get no further, we've got an improved guess.
Next our well-coached SATist rules out 27.5% because its the kind of "obvious" misleading answer SAT writers love (subtract 20% and 25% from 100%, get 55% and divide by 2: Wrong!). So now you have a 1 in 2 shot of being right and you still know nothing about the math the question supposedly tests.
But our SATist can do better. Consider: at the most, Mary and Dolly ate 20%+25%=45% (actually they ate less) leaving at least 55% for Max and Bob - so Bob ate at least 27.5%, which rules out 20%. That leaves just one answer, which miraculously is the right one: 30%.
In other words, a middle-class kid whose family can afford a good SAT tutor can get this question right by pure "test logic", without ever having to understand either the math supposedly required nor the implications of the tricky "what was left" wording!
And before anybody says "well, that was just one question", remember that this is one question chosen ENTIRELY AT RANDOM by somebody else, not by me, for totally different reasons - and I was able to reason my way through it without math. In fact, many many SAT questions are like this (the infamous donut question for instance), and even if you can't get to the right answer you can enormously improve your odds of guessing right.
Now somebody might say, "wow Carl, that was some pretty clever reasoning - isn't that a useful skill in its own right?". And I would say no, for the simple reason that it only worked because I knew the answer had to be one of the the 5 on offer. Life is not a multiple choice test.
In well-written tests, students are required to show their reasoning and working; and in the best math tests, you get credit for working the problem correctly, even if you make an arithmetical error along the way. Unfortunately, marking such tests requires actual human beings to read the work, you can't just shove it through an optical reader. If you put ease of marking as your primary concern, you get exactly the college entrants you deserve. (Draw your own analogy to elections in the space provided below, if you wish).
P.S. For the truly dedicated test debunker, there's an even quicker route to the right answer.
Suppose its late in the test, and you only have 60 seconds left. You don't even have time to read the question properly. But you know two things: Max and Bob ate the same amount, and SAT setters love to include answers that are half, double, or the reciprocal of the right answer to catch careless math or logic errors. Quick now, 30 seconds left: are there two answers that look like the right answer and a trick answer? Yes! 30% and 60%! Which one is right? It can't be 60% (Bob and Max can't both eat over 50%) so it must be 30%. Congratulations! You just got one more SAT point without even understanding the question!
This question is broken, SATs are broken, life is not a multiple choice test.
For my GMAT some years ago I bought practice books by Princeton and by Barrons. The Princeton book was terrible - their grammar section had (ahem) grammar errors. I was loathe to try the exercises in the math section - by then I had completely lost trust in the book.
The Barrons book was excellent - the exercises reflected the range of questions that I was asked during the actual test. The explanations by the Barrons writers were very instructive as well.
Stay away from Princeton books!
Getting credit for a wrong answer based on mostly-correct work feels great in school, but I'd hate to drive across a bridge built by the "good work but wrong answer" student.
It has already been said that life is not multiple-choice. There are also few cases for "partial credit."
The right answer is this :
Start out with 100% Mary ate 20% so that leaves 80% then Dolly ate 25% of the 80% that leaves 60%
(1/4 of 80= 20,80-20=60)
Then Bob and Max split the rest: 60% split in half is 30%
The answer is 30%
Carl Zetie, there is something they do on standardized tests that renders your long explanation of SAT skills mostly useless.
It looks like this:
E. None of the above
Comments on this entry are closed
Previous: Sugar/sweetener packets | Main | Next: Travel game package
At least he split it with Max. I wonder what Max thought.
Posted by: Trevor Hall at December 3, 2004 10:55 AM