One of my hobbies is doing multiple choice tests in subjects I know nothing about and seeing how high I can score. Today I got 45% in an online test of Russian (http://www.lidenz.ru/online-test/), putting me at A2 level (i.e. one above beginner) and my language was described as follows:
"You will be able to understand and use sentences and frequently used expressions related to areas of everyday speech. (e.g. very basic personal and family information, shopping, local geography). You will be able to participate in a conversation concerning personal interests."
No I can't. I don't know any Russian - I can't even read the alphabet. Yet I managed to score correctly in nearly half the questions, a full 20% higher than what would be expected by chance.
Also today I did an advanced Dutch test, and was placed in the 3rd level (pre-intermediate)
http://www.englishacademy.be/eng-onlinetest.asp
Two years ago I did a test for doctors applying for immigration to Australia (Australian Medical Council - sadly the test has been removed now), I got 23/50, two short of the pass mark. Again I know nothing about medicine - I couldn't understand the language in most of the questions, let alone the answers. Added to this, there were 5 choices, giving an expected random hit of just 10/50.
How is this done? Well, its all to do with the Multiple Choice Test.
If somebody asks me a question in Russian, I am certain to give the wrong answer, because my knowledge of Russian is zero. But if I am given me four choices, then I have at least a one-in-four chance of getting it right. My success in the test depends, not on my skill in producing a complex foreign tongue, but simply my skill in picking a, b, c or d. People think that if there are four choices, then a person with zero knowledge must have only a one-in-four chance of getting it right. This is only true if each option is equally plausible. It only takes one option to seem unlikely to increase your chances to 33%; two unlikely answers and we're up to fifty-fifty.
So, what are the secrets? Here is my easy guide to bluffing to multiple choice test:
Choose the answer that is most like the others. Here is an example from a real test:
a. ground
b. arena
c. pound
d. stadium
A weakness of Multiple Choice is that writers naturally write distractors (wrong options) that are similar to the right answer. Answer a above is the only option that has something in common with all the other three (why would pound be an option when the only thing it has in common with the others is that it rhymes with ground?). Knowing this, it is often possible to triangulate the correct answer, or at least improve your odds.
Here is set of options from the Russian test:
a. входить - войдите
b. входи - войти
c. войти - входите
d. войди - входить
It looks tough at first sight, but I chose c because it is the only one where both words also occur in the other options. This is the correct answer.
Often there is one option that is obviously different from the others, or a nonsense answer. At other time two anwers look very similar: the correct answer is likely to be one of these two.
Another trap that question setters sometime fall into is making the correct answer longer in length than the distractors - this accounted for many of the correct answers in the Australian doctor's test.
Multiple choice is one of the most common formats for testing, often in important and high-stakes exams (no doubt because it's administratively easy to mark), but the margin between chance and skilled performance is sometimes so narrow that it's quite easy for these tests to have compromised validity.
Subscribe to:
Post Comments (Atom)
That's pretty impressive. You should be a TOEIC teacher!
ReplyDelete