How to Write Better Tests

Pop quiz!

Multiple choice questions are:
A) Easy to write
B) Kind of a cliché
C) Often too easy to guess (eliminate the obvious and it comes down to a 50/50)
D) Not the best way to test mastery

We've all taken tests. If you’re reading this, you’ve probably even written some yourself. Why is it that so many test questions are so bad?

Who’s the test for anyway?
True or false? This type of question is even easier.

I think the problem is perspective. When people write tests, they’re doing so from the perspective of someone who knows the answer and is looking for the right question. On the other hand, when someone takes a test, they’re doing it from the perspective of someone who has the question and is looking for the right answer.
I’d like you to try something for me, friends and colleagues. I’d consider it a personal favour. Next time you’re writing a test, start from the question, not the answer.

It’s a start
It doesn't stop there. Why did I make you do that, if that wasn’t the solution? It’s because I’m sadistic. Also, it’s a great place to start. If you stop there, your multiple choice and true/false questions will be better.
That’s not enough. The good news is that this exercise makes it much easier to take the next step.
Why settle for multiple choice and true/false questions (or matching or fill-in-the-blank) when you can use something better?

What does better look like?
Better test items require a little more thought. Beyond that it’ll depend on what you’re doing.

Classroom training
Not so good: multiple choice, true/false
Better: essay or open-ended questions
Best: hands-on, role-playing

e-Learning
Not so good: multiple choice, true/false
Better: hot spot questions
Best: simulation, role-playing

What makes for the best test items?
The best test of all would be on-the-job performance. That said, it’s not always practical (or indeed safe) to use real world tasks to test competence (or proficiency or even mastery). When the consequences of failure are too great, that’s when you want to go with the next best thing, a simulation. For example, pilots log time in flight simulators, surgeons practice with cadavers and programmers code first in non-production environments.

Now, if you've taken my advice (you have, haven’t you?), you’re already halfway there. Instead of looking for test questions to fit the answers you’re looking for (“Which of the following is NOT a feature of our new widget?”), you’re looking for answers to fit the questions (“What are the benefits of our new widget?”). Now you just need to make it real.
Think about it this way, which of these lawyers would you want to defend you in a murder trial?

A. The one who got his license by answering multiple choice questions
B. The one who got his license by writing essays that mimicked closing arguments
C. The one who got his license by performing mock trials in realistic courtroom settings
I know who I’d want!

Conclusion
Have you made the jump to better testing? I’d love to hear how it went.
Don’t think this will work for you? Please tell me why in the comments.