In their seminal work, Clegg and Cashin admit, “the process [of designing exams] is not only difficult; it is also frustrating and often ineffective.” Too often, our exams don’t align with our priorities for learning. They don’t give students a chance to show how much they've learned—so we can't tell how well we've taught and our students don’t have a clear sense of what they know. When you give an exam, you’re gathering data on your students’ progress, so you want it to be both a valid and reliable measure. The first step is to articulate exactly what you want to assess. Ory and Ryan urge, you have to make sure you’re “testing what you want to be testing”—that is, what you think your students should know and/or be able to do by this point in the course. If your exam is not aligned with your goals, “you will not be able to evaluate [your students’] progress with any accuracy,” warn Clegg and Cashin. This means you have to start with a good idea of what successful learning looks like and figure out how your students can give you evidence of that learning.
You’ll also need to think about how much time and priority you've allotted to particular concepts. The most important ideas deserve the bulk of our attention, student effort, and class time, so they merit a corresponding proportion of the exam. As the research on the “testing effect” reminds us, if we test primarily on minutiae, our students may leave the course remembering those details, rather than the foundational concepts. Part of testing what we want to test means testing students on their mastery of the material, rather than their ability to guess what’s important to us.
After you’ve identified what you want to know about your students' learning and what evidence you want to collect of that learning, consider what you need to ask your students to do to give you that evidence. You may decide to use questions from a publisher’s test bank to help you design your exam. These test banks tend to focus on recognition and other lower-order thinking skills, so you’ll need to decide what proportion of your exam will test lower- and higher-order thinking skills. If you decide to write your exam questions, these guides from our colleagues at the University of Illinois CITL and Vanderbilt University can help you draft a variety of different types of questions.
Finally, as you design your exam for tests of all shapes and sizes, it's important to consider the following guidelines, outlined by Barbara G. Davis in Tools for Teaching:
- Make sure your instructions are clear—ask a colleague or graduate student to review them. Keep in mind that these instructions need to guide stressed-out students, so they probably need to be simpler and more precise than you expect.
- Advise students on how much time to spend on each section. How much of their time a section of the exam deserves should be proportionate with its weight. (This also means that time-consuming problems should be heavily weighted, so you’ll need to decide how important each task is for measuring student learning).
- Include a few easy items first, as this will help alleviate students’ anxiety and increase their confidence.
- Challenge your best students with one very difficult (but reasonable) question. But please locate it toward the end of the exam.
- Review the timing. Rushing students jeopardize the validity of your exam results, as you end up assessing how quickly students can demonstrate what you think they should know and/or be able to do. Unless one of your course goals entails generating fast responses, a good rule of thumb is to allow at least 3 to 4 times as long as it takes you or a graduate student to complete the exam. We often want to put too many questions on an exam, forgetting that students need time to figure out exactly what we’re asking.
- Make sure the layout is clear, easy to navigate, includes enough space for responses, etc.