Q: We are interested in finding information about discarding exam questions. Do you have any information about this? What kind of benchmarks should we be looking for?
A: I have attached a link to a website that gives very detailed information on how to determine whether questions on your assessments should be reviewed.
As a rule of thumb, some experts suggest that if 50% or more students choose the wrong answer on a question, it should be reviewed and possibly revised. The question may be ambiguous, poorly constructed, or misleading and may not adequately measure the student's level of understanding of content.
Other situations your might consider:
a. More students in the top 25% of test takers choose the same wrong answer on a question. This question should be reviewed and possibly revised. It may be ambiguous.
b. In a multiple choice question, nearly equal numbers of students chose each answer option. This suggests the question may be ambiguous, the material may not have been clearly presented or the material was not covered in the same way that the information is requested. This question should be reviewed.
c. A higher than average number of all students select the wrong answer on a question. The question should be reviewed. If you determine that the question is difficult but well constructed and necessary to evaluate whether students have met course objectives, you keep this question.
d. A very high number of students select the correct answer on a question. You should review the question as it may be too easy and may not adequately allow you to evaluate whether students have an adequate level of mastery of your content.
Here is a link to some additional information. http://www.washington.edu/oea/services/scanning_scoring/scoring/item_analysis.html
Questions
Discarding Exam Questions
May 23, 2018 12:44:58 AM
by
Jordan Kivley
|