Item analysis report
An item analysis includes two statistics that can help you analyze the effectiveness of your test questions. The question difficulty is the percentage of students who selected the correct response. The discrimination (item effectiveness) indicates how well the question separates the students who know the material well from those who don’t.
Question difficulty
Question difficulty is defined as the proportion of students selecting the correct answer. The most effective questions in terms of distinguishing between high and low scoring students will be answered correctly by about half of the students. In practical terms, questions in most classroom tests will have a range of difficulties from low or easy (.90) to high or very difficult (.40). Questions having difficulty estimates outside of these ranges may not contribute much to the effective evaluation of student performance.
Very easy questions may not sufficiently challenge the most able students. However, having a few relatively easy questions in a test may be important to verify the mastery of some course objectives. Keep tests balanced in terms of question difficulty.
Very difficult questions, if they form most of a test, may produce frustration among students. Some very difficult questions are needed to challenge the best students.
Question discrimination
The discrimination index (item effectiveness) is a kind of correlation that describes the relationship between a student’s response to a single question and his or her total score on the test. This statistic can tell you how well each question was able to differentiate among students in terms of their ability and preparation.
As a correlation, question discrimination can theoretically take values between -1.00 and +1.00. In practical terms values for most classroom tests range between near 0.00 to values near .90.
If a question is very easy so that nearly all students answered correctly, the questions discrimination will be near zero. Extremely easy questions cannot distinguish among students in terms of their performance.
If a question is extremely difficult so that nearly all students answered incorrectly, the discrimination will be near zero.
The most effective questions will have moderate difficulty and high discrimination values. The higher the value of discrimination is, the more effective it is in discriminating between students who perform well on the test and those that don’t.
Questions having low or negative values of discrimination need to be reviewed very carefully for confusing language or an incorrect key. If no confusing language is found then the course design for the topic of the question needs to be critically reviewed.
A high level of student guessing on questions will result in a question discrimination value near zero.
Steps in a review of an item analysis report
Review the difficulty and discrimination of each question.
For each question having low values of discrimination review the distribution of responses along with the question text to determine what might be causing a response pattern that suggests student confusion.
If the text of the question is confusing, change the text or remove the question from the course database. If the question text is not confusing or faulty, then try to identify the instructional component that may be leading to student confusion.
Carefully examine the questions that discriminate well between high and low scoring students to fully understand the role that instructional design played in leading to these results. Ask yourself what aspects of the instructional process appear to be most effective.
An item analysis includes two statistics that can help you analyze the effectiveness of your test questions. The question difficulty is the percentage of students who selected the correct response. The discrimination (item effectiveness) indicates how well the question separates the students who know the material well from those who don’t.
Question difficulty
Question difficulty is defined as the proportion of students selecting the correct answer. The most effective questions in terms of distinguishing between high and low scoring students will be answered correctly by about half of the students. In practical terms, questions in most classroom tests will have a range of difficulties from low or easy (.90) to high or very difficult (.40). Questions having difficulty estimates outside of these ranges may not contribute much to the effective evaluation of student performance.
Very easy questions may not sufficiently challenge the most able students. However, having a few relatively easy questions in a test may be important to verify the mastery of some course objectives. Keep tests balanced in terms of question difficulty.
Very difficult questions, if they form most of a test, may produce frustration among students. Some very difficult questions are needed to challenge the best students.
Question discrimination
The discrimination index (item effectiveness) is a kind of correlation that describes the relationship between a student’s response to a single question and his or her total score on the test. This statistic can tell you how well each question was able to differentiate among students in terms of their ability and preparation.
As a correlation, question discrimination can theoretically take values between -1.00 and +1.00. In practical terms values for most classroom tests range between near 0.00 to values near .90.
If a question is very easy so that nearly all students answered correctly, the questions discrimination will be near zero. Extremely easy questions cannot distinguish among students in terms of their performance.
If a question is extremely difficult so that nearly all students answered incorrectly, the discrimination will be near zero.
The most effective questions will have moderate difficulty and high discrimination values. The higher the value of discrimination is, the more effective it is in discriminating between students who perform well on the test and those that don’t.
Questions having low or negative values of discrimination need to be reviewed very carefully for confusing language or an incorrect key. If no confusing language is found then the course design for the topic of the question needs to be critically reviewed.
A high level of student guessing on questions will result in a question discrimination value near zero.
Steps in a review of an item analysis report
Review the difficulty and discrimination of each question.
For each question having low values of discrimination review the distribution of responses along with the question text to determine what might be causing a response pattern that suggests student confusion.
If the text of the question is confusing, change the text or remove the question from the course database. If the question text is not confusing or faulty, then try to identify the instructional component that may be leading to student confusion.
Carefully examine the questions that discriminate well between high and low scoring students to fully understand the role that instructional design played in leading to these results. Ask yourself what aspects of the instructional process appear to be most effective.
Makkah Time
Posts
