We use different methods of assessment in relation to education and one of the challenges that we face would be to assure the test items behave as we expect it to behave in an exam setting. The intended behavior of such test items could be gathered according to the following queries,
Are the items too difficult or too easy for the candidates
Can it distinguish between the knowledgeable student and a student who is not
Whether the distracters are functioning as expected or have they failed in their task
In order to answer these questions, the educators need to perform an item analysis and this would be rather useful in preparing MCQ banks for the future use as it would give the best and the most reliable set of test items for the educators to work with.
There are several theories behind the procedures adapted in doing an item analysis and among them ‘Item response theory’ is widely used in the modern time.
Item response theory
In the item response theory, it is believed that, the performance of an individual in relation to a particular test item would have a correlation with the persons performance as a whole based on his or her latent trait which would decide the persons overall ability.
As such, the theorists have formulated three parameters that needs to be assessed with regard to a test item and these include,
-Difficulty – The level of ability that requires answering a test item correctly
-Discrimination – Ability of a test item to separate candidates with similar abilities
-Chance – The probability of a candidate guessing the answer correctly than based on the ability
Difficulty factor:
‘Difficulty’ of an item can be assessed using the SPSS statistical package thorough comparing the percentage of students in the upper 1/3rd of the class who have answered correctly with the percentage of students in the lower 1/3rd of the class whom have answered correctly.
Thus, the software will produce a ‘difficulty index’ which can be used to analyze each item.
According to literature, the difficulty factor can be derived through the following formulae as well.
DF = C/N
DF = Difficulty factor
C = Number of correct responses
N = Number of respondents (n)
If the DF = 1, all students have answered correctly and it would not be a suitable test item to discriminate between the students. Similarly, a DF of 0 would also be bad as none of the respondents have answered it correctly. Accordingly, the experts suggest that a DF of 0.3 -0.7 would be the range for a good test item with 0.5 being the optimal value.
Discriminatory index:
The discriminatory index of a particular test item can be derived through the following formulae.
DI = (a-b)/n
DI = Discriminatory index
a= Number of correct responses made by the students in or above the 75th percentile
b= Number of correct responses made by the students in or below the 25th percentile
n= Number of respondents in or above the 75th percentile
As such, a discriminatory index of 0 would mean that both groups have performed similarly and therefore the discriminatory ability of the test item would be none. In general, the higher the DI more discriminatory the test item would become.
