Wednesday, June 27, 2012
Non-Iterative PROX and Winsteps
Winsteps ignores extreme values (when all students mark the right or wrong answer on an item or a student marks all right or wrong answers on all items on the test) when estimating measures. Winsteps prints extreme values in the student and item reports and on the person-item bar chart (100% is replaced with a bit smaller number). This makes the reports convenient for classroom use, but prohibits comparison of black box input-outputs from different methods of estimating measures, unless extreme values are deleted prior to running Winsteps.
[UEXTREME=Yes has just been added to Winsteps 3.74.0 to include extreme values, 15 April 2012, as I am writing this.] The PROX routine in Power Up Plus 5.10 (PUP) deletes extreme values so no manual cleanup is needed.
Data from the 24-student by 24-item Nursing1 test was selected to compare non-iterative PROX and Winsteps. PUP Table 10 shows 22 students and 21 items remaining after excluding extreme values. The average test score for these students, preparing for a standardized test, was 80%.
The sequence of student ability and item difficulty plots, for PROX and Winsteps, in every case, appear in order at a resolution of 1/10 measure. Sixteen of the twenty points are identical for the two methods of estimation. There is little question that corresponding student ability and item difficulty values are being accurately plotted.
The black box charts show that the mean for item difficulty was successfully moved to the 50% (zero logit) position by both methods for estimating measures. This required a shift (-1.62 logit) and an expansion factor (1.11 logit) for PROX. The average input value (20%) was changed to an average output value of 50%, a change of 30% when converting the negative item count to a positive student expected score.
Student ability changed very little as only the expansion factor (1.18 logit) was applied by PROX. Winsteps had to make similar changes in its two-stage estimation.