PUP-IRT Winsteps Unexpectedness

#28

PUP-IRT cannot automatically create fully colored tables from the default settings of Winsteps. You must first run Winsteps and observe the ST. RES. results in Output Table 6.6. If the absolute value has not fallen below 1.5, then add UCOUNT= as suggested below:

UCOUNT=200   ;The default value is 50. You must reset to

            ;a larger value to color all MOST UNEXPECTED

                        ;(>2.0 ST. RES) and even larger to color all

                        ;LESS UNEXPECTED (1.5 - 2.0 ST. RES) listed

            ;in column five on Winsteps Table 6.6 in

                        ;Output Table 6. PERSON (row) fit order.

&END

1                        ;item labels

2

In this example, UCOUNT = 100 listed all the MOST UNEXPECTED, and UCOUNT = 200 listed all the LESS UNEXPECTED.

All students do not learn and retain specific knowledge and specific skills with equal ability. The unexpectedness colored on PUP Table 3c is an on-average value. About half of the students on this Fall88 test have an omit colored Most Unexpected. This in no way means that each of these students should have marked. It does say that, in general, it is most unexpected for a student with this ability, on-average, to omit (blue) a question with this difficulty, on-average. We cannot determine the specific reason each student omitted from PUP Table 3c.

PUP Table 3a, Mastery, Unfinished, and Discriminating, a test maker view of the test results, provides more information.

PUP Table 3b, Expected, Guessing, Misconception, and Discriminating, a test taker view of the test, splits the omits between Expected and Discriminating among higher scoring students.