Dichotomous Rasch Model Audit

                                                             41

The internal black box audit tool, introduced in the previous post, provides a clear view of what is happening on the logit scale during convergence. Three sets of answer sheets provide the needed data to explore these happenings with average test scores of 80%, 59%, and 48%. These three values as associated with mastery, passing in the classroom, and optimum for psychometric analysis.

Student and Item Data
	Score	Initial Standard Deviation	Number
Nursing1	80:20	0.70:0.95	22:21
EDS - Moulton	59:41	0.97:1.63	9:10
Cantrell	48:52	0.85:2.20	34:14

The Nursing1 chart shows the input values evenly spaced with the exception of a very small ever-expanding distribution. This “error” is an artifact of converting normal values to logit values. The input item difficulty values are shifted to comparable student ability values in an orderly manner, that is, the lines are parallel. The average item difficulty logit value is relocated to zero. This data set represents a good fit to the Rasch model, given my current understanding. The relative positions of student ability and item difficulty remain stable from input to output.

The EDS data shows student ability and item difficulty expanding at two different rates. Here the initial standard deviations differ more than in the Nursing1 data. The result is that the relative positions of student ability and item difficulty changed from input to output.

The Cantrell data are centered near the 50% normal or zero logit location. There was no need to relocate item difficulties, as was done with the Nursing1 data. There was a need to respond to the two very different initial standard deviations. This caused the student abilities to be expanded far faster than the item difficulties.

A second observation on the Cantrell data relates PUP non-iterative PROX and Winsteps results. Winsteps results at four JMLE iterations compared to PUP PROX results. Winsteps then continued ten more JMLE iterations before convergence was called. This both increased the expansion of the distributions but also increased the change in the relative locations of student ability and item difficulty from input to output.

The destabilizing factor seems to be the relative spread of the two distributions for student ability and item difficulty. Statistically the distributions (standard deviations) are being matched, when converging, for the Cantrell and EDS data, but the process changes the relative individual locations.

Good data must then have similar distributions (standard deviations). Also the values need to fall within about -2 to +2 logits (12% to 88% normal).

I am calling the Nursing1 data a good fit for dichotomous Rasch model analysis based on two observations: 1. Both PUP PROX and Winsteps obtained the same results (non-iterative PROX and iterative PROX:JMLE). 2. The relative location of student ability and item difficulty remained stable from input to output.