41
The internal black box audit tool, introduced in the
previous post, provides a clear view of what is happening on the logit scale during convergence.
Three sets of answer sheets provide the needed data to explore these happenings
with average test scores of 80%, 59%, and 48%. These three values as associated
with mastery, passing in the classroom, and optimum for psychometric analysis.
Student and Item Data


Score

Initial Standard Deviation

Number


Nursing1

80:20

0.70:0.95

22:21

EDS  Moulton

59:41

0.97:1.63

9:10

Cantrell

48:52

0.85:2.20

34:14

The Nursing1 chart shows the input values evenly spaced with
the exception of a very small everexpanding distribution. This “error” is an
artifact of converting normal values to logit values. The input item difficulty
values are shifted to comparable student ability values in an orderly manner,
that is, the lines are parallel. The average item difficulty logit value is
relocated to zero. This data set represents a good fit to the Rasch model,
given my current understanding. The relative positions of student ability and
item difficulty remain stable from input to output.
The EDS data shows student ability and item difficulty
expanding at two different rates. Here the initial standard deviations differ
more than in the Nursing1 data. The result is that the relative positions of
student ability and item difficulty changed from input to output.
The Cantrell data are centered near the 50% normal or zero
logit location. There was no need to relocate item difficulties, as was done
with the Nursing1 data. There was a need to respond to the two very different
initial standard deviations. This caused the student abilities to be expanded
far faster than the item difficulties.
A second
observation on the Cantrell data relates PUP noniterative PROX and Winsteps
results. Winsteps results at four JMLE iterations compared to PUP PROX results.
Winsteps then continued ten more JMLE iterations before convergence was called.
This both increased the expansion of the distributions but also increased the
change in the relative locations of student ability and item difficulty from
input to output.
The destabilizing factor seems to be the relative spread of
the two distributions for student ability and item difficulty. Statistically
the distributions (standard deviations) are being matched, when converging, for
the Cantrell and EDS data, but the process changes the relative individual
locations.
Good data must then have similar distributions (standard
deviations). Also the values need to fall within about 2 to +2 logits (12% to
88% normal).
I am calling the Nursing1 data a good fit for dichotomous
Rasch model analysis based on two observations: 1. Both PUP PROX and Winsteps
obtained the same results (noniterative PROX and iterative PROX:JMLE). 2. The
relative location of student ability and item difficulty remained stable from
input to output.
No comments:
Post a Comment