39

The
EDS Excel example by Dr. Mark H. Moulton is a complete functional solution for latent student ability and item difficulty calibration. However,
plugging numbers into algorithms is not the same as understanding what is
happening. This JMLE discussion makes use of four of the EDS charts with no missing marks.

The
observed raw values chart (Chart 1) is identical to iterative PROX: raw scores
are converted into student ability logits [logit = ln(right/wrong)] and items
are converted into difficulty logits [logit = ln(wrong/right). Again the item
difficulty mean is subtracted from each item to shift the item difficulty
distribution to center it on the zero logit location; the first step in
converging student ability and item difficulty estimates.

The
Rasch model [expected probability = exp(person logit – Item logit)/(1 + exp(person
logit – item logit)] is then applied to the raw value chart (Chart 1) marginal logit
cells to generate an expected probability value for each internal expected
value chart cell (Chart 2). The 0 and 1 for wrong and right in the raw values
chart (Chart 1) are replaced with the probability of a student with a given
ability being able to mark correctly 50% of the time an item with a given
difficulty in the expected value chart (Chart 2).

The
marginal logit raw values (Chart 1) control the pattern of the probability values
within the expected value chart cells (Chart 2). The variance [variance =
probability * (1 – probability)] of expected values chart (Chart 3) has the
same cell pattern. Therefore all students with the same score, or items with
the same difficulty, receive the same expected probability value and the same
variance value.

Subtracting
the expected probability values (Chart 2) from the observed raw values (Chart 1) [0 and 1] fills the internal residuals
chart cells (Chart 4) [residuals (Chart 4) = observed (Chart 1) – expected (Chart
2)]. Filling in Chart 4 marginal cells will complete the first JMLE iteration.

The
logit distributions expand as the process of convergence progresses. Student
ability expands faster than item difficulty. Convergence must not use too large
or too small of steps. A scheme is needed that senses the approach of
convergence and that makes an expansion step that does not overshoot the point
of convergence; where student ability matches item difficulty resulting in a
right mark 50% of the time.

The
approach of convergence is monitored by first summing the residuals for each
person and each item in Chart 4. Some are positive and some are negative.
Squaring turns all of them positive. The sum of positive squared person
residuals is then used to monitor the approach of convergence; the point when
the sum of squared residuals has a value of or near zero.

The
last thing we need is to control the size of change made with each iteration.
In general the change is something less than the current residual value. The
sum of residuals for each person and each item is standardized by dividing by
the respective sum of variances for each person and item; this is in contrast
to PROX where item variance is applied to person logits and person variance is
applied to item logits. The standardized value is then combined with logit
measures that are also standardized values. The ninth iteration expansion
values (left) are less than one percent of those in the first iteration (right), in this
example.

The
final step in each iteration is to fill the marginal cells with expanded person
and item logit values. The standardized residuals are subtracted from the
current person and item logit values. This expands their locations on the logit
scale.

The
next iteration again fills an expected value chart (Chart 2) using the Rasch
model to create new probability values from the old (Chart 4) logit values.
Then a new variance chart (Chart 3) and a new residuals chart (Chart 4)
complete each iteration.

Again,
there is no need for pixy dust but there are still lingering questions. The
perfect Rasch model requires near perfect item calibration and latent student
ability estimates on a nearly perfect linear scale. The unsettling alternative
is a skilled operator who can deliver desired results.

## No comments:

## Post a Comment