Moulton JMLE Example

                                                              39

The EDS Excel example by Dr. Mark H. Moulton is a complete functional solution for latent student ability and item difficulty calibration. However, plugging numbers into algorithms is not the same as understanding what is happening. This JMLE discussion makes use of four of the EDS charts with no missing marks.

The observed raw values chart (Chart 1) is identical to iterative PROX: raw scores are converted into student ability logits [logit = ln(right/wrong)] and items are converted into difficulty logits [logit = ln(wrong/right). Again the item difficulty mean is subtracted from each item to shift the item difficulty distribution to center it on the zero logit location; the first step in converging student ability and item difficulty estimates.

The Rasch model [expected probability = exp(person logit – Item logit)/(1 + exp(person logit – item logit)] is then applied to the raw value chart (Chart 1) marginal logit cells to generate an expected probability value for each internal expected value chart cell (Chart 2). The 0 and 1 for wrong and right in the raw values chart (Chart 1) are replaced with the probability of a student with a given ability being able to mark correctly 50% of the time an item with a given difficulty in the expected value chart (Chart 2).

The marginal logit raw values (Chart 1) control the pattern of the probability values within the expected value chart cells (Chart 2). The variance [variance = probability * (1 – probability)] of expected values chart (Chart 3) has the same cell pattern. Therefore all students with the same score, or items with the same difficulty, receive the same expected probability value and the same variance value.

Subtracting the expected probability values (Chart 2) from the observed raw values (Chart 1)  [0 and 1] fills the internal residuals chart cells (Chart 4) [residuals (Chart 4) = observed (Chart 1) – expected (Chart 2)]. Filling in Chart 4 marginal cells will complete the first JMLE iteration.

The logit distributions expand as the process of convergence progresses. Student ability expands faster than item difficulty. Convergence must not use too large or too small of steps. A scheme is needed that senses the approach of convergence and that makes an expansion step that does not overshoot the point of convergence; where student ability matches item difficulty resulting in a right mark 50% of the time.

The approach of convergence is monitored by first summing the residuals for each person and each item in Chart 4. Some are positive and some are negative. Squaring turns all of them positive. The sum of positive squared person residuals is then used to monitor the approach of convergence; the point when the sum of squared residuals has a value of or near zero.

The last thing we need is to control the size of change made with each iteration. In general the change is something less than the current residual value. The sum of residuals for each person and each item is standardized by dividing by the respective sum of variances for each person and item; this is in contrast to PROX where item variance is applied to person logits and person variance is applied to item logits. The standardized value is then combined with logit measures that are also standardized values. The ninth iteration expansion values (left) are less than one percent of those in the first iteration (right), in this example.

The final step in each iteration is to fill the marginal cells with expanded person and item logit values. The standardized residuals are subtracted from the current person and item logit values. This expands their locations on the logit scale.

The next iteration again fills an expected value chart (Chart 2) using the Rasch model to create new probability values from the old (Chart 4) logit values. Then a new variance chart (Chart 3) and a new residuals chart (Chart 4) complete each iteration.

Again, there is no need for pixy dust but there are still lingering questions. The perfect Rasch model requires near perfect item calibration and latent student ability estimates on a nearly perfect linear scale. The unsettling alternative is a skilled operator who can deliver desired results.