Separate Versus Concurrent Estimation of IRT Item Parameters in the Common Item Equating Design

A C T Rese&rcli R eport Series 99Separate Versus Concurrent Estimation of IRT Item Parameters in the Common Item Equating DesignBradley A. Hanson Anton A. BegumDecemlber 1999 For additional copies write:ACT Research Report Series PO Box 168Iowa City, Iowa 52243-0168© 1999 by ACT, Inc. All rights reserved. Separate Versus Concurrent Estimation of IRT Item Parameters in the Common Item Equating DesignBradley A. Hanson ACT, Inc.Anton A. Beguin University of Twente A bstractIRT item parameters can be estimated using data from a common item equating de­sign either separately for each form, or concurrently across forms. This paper reports the results of a simulation study of separate versus concurrent item parameter estimation. Using simulated data from a test with 60 dichotomous items, four factors were consid­ered: 1) program (MULTILOG versus BILOG-MG), 2) sample size per form (3000 versus 1000). 3) number of common items (20 versus 10). and 4) equivalent versus nonequivalent groups taking the two forms (no mean difference versus a mean difference of 1 standard deviation). In addition, four methods of item parameter scaling were used in the separate estimation condition: two item characteristic curve methods (Stocking-Lord and Haebara), and two moment methods (Mean/Mean and Mean/Sigma). Although concurrent estima­tion resulted in less error than separate estimation more times than not. it is argued that the results of this study, together with other research on this topic, are not sufficient to recommend completely avoiding separate estimation in favor of concurrent estimation. Separate Versus Concurrent Estimation of IRT Item Parameters in the Common Item Equating DesignThe latent variable in many IRT (item response theory) models is unidentified up to a linear transformation. This means that if the latent variable is linearly transformed then an appropriate linear transformation can be made to the item parameters so that the model produces exactly the same fitted probabilities. In practice, a scale or metric for the IRT latent variable and estimated item parameters is determined by constraints imposed by the software used for parameter estimation. For the software studied in this paper the scale of the latent variable is determined by assuming the mean and standard deviation of the latent variable distribution used in the marginal maximum likelihood estimation are 0 and 1, respectively.For example, the probability of a correct response for a dichotomous item i given by the three-parameter logistic IRT model (Lord, 1980) at latent variable value 8 isP{6 | a^b^Ci) — ci + i + e_ li7a.(0_6.) 5 (!)where a*, 6Z. and c* are item parameters for item i. Let the latent variable be linearly transformed by 0* = A9 + £ , let a* = cii/A, and let b* = Abi + B. Substituting a* for a*, b* for bi, and 6* for 6 in Equation 1 will produce exactly the same probability of a correct response to item i as using a,, £>*, and ${. For any linear transformation of the latent variable, a corresponding linear transformation of the item parameters a* and 6* for any item i can be found to produce exactly the same probability of a correct response. Thus, the scale and location of the latent variable in the three-parameter logistic model are unidentified (any linear transformation of the latent variable produces exactly the same model fit).In the common item nonequivalent groups equating design two forms of a test with some items in common are administered to samples from two populations. This paper compares two alternative procedures for producing item parameter estimates on a common scale in a common item nonequivalent groups equating design: concurrent and separate estimation. In concurrent estimation item parameters for all items on both forms are estimated simultaneously in one run of the estimation software. Estimating parameters for all items simultaneously assures that all parameter estimates are on the same scale. Estimation software that can handle multiple groups of examinees is required to properly perform concurrent estimation in the nonequivalent groups design. 2In separate estimation the item param eters for the two forms are estim ated using two separate runs of the estimation software. The item param eter estim ates for the two forms will not be on the same scale. This is due to the fact th at constraining the scale of the latent variable by fixing the mean and standard deviation of the latent variable distribution will result in different scales when samples from different populations are used for item param eter estimation. In separate estimation the two sets of item param eter estim ates for the commo