Application of a Polynomial Loglinear Model to Assessing Differential Item Functiong for Common Items in the Common-Item Equating Design

A C T R e s e a r c h R e p o r t S e r ie s 9 7 - 1Application of a Polynomial Loglinear Model to Assessing Differential Item Functioning for Common Items in the Common-Item Equating DesignBradley A. Hanson Zachary S. Feinstein1 S T M arch 1 9 9 7 For additional copies write:ACT Research Report Series PO Box 168Iowa City, Iowa 52243-0168 1997 by ACT, Inc. All rights reserved. Application of a Polynomial Loglinear Model to Assessing Differential Item Functioning for Common Items in the Common-Item Equating DesignBradley A. Hanson Zachary S. Feinstein Errata forApplication of a Polynomial Loglinear Model to Assessing Differential Item Functioning for Common Items in the Common-Item Equating DesignACT Research Report Series 97-1Bradley A. Hanson Zachary S. Feinstein vThe third line after Equation 6 on page 3 should read as follows:at each level of the matching variable. If 9\\k = 1 then group and item response are independent AbstractLoglinear and logit models that have been suggested for studying differential item functioning (DIF) are reviewed, and loglinear formulations of the logit models are given. A polynomial loglinear model for assessing DBF is introduced which incorporates scores on the matching variable and item responses. The polynomial loglinear model contains far fewer parameters than loglinear models which treat the matching variable and item response as nominal. Compared to logit models that have been presented for investigating DIF, the polynomial loglinear model is easier to generalize to the case of more than two groups and more the two response categories, and can model more complex forms of DEF. The use of DIF methodology to investigating whether common items in the common- item equating design are functioning differently across test forms is discussed. Examples are given of using the polynomial loglinear model to investigate DIF for a test containing dichotomous and polytomous items, and for investigating DIF for common items in a common-item nonequivalent groups equating design.Acknowledgement. The authors thank Mary Pommerich and Ron Cope for helpful comments on an earlier version of this paper, and thank Judy Spray for providing the data used in one of the examples. Application of a Polynomial Loglinear Model to Assessing Differential Item Functioning for Common Items in the Common-item Equating DesignThere are many procedures a researcher may use to examine the validity of a test, so as to prevent bias from inadvertently affecting a sub-group of examinees the test is intended for. Procedures of this type are part of the process of construct validation. One aspect of investigating the validity of a test for various groups of examinees is the investigation of the test items for differential item functioning. Differential item functioning (DIF) is said to exist when an item is functioning differently for two or more groups of examinees, within the population the test is intended for. DIF manifests itself by differential response to an item based on the group an examinee belongs to, when conditioned on the latent variable being measured by the test the item is a part of. Differential item functioning is defined conditioned on the latent variable measured by the test. This is in contrast to differences in responding to an item among groups when averaged across levels of the latent variable. These marginal differences in item performance may reflect legitimate differences between the groups on the latent variable measured by the test (denoted impact), and do not necessarily represent DIF.The first part of this paper presents a definition of DIF and reviews loglinear and logit models used for assessing DIF. Loglinear formulations of logit models that have been suggested for studying DIF are presented.The second part of this paper presents a polynomial loglinear model for assessing DIF which incorporates numerical scores for the item response variable and conditioning variable. An example using the polynomial loglinear model for investigating DIF on a test with both dichotomous and polytomous items is given.The third part of this paper discusses applying DIF methodology to investigating whether common items in a common-item equating design are functioning differently across test forms. In the common-item equating design the forms of a test to be equated are administered to different groups along with a common set of items. For common-item equating to provide valid results it is important that the common items function the same on all test forms. DIF techniques can be used to investigate whether common items are functioning differently on different forms of a test. An example is presented applying the polynomial loglinear model discussed in the second part of the paper to investigate DIF for common items in a common-item nonequivalent groups equating design.Definition of DIFThe data used to investigate DIF for