Differential Test and Item Performance of Talented Youth and High School Seniors on the ACT

ACT Research Report Series90-9Differential Test and Item Performance of Talented Youth and High School Seniors on the ACTAllen E. Doolittle Catherine J. WelchOctober 1990 For additional copies write: ACT Research Report Series P.O. Box 168 Iowa City, lowa 52243©1990 by The American College Testing Program. All rights reserved. DIFFERENTIAL TEST AND ITEM PERFORMANCE OF TALENTED YOUTH AND HIGH SCHOOL SENIORS ON THE ACTAllen E. Doolittle Catherine J. WelchThe American College Testing Program ABSTRACTThe performance of academically talented seventh grade students on the ACT Assessment (ACT) was investigated with respect to college-bound, twelfth grade examinees. Differential test and item performance were examined based on one form of the ACT, and the results were cross-validated on a different form. Although the twelfth grade examinees outperformed the seventh graders on all tests, the differences were greatest on the Mathematics and Social Studies Reading tests. In addition, systematic performance differences between the two groups of students were found at the item level. The nature and implications of these differences are discussed.2 Differential Test and Item Performance of Talented Youth and High School Seniors on the ACTThe early identification of academically talented students has long been of interest to educators. One approach has been to identify able students on the basis of their performance on one of the major standardized achievement batteries that are administered in school systems throughout the country. However, these achievement batteries are typically designed to measure performance for a wide range of students and, as a result, do not differentiate particularly well at the top end of the score scale. Beginning in 1971 with a mathematics talent search sponsored by Johns Hopkins University (Stanley, 1976), the SAT has been used as an "out-of-level" test to provide adequate ceiling for very able seventh graders. Subsequently, four regional, university-based talent searches have been developed in the United States, all using the SAT as an important part of the identification process. Duke University*s Talent Identification Program (TIP) is the regional search that focuses on 16 southeastern and midwestern states, bordered by Iowa and North Carolina in the north and Texas and Florida in the south.Because the ACT Assessment (ACT) is the most commonly used college admissions test in the TIP region, the use of the ACT as an alternative for selecting academically talented seventh graders has recently been explored. Beginning in 1987, TIP and ACT began working cooperatively to investigate the ACT for this3 purpose. (Dreyden & Stanley, 1988, Maxey & Dreyden, 1988? Sawyer & Brounstein, 1988? Stanley & York? 1988.)Dreyden and Stanley (1988) investigated TIP examinees (seventh graders) and found that the population was very homogeneous in terms of test performance and had uniformly positive attitudes toward academic achievement and self perception. The observed homogeneity of the population on these characteristics was consistent for TIP examinees from different gender, race, and income level groups.Maxey and Dreyden (1988) investigated the statistical relationships between ACT scores and scores on standardized achievement tests used in elementary and junior high schools. Generally they found fairly weak correlations and attributed these results to a lack of differentiation by the achievement tests at the top end of the scale. These results are consistent with the rationale for using either the ACT or the SAT rather than the achievement tests to identify talented students at the top end of the distribution.Stanley and York (1988) summarized part of the deliberation process for deciding to offer the ACT as an alternative to the SAT for qualifying TIP applicants. They reported that the ACT and SAT seemed to be equally valid for TIP selection and that the ACT was a particularly good alternative because of the accessibility of ACT test centers in the TIP region.4 The objective of this research was to examine the performance in detail of very able seventh graders relative to college-bound high school students on the ACT.METHODOLOGYResearch PlanThere were two stages to this research, each involving an independent data set. The first stage was designed to identify areas of differential performance on one form of the ACT, and the second stage was to confirm or clarify the observations from the first stage on comparable, but different students and a second, equated form of the ACT.Data SourceThe TIP examinee population consisted of seventh-grade students in the 16-state region covered by the program who had accepted an invitation to take the test. These students all had scores in the upper 3 percent of any of several nationally standardized academic achievement tests. Approximately 55% of these students were female.The high-school examinee population co