Explore the relationship between English learners' English proficiency and ACT test scores - State 2

Exploring Relationships BetweenEnglish Proficiency and ACT®Test State 2 JOANN L. MOORE, JOYCE Z. SCHNIEDERS, DONGMEI LI, AND CHEN Conclusions This report contains the findings from State 2 of a two-state study investigating relationshipsbetween English proficiency and the performance of English learners (ELs) on the ACT®test.For both states, we found that the English proficiency level of ELs was a significant predictor ofperformance on the ACT, such that students with lower levels of English proficiency tended tohave lower ACT scores, and students with higher levels of English proficiency tended to havehigher ACT scores. Additional analyses from State 2 suggest that the test scores of studentsscoring at the lowest English proficiency levels may reflect some amount of construct-irrelevant So What? It is important that users of test scores consider not only the knowledge and skills that are beingmeasured by the test but also other contextual information that may be relevant wheninterpreting scores. For English learners, limited English proficiency can affect students’ abilityto adequately access the test content and accurately respond to the test items, leading to Now What? Future research should replicate these findings with other testing programs and in other statesto further build up evidence of how well we are measuring the knowledge and skills of English About the Authors Acknowledgments The authors would like to thank Jeff Allen and HeatherBuzick for their comments on earlier drafts of this report. Joann L. Moore, PhD Joann L. Moore is a lead research scientist inAcademic Research at ACT. Her researchfocuses on prediction of secondary and Joyce Z. Schnieders, PhD Joyce Z. Schnieders is a research scientist inSkills Research & Development at ACT. Herresearch interests include K–postsecondary Dongmei Li, PhD Dongmei Li is a lead psychometrician at ACTspecializing in scaling, equating, and growth Chen Qiu, PhD Chen Qiu is a psychometrician at ACTspecializing in statistical modeling and Abstract This study examined relationships between English proficiency and the performance of Englishlearners (ELs) on the ACT®test. Using a sample of students from a U.S. state who took bothWIDA ACCESS English proficiency assessments and the ACT in Grade 11 during the 2018–19to 2022–23 school years, we found that ACCESS Reading scores have a strong relationshipwith performance on the ACT across all test sections and the Composite score. Students with ELs also tended to have lower Grade 11 GPAs than non-ELs, and for ELs, correlations betweenACT scores and Grade 11 GPA increased as ACCESS Reading level increased. In fact,correlations were small and near zero for ELs at Levels 1 and 2. ELs also tended to earn fewercredit hours and were less likely to take advanced courses than non-ELs, especially ELs at Psychometric analyses on a subset of the study sample found that the ACT scores of ELs withlower levels of reading proficiency (Levels 1–3) showed some evidence of differential itemfunctioning (DIF) and had lower reliability compared to the scores of ELs with higher levels of Regression models showed that ACCESS Reading scores were strong predictors of ACTperformance across all test sections and the Composite score. Use of testing supports on theACT had a significant positive relationship with the performance of ELs, even after we controlled Overall, these results indicate that the English proficiency levels of ELs are strongly related toperformance on the ACT. Caution should be taken when interpreting the scores of ELs with thelowest ELP levels (ACCESS Reading levels of 1–2) because limited English proficiency may beintroducing construct-irrelevant variance in the scores. However, we also found evidence that Introduction This report contains the findings from State 2 of a two-state study investigating relationshipsbetween English proficiency and the performance of English learners (ELs) on the ACT®test.Both participating states have been administering the ACT to all Grade 11 students for multiple ACCESS data were obtained either from the state or from WIDA with the state’s permission.Parallel analyses were conducted across the two states where possible, and Grade 11 highschool transcript data provided by State 2 allowed for additional analyses related to gradesearned, credit hours earned, and coursework taken. Additional details about the State 2 data ACT began offering testing supports (also referred to as language supports or accommodations)to ELs in the fall of 2017. These supports include extended time (one and one half, single day),translated test instructions provided by ACT, and use of an approved word-to-word bilingualdictionary without definitions. ACT has conducted research investigating ACT performance,score gains, and psychometric properties of the scores of ELs taking the ACT with or without In general, the research cited above has found that ELs (with or without testing supports) ten