Development and Validation of a Preliminary ELA Readiness Benchmark Based on the ACT ELA Score

A C T Re s e A R C h Re p o R T se R i e s 2 0 1 7 - 9 Development and Validation of a Preliminary ELA Readiness Benchmark based on the ACT ELA Score Justine Radunzel, PhD Paul Westrick, PhD Dina Bassiri, PhD Dongmei Li, PhD Justine Radunzel is a principal research scientist in Statistical and Applied Research at ACT specializing in postsecondary outcomes research and validity evidence for the ACT test. Paul Westrick is a senior research scientist in Statistical and Applied Research at ACT specializing in postsecondary outcomes research and validity evidence for the ACT test. Dina Bassiri is a senior research scientist in Statistical and Applied Research at ACT specializing in educational outcomes research and student growth models. Dongmei Li is a principal psychometrician in Psychometrics Research at ACT specializing in equating, scaling, and growth modeling. Acknowledgments The authors thank Krista Mattern, Jeff Allen, Joann Moore, and Deborah Harris for their input on the methods and their suggestions on earlier drafts of this report. The authors also thank Cathy Lacina for her work on assigning course content codes and Jizhi Ling for data analysis support. © 2017 by ACT, Inc. All rights reserved. R1640 Table of Contents Abstract .................................................................. vi Development and Validation of a Preliminary ELA Readiness Benchmark based on the ACT ELA Score ...................................................... 1 The ACT ELA Score ....................................................... 2 Readiness Benchmarks.................................................... 2 Current Study.............................................................. 4 Study 1: Course Taking Patterns in English and the Social Sciences................ 4 Sample................................................................. 4 Results................................................................. 7 Study 2: Development of an ELA Readiness Benchmark ......................... 10 Sample................................................................ 10 Analyses............................................................... 12 Results .................................................................. 13 Descriptive Results....................................................... 13 Model-Based Results..................................................... 13 Study 3: Validation of ACT ELA Readiness Benchmark .......................... 14 Sample................................................................ 15 Analyses............................................................... 17 Results................................................................ 18 ACT ELA Benchmark Attainment for Recent ACT-Tested High School Graduates...... 20 Discussion ............................................................... 22 Appendix A............................................................... 25 An Empirical Validation of the Use of Concorded Writing Scores in Estimating ELA Scores for Students with Former ACT Writing Test Scores .................... 25 Data and Method........................................................ 25 Results................................................................ 26 Appendix B............................................................... 29 References ............................................................... 33 iii ACT Research Report Development and Validation of a Preliminary ELA Readiness Benchmark based on the ACT ELA Score List of Tables Table 1. Summary of Student Characteristics of English Sample for Study 1 ............. 6 Table 2. Summary of Student Characteristics of Social Science Sample for Study 1........ 7 Table 3. Distribution (%) of Students’ First College English Course by Major Category.................................................... 8 Table 4. Distribution (%) of Students’ First College Social Science Course by Major Category.................................................... 9 Table 5. Summary of Institutional Characteristics of Study 2......................... 11 Table 6. Summary of Student Characteristics of Study 2............................ 12 Table 7. Summary of Institutional Characteristics of Study 3......................... 16 Table 8. Summary of Student Characteristics of Study 3 ............................ 17 Table 9. Students’ Estimated Probability of Earning a Cumulative GPA of 3.0 or higher by ELA Benchmark Attainment.................................. 18 Table 10. ACT College Readiness Benchmark Attainment for 2016 ACT-Tested High School Graduates.............................................. 21 Table 11. Cross-tabulation of ACT ELA Benchmark Attainment by ACT English and Reading Benchmark Attainment for 2016 ACT-Tested High School Graduates who Took the Current ACT Writing Test............