从黑板到聊天机器人：评估生成性人工智能对尼日利亚学习成果的影响（英）2025

11125 Produced by the Research Support TeamAbstractThe Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about developmentissues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry thenames of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely thoseof the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank andits affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent.Policy Research Working Paper11125This study evaluates the impact of a program leveraginglarge language models for virtual tutoring in secondaryeducation in Nigeria. Using a randomized controlled trial,the program deployed Microsoft Copilot (powered byGPT-4) to support first-year senior secondary students inEnglish language learning over six weeks. The interventiondemonstrated a significant improvement of 0.31 standarddeviation on an assessment that included English topicsaligned with the Nigerian curriculum, knowledge of artifi-cial intelligence and digital skills. The effect on English, themain outcome of interest, was of 0.23 standard deviations.This paper is a product of the Education Global Department. It is part of a larger effort by the World Bank to provideopen access to its research and make a contribution to development policy discussions around the world. Policy ResearchWorking Papers are also posted on the Web at http://www.worldbank.org/prwp. The authors may be contacted atdesimone@worldbank.org,ftiberti@worldbank.org,mbarronrodriguez@worldbank.org,fmanolio@worldbank.org,wmosuro@worldbank.org, and edikoru@worldbank.org. Cost-effectiveness analysis revealed substantial learninggains, equating to 1.5 to 2 years of ’business-as-usual’schooling, situating the intervention among some of themost cost-effective programs to improve learning outcomes.An analysis of heterogeneous effects shows that while theprogram benefits students across the baseline ability dis-tribution, the largest effects are for female students, andthose with higher initial academic performance. The find-ings highlight that artificial intelligence-powered tutoring,when designed and used properly, can have transformativeimpacts in the education sector in low-resource settings. FromChalkboardstoChatbots:EvaluatingtheImpactofGenerativeAIonLearningOutcomesinNigeria*Mart´ınDeSimone,FedericoTiberti,MariaBarronRodriguez,FedericoManolio,WuraolaMosuro,EliotJolomiDikoru.†Keywords:large-languagemodels,adaptivelearning,artificialintelligence,educa-tiontechnology,secondaryeducation,teachingattherightlevel.JELClassification:C93,I21,J24,O15,O33.*The team would like to thank Scherezad Latif and Halil Dundar, Education Practice Managers, WorldBank. The team extends its appreciation to Dr. Joan Osa Oviawe and Jennifer Aisuan, for their collaborationthroughout the implementation of the pilot, as well as Alex Twinomugisha, Robert Hawkins, and Crist ´obalCobo for their support with the intervention. The team thanks those who provided comments to a previousversion of this paper, including David Evans, Halsey Rogers, Carolina Lopez, Francisco Haimovich, DanielRodriguez-Segura, Noah Yarrow, Juan Bar ´on, and Lucas Gortazar. The team acknowledges the financialsupport received from the Mastercard Foundation.†De Simone:The World Bank.E-mail:mdesimone@worldbank.org.Tiberti:The World Bank.E-mail:ftiberti@worldbank.org.Barron:The World Bank.E-mail:mbarronrodriguez@worldbank.org.Manolio: The World Bank. E-mail: fmanolio@worldbank.org. Mosuro: The World Bank. E-mail: wmo-suro@worldbank.org. Dikoru: The World Bank. E-mail: edikoru@worldbank.org. 1IntroductionThe global education sector is grappling with a learning crisis. According to the LearningPoverty Index, approximately 70% of 10-year-olds in low- and middle-income countriescannot read and understand an age-appropriate text (World Bank, 2022). These deficitsin learning accumulate and become particularly acute at the secondary school level, asevidenced by numerous international, regional, and national assessments.In his seminal 1984 study, Bloom demonstrated that students receiving one-on-one tu-toring outperformed their peers in traditional classroom settings by an average of twostandard deviations (Bloom, 1984). Subsequent studies have consistently confirmed thesignificant benefits of one-on-one tutoring (Nickow et al., 2020). The challenge, however,is that implementing one-on-one tutoring at scale is costly and unaffordable for most ed-ucation systems. Bloom referred to this challenge as the “two-sigma problem”: how toreplicate the gains of personalized tutoring at scale in a cost-effective manner.This paper examines whether generative artificial intelligence, specifically large lang