
The Independent Evaluation Group’s Journey of Discovery Table of Contents Abbreviations3 Acknowledgments 3 1.Introduction: A Journey of Discovery4 Vision Statement8Key Values and Guiding Principles8Rethinking Evaluators and Evaluation Knowledge Brokers’ Roles and Workflows10Maturity Model10 Objective: Artificial Intelligence Use in Evaluation Practice12Objective: Artificial Intelligence Use in Validation Practice16Objective: Artificial Intelligence to Leverage and Broker Independent Evaluation Group Knowledge18 Governance19Staffing, Capacity, and Resources25Data and Information Technology Ecosystems27Partnerships and Strategic Engagements30References31Appendix A.Stocktaking, Future State, and Gaps32Appendix B.World Bank Group Main Guidelines on the Responsible Use of Artificial Intelligence36FiguresFigure 1.1.Artificial Intelligence Strategy Framework6Figure 3.1.Scaling Artificial Intelligence Use13Figure 4.1.Artificial Intelligence Tool Ecosystem29TablesTable 1.1.Gaps to Be Addressed by the Artificial Intelligence Strategy7Table 2.1.Values and Guiding Principles of Embedding Artificial Intelligence in Evaluation Practice9Table 2.2.Maturity Levels11Table 4.1.Engagement Spectrum30 Abbreviations AIartificial intelligenceAIRBArtificial Intelligence Review BoardECGEvaluation Cooperation GroupIEGIndependent Evaluation GroupIFCInternational Finance CorporationITinformation technologiesLLMlarge language modelsTTLtask team leader Acknowledgments The content of this strategy was prepared by a working group of Independent EvaluationGroup staff led by Estelle Raimondo (Head of Methods, Methods Advisory Function), andcomposed of Jenny Gold (senior evaluation officer, Human Development and EconomicManagement), Kristin Strohecker (program manager, Knowledge and Communications),Leonardo Bravo (senior evaluation officer, Private Sector, Finance, IFC, and MIGA), andDouglas Glandon (senior evaluation officer, Evaluation Capacity Building). The working groupbenefited from valuable input from the members of the IEG Data Science Community ofPractice, the IEG Task Team Leaders Community of Practice, and the IEG Extended LeadershipTeam, which endorsed the document. The working group is also grateful to the manycolleagues from the World Bank Group and the Evaluation Cooperation Group who wereconsulted throughout the preparation process for their useful insights. 1. INTRODUCTION A JOURNEY OFDISCOVERY Background The Independent Evaluation Group (IEG) has long been ona journey of discovery to leverage data science and artificialintelligence (AI) for evaluation practice. In this context,the adoption of the AI strategy represents not a point ofdeparture but rather an important milestone on a longertrajectory. This strategy builds on a rich experience sharedamong IEG teams who have been exploring the applicationof discriminative and generative AI in the evaluation field.Years of experimentation—whether leveraging text orimages as data—have demonstrated the high potential ofdata science and AI in answering evaluation questions ofincreasingly complex evaluands. The experiments furtherindicate that the use of data science and AI must beanchored in (not become a substitute for) the fundamentalprinciples of high-quality evaluation. The foundationalbuilding blocks of evaluation—rigorous design, depth ofstakeholder engagement, ethical responsibilities, and robusttheories—remain at the heart of what it takes to deliverimpactful evaluative evidence to inform decision-makingand move the development agenda forward. The AI strategy reflects both the explicit and the tacitknowledge accrued during years of trial and error,exploration, and collective learning across a range ofapplications. It builds on strong networks (for example, theData Science Community of Practice) and renewed waysof working together in interdisciplinary teams, with datascientists and analysts playing an increasingly pivotal role.The strategy is informed by the expertise of the evaluationsupport teams—the Methods Advisory Function, Data andSystems, and Knowledge Management. Context and Driving Forces We prepared the AI strategy as the fields ofinternational development and evaluation struggledto keep pace with the speed of AI development andthe rapid embedment of AI technology in mostcorners of our ecosystems. Objectives and Scopeof the ArtificialIntelligence Strategy The World Bank Group has been leading theadoption of AI-enabled solutions to developmentchallenges. Its information technology (IT) systemsand teams were early adopters of enterprisegenerative AI tools, and they enabled broad accessamong Bank Group staff and teams.IEG hasbenefited from these early and large investments andworked closely with the Information and TechnologySolutions department to customize tools specificallyfor evaluation practice. Accordingly, IEG haspositioned itself at the forefront of AI adoption amongpeer evaluation functions and intends to continuepioneering in this space.