人工智能战略：独立评估小组的发现之旅

Table of Contents Abbreviations Acknowledgments 1.Introduction: A Journey of Discovery Background4Context and Driving Forces5Objectivesand Scope of the Artificial Intelligence Strategy5Defining Artificial Intelligence for Evaluation Practice6Where We Are Today6 2.Where We Want to Go Vision Statement8Key Values and Guiding Principles8Rethinking Evaluators and Evaluation Knowledge Brokers’ Roles and Workflows10 3.How We Will Get There Objective: Artificial Intelligence Use in Evaluation PracticeObjective: Artificial Intelligence Use in Validation Practice 4.BuildingBlocks Appendix A.Stocktaking, Future State, and Gaps Appendix B.World Bank Group Main Guidelines on the Responsible Use of Artificial Intelligence Abbreviations AIartificial intelligenceAIRBArtificial Intelligence Review BoardECGEvaluation Cooperation Group IEGIndependent Evaluation Group Acknowledgments The content of this strategy was prepared by a working group of Independent EvaluationGroup staff led by Estelle Raimondo (Head of Methods, Methods Advisory Function), andcomposed of Jenny Gold (senior evaluation officer, Human Development and Economic Background The Independent Evaluation Group (IEG) has long been ona journey of discovery to leverage data science and artificialintelligence (AI) for evaluation practice. In this context,the adoption of the AI strategy represents not a point ofdeparture but rather an important milestone on a longertrajectory. This strategy builds on a rich experience sharedamong IEG teams who have been exploring the application 1. INTRODUCTION The AI strategy reflects both the explicit and the tacitknowledge accrued during years of trial and error,exploration, and collective learning across a range ofapplications. It builds on strong networks (for example, the Context and Driving Forces We prepared the AI strategy as the fields ofinternational development and evaluation struggledto keep pace with the speed of AI development and Objectives and Scope In adopting this AI strategy, IEG affirms itscommitment to being deliberate and ambitious inits scale-up of the use of AI for evaluation practiceand knowledge brokering. It intends to createbroader ownership of this effort and more evenadoption of these innovative practices acrossIEG units and teams. IEG aims to provide clarity to Concurrently, the regulatory environment—norms,guardrails, and robust governance mechanisms—is falling into place, in the Bank Group and beyond, Where We Are Today IEG has been experimenting with and using AI techniquesin evaluation and for brokering evaluation knowledge sinceat least 2018. Given the widespread availability of generative The framework guiding IEG’s strategic approach to AI isdriven by a vision to thoughtfully embed AI at the core ofits practice and set a global benchmark for producing andbrokering high-quality, AI-enabled evaluative evidence.The strategy defines two objectives drawn from this vision, as Defining ArtificialIntelligence AI refers to computational methods thatlearn patterns from data or representknowledge in ways that enable systemsto perform tasks that typically requirehuman cognitive capabilities—such as understanding, generating, classifying,predicting, or making recommendations—without being explicitly programmed for To determine how to move forward, a stocktaking of effortsin each area of the framework has been conducted, followedby a comparison with the desired future state to identify Table 1.1. STEPS ON THISJOURNEY As we continue this journey of discovery with theuse of AI in IEG evaluation, we are picturing wherewe would like to be in three years and delineating 2. WHERE WE WANT TO GO Vision Statement By thoughtfully embedding AI at the core of itsevaluation practice, IEG will set a global benchmark Key Values and This scale-up effort is anchored in several coreconcepts (values) and guiding principles that will Rethinking Evaluatorsand Evaluation Maturity Four interconnected transformations underpin therethinking of the evaluator’s role and the workflows As for any voyage of discovery, having a wayto determine whether we are on track andtraveling at the right pace is important.To help us self-evaluate and perform somehealth checks on our strategy implementation,we have adopted a maturity framework withfive levels. Although adoption and practicesvary across IEG, the goal is to move toward aninstitutionalized use in the next three to fiveyears. Informed by the stocktaking exerciseconducted as a preliminary step of this •Transforming the evaluation cycle. IEG will experiment with and scale successfulAI applications across all stages of the evaluationcycle—from design and data collection to analysis, •Transforming how we work. AI is expected to reshape nearly every aspect ofevaluation teamwork. Evaluators will integrate theirexpertise with new tools, workflows, and mindsets •Transforming evaluative products. Embedding AI into evaluation practice is expectedto r