Staffing to Support Evidence Culture and Capacity in the Federal Government

EVIDENCECOLLABORATIVE.ORG POLICY BRIEF Staffing to Support Evidence Culture and Capacity in the Federal Government Molly M. Scott Wilton Oliver Urban Institute Urban Institute October 2018 2 EVIDENCECOLLABORATIVE.ORG | EVIDENCE- BASED POLICYMAKING COLLABORATIVE Introduction Policymakers and administrators in many federal agencies are increasingly interested in how they can best ensure the quality and value of federal programs and services. Congressional establishment of the Commission on Evidence-Based Policymaking was part of this growing interest. The bipartisan commission’s report, released in September 2017, included 22 recommendations for improving access to data, strengthening privacy protections, and improving agency evidence-building capacity (Commission on Evidence-Based Policymaking 2017). In addition to the commission report, the White House budget documents presenting the administration’s proposals and priorities for fiscal years 2018 and 2019, and the 2018 President’s Management Agenda, include chapters on evidence (OMB 2017, OMB 2018a, OMB 2018b). The 2019 budget is subtitled “Efficient, Effective, Accountable”—terms that underlie evidence-based policymaking. President Trump’s 2018 reform plan1 echoes these sentiments and specifically stresses the need for learning agendas in federal agencies. The commission’s recommendations and the White House budget and management documents are motivating some agencies to strengthen their existing evidence capacity. But there is no single process for accomplishing these goals. Every federal agency has its own experience with and resources for research, evaluation, and analysis; institutional culture regarding evidence; and staff analytical skills and capacity. Some agencies have a long history of sponsoring and using evaluations and dedicating funding to do so; others have only minimal analytic experience and few resources. Building a culture of evidence is often driven by department or cabinet leadership and statutory provisions. However, efforts to build or strengthen a culture of evidence can also occur within a single office or subagency. Throughout this brief, the term “federal agencies” is used generically to include both departmental and subagency staff charged with building evidence. This brief provides guidance to chief evaluation officers2 and others leading evidence capacity development within federal agencies on how agencies with different resources might structure and staff their supporting functions. We begin with a short section to define what we mean by evidence culture and capacity. We then describe the different stages of the evidence process before outlining the key competencies needed to support it. Last, we offer different examples of how existing federal agencies have chosen to staff and structure their evidence functions. Definitions Evidence Culture Organizational culture is a somewhat abstract concept that generally refers to shared beliefs, viewpoints, or norms that reflect and define how procedures and activities are carried out. The fiscal year 2018 budget EVIDENCE- BASED POLICYMAKING COLLABORATIVE | EVIDENCECOLLABORATIVE.ORG 3 included a discussion of “evidence culture” in terms of using information for continuous learning and effectiveness: With a strong evidence infrastructure and culture, agencies constantly (1) ask and answer questions that help them find, implement, and sustain effective programs and practices, (2) identify and improve or eliminate ineffective programs and practices, (3) test promising programs and practices to see if they are effective and can be replicated, and (4) find lower cost ways to achieve better results. (OMB 2017, 55) Thus, from a functional perspective, an agency can establish a culture of evidence by carrying out activities that support, produce, and use evidence, and by integrating it into agency procedures and operations. This culture should transcend political ideology and be sustained across presidencies. Evidence Capacity Agencies can develop the capacity to support a culture of evidence by establishing functional and procedural activities related to research, analysis, and evaluation. The evidence chapter in the White House 2019 budget, like the previous five years’ budget documents, encourages agencies to adopt institutional or infrastructural strategies to improve their evidence activities. More specifically, agencies are urged to  adhere to “evaluation principles and practices . . . [that] include rigor, relevance, independence, transparency and ethics”;  have “designated evaluation officials and offices”;  develop “multiyear learning agendas” to plan and focus evidence-building activities and priorities;  strengthen “interagency coordination” to build and use evidence;  leverage available “funding flexibilities and set-asides”;  “improv[e] data access and governance for evidence-building” by investing in improving the quality and appropriate integr