调查模式对数据质量的影响：来自尼日利亚的实验证据

Policy Research Working Paper The Effect of Survey Mode on Data Quality Experimental Evidence from Nigeria Yannick MarkhofPhilip WollburgAmparo Palacios-LopezPauline CastaingAkiko Sagesaka A verified reproducibility package for this paper isavailable athttp://reproducibility.worldbank.org,clickherefor direct access. Policy Research Working Paper11302 Abstract This paper uses a large-scale experiment in rural Nigeriato study the role of survey mode—in-person versus overthe phone—in survey measurement and data quality.The experimental design isolates mode effects from othercommon sources of errors in surveys and covers 20 out-come measures across topics such as health, labor, shocks,wellbeing, and food security. The findings indicate consis- without phones. A within-respondent design enables cap-turing the full, respondent-level distribution of mode effectsand finds them to vary much more than the averages reveal.Respondents with higher education levels are less proneto mode effects, whereas mode effects sharply increase inprevalence as respondents face more answer options. As thereliance on phone surveys in low- and middle-income coun- This paper is a product of the Development Data Group, Development Economics. It is part of a larger effort by theWorld Bank to provide open access to its research and make a contribution to development policy discussions aroundthe world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The authorsmay be contacted at pwollburg@worldbank.org. A verified reproducibility package for this paper is available athttp:// The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about developmentissues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry thenames of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those The Effect of Survey Mode on Data Quality: Experimental Evidence fromNigeria Yannick Markhof1ⓡPhilip WollburgⓡAmparo Palacios-LopezⓡPauline CastaingⓡAkikoSagesakaⓡIvette Contreras Keywords:Data quality, Survey data, Phone survey, Measurement error, Survey modeJEL codes:C81, C83, O12 1Introduction High-quality socioeconomic data is fundamental for advancing research and for guiding, targeting, andmonitoring humanitarian and development interventions. In low- and middle-income countries (LMICs),surveys remain a critical source of representative data and a key tool for “ground truthing” socioeconomicoutcomes. In recent years, phone surveys have become an increasingly important mode of data collectionin these settings, particularly following the COVID-19 pandemic, which temporarily forced a shift from in-person to remote interviews (Gourlay et al., 2021a; Zezza et al., 2023). Their appeal is clear: phone surveysare less costly than traditional face-to-face surveys and can be implemented quickly and flexibly to providetimely and higher-frequency data. As such, phone surveys offer a pragmatic response to two converging Yet, as phone surveys have proliferated, concerns about their data quality remain. Like all data sources,phone (and in-person) surveys are subject to errors from multiple sources, but the survey mode itself isincreasingly recognized as an important source of measurement error and bias. Despite growing evidenceon phone survey data quality, important gaps remain in understanding the magnitude, direction, and drivers Literature on mode effects is an active field in its nascency, with limited and somewhat inconclusiveexperimental evidence . Mode effects have been documented for outcomes such as agricultural yields andproduction(Anderson et al., 2024; Kilic et al., 2021), consumption (Abate et al., 2023), microenterprisedata(Garlick et al., 2020), and contraceptive use (Greenleaf et al., 2020), while other outcomes such asdietary diversity and health appear less sensitive to survey mode (Abate et al., 2023; Lamanna et al., 2019;Markhof et al., 2025). Findings on direction vary: for example, Abate et al. (2023) report lower consumptionand higher poverty in phone interviews, whereas Greenleaf et al. (2020) find overreporting of contraceptive This study provides experimental evidence on survey mode effects based on a large-scale, multi-mode studyin rural Nigeria. As part of the experiment, we distributed phones to 937 households in randomly selectedrural villages in Nigeria and interviewed them both in person and by phone, using identical questions andrandomizing the order of interview modes. This design enables two complementary strategies to identifymode effects: a between-respondent design that compares first interviews across randomized phone and in- distinguishing them from other confounders and sources of errors including coverage errors,2selection effects,3timing effects,4and o