Decision-Centered Evaluation of Machine Learning Poverty Maps Using Mobile Phone and Satellite Data

Abstract

Identifying the poorest communities is essential for poverty alleviation, but household surveys and censuses are costly and infrequent. Machine learning offers alternative poverty estimates from mobile phone and satellite data, yet average prediction accuracy alone does not show whether maps support targeting under limited budgets. We apply a decision-centered evaluation to 13,985 Grama Niladhari divisions in Sri Lanka, combining call detail records (CDRs), remote sensing (RS), and CNN-derived Landsat 8 embeddings. We assess recovery of the poorest administrative units, compare random and spatially grouped validation, and examine errors in socioeconomically atypical communities. Against a census-derived asset index (PC1), Random Forest achieves Recall@25\% of 0.830, compared with 0.450 for nighttime lights. Random splitting raises recall by 4.1 percentage points relative to Divisional Secretariat Division (DSD)-grouped holdouts. The combined model recovers 86\% of the 25 poorest DSDs by PC1, versus 69\% for RS-only and 67\% for CDR-only models. Spatially isolated divisions have 10.8\% higher prediction error. Stronger isolation--error association in RS-only models is consistent with spatial smoothing, although its cause remains unconfirmed. These findings support evaluating poverty maps by targeting performance and geographic transfer, while recognising that agreement with an asset index does not establish consumption-poverty accuracy.