AI News, Stanford scientists combine satellite data, machine learning to map poverty
Stanford scientists combine satellite data, machine learning to map poverty
“We have a limited number of surveys conducted in scattered villages across the African continent, but otherwise we have very little local-level information on poverty,” said study co-author Marshall Burke, an assistant professor of Earth system science at Stanford and a fellow at the Center on Food Security and the Environment.
“At the same time, we collect all sorts of other data in these areas – like satellite imagery – constantly.” The researchers sought to understand whether high-resolution satellite imagery – an unconventional but readily available data source – could inform estimates of where impoverished people live.
“This makes it hard to extract useful information from the huge amount of daytime satellite imagery that’s available.” Because areas that are brighter at night are usually more developed, the solution involved combining high-resolution daytime imagery with images of Earth at night.
“And since it’s cheap and scalable – requiring only satellite images – it could be used to map poverty around the world in a very low-cost way.” Co-authors of the study, titled “Combining satellite imagery and machine learning to predict poverty”, include Michael Xie from Stanford’s Department of Computer Science and David Lobell and W.
Scientists combine satellite data, machine learning to map poverty
In the current issue of Science, Stanford researchers propose an accurate way to identify poverty in areas previously void of valuable survey information.
'We have a limited number of surveys conducted in scattered villages across the African continent, but otherwise we have very little local-level information on poverty,' said study coauthor Marshall Burke, an assistant professor of Earth system science at Stanford and a fellow at the Center on Food Security and the Environment.
The researchers sought to understand whether high-resolution satellite imagery -- an unconventional but readily available data source -- could inform estimates of where impoverished people live.
'There are few places in the world where we can tell the computer with certainty whether the people living there are rich or poor,' said study lead author Neal Jean, a doctoral student in computer science at Stanford's School of Engineering.
Combining satellite imagery and machine learning to predict poverty
Our transfer learning model is strongly predictive of both average household consumption expenditure and asset wealth as measured at the cluster level across multiple African countries.
Cross-validated predictions based on models trained separately for each country explain 37 to 55% of the variation in average household consumption across four countries for which recent survey data are available (Fig.
Models trained on pooled consumption or asset observations across all countries (hereafter “pooled model”) perform similarly, with cross-validated predictions explaining 44 to 59% of the overall variation in these outcomes (fig.
This high overall predictive power is achieved despite a lack of temporal labels for the daytime imagery (i.e., the exact date of each image is unknown), as well as imperfect knowledge of the location of the clusters, as up to 10 km of random noise was added to cluster coordinates by the data collection agencies to protect the privacy of survey respondents.
We find that differences in the outcome being measured, rather than differences in survey design or direct identification of key assets in daytime imagery, likely explain these performance differences (see supplementary materials 2.1 and fig.
Finally, asset-estimation performance of our model in Rwanda surpasses performance in a recent study using cell phone data to estimate identical outcomes (11) (cluster-level r2 = 0.62 in that study, and r2 = 0.75 in our study;
To test whether our transfer learning model improves upon the direct use of nightlights to estimate livelihoods, we ran 100 trials of 10-fold cross-validation separately for each country and for the pooled model, each time comparing the predictive power of our transfer learning model to that of nightlights alone.
(C) Comparison of r2 of models trained on correctly assigned images in each country (vertical lines) to the distribution of r2 values obtained from trials in which the model was trained on randomly shuffled images (1000 trials per country).
4, C and D, the r2 values obtained using “correct” daytime imagery are much higher than any of the r2 values obtained from the reshuffled images, for both consumption and assets, indicating that our model’s level of predictive performance is unlikely to have arisen by chance.
Examining whether a particular model generalizes across borders is useful for understanding whether accurate predictions can be made from imagery alone in areas with no survey data—an important practical concern given the paucity of existing survey data in many African countries (see Fig.
These results indicate that, at least for our sample of countries, common determinants of livelihoods are revealed in imagery, and these commonalities can be leveraged to estimate consumption and asset outcomes with reasonable accuracy in countries where survey outcomes are unobserved.
Using Machine Learning to Map Poverty from Satellite Imagery
From water levels to population density, suburban sprawl to the species of trees growing in individual forests around the world, satellite imagery can map a nearly unlimited amount of data.
By using satellite imagery the amount of work on the ground is reduced, which also reduces costs and risk factors associated with working in poverty-stricken parts of the world.
The same picture can be used for a nearly unlimited number of purposes by any organization with access to the satellite images and the ability to analyze the data contained within.
these indicators include access to water, location in comparison to an urban center and food sources, and if agriculture is viable part of the economy nearby.
People looking at thousands of images would take a long time, and people may miss some of the important big picture data when they were looking at such small images.
- On Monday, September 23, 2019
Neal Jean, " "Combining satellite imagery and machine learning to predict poverty"
Neal Jean, Michael Xie, Stefano Ermon, Matt Davis, Marshall Burke, David Lobell "Combining satellite imagery and machine learning to predict poverty" ...
Fighting Poverty with Data
Speaker: Josh Blumenstock, Assistant Professor, School of Information Full Title: Fighting Poverty with Data: Research at the Intersection of Machine Learning ...
What is poverty mapping? by Nikos Tzavidis
The historical origins of poverty research and the more recent interest in estimating poverty at disaggregated geographical levels. Although the aim is not to ...
Processing a Petabyte of Planetary Pixels with Python | SciPy 2016 | Samuel Skillman
In the last 40 years over a petabyte of publicly available earth observation imagery has been produced. In the near future, many petabytes of imagery per year ...
Predicting Poverty with Deep Learning, Neal Jean, PhD Student, Stanford at AAI16
Predicting Poverty with Deep Learning, Neal Jean, PhD Student, Stanford University at Applied AI Conference 2016 by BootstrapLabs Neal Jean, a PhD student ...
How Analytics can Play a Major Role in Policy Making | Predictive Analytics using Satellite Imagery
Learn More: With the help of a sample satellite imagery, Dr.Bappaditya Mukhopadhyay, one of the Top 10 Analytics Academicians in India ..
CVPR18: Tutorial: Part 4: Generative Adversarial Networks
Organizers: Jun-Yan Zhu Taesung Park Mihaela Rosca Phillip Isola Ian Goodfellow. Description: Generative adversarial networks (GANs) have been at the ...
Stefano Ermon Interview - Domain Knowledge in Machine Learning Models for Sustainability
My guest this week is Stefano Ermon, Assistant Professor of Computer Science at Stanford University, and Fellow at Stanford's Woods Institute for the ...
Zia Ahmed: Application of machine learning in agricultural research ...
Application of machine learning in agricultural research: Precision nutrient management, digital soil mapping and yield prediction Zia Ahmed, Soil and Crop ...
A Python API for Earth | SciPy 2017 | Sam Skillman
At Descartes Labs, we use the commercial cloud to run global-scale machine learning applications over satellite imagery. Our core software stack builds upon ...