Prediction, Machine Learning, and Individual Lives: an Interview with Matthew Salganik

Matthew Salganik,Cynthia Rudin,Lauren Maffeo

doi:10.1162/99608f92.eecdfa4e

Abstract

Machine learning techniques are increasingly used throughout society to predict individual’s life outcomes. However, research published in the Proceedings of the National Academy of Sciences raises questions about the accuracy of these predictions. Led by researchers at Princeton University, this mass collaboration involved 160 teams of data and social scientists building statistical and machine learning models to predict six life outcomes for children, parents, and families. They found that none of the teams could make very accurate predictions, despite using advanced techniques and having access to a rich dataset. This interview of Matthew Salganik, the study’s lead author and a professor of Sociology at Princeton University, was conducted by Lauren Maffeo, Associate Principal Analyst at Gartner, and Cynthia Rudin, a professor of Computer Science, Electrical and Computer Engineering, and Statistical Science at Duke University. It provides an overview of the study’s goals, research methods, and results. The interview also includes key takeaways for policy leaders who wish to use machine learning to predict and improve life outcomes for people.

Highlights

Machine learning techniques are increasingly used throughout society to predict individual’s life outcomes
Lauren Maffeo (LM): Matthew, what is the Fragile Families Challenge and what were you trying to accomplish? Matthew Salganik (MS): The Fragile Families Challenge is a scientific mass collaboration designed to answer one question: how predictable are life trajectories? That is, given some data about a person, how accurately can we predict what will happen to that person in the future? Cynthia Rudin (CR): How did you attempt to quantify that? MS: We measured the predictability of life outcomes by combining a high-quality dataset from the social sciences, a research design from machine learning, and 457 researchers from around the world (Salganik et al, 2020a)
We found that the most accurate models were only slightly better than a simple, four-variable regression model, where the four variables were chosen by a domain expert

Summary

Introduction

Machine learning techniques are increasingly used throughout society to predict individual’s life outcomes. Led by researchers at Princeton University, this mass collaboration involved 160 teams of data and social scientists building statistical and machine learning models to predict six life outcomes for children, parents, and families. MS: We measured the predictability of life outcomes by combining a high-quality dataset from the social sciences, a research design from machine learning, and 457 researchers from around the world (Salganik et al, 2020a).

Results

Conclusion