Interpretable Machine Learning Models for Clinical Decision-Making in a High-Need, Value-Based Primary Care Setting

Abstract

Summary While health care organizations are increasingly interested in using artificial intelligence (AI), there is a significant lack of literature on the real-world application of a risk stratification AI tool in primary care. Oak Street Health, a network of more than 80 primary care centers in medically underserved communities, successfully implemented a machine learning–based risk stratification tool across their organization that outperformed prior backward-looking approaches in identifying high-risk patients. The data science team collaborated with an interdisciplinary set of stakeholders to test, iterate, and implement the tool into clinical practice. Early feedback from Oak Street Health’s primary care providers (physicians and nurse practitioners) and nonproviders (social workers) suggests that the display of top risk factors based on model predictions created a broadly interpretable and actionable risk stratification tool in caring for the highest-risk patients.