When Managers Meet Models: Integrating Human Judgment and Analytics

Abstract

Data analytics and machine learning are increasingly prevalent in emerging forecasting practice. Despite the growth in model-based forecasting, practitioners continue to employ human judgment to incorporate contextual information to improve the accuracy of model forecasts. Our research uses an experiment to examine how human judgment and statistical models may be best integrated to improve forecast accuracy within heterogeneous forecasting environments. Our findings suggest that human judgment provides a significant benefit to forecasting. Specifically, integrated forecasts (i.e., forecasts that combine human judgment with computational analytics) can substantially improve forecast accuracy compared to non-integrated forecasts. We find however that this improvement in accuracy is dependent on the method of integration. Human guided machine learning is the most effective method of integration in comparison to other methods commonly used in practice and studied in the academic literature. In keeping with theory, we call for further empirical testing of forecasting methods that leverage the strengths of human judgment and the strengths of models, and urge study of implementation issues in practice.