Integrating Machine Learning and Human Judgment: A Study on Demand Planning in the Field

Abstract

While firms use automated machine learning algorithms in their demand planning processes, human judgment continues to feature in these processes. This research examines two methods of integrating machine learning and human judgment into demand planning. We implement a field study at a large, multinational firm testing interactive machine learning (IML) in which humans estimate adjustment quantities due to special events, and human-guided machine learning (HGML) in which humans merely input information about the special events. Analyzing the results of over three million data-points across five product categories reveals that demand forecasts using IML and HGML are more accurate than the current demand planning process used in the firm. These findings suggest that using an appropriate process to integrate machine learning and human judgment—IML and HGML—provides a significant benefit to demand planning. Furthermore, the paper provides insights on how to implement IML and HGML in practice.