Large-scale Probabilistic Forecasting of Consumer Engagement of CPG Products using Heterogeneous Web Data

Abstract

Consumer Packaged Goods (CPG) play a pivotal role in customer-centric industries. Understanding the distinctive features of such products and how customers engage with them is essential for CPG manufacturers to create customer-winning products. In this study, we explore an innovative data-centric approach to forecast customer engagement for CPG products by monitoring their digital evolution, particularly in the Snacks category in the USA. Traditional methods for consumer analysis such as surveys, focus groups are time consuming, costly and ineffective with limited scope. They also lack a forecasting component, making it difficult for CPG companies to make forward-looking decisions. However, with the emergence of big data, we can leverage user generated public data from social media, web search, e-commerce platforms to estimate consumer engagement and make long-term forecasts. To achieve this, we propose a systematic approach to accumulate and prepare large datasets from heterogeneous web sources for CPG products. We then use state-of-the-art deep learning based time series forecasting models to efficiently train and predict consumer engagement for the next 12 months, benchmarking their computational efficiency and forecasting performance. Our findings indicate that the DeepAR model outperforms all other models, with the lowest NRMSE (=0.378), RMSE (=14.848) MSE (=220.457), MASE (=0.871), and sMAPE (=0.306) values. Furthermore, we demonstrate methods for computing single forecasting points and prediction intervals using the forecasted sample distribution from the probabilistic models. The proposed approach will provide CPG businesses with valuable insights to make informed decisions about product development, marketing strategies, and supply chain management.