Refining Sentiment Predictions: Obtaining an Unbiased Business Sentiment Index from Japanese Newspapers

Abstract

Economic indices, such as the consumer price index, gross domestic product, and purchasing managers’ index, are typically computed based on economic data (e.g., prices of goods and incomes) and surveys. Recently, there has been considerable research to predict these indices using alternative data. One such approach is to utilize textual data, specifically newspaper articles, to predict a business sentiment index (BSI). However, newspapers often exhibit biases, which may impact the resulting index. This potential issue has not received adequate attention. Thus, this work first investigates whether these biases do influence the predicted indices and then proposes an approach to obtaining an unbiased business sentiment index. For this purpose, we analyze multiple Japanese newspapers, including one financial and three general papers with varied political orientations. We then predict a monthly BSI for each newspaper based on its content using a Transformer-based sentiment analysis model. To achieve an unbiased BSI, we employ the dynamic factor model. This model decomposes the time series of BSIs into a common dynamic factor shared across different newspapers and specific dynamic factors unique to individual newspapers. Our experiment indicates that the magnitudes of the predicted BSIs and the specific dynamic factors closely align with the political stances of the newspapers. Furthermore, the common dynamic factor exhibits a stronger and more positive correlation with composite indices and a survey-based BSI, namely the Tankan index.