A Standardization Method of Individual Rating Fluctuation in Social Listening Data

Abstract

In recent years, social listening data analysis is adopted in various business areas as a measure to analyze customers’ voices. While sentiment information systems in social listening data analysis have been highly developed, their verification data requires sentiment score by humans. Sentiment means emotions such as Joy, anger and so on. Sentiment score by humans, however, depends on individuals and is not consistent enough. Thus, the accuracy of evaluation is uncertain. If verification data has uncertainty, it is apparent that uncertainty also occurs in the verification of systems. It is therefore important to exclude the uncertainty of verification data as much as possible. In this paper, we studied a standardization method for reducing such uncertainty and its effect.We set three choices, “Positive,” “Negative,” and “Not Sure,” for the verification data used in this study, and sentiment score evaluated the data as social listening data. We first showed an example of uncertainty in the verification data. We then proposed a correction method for reducing uncertainty in the verification data. The method adopted correction formulas using the average sentiment score of all the subjects as the norm. We used these formulas to suppress the bias of sentiment score of each individual subject.Consequently, compared with the result processed by simple decision by a majority, the frequency of “Not Sure” and the amount of mixed data of different sentiment scores decreased. Namely the method was effective in reducing the discarded data of indecisive sentiment score. Furthermore, the variance of data decreased consequently. These results showed that the standardization method was effective in reducing uncertainty in verification data.