Avoiding Algorithm Error in Computer-Aided Text Analysis: Selecting Text Analysis Software

Abstract

The use of computer-aided text analysis as a tool to quantify qualitative information continues to grow in management research. Academia increasingly develops guiding principles that aid scholars in addressing reliability and validity issues. However, there is little guidance in selecting a proper software solution that aligns the features of the software with the features of the underlying construct. Further guidelines are particularly relevant because computer-aided text analysis is highly software dependent. This paper demonstrates a systematic software selection process using two illustrative constructs: a theoretically complex construct (value- based management sophistication) and a more straightforward construct (firm’s future orientation). The paper develops generalizable principles for selecting proper software solutions. After reviewing relevant literature, this paper compares the algorithmic proximity of frequently applied software solutions in management research, performing analysis of variance and convergent validity analysis. The results demonstrate that the algorithmic performance depends on the construct type. The analysis of variance indicates statistically significant differences in mean values between various software solutions. However, these differences do not lead to substantial deviations in convergent validity results between the applied software solutions. This paper contributes to the literature by assisting scholars in choosing a proper software solution when conducting computer-aided text analysis.