Enveloping Intelligent Tools: Algorithmic Accountability in Academic Research

Abstract

Scholars in management, organizations, and information systems disciplines have been increasingly relying on various machine learning-based techniques to analyze large amounts of unstructured textual data. While reliance on such intelligence tools may help amplify the knowledge-producing capacities of researchers, it also poses challenges for the researchers in fostering algorithmic accountability as demonstrated by our systematic methodological review of papers using topic modeling in their studies in leading journals. Drawing on the literature on envelopment in robotics, we propose an envelopment framework to help researchers systematically think about fostering accountability while employing intelligent tools in research. Our framework conceptualizes research designs in the form of four categories of envelopes depending on their breadth and depth: The breadth of an envelope characterizes the alignment between the data and the phenomenon that the data are believed to represent, while the depth of envelope characterizes the role topic modeling plays in contributing to the chain of evidence in the study. Our core thesis is that the higher the volume of an envelope the more the choices the researchers need to account for while employing intelligent tools in research. Although, the exact nature of accountability expectations varies across research phases. Our paper contributes to the emerging interdisciplinary scholarship on intelligent tools and accountability in academic research in the digital age.