Numerical data quality in IS research and the implications for replication

Abstract

We argue that there are major, persistent numerical data quality issues in IS academic research. These issues undermine the ability to replicate our research – a critical element of scientific investigation and analysis. In IS empirical and analytics research articles, the amount of space devoted to the details of data collection, validation, and/or quality pales in comparison to the space devoted to the evaluation and selection of relatively sophisticated model form(s) and estimation technique(s). Yet erudite modeling and estimation can yield no immediate value or be meaningfully replicated without high quality data inputs. The purpose of this paper is: 1) to detail potential quality issues with data types currently used in IS research, and 2) to start a wider and deeper discussion of data quality in IS research. No data type is inherently of low quality and no data type guarantees high quality. As researchers, our empirical research must always address data quality issues and provide the information necessary to determine What, When, Where, How, Who, and Which.