Experimental research and multivariate statistical analyses in business information systems research - a meta-review

Abstract

Experimental research has been widely used within the information systems IS field. Using a meta-review, this study examines the use of experimental research in the IS field during the period 1990 to 2013 and investigates the utilisation of statistical techniques, particularly multivariate analyses. This study provides important information regarding the pattern of experimental research in the IS field i.e., experimental designs, treatments, tasks, incentives and participants. This study finds that multivariate analysis of variance MANOVA is frequently used as the multivariate statistical technique in experimental IS research. While the use of multivariate analyses is common in recent experimental IS research, concerns remain about the potential inappropriate use of measurements and violations of statistical assumptions required by the various multivariate analyses, for example, using ordinal scales rather than interval scales and issues surrounding the normality of data. This article is intended to offer guidelines to help future researchers be aware of the statistical issues and to mitigate the shortcomings that arise due to inappropriate use of measurements and violations of the statistical assumptions in experimental IS research.