A comparative study of the reliability of nine statistical software packages

Abstract

The reliabilities of nine software packages commonly used in performing statistical analysis are assessed and compared. The (American) National Institute of Standards and Technology (NIST) data sets are used to evaluate the performance of these software packages with regard to univariate summary statistics, one-way ANOVA, linear regression, and nonlinear regression. Previous research has examined various versions of these software packages using the NIST data sets, but typically with fewer software packages than used in this study. This study provides insight into a relative comparison of a wide variety of software packages including two free statistical software packages, basic and advanced statistical software packages, and the popular Excel package. Substantive improvements from previous software reliability assessments are noted. Plots of principal components of a measure of the correct number of significant digits reveal how these packages tend to cluster for ANOVA and nonlinear regression.