Predictive performance of psychological tests: Is it better to use items than subscales?

Abstract

Using psychological tests to predict outcomes involves generating a prediction rule from these tests. For multidimensional tests, the standard approach to generate a prediction rule is to use the subscale scores of the test as predictor variables in a regression model to estimate an outcome value for each individual. The coefficients in this model are estimated with ordinary least squares and the predictive performance of the rule is estimated out-of-sample. Recently, studies used the separate items as predictors and estimated the regression coefficients with statistical learning methods to improve the predictive performance of these tests. However, it is unclear whether this approach is always beneficial. The aim is to identify factors that influence the decision whether to use items or subscales in a prediction rule, or letting the data decide between these two types of rules. Several statistical methods are used to derive the prediction rules: ordinary least squares, factor score regression, elastic net, supervised principal components, and principal covariates regression. Data from two empirical studies is analyzed and a simulation study is performed. Overall, results showed that, contrary to earlier findings, item rules are not always better than subscale rules. Subscale rules from elastic net often performed best.