Predictive model selection in partial least squares path modeling (PLS-PM)

Abstract

Predictive model selection metrics are used to select models with the highest out-of-sample predictive power among a set of models. R2 and related metrics, which are heavily used in partial least squares path modeling, are often mistaken as predictive metrics. We introduce information theoretic model selection criteria that are designed for out-of-sample prediction and which do not require creating a holdout sample. Using a Monte Carlo study, we compare the performance of frequently used model evaluation criteria and information theoretic criteria in selecting the best predictive model under various conditions of sample size, effect size, loading patterns, and data distribution.