Adaptive Bottom-Up Space Exploration in model population analysis: An agile variable selection algorithm for PLS models

Abstract

All variable selection algorithms for partial least squares (PLS) regression models based on model population analysis, including variable iterative space shrinkage approach (VISSA), take an iterative top-down space shrinkage approach. The time efficiency of this unidirectional VISSA is not promising as it drains much of the valuable time while evaluating sub-models of irrelevant size while shrinking variable space in a step-wise manner. In this work, two variants of Adaptive Bottom-Up Space Exploration (ABUSE) approach have been proposed. Both variants of ABUSE, based on VISSA framework, adopts a low weight (e.g. 0.005) of variable selection to formulate short length sub-model populations. When average fitness of sub-model population stops improving in reweighted sampling, the weight of variables appearing in “best-fit” sub-model is increased to 0.5, while maintaining others (but not expunged like VISSA) at the same low selection frequency for next round of iteration. The first algorithmic variant enforces the weight vector manipulation once in a binary matrix sampling, while the other enforces this weight manipulation in every cycle of weighted binary matrix sampling. The proposed methods offered better fitness, outcome stability and algorithmic efficiency particularly for large benchmark NIR data sets. Choice of variable weight is critical as at higher weight, though still better than VISSA, the algorithm tends to project more variables with a deterioration of model fitness.