Topological techniques in model selection

Abstract

The LASSO is an attractive regularisation method for linear regression that combines variable selection with an efficient computation procedure. This paper is concerned with enhancing the performance of LASSO for square-free hierarchical polynomial models when combining validation error with a measure of model complexity. The measure of the complexity is the sum of Betti numbers of the model which is seen as a simplicial complex, and we describe the model in terms of components and cycles, borrowing from recent developments in computational topology. We study and propose an algorithm which combines statistical and topological criteria. This compound criterion would allow us to deal with model selection problems in polynomial regression models containing higher-order interactions. Simulation results demonstrate that the compound criteria produce sparser models with lower prediction errors than the estimators of several other statistical methods for higher order interaction models.