An Approach to Sparse Model Selection and Averaging

Abstract

Parameter estimation when the true model structure is unknown is a commonly occurring task in measurement problems. In a sparse modeling scenario, the number of possible models grows exponentially with the total number of parameters. The full set of models therefore becomes computationally infeasible to handle. We propose a method, based on successive model reduction, for finding a sound and computationally feasible set of sparse linear regression models. Once this set of models has been found, standard model selection or model averaging techniques can be applied. We demonstrate the performance of our method by some numerical examples.