Deterministic bridge regression for compressive classification

Abstract

Pattern classification using a compact representation is a crucial component of machine intelligence. Specifically, it is essential to learn a model with well-regulated parameters to achieve good generalization. Bridge regression provides a mechanism for regulating parameters through a penalized ℓp-norm. However, due to the nonlinear nature of the formulation, an iterative numerical search is typically used to solve the optimization problem. In this work, we propose an analytic solution for bridge regression based on solving a penalized error formulation using an approximated ℓp-norm. The solution is presented in primal form for over-determined systems and in dual form for under-determined systems. The primal form is suitable for low-dimensional problems with a large number of data samples, while the dual form is suitable for high-dimensional problems with a small number of data samples. We also extend the solution to problems with multiple classification outputs. Numerical studies using simulated and real-world data demonstrate the effectiveness of our proposed solution.