Ridge regression with adaptive additive rectangles and other piecewise functional templates

Abstract

We propose a penalization algorithm for functional linear regression models, where the coefficient function β is shrunk towards a data-driven shape template γ. To the best of our knowledge, we employ the nonzero centered L2 penalty in a novel manner, as the center of the penalty γ is also optimized while being constrained to belong to a class of piecewise functions Γ, by restricting its basis expansion. This indirect penalization allows the user to control the overall shape of β, by imposing his prior knowledge on γ through the definition of Γ, without limiting the flexibility of the estimated model. In particular, we focus on the case where γ is expressed as a sum of q rectangles that are adaptively positioned with respect to the regression error. As the problem of finding the optimal knot placement of a piecewise function is nonconvex, we also propose a novel parametrization that allows to reduce the number of variables in the global optimization scheme, resulting in a fitting algorithm that alternates between approximating a suitable template and solving a convex ridge-like problem. The predictive power and interpretability of our method is shown on multiple simulations and two real world case studies.