A Simulation Study Comparing Knot Selection Methods With Equally Spaced Knots in a Penalized Regression Spline

Nehemias Ulloa,Victoria Miller,Eduardo L. Montoya

doi:10.5539/ijsp.v3n3p96

Nehemias Ulloa, Victoria Miller + Show 1 more

Open Access

https://doi.org/10.5539/ijsp.v3n3p96

Copy DOI

Abstract

Penalized regression splines are a commonly used method to estimate complex non-linear relationships between two variables. The fit of a penalized regression spline to the data depends on the number of knots, knot placement, and the value of the smoothing parameter. In this paper, we use a simulation study to compare knot selection methods with equidistant knots in a penalized regression spline model. We found that one method generally performed better than others. The results provide guidance in selecting the number of equidistant knots in a penalized regression spline model.

Highlights

IntroductionUsing a penalized regression spline, Ruppert (2002) compares the performance of two different algorithms (Myopic and Full Search) to select the number of knots when placing the knots at quantiles of the xi’s with generalized cross validation for smoothing parameter selection and found that both algorithms perform well
Consider the general regression model with a single explanatory variable that takes the form yi = g(xi) + εi, for i = 1, ..., n, (1)where xi ∈ [a, b], yi is a response variable, xi is a covariate, g(x) is the regression function dependent on the covariate, n is the number of observations, and εi i∼id N(0, σ2) for all i
The cubic penalized regression spline estimate obtained by minimizing fitting criterion (2) is dependent on the value of λ, K, and the location of the knots

Summary

Introduction

Using a penalized regression spline, Ruppert (2002) compares the performance of two different algorithms (Myopic and Full Search) to select the number of knots when placing the knots at quantiles of the xi’s with generalized cross validation for smoothing parameter selection and found that both algorithms perform well. We build upon the analysis of Ruppert (2002) and Ruppert et al (2003) in regards to selecting the number of knots in a penalized regression spline; a notable difference is that we use spaced knots and several smoothing parameter selection methods under various simulation settings.

A Regression Spline With a Truncated Power Basis

Selecting the Smoothing Parameter

Knot Selection Methods

Myopic Algorithm

Full Search Algorithm

A Simulation Study

Conclusions