We discuss the use of smoothing splines (SS) and least squares splines (LSS) in nonparametric regression on geomagnetic data. The distinction between smoothing splines and least squares splines is outlined, and it is suggested that in most cases the smoothing spline is, a preferable function estimate. However, when large data sets are involved, the smoothing spline may require a prohibitive amount of computation; the alternative often put forward when moderate or heavy smoothing is -desired is the least squares spline. This may not be capable of modeling the data adequately since the smoothness of the resulting function can be controlled only by the number and position of the knots. The computational efficiency of the least squares spline may be retained and its principal disadvantage overcome, by adding a penalty term in the square of the second derivative to the minimized functional. We call this modified form a penalized least squares spline, (denoted by PS throughout this work), and illustrate its use in the removal of secular trends in long observatory records of geomagnetic field components. We may compare the effects of smoothing splines, least squares splines, and penalized least squares splines by treating them as equivalent variable-kernel smoothers. As Silverman has shown, the kernel associated with the smoothing spline is symmetric and is highly localized with small negative sidelobes. The kernel for the least squares spline with the same fit to the data has large oscillatory sidelobes that extend far from the central region; it can be asymmetric even in the middle of the interval. For large numbers of data the penalized least squares spline can achieve essentially identical performance to that of a smoothing spline, but at a greatly reduced computational cost. The penalized spline estimation technique has potential widespread applicability in the analysis of geomagnetic and paleomagnetic data. It may be used for the removal of long term trends in data, when either the trend or the residual is of interest.