Smoothing Parameter Selection Research Articles

Generalized Cross Validation (GCV) in Smoothing Spline Nonparametric Regression Models

A nonparametric regression model is utilized if the the regression curve does not contain information about the accepted shape and accepted curve is exist in a function. If any curve is given without the limitation of a certain functional form, a rough and non-unique curve will result. The smoothing spline can be utilized to remove rough curves in some segments by following a curve pattern. An approach that combines nonparametric regression and smoothing spline is known as the smoothing spline nonparametric regression model. The problem in estimating is the selection and determination of smoothing parameters obtained by taking into the sum of knots used and the position of the knots so that the Generalized Cross-Validation (GCV) method is required. A study was conducted on the smoothing spline nonparametric regression model on GCV. The method used in research is a literature study obtained from some articles, journals, and books that support research achievement. The results showed that with the GCV method the minimum GCV value was obtained which would determine how well the smoothing parameters shown by the estimator did not change significantly even though the number and position of the knots varied.

Robust fitting for generalized additive models for location, scale and shape

The validity of estimation and smoothing parameter selection for the wide class of generalized additive models for location, scale and shape (GAMLSS) relies on the correct specification of a likelihood function. Deviations from such assumption are known to mislead any likelihood-based inference and can hinder penalization schemes meant to ensure some degree of smoothness for nonlinear effects. We propose a general approach to achieve robustness in fitting GAMLSSs by limiting the contribution of observations with low log-likelihood values. Robust selection of the smoothing parameters can be carried out either by minimizing information criteria that naturally arise from the robustified likelihood or via an extended Fellner–Schall method. The latter allows for automatic smoothing parameter selection and is particularly advantageous in applications with multiple smoothing parameters. We also address the challenge of tuning robust estimators for models with nonlinear effects by proposing a novel median downweighting proportion criterion. This enables a fair comparison with existing robust estimators for the special case of generalized additive models, where our estimator competes favorably. The overall good performance of our proposal is illustrated by further simulations in the GAMLSS setting and by an application to functional magnetic resonance brain imaging using bivariate smoothing splines.

On the Number of Independent Pieces of Information in a Functional Linear Model with a Scalar Response

In a functional linear model (FLM) with scalar response, the parameter curve quantifies the relationship between a functional explanatory variable and a scalar response. While these models can be ill-posed, a penalized regression spline approach may be used to obtain an estimate of the parameter curve. The penalized regression spline estimate will be dependent on the value of a smoothing parameter. However, the ability to obtain a reasonable parameter curve estimate is reliant on how much information is present in the covariate functions for estimating the parameter curve. We propose to quantify the information present in the covariate functions to estimate the parameter curve. In addition, we examine the influence of this information on the stability of the parameter curve estimator and on the performance of smoothing parameter selection methods in a FLM with a scalar response.

An asymptotic and empirical smoothing parameters selection method for smoothing spline ANOVA models in large samples.

Large samples are generated routinely from various sources. Classic statistical models, such as smoothing spline ANOVA models, are not well equipped to analyse such large samples because of high computational costs. In particular, the daunting computational cost of selecting smoothing parameters renders smoothing spline ANOVA models impractical. In this article, we develop an asympirical, i.e., asymptotic and empirical, smoothing parameters selection method for smoothing spline ANOVA models in large samples. The idea of our approach is to use asymptotic analysis to show that the optimal smoothing parameter is a polynomial function of the sample size and an unknown constant. The unknown constant is then estimated through empirical subsample extrapolation. The proposed method significantly reduces the computational burden of selecting smoothing parameters in high-dimensional and large samples. We show that smoothing parameters chosen by the proposed method tend to the optimal smoothing parameters that minimize a specific risk function. In addition, the estimator based on the proposed smoothing parameters achieves the optimal convergence rate. Extensive simulation studies demonstrate the numerical advantage of the proposed method over competing methods in terms of relative efficacy and running time. In an application to molecular dynamics data containing nearly one million observations, the proposed method has the best prediction performance.

Baseline drift estimation for air quality data using quantile trend filtering

We address the problem of estimating smoothly varying baseline trends in time series data. This problem arises in a wide range of fields, including chemistry, macroeconomics and medicine; however, our study is motivated by the analysis of data from low cost air quality sensors. Our methods extend the quantile trend filtering framework to enable the estimation of multiple quantile trends simultaneously while ensuring that the quantiles do not cross. To handle the computational challenge posed by very long time series, we propose a parallelizable alternating direction method of multipliers (ADMM) algorithm. The ADMM algorthim enables the estimation of trends in a piecewise manner, both reducing the computation time and extending the limits of the method to larger data sizes. We also address smoothing parameter selection and propose a modified criterion based on the extended Bayesian information criterion. Through simulation studies and our motivating application to low cost air quality sensor data, we demonstrate that our model provides better quantile trend estimates than existing methods and improves signal classification of low-cost air quality sensor output.

Kernel smoothed probability mass functions for ordered datatypes

We propose a kernel function for ordered categorical data that overcomes limitations present in ordered kernel functions appearing in the literature on the estimation of probability mass functions for multinomial ordered data. Some limitations arise from assumptions made about the support of the underlying random variable. Furthermore, many existing ordered kernel functions lack a particularly appealing property, namely the ability to deliver discrete uniform probability estimates for some value of the smoothing parameter. We propose an asymmetric empirical support kernel function that adapts to the data at hand and possesses certain desirable features. There are no difficulties arising from zero counts caused by gaps in the data while it encompasses both the empirical proportions and the discrete uniform probabilities at the lower and upper boundaries of the smoothing parameter. We propose likelihood and least-squares cross-validation for smoothing parameter selection and study their asymptotic and finite-sample behaviour.

Generalized Link-Based Additive Survival Models with Informative Censoring

Time to event data differ from other types of data because they are censored. Most of the related estimation techniques assume that the censoring mechanism is noninformative while in many applications it can actually be informative. The aim of this work is to introduce a class of flexible survival models which account for the information provided by the censoring times. The baseline functions are estimated non-parametrically by monotonic P-splines, whereas covariate effects are flexibly determined using additive predictors. Parameter estimation is reliably carried out within a penalized maximum likelihood framework with integrated automatic multiple smoothing parameter selection. We derive the -consistency and asymptotic normality of the noninformative and informative estimators, and shed light on the efficiency gains produced by the newly introduced informative estimator when compared to its non-informative counterpart. The finite sample properties of the estimators are investigated via a Monte Carlo simulation study which highlights the good empirical performance of the proposal. The modeling framework is illustrated on data about infants hospitalized for pneumonia. The models and methods discussed in the article have been implemented in the R package GJRM to allow for transparent and reproducible research. Supplementary materials for this article are available online.

Bootstrap bandwidth selection in time-varying coefficient models with jumps

The time-varying coefficient models (TVCMs) are very important tools to explore the hidden structure between the time series response variable and its predictors. The assumption that the coefficient functions are smooth can be restrictive in practice. The TVCMs with jumps are needed to be considered. Selection of smoothing parameters plays a critical role in assessing the performance of estimations of coefficient functions. In this article, we first develop a nonparametric two-step estimation procedure for estimating a finite number of jumps of the coefficient functions. Then, based on the estimated jumps, we propose a bootstrapping bandwidth selection procedure in the TVCMs with jumps. Monte Carlo simulations are conducted to evaluate the finite sample performance of the proposed data-driven estimation and bootstrap bandwidth selection procedures. As an illustration, the proposed methodologies are further applied to a real data example.

The effect of end-point adjustments on smoothing splines used for tree-ring standardization

The cubic smoothing spline has been a popular method for detrending tree-ring data since the 1980s. The common implementation of this procedure (e.g., ARSTAN, dplR) uses a unique method for determining the smoothing parameter that is widely known as the %n criterion. However, this smoothing parameter selection method carries the assumption that end point effects are ignorable. In this paper, we complete the mathematical derivation and show how the original method differs from the complete version, both in the interpretation of the smoothing parameter and in the spline fit. Frequency response curves (FRC) demonstrate how the smoothing parameter is affected by the original assumption. For example, the FRC results indicate that a tree core of 250-year length has a 14% difference in the cut-off frequency when looking at the 67%n criterion. The FRC analysis shows that the existing approach produces a more flexible fit than anticipated, i.e., it is removing more variance than previously thought. For example, a 67%n spline under the existing approach corresponds to a 53%n spline fit. By using both simulated tree-core sequences and a dataset from a Midwest forest, we discuss which conditions result in greater differences between the spline fits and which conditions will have small differences. Tree-core sequences that have more curvature, such as a large-amplitude growth release, will lead to greater differences. Finally, we provide approximations to the end-point effect procedure. For example, using an 83%n criterion under the original approach produces a spline fit approximating the 67%n fit under the complete approach. These approximations could be easily implemented within existing programs like ARSTAN.

A local maximum likelihood model of crop yield distributions

AbstractIn this note, we propose a local maximum likelihood estimator for spatially‐dependent distributions. Our estimator adopts the Poisson regression approach for density ratio models and incorporates spatial smoothing via local regression. We also present a method of smoothing parameter selection. We illustrate this easy‐to‐implement estimator with an application to the estimation of corn yield distributions of Iowa counties. The usefulness of the approach is further demonstrated via an application to the estimation of crop insurance premium.

Computation and application of robust data-driven bandwidth selection for gradient function estimation

The significance of gradient estimates in nonparametric regression cannot be neglected as it is a critical process for executing marginal effect in empirical application. However, the performance of resulting estimates is closely related to the selection of smoothing parameters. The existing methods of parameter choice are either too complicated or not robust enough. For improving the computational efficiency and robustness, a data-driven bandwidth selection procedure is proposed in this paper to compute the gradient of unknown function based on local linear composite quantile regression. Such bandwidth selection method can solve the difficulty of the infeasible selection program that requires the direct observation of true gradient. Moreover, the leading bias and variance of the estimated gradient are obtained under certain regular conditions. It is shown that the bandwidth selection method processes the oracle property in the sense that the selected bandwidth is asymptotically equivalent to the optimal bandwidth if the true gradient is known. Monte Carlo simulations and a real example are conducted to demonstrate the finite sample properties of the suggested method. Both simulation and application corroborate that our technique delivers more effective and robust derivative estimator than some existing approaches.

DoFORC tool for calculating first-order reversal curve diagrams of noisy scattered data

The first-order reversal curve (FORC) diagram method is one of the most successful characterization techniques used to characterize complex hysteretic phenomena not only in magnetism but also in other areas of science like in ferroelectricity, geology, archeology, in spin-transition materials, etc. Because the definition of the FORC diagram involves a second-order derivative, the main problem in their numerical calculation is that the derivative of a function for which only discrete noise-contaminated data values are available magnifies the noise that is inevitably present in measurements. In this paper, we present the doFORC tool for calculating FORC diagrams of noise scattered data. It can provide both a smooth approximation of the measured magnetization and all its partial derivatives. Even if doFORC is mainly dedicated to FORC diagrams’ computation, it can process a general set of arbitrarily distributed two-dimensional points. doFORC is a free, portable application working on various operating systems, with an easy to use graphical interface, with four regression methods implemented to obtain a smooth approximation of the data which may then be differentiated to obtain approximations for derivatives. In order to perform the diagnostics and goodness of fit, doFORC computes residuals to characterize the difference between the observed and predicted values, generalized cross-validation to measure the predictive performance, two information criteria to quantify the information that is lost by using an approximate model, and three degrees of freedom to compare different amounts of smoothing being performed by different smoothing methods. Based on these, doFORC can perform automatic smoothing parameter selection.

MapGAM: Mapping geographic disparities in health outcomes using individual-level epidemiologic data

Event Abstract Back to Event MapGAM: Mapping geographic disparities in health outcomes using individual-level epidemiologic data Veronica Vieira1* 1 University of California, Irvine, United States As location is often an important predictor of disease risk, epidemiological studies and disease registries now routinely collect residential histories. Growing public health concerns of geographic disparities in health outcomes has prompted the need for flexible approaches to investigate the role of risk factors in spatial variation. MapGAM is a user-friendly R package that provides researchers and practitioners with a unified methodology for spatially analyzing individual-level geographic data and mapping the resulting point estimates and confidence intervals within R. MapGAM uses generalized additive models (GAMs) with a non-parametric bivariate smooth term of location to analyze geographic patterns in common epidemiologic datasets including case-control and survival data. The association between health outcomes and location can be estimated while simultaneously assessing the contribution from spatially-varying predictors such as socioeconomic characteristics or environmental exposures. MapGAM also includes convenient functions for efficient control sampling over space, optimal selection of smoothing parameters, and prediction of spatial associations for a continuous study area. We demonstrate the utility of MapGAM in different epidemiologic settings where the role of environmental exposures on geographic variation is assessed. The package is freely available under a GNU General Public License from the Comprehensive R Archive Network at https://CRAN.R-project.org/package=MapGAM. Keywords: Epidemiology, Disease cluster, Additive modeling, Residential histories, Spatio-temperal pattern Conference: GeoVet 2019. Novel spatio-temporal approaches in the era of Big Data, Davis, United States, 8 Oct - 10 Oct, 2019. Presentation Type: Keynote Topic: Emerging GIS, data science and sensor technologies adapted to animal, plant and human health, including precision medicine and precision farming Citation: Vieira V (2019). MapGAM: Mapping geographic disparities in health outcomes using individual-level epidemiologic data. Front. Vet. Sci. Conference Abstract: GeoVet 2019. Novel spatio-temporal approaches in the era of Big Data. doi: 10.3389/conf.fvets.2019.05.00001 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 30 Jul 2019; Published Online: 27 Sep 2019. * Correspondence: Dr. Veronica Vieira, University of California, Irvine, Irvine, United States, vvieira@uci.edu Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Veronica Vieira Google Veronica Vieira Google Scholar Veronica Vieira PubMed Veronica Vieira Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.

Estimating daily climatological normals in a changing climate

Climatological normals are widely used baselines for the description and the characterization of a given meteorological situation. The World Meteorological Organization (WMO) standard recommends estimating climatological normals as the average of observations over a 30-year period. This approach may lead to strongly biased normals in a changing climate. Here we propose a new method with which to estimate daily climatological normals in a non-stationary climate. Our statistical framework relies on the assumption that the response to climate change is smooth over time, and on a decomposition of the response inspired by the pattern scaling assumption. Estimation is carried out using smoothing splines techniques, with a careful examination of the selection of smoothing parameters. The new method is compared, in a predictive sense and in a perfect model framework, to previously proposed alternatives such as the WMO standard (reset either on a decadal or annual basis), averages over shorter periods, and hinge fits. Results show that our technique outperforms all alternatives considered. They confirm that previously proposed techniques are substantially biased—biases are typically as large as a few tenths to more than 1 $$^{\circ }\text{C}$$ by the end of the century—while our method is not. We argue that such “climate change corrected” normals might be very useful for climate monitoring, and that weather services could consider using two different sets of normals (i.e. both stationary and non-stationary) for different purposes.

A trivariate additive regression model with arbitrary link functions and varying correlation matrix

In many empirical situations, modelling simultaneously three or more outcomes as well as their dependence structure can be of considerable relevance. Copulae provide a powerful framework to build multivariate distributions and allow one to view the specification of the marginal responses’ equations and their dependence as separate but related issues. We propose a generalizationof the trivariate additive probit model where the link functions can in principle be derived from any parametric distribution and the parameters describing the residual association between the responses can be made dependent on several types of covariate effects (such as linear, nonlinear, random, and spatial effects). All the coefficients of the model are estimated simultaneously within a penalized likelihood framework that uses a trust region algorithm with integrated automatic multiple smoothing parameter selection. The effectiveness of the model is assessed in simulation as well as empirically by modelling jointly three adverse birth binary outcomes in North Carolina. The approach can be easily employed via the gjrm() function in the R package GJRM.

Transformation-Kernel Estimation of Copula Densities

ABSTRACTThe standard kernel estimator of copula densities suffers from boundary biases and inconsistency due to unbounded densities. Transforming the domain of estimation into an unbounded one remedies both problems, but also introduces an unbounded multiplier that may produce erratic boundary behaviors in the final density estimate. We propose an improved transformation-kernel estimator that employs a smooth tapering device to counter the undesirable influence of the multiplier. We establish the theoretical properties of the new estimator and its automatic higher-order improvement under Gaussian copulas. We present two practical methods of smoothing parameter selection. Extensive Monte Carlo simulations demonstrate the competence of the proposed estimator in terms of global and tail performance. Two real-world examples are provided. Supplementary materials for this article are available online.

Improved model based fault detection technique and application to humanoid robots

One of the issue of significant interest for robotics is the fault detection, specifically when we have application in risky circumstances. Robotic systems required a capacity to efficiently identify and endure some defects so that they can keep achieving the required tasks while avoiding instantaneous repairing process. Consequently, we aim in this work to propose a systematic approach for state estimation and fault detection technique to enhance the operation of humanoid robots (HR) systems using an extended Kalman filter (EKF)-based multiscale optimized exponentially weighted moving average chart (MS-OEWMA). The objectives of this work are sixfold: (1) apply EKF technique to estimate the state variables in HR systems. The EKF is among the most popular nonlinear state estimation methods; (2) use dynamical multiscale representation for obtaining accurate settled characteristics; (3) propose a new optimized EWMA (OEWMA) based on the best selection of both smoothing parameter (λ) and control width L; (4) combine the advantages of state estimation technique with MS-OEWMA chart to improve the monitoring of HR systems; (5) investigate the effect of fault types (change in variance and mean in shift) and fault sizes on the monitoring performances; (6) validate the developed technique using two robot models: inverted pendulum and five-bar linkage. The detection results are evaluated using three fault detection metrics: missed detection rate (MDR), false alarm rate (FAR) and out-of-control average run length (ARL1).

Optimization of sampling rate and smoothing improves classification of high frequency power in electrocorticographic brain signals

Objective. High-frequency band (HFB) activity, measured using implanted sensors over the cortex, is increasingly considered as a feature for the study of brain function and the design of neural-implants, such as Brain-Computer Interfaces (BCIs). One common way of extracting these power signals is using a wavelet dictionary, which involves the selection of different temporal sampling and temporal smoothing parameters, such that the resulting HFB signal best represents the temporal features of the neuronal event of interest. Typically, the use of neuro-electrical signals for closed-loop BCI control requires a certain level of signal downsampling and smoothing in order to remove uncorrelated noise, optimize performance and provide fast feedback. However, a fixed setting of the sampling and smoothing parameters may lead to a suboptimal representation of the underlying neural responses and poor BCI control. This problem can be resolved with a systematic assessment of parameter settings. Approach. With classification of HFB power responses as performance measure, different combinations of temporal sampling and temporal smoothing values were applied to data from sensory and motor tasks recorded with high-density and standard clinical electrocorticography (ECoG) grids in 12 epilepsy patients. Main results. The results suggest that HFB ECoG responses are best performed with high sampling and subsequent smoothing. For the paradigms used in this study, optimal temporal sampling ranged from 29 Hz to 50 Hz. Regarding optimal smoothing, values were similar between tasks (0.1–0.9 s), except for executed complex hand gestures, for which two optimal possible smoothing windows were found (0.4–0.6 s and 0.9–2.7 s). Significance. The range of optimal values indicates that parameter optimization depends on the functional paradigm and may be subject-specific. Our results advocate a methodical assessment of parameter settings for optimal decodability of ECoG signals.

A model-averaging approach for smoothing spline regression

ABSTRACTThis article considers nonparametric regression problems and develops a model-averaging procedure for smoothing spline regression problems. Unlike most smoothing parameter selection studies determining an optimum smoothing parameter, our focus here is on the prediction accuracy for the true conditional mean of Y given a predictor X. Our method consists of two steps. The first step is to construct a class of smoothing spline regression models based on nonparametric bootstrap samples, each with an appropriate smoothing parameter. The second step is to average bootstrap smoothing spline estimates of different smoothness to form a final improved estimate. To minimize the prediction error, we estimate the model weights using a delete-one-out cross-validation procedure. A simulation study has been performed by using a program written in R. The simulation study provides a comparison of the most well known cross-validation (CV), generalized cross-validation (GCV), and the proposed method. This new method is straightforward to implement, and gives reliable performances in simulations.

Estimation in monotone single‐index models

Single‐index models are popular regression models that are more flexible than linear models and still maintain more structure than purely nonparametric models. We consider the problem of estimating the regression parameters under a monotonicity constraint on the unknown link function. In contrast to the standard approach of using smoothing techniques, we review different “non‐smooth” estimators that avoid the difficult smoothing parameter selection. For about 30 years, one has had the conjecture that the profile least squares estimator is an ‐consistent estimator of the regression parameter, but the only non‐smooth argmin/argmax estimators that are actually known to achieve this ‐rate are not based on the nonparametric least squares estimator of the link function. However, solving a score equation corresponding to the least squares approach results in ‐consistent estimators. We illustrate the good behavior of the score approach via simulations. The connection with the binary choice and current status linear regression models is also discussed.