Optimal Smoothing Parameter Research Articles

Recursive bias estimation for multivariate regression smoothers

This paper presents a practical and simple fully nonparametric multivariate smoothing procedure that adapts to the underlying smoothness of the true regression function. Our estimator is easily computed by successive application of existing base smoothers (without the need of selecting an optimal smoothing parameter), such as thin-plate spline or kernel smoothers. The resulting smoother has better out of sample predictive capabilities than the underlying base smoother, or competing structurally constrained models (GAM) for small dimension (3 < d < 8) and moderate sample size (n < 800). Moreover our estimator is still useful when (d> 10) and to our knowledge, no other adaptive fully nonparametric regression estimator is available without constrained assumption such as additivity for example. On a real example, the Boston Housing Data, our method reduces the out of sample prediction error by 20 %. An R package ibr, available at CRAN, implements the proposed multivariate nonparametric method in R.

UNEXPECTED PROPERTIES OF BANDWIDTH CHOICE WHEN SMOOTHING DISCRETE DATA FOR CONSTRUCTING A FUNCTIONAL DATA CLASSIFIER.

The data functions that are studied in the course of functional data analysis are assembled from discrete data, and the level of smoothing that is used is generally that which is appropriate for accurate approximation of the conceptually smooth functions that were not actually observed. Existing literature shows that this approach is effective, and even optimal, when using functional data methods for prediction or hypothesis testing. However, in the present paper we show that this approach is not effective in classification problems. There a useful rule of thumb is that undersmoothing is often desirable, but there are several surprising qualifications to that approach. First, the effect of smoothing the training data can be more significant than that of smoothing the new data set to be classified; second, undersmoothing is not always the right approach, and in fact in some cases using a relatively large bandwidth can be more effective; and third, these perverse results are the consequence of very unusual properties of error rates, expressed as functions of smoothing parameters. For example, the orders of magnitude of optimal smoothing parameter choices depend on the signs and sizes of terms in an expansion of error rate, and those signs and sizes can vary dramatically from one setting to another, even for the same classifier.

An Analysis on Mammograms using KDA in Multi Transform Domain

a robust Computer Aided Diagnosis (CADx) system for mammograms analysis has been the challenging task for years. The slight difference in X-ray attenuation between normal and abnormal glandular tissues makes the mammogram diagnosis complicated. The proposed CAD system discriminates the abnormal severity of mammograms into normal-benign (non-cancerous, non-spreadable), normal- malign (cancerous) ans benign-,malign, using wavelet features combined with spectral domain. Classification is performed on 206 different mammogram images from Mias database using Kernel Discriminant Analysis (KDA). KDA affords a non-parametric statistical approach with parzen window density estimation to estimate density function from a given sample dataset. The study reveals that the optimal smoothing parameters are increasing functions of the sample size of the complementary classes, features used to classify and value of the bandwidth. KeywordsMammograms, KDA, Accuracy, MCC

Dynamic functional data analysis with non-parametric state space models

In this article, we introduce a new method for modelling curves with dynamic structures, using a non-parametric approach formulated as a state space model. The non-parametric approach is based on the use of penalised splines, represented as a dynamic mixed model. This formulation can capture the dynamic evolution of curves using a limited number of latent factors, allowing an accurate fit with a small number of parameters. We also present a new method to determine the optimal smoothing parameter through an adaptive procedure, using a formulation analogous to a model of stochastic volatility (SV). The non-parametric state space model allows unifying different methods applied to data with a functional structure in finance. We present the advantages and limitations of this method through simulation studies and also by comparing its predictive performance with other parametric and non-parametric methods used in financial applications using data on the term structure of interest rates.

The smoothness of HASM

To smooth noises inherent in uniformly sampled dataset, the smoothness of high accuracy surface modeling (HASM) was explored, and a smoothing method of HASM (HASM-SM) was developed based on a penalized least squares method. The optimal smoothing parameter of HASM-SM was automatically obtained by means of the generalized cross-validation (GCV) method. For an efficient smoothing computation, discrete cosine transform was employed to solve the system of HASM-SM and to estimate the minimum GCV score, simultaneously. Two examples including a numerical test and a real-world example were employed to compare the smoothing ability of HASM-SM with that of GCV thin plate smoothing spline (TPS) and kriging. The numerical test indicated that the minimum GCV HASM-SM is averagely more accurate than TPS and kriging for noisy surface smoothing. The real-world example of smoothing a lidar-derived Digital Elevation Model (DEM) showed that HASM-SM has an obvious smoothing effect, which is on a par with TPS. In conclusion, HASM-SM provides an efficient tool for filtering noises in grid-based surfaces like remote sensing–derived images and DEMs.

Improvement of the Contour Method for Measurement of Near-Surface Residual Stresses from Laser Peening

A study was conducted to develop a methodology to obtain near-surface residual stresses for laser-peened aluminium alloy samples using the contour method. After cutting trials to determine the optimal cut parameters, surface contours were obtained and a new data analysis method based on spline smoothing was applied. A new criterion for determining the optimal smoothing parameters is introduced. Near-surface residual stresses obtained from the contour method were compared with X-ray diffraction and incremental hole drilling results. It is concluded that with optimal cutting parameters and data analysis, reliable near-surface residual stresses can be obtained by the contour method.

On Kolmogorov asymptotics of estimators of the misclassification error rate in linear discriminant analysis

We provide a fundamental theorem that can be used in conjunction with Kolmogorov asymptotic conditions to derive the first moments of well-known estimators of the actual error rate in linear discriminant analysis of a multivariate Gaussian model under the assumption of a common known covariance matrix. The estimators studied in this paper are plug-in and smoothed resubstitution error estimators, both of which have not been studied before under Kolmogorov asymptotic conditions. As a result of this work, we present an optimal smoothing parameter that makes the smoothed resubstitution an unbiased estimator of the true error. For the sake of completeness, we further show how to utilize the presented fundamental theorem to achieve several previously reported results, namely the first moment of the resubstitution estimator and the actual error rate. We provide numerical examples to show the accuracy of the succeeding finite sample approximations in situations where the number of dimensions is comparable or even larger than the sample size.

Adaptive evolutionary clustering

In many practical applications of clustering, the objects to be clustered evolve over time, and a clustering result is desired at each time step. In such applications, evolutionary clustering typically outperforms traditional static clustering by producing clustering results that reflect long-term trends while being robust to short-term variations. Several evolutionary clustering algorithms have recently been proposed, often by adding a temporal smoothness penalty to the cost function of a static clustering method. In this paper, we introduce a different approach to evolutionary clustering by accurately tracking the time-varying proximities between objects followed by static clustering. We present an evolutionary clustering framework that adaptively estimates the optimal smoothing parameter using shrinkage estimation, a statistical approach that improves a naive estimate using additional information. The proposed framework can be used to extend a variety of static clustering algorithms, including hierarchical, k-means, and spectral clustering, into evolutionary clustering algorithms. Experiments on synthetic and real data sets indicate that the proposed framework outperforms static clustering and existing evolutionary clustering algorithms in many scenarios.

Data-driven bandwidth choice for gamma kernel estimates of density derivatives on the positive semi-axis

In some applications it is necessary to estimate derivatives of probability densities defined on the positive semi-axis. The quality of nonparametric estimates of the probability densities and their derivatives are strongly influenced by smoothing parameters (bandwidths). In this paper an expression for the optimal smoothing parameter of the gamma kernel estimate of the density derivative is obtained. For this parameter data-driven estimates based on methods called “rule of thumb” and “cross-validation” are constructed. The quality of the estimates is verified and demonstrated on examples of density derivatives generated by Maxwell and Weibull distributions.

Nonparametric inference of the hemodynamic response using multi-subject fMRI data

Estimation and inferences for the hemodynamic response functions (HRF) using multi-subject fMRI data are considered. Within the context of the General Linear Model, two new nonparametric estimators for the HRF are proposed. The first is a kernel-smoothed estimator, which is used to construct hypothesis tests on the entire HRF curve, in contrast to only summaries of the curve as in most existing tests. To cope with the inherent large data variance, we introduce a second approach which imposes Tikhonov regularization on the kernel-smoothed estimator. An additional bias-correction step, which uses multi-subject averaged information, is introduced to further improve efficiency and reduce the bias in estimation for individual HRFs. By utilizing the common properties of brain activity shared across subjects, this is the main improvement over the standard methods where each subject's data is usually analyzed independently. A fast algorithm is also developed to select the optimal regularization and smoothing parameters. The proposed methods are compared with several existing regularization methods through simulations. The methods are illustrated by an application to the fMRI data collected under a psychology design employing the Monetary Incentive Delay (MID) task.

Kernel smoothers and bootstrapping for semiparametric mixed effects models

While today linear mixed effects models are frequently used tools in different fields of statistics, in particular for studying data with clusters, longitudinal or multi-level structure, the nonparametric formulation of mixed effects models is still quite recent. In this paper we discuss and compare different nonparametric estimation methods. In this context we introduce a computationally inexpensive bootstrap method, which is used to estimate local mean squared errors, to construct confidence intervals and to find locally optimal smoothing parameters. The theoretical considerations are accompanied by the provision of algorithms and simulation studies of the finite sample behavior of the methods. We show that our confidence intervals have good coverage probabilities, and that our bandwidth selection method succeeds to minimize the mean squared error for the nonparametric function locally.

Rapid Determination of Chemical Oxygen Demand in Sugar Refinery Wastewater by Short-Wave Near-Infrared Spectroscopy

Near-infrared (NIR) spectroscopy combined with partial least squares (PLS) regression and Savitzky-Golay (SG) smoothing was successfully applied for the rapid no-reagent determination of chemical oxygen demand (COD) in sugar refinery wastewater. Among 141 samples, 60 were randomly selected as the validation set. The remaining 81 samples were divided into the calibration set (50 samples) and the prediction set (31 samples) for a total of 20 times with certain similarities. The results showed that the short-wave NIR region (780 nm to 1100 nm) can be used as information waveband of COD in sugar refinery wastewater. The optimal SG smoothing parameters and PLS factors OD, DP, NSP, and F were 5, 5, 7, and 6, respectively. The modeling effects M-SEPAve, M-RP,Ave, M-SEPStd, and M-RP,Stdwere 24.3 mg/L, 0.971, 2.7 mg/L, and 0.007, respectively. The validation effects V-SEP and V-RP were 25.9 mg/L and 0.963, respectively. The results indicated that the method has good prediction effect and stability, and provided a reliable analysis model and valuable references for designing specialized instruments.

Optimal Design of a EWMA Chart to Monitor the Normal Process Mean

EWMA(exponentially weighted moving average) charts and CUSUM(cumulative sum) charts are very effective to detect small shifts in the process mean. These charts have some control-chart parameters that allow the charts and be tuned and be more sensitive to certain shifts. The EWMA chart requires users to specify the value of a smoothing parameter, which can also be designed for the size of the mean shift. However, the size of the mean shift that occurs in applications is usually unknown and EWMA charts can perform poorly when the actual size of the mean shift is significantly different from the assumed size. In this paper, we propose the design procedure to find the optimal smoothing parameter of the EWMA chart when the size of the mean shift is unknown.

ON MULTIRESPONSE SEMIPARAMETRIC REGRESSION MODEL

The modeling between predictors and response in statistics sometimes deals with more than one response or multiresponse situation. Furthermore, it can be happen that some predictors have linear relationship with the responses and the others predictor have unknown relationship. To overcome this modeling problem we proposed multiresponse semiparametric regression model. This model has more than one response and contains both parametric and nonparametric model. This study focuses on how to estimate parameter in multiresponse semiparametric regression. The weighted penalized least squares method is used to fit the model. This method produce partial spline estimator for nonparametric model and by applying some assumptions the estimator is polynomial natural spline. The performance of this estimator depends on smoothing parameter. So, we also proposed G criteria as modification of generalized cross validation in the context of multiresponse semiparametric regression to choose the optimal smoothing parameter. Using simulation data, it can be shown that this model can work well to describe relationship between some predictors and several responses.

FTIR/ATR Spectroscopy with Stability for Determination of HGB in Human Whole Blood

Fourier transform infrared spectroscopy (FTIR) and attenuated total reflection (ATR) combined with partial least-squares (PLS) and Savitzky-Golay (SG) smoothing were successfully applied for rapid no-reagent determination of hemoglobin (HGB) in human whole blood. The optimal SG smoothing parameters OD, DP, NSP and PLS factor were 1, 2, 49 and 14, respectively. The modeling effect M-SEPAve, M-RP,Ave, M-SEPStd, and M-RP,Std were 6.73g/L, 0.850, 0.306g/L, and 0.015, respectively. The validation effect V-SEP and V-RP were 6.83g/L and 0.864, respectively. The method provided a reliable model with stability for clinic application of FTIR/ATR.

On asymptotically optimal wavelet estimation of trend functions under long-range dependence

We consider data-adaptive wavelet estimation of a trend function in a time series model with strongly dependent Gaussian residuals. Asymptotic expressions for the optimal mean integrated squared error and corresponding optimal smoothing and resolution parameters are derived. Due to adaptation to the properties of the underlying trend function, the approach shows very good performance for smooth trend functions while remaining competitive with minimax wavelet estimation for functions with discontinuities. Simulations illustrate the asymptotic results and finite-sample behavior.

Model Optimization for near-Infrared Spectroscopy Analysis of Chemical Oxygen Demand of Wastewater

A directly rapid quantitative analysis method for chemical oxygen demand (COD) of wastewater samples was established by near-infrared (NIR) spectroscopy and partial least square (PLS) method. The optimization of Savitzky-Golay (SG) smoothing modes combined with PLS factor was applied to optimize the model of NIR spectroscopy analysis here. The waveband used for modeling was the combination of 400-1878 nm and 2088-2338 nm. The optimal smoothing parameters were the 5thderivative smoothing, 5thdegree polynomial, 17 smoothing points, the optimal PLS factor, root mean squared error of predication (RMSEP) and correlation coefficient of predication (RP) were 7, 33.2 (mg/L) and 0.929 respectively, which was obviously superior to the direct PLS model without SG smoothing and ones based on the whole spectral collecting region 400-2500 nm. This demonstrated that NIR spectroscopy can be applied to the rapid determination of COD of wastewater, large-scale simultaneous optimization selection of SG smoothing parameters and PLS factor can be effectively applied to the model optimization of NIR analysis.

Estimating Optimal Hodrick-Prescott Filter Smoothing Parameter for Turkey

Bu çalışma, 1987-2007 dönemi çeyreklik reel Gayrisafi Yurt İçi Hasıla (GSYİH) verilerini kullanarak, Türkiye için Hodrick-Prescott filtresi düzgünleştirme parametresini tahmin etmektedir. Tahmin için kullanılan alternatif yöntemlerden, Pedersen (2001) yöntemi ve Dermoune ve diğ. (2008) yöntemi söz konusu parametrenin Türkiye ekonomisi için sırasıyla 98 ve 19 olduğunu göstermektedir. Tahmin edilen bu değerleri kullanarak elde ettiğimiz sonuçlara göre: (1) Türkiye ekonomisinin reel GSYİH uzun dönemli eğilimi, Aguiar ve Gopinath (2007)’nin gelişmekte olan ülkeler için yaptıkları öngörülerle tutarlı olarak, oldukça dalgalıdır. (2) 1600 değeri ile elde edilenlere kıyasla, çalışmada tahmin edilen düzgünleştirme parametreleri GSYİH ve bileşenlerinin iş çevirimleri frekanslarında gösterdiği mutlak oynaklıkların daha küçük olduğunu göstermektedir. (3) Gerek 1600 ile, gerekse tahmin edilen parametrelerle elde edilen sonuçlar, iş çevrimleri frekanslarında GSYİH’nin bileşenlerinin GSYİH’ye göre göreli oynaklığının benzer büyüklüklerde olduğunu göstermektedir

FTIR/ATR Spectroscopy Analysis of COD in Wastewater Based on Combination Optimization of SG Smoothing Modes and PLS Factor

A directly rapid quantitative analysis method for chemical oxygen demand (COD) of wastewater was established by Fourier transform infrared (FTIR) spectroscopy and attenuated total reflection (ATR) technology. The combination selection of Savitzky-Golay (SG) smoothing modes combined with partial least squares (PLS) factor was applied to the model optimization here. The optimal smoothing parameters were 2ndorder derivative smoothing, 6thdegree polynomial and 51 smoothing points. The corresponding optimal PLS factor, root mean squared error of predication (RMSEP) and correlation coefficient of predication (RP) were 11, 24.4 (mg/L) and 0.968 respectively, which was obviously superior to the direct PLS model without SG smoothing. This demonstrated that the combination selection of SG smoothing parameters and PLS factor can be effectively applied to the model optimization for FTIR/ATR analysis of COD in wastewater.

A non-iterative optimization method for smoothness in penalized spline regression

Typically, an optimal smoothing parameter in a penalized spline regression is determined by minimizing an information criterion, such as one of the C p , CV and GCV criteria. Since an explicit solution to the minimization problem for an information criterion cannot be obtained, it is necessary to carry out an iterative procedure to search for the optimal smoothing parameter. In order to avoid such extra calculation, a non-iterative optimization method for smoothness in penalized spline regression is proposed using the formulation of generalized ridge regression. By conducting numerical simulations, we verify that our method has better performance than other methods which optimize the number of basis functions and the single smoothing parameter by means of the CV or GCV criteria.