Smoothing Parameter Selection Research Articles

The Optimization of a Pipeline Temperature Monitoring Method Based on Non-Local Means with the Black Widow Optimization Algorithm

The accuracy of pipeline temperature monitoring using the Brillouin Optical Time Domain Analysis system depends on the Brillouin Gain Spectrum in the Brillouin Optical Time Domain Analysis system. The Non-Local Means noise reduction algorithm, due to its ability to use the data patterns available within the two-dimensional measurement data space, has been used to improve the Brillouin Gain Spectrum in the Brillouin Optical Time Domain Analysis system. This paper studies a new Non-Local Means algorithm optimized through the Black Widow Optimization Algorithm, in view of the unreasonable selection of smoothing parameters in other Non-Local Means algorithms. The field test demonstrates that, the new algorithm, when compared to other Non-Local Means methods, excels in preserving the detailed information within the Brillouin Gain Spectrum. It successfully restores the fundamental shape and essential characteristics of the Brillouin Gain Spectrum. Notably, at the 25 km fiber end, it achieves a 3 dB higher Signal-to-Noise Ratio compared to other Non-Local Means noise reduction algorithms. Furthermore, the Brillouin Gain Spectrum values exhibit increases of 9.4% in Root Mean Square Error, 12.5% in Sum of Squares Error, and 10% in Full Width at Half Maximum. The improved method has a better denoising effect and broad application prospects in pipeline safety.

Spectral Estimate for Stable Signals with P-Adic Time and Optimal Selection of Smoothing Parameter

The spectral density of stable signals with p-adic times is already estimated under various conditions. The estimate is made by constructing a periodogram that is subsequently smoothed by a spectral window. It is clear that the convergence rate of this estimator depends on the bandwidth of the spectral window (called the smoothing parameter). This work gives a method to select the smoothing parameter in an optimal way, i.e. the estimator converges to the spectral density with the bestrate. The method is inspired by the cross-validation method, which consists in minimizing the estimate of the integrated square error.

The Efficiency of the Proposed Smoothing Method over the Classical Cubic Smoothing Spline Regression Model with Autocorrelated Residual

Spline smoothing is a technique used to filter out noise in time series observations when predicting nonparametric regression models. Its performance depends on the choice of the smoothing parameter. Most of the existing smoothing methods applied to time series data tend to over fit in the presence of autocorrelated errors. This study aims to determine the optimum performance value, goodness of fit and model overfitting properties of the proposed Smoothing Method (PSM), Generalized Maximum Likelihood (GML), Generalized Cross-Validation (GCV), and Unbiased Risk (UBR) smoothing parameter selection methods. A Monte Carlo experiment of 1,000 trials was carried out at three different sample sizes (20, 60, and 100) and three levels of autocorrelation (0.2, 05, and 0.8). The four smoothing methods' performances were estimated and compared using the Predictive Mean Squared Error (PMSE) criterion. The findings of the study revealed that: for a time series observation with autocorrelated errors, provides the best-fit smoothing method for the model, the PSM does not over-fit data at all the autocorrelation levels considered ( the optimum value of the PSM was at the weighted value of 0.04 when there is autocorrelation in the error term, PSM performed better than the GCV, GML, and UBR smoothing methods were considered at all-time series sizes (T = 20, 60 and 100). For the real-life data employed in the study, PSM proved to be the most efficient among the GCV, GML, PSM, and UBR smoothing methods compared. The study concluded that the PSM method provides the best fit as a smoothing method, works well at autocorrelation levels (ρ=0.2, 0.5, and 0.8), and does not over fit time-series observations. The study recommended that the proposed smoothing is appropriate for time series observations with autocorrelation in the error term and econometrics real-life data. This study can be applied to; non – parametric regression, non – parametric forecasting, spatial, survival, and econometrics observations.

On some techniques of selecting spline smoothing parameters for a correlated dataset with autocorrelation structure in the residual

Residuals are minimized in a correlated dataset by selecting a smoothing parameter with optimum performance in the smoothing spline. The selection methods utilized in this study include Generalized Maximum Likelihood (GML), Generalized Cross-Validation (GCV), Unbiased Risk (UBR), and the Proposed Smoothing Method (PSM). The aim of this study is to compare the smoothing parameter selection ability of the four parameter selection methods for a correlated dataset with autocorrelation structure in the error term. To achieve this purpose, a Monte-Carlo simulation was conducted by utilizing program written in R-4.2.2. The performance of the parameter selection methods were evaluated using predictive Mean Squared Error (PMSE). Findings from the study indicated that GCV and GML were mostly affected by the presence of auto correlation in the residual and therefore had an asymptotically similar behavioural pattern. The estimators conformed to the asymptotic properties of the smoothing parameter selection methods considered; this is noticed in all the sample sizes and at all the smoothing parameters. The result also showed that; the most consistent and efficient among the four spline smoothing parameter selection methods considered in this study based on sample size and performance in the presence of autocorrelated residual error is the proposed smoothing method (PSM) because it does not undersmooth relative to the other smoothing method especially for small sample and medium sample size of 50 and 100.

The Efficiency of the Proposed Smoothing Method over the Classical Cubic Smoothing Spline Regression Model with Autocorrelated Residual

Spline smoothing is a technique used to filter out noise in time series observations when predicting nonparametric regression models. Its performance depends on the choice of the smoothing parameter. Most of the existing smoothing methods applied to time series data tend to overfit in the presence of autocorrelated errors. This study aims to determine the optimum performance value, goodness of fit and model overfitting properties of the proposed Smoothing Method (PSM), Generalized Maximum Likelihood (GML), Generalized Cross-Validation (GCV), and Unbiased Risk (UBR) smoothing parameter selection methods. A Monte Carlo experiment of 1,000 trials was carried out at three different sample sizes (20, 60, and 100) and three levels of autocorrelation (0.2, 05, and 0.8). The four smoothing methods' performances were estimated and compared using the Predictive Mean Squared Error (PMSE) criterion. The findings of the study revealed that: for a time series observation with autocorrelated errors, provides the best-fit smoothing method for the model, the PSM does not over-fit data at all the autocorrelation levels considered ( the optimum value of the PSM was at the weighted value of 0.04 when there is autocorrelation in the error term, PSM performed better than the GCV, GML, and UBR smoothing methods were considered at all-time series sizes (T = 20, 60 and 100). For the real-life data employed in the study, PSM proved to be the most efficient among the GCV, GML, PSM, and UBR smoothing methods compared. The study concluded that the PSM method provides the best fit as a smoothing method, works well at autocorrelation levels (ρ=0.2, 0.5, and 0.8), and does not over fit time-series observations. The study recommended that the proposed smoothing is appropriate for time series observations with autocorrelation in the error term and econometrics real-life data. This study can be applied to; non – parametric regression, non – parametric forecasting, spatial, survival, and econometrics observations.

Vine copula Granger causality in quantiles

ABSTRACT Causal relationships between time series are typically examined by testing for Granger causality. Although many studies have tested Granger causality in mean, non-causality in mean does not need to carry over to other distribution characteristics or different parts of the distribution. This scenario has motivated many researchers to investigate causal relations from the perspective of conditional quantiles. Several methods have been proposed for both parametric and nonparametric modelling frameworks. Parametric methods have limitations in detecting nonlinear causality, whereas nonparametric methods have difficulty selecting smoothing parameters that significantly affect detection performance. To overcome the difficulties of both parametric and nonparametric Granger causality tests in quantiles, we propose a vine copula Granger causality test in quantiles using the semiparametric time-series modelling technique. The proposed test overcomes shortcomings in parametric modelling and has a computational advantage over nonparametric tests. Our test shows good performance in terms of size and power when using various simulated data. Finally, we illustrate our test using recent cryptocurrency data.

Percentile Bootstrap Interval on Univariate Local Polynomial Regression Prediction

This study offers a new technique for constructing percentile bootstrap intervals to predict the regression of univariate local polynomials. Bootstrap regression uses resampling derived from paired and residual bootstrap methods. The main objective of this study is to perform a comparative analysis between the two resampling methods by considering the nominal coverage probability. Resampling uses a nonparametric bootstrap technique with the return method, where each sample point has an equal chance of being selected. The principle of nonparametric bootstrapping uses the original sample data as a source of diversity in contrast to parametric bootstrapping, where the variety comes from generating a particular distribution. The simulation results show that the paired and residual bootstrap interval coverage probabilities are close to nominal coverage. The results showed no significant difference between paired bootstrap interval and percentile residual. Increasing the bootstrap sample size sufficiently large gives the scatterplot smoothness of the confidence interval. Applying the smoothing parameter by choice gives a second-order polynomial regression with a smoother distribution than the first-order polynomial regression. The scatterplot shows that the second-degree polynomial regression can capture the data curvature feature compared to the first-degree polynomial. The bands made from second-degree polynomials give a narrower width than first-degree polynomials. In contrast, applying optimal smoothing parameters to the model provides different conclusions by using smoothing parameters based on choice. In addition to the differences based on the scatterplot, the bootstrap estimates of the coverage probability are also other. Selecting smoothing parameters based on a particular value provides probability coverage with the paired bootstrap method for the first-degree local polynomial regression is 0.93, while the second-degree local polynomial is 0.96. The probability of coverage based on the residual bootstrap method for the first-degree local polynomial regression is 0.95, while the second-degree local polynomial is 0.96. The probability coverage based on the optimal parameters of the paired bootstrap method for the first-degree local polynomial regression is 0.945, while the second-degree local polynomial is 0.93. The residual bootstrap method gives the first-degree local polynomial regression of 0.95, while the second-degree local polynomial is 0.93. In general, both bootstrap methods work well for estimating prediction confidence intervals.

Bootstrap-t Confidence Interval on Local Polynomial Regression Prediction

In local polynomial regression, prediction confidence interval estimation using standard theory will give coverage probability close to exact coverage probability. However, if the normality assumption is not met, the bootstrap method makes it possible to apply it. The working principle of the bootstrap method uses the resampling method where the sample data becomes a population and there is no need to know the distribution of the sample data is normal or not. Indiscriminate selection of smoothing parameters allows scatterplot results from local polynomial regressions to be rough and can even lead to misleading statistical conclusions. It is necessary to consider the optimal smoothing parameters to get local polynomial regression predictions that are not overfitting or underfitting. We offer two new algorithms based on the nested bootstrap resampling method to determine the bootstrap-t confidence interval in predicting local polynomial regression. Both algorithms consider the search for optimal smoothing parameters. The first algorithm performs paired and residual bootstrap samples, and the second algorithm performs based on residuals with residuals. The first algorithm provides a scatterplot and reasonable coverage probability on relatively large sample data. In contrast, the second algorithm is more powerful for each data size, including for relatively small sample data sizes. The mean of the bootstrap-t confidence interval coverage probability shows that the second algorithm for second-degree local polynomial regression is better than the other three. However, the larger the sample data size gives, the closer the closer the average coverage probability of the two algorithms is to the nominal coverage probability.

Copula link-based additive models for bivariate time-to-event outcomes with general censoring scheme

Bivariate survival outcomes arise frequently in applied studies where the occurrence of two events of interest are associated. Often the exact event times are unknown due to censoring which can manifest in various forms. A general and flexible copula regression model that can handle bivariate survival data subject to various censoring mechanisms, which include a mixture of uncensored, left-, right-, and interval-censored data, is proposed. The proposal permits to specify all model parameters as flexible functions of covariate effects, flexibly model the baseline survival functions by means of monotonic P-splines, characterise the marginals via transformations of the survival functions which yield, e.g., the proportional hazards and odds models as special cases, and model the dependence between events using a wide variety of copulae. The algorithm is based on a computationally efficient and stable penalised maximum likelihood estimation approach with integrated automatic multiple smoothing parameter selection. The proposed model is evaluated in a simulation study and illustrated using data from the Age-Related Eye Disease Study. The modelling framework has been incorporated in the newly-revised R package GJRM, hence allowing any user to fit the desired model(s) and produce easy-to-interpret numerical and visual summaries.

A critical review of univariate non-parametric estimation of first derivatives

This paper gives new guidance for selection of univariate non-parametric derivative estimators with non-iid errors in finite samples. It is shown via an extensive set of Monte Carlo simulations that the generalized criterion of Charnigo et al. (A generalized criterion for derivative estimation. Technometrics. 2011;53:238–253.) with spline smoothing performs the best in situations with minimal noise. For increased noise, generalized cross-validation of Craven and Wahba (Smoothing noisy data with spline functions. Numer Math. 1978;31:377–403) with P spline smoothing and the improved Akaike Information Criterion of Hurvich et al. (Smoothing parameter selection in nonparametric regression using an improved Akaike information criterion. J R Stat Soc Ser B. 1998;60:271–293.) with P spline smoothing are preferred. In the class of kernel smoothing and local regression methods, the local-cubic estimator of Henderson et al. (Gradient-based smoothing parameter selection for nonparametric regression estimation. J Econom. 2015;184:233–241.) generally outperforms its competitors. An internal meta-analysis separately favours the generalized method and P spline smoothing. The empirical example given provides support for use of the local-cubic estimator.

Vine copula Granger causality in mean

Ever since the Granger causality test was proposed in 1969, financial market researchers have used it heavily to determine whether the past of a one-time series facilitates the future prediction of another time series. However, as many researchers have noted, the traditional Granger causality test based on the vector autoregression model has limitations in detecting nonlinear causality. To relax the parametric model assumptions of the Granger causality test, nonparametric versions have been proposed to use the advantage of detecting nonlinear Granger causality but have shown difficulty in selecting smoothing parameters that significantly affect detection performance. To overcome the difficulties of both parametric and nonparametric Granger causality tests, we propose the vine copula Granger causality test in mean based on the semiparametric time-series modeling technique. The proposed test overcomes the shortcomings of parametric modeling and has a computational advantage over the nonparametric tests. Our test shows good size and power performance with various simulated data and a real data.

On bandwidth selection problems in nonparametric trend estimation under martingale difference errors

In this paper, we are interested in the problem of smoothing parameter selection in nonparametric curve estimation under dependent errors. We focus on kernel estimation and the case when the errors form a general stationary sequence of martingale difference random variables where neither linearity assumption nor “all moments are finite” are required. We compare the behaviors of the smoothing bandwidths obtained by minimizing either the unknown average squared error, the theoretical mean average squared error, a Mallows-type criterion adapted to the dependent case and the family of criteria known as generalized cross validation (GCV) extensions of the Mallows’ criterion. We prove that these three minimizers and those based on the GCV family are first-order equivalent in probability. We give also a normal asymptotic behavior of the gap between the minimizer of the average squared error and that of the Mallows-type criterion. This is extended to the GCV family. Finally, we apply our theoretical results to a specific case of martingale difference sequence, namely the Auto-Regressive Conditional Heteroscedastic (ARCH(1)) process. A Monte-Carlo simulation study, for this regression model with ARCH(1) process, is conducted.

Semi and Nonparametric Conditional Probability Density, a Case Study of Pedestrian Crashes

Background: Kernel-based methods have gained popularity as employed model residual’s distribution might not be defined by any classical parametric distribution. Kernel-based method has been extended to estimate conditional densities instead of conditional distributions when data incorporate both discrete and continuous attributes. The method often has been based on smoothing parameters to use optimal values for various attributes. Thus, in case of an explanatory variable being independent of the dependent variable, that attribute would be dropped in the nonparametric method by assigning a large smoothing parameter, giving them uniform distributions so their variances to the model’s variance would be minimal. Objectives: The objective of this study was to identify factors to the severity of pedestrian crashes based on an unbiased method. Especially, this study was conducted to evaluate the applicability of kernel-based techniques of semi- and nonparametric methods on the crash dataset by means of confusion techniques. Methods: In this study, two non- and semi-parametric kernel-based methods were implemented to model the severity of pedestrian crashes. The estimation of the semi-parametric densities is based on the adoptive local smoothing and maximization of the quasi-likelihood function, which is similar somehow to the likelihood of the binary logit model. On the other hand, the nonparametric method is based on the selection of optimal smoothing parameters in estimation of the conditional probability density function to minimize mean integrated squared error (MISE). The performances of those models are evaluated by their prediction power. To have a benchmark for comparison, the standard logistic regression was also employed. Although those methods have been employed in other fields, this is one of the earliest studies that employed those techniques in the context of traffic safety. Results: The results highlighted that the nonparametric kernel-based method outperforms the semi-parametric (single-index model) and the standard logit model based on the confusion matrices. To have a vision about the bandwidth selection method for removal of the irrelevant attributes in nonparametric approach, we added some noisy predictors to the models and a comparison was made. Extensive discussion has been made in the content of this study regarding the methodological approach of the models. Conclusion: To summarize, alcohol and drug involvement, driving on non-level grade, and bad lighting conditions are some of the factors that increase the likelihood of pedestrian crash severity. This is one of the earliest studies that implemented the methods in the context of transportation problems. The nonparametric method is especially recommended to be used in the field of traffic safety when there are uncertainties regarding the importance of predictors as the technique would automatically drop unimportant predictors.

Selection of Optimal Smoothing Parameters in Mixed Estimator of Kernel and Fourier Series in Semiparametric Regression

In this article, we propose a new method of selecting smoothing parameters in semiparametric regression. This method is used in semiparametric regression estimation where the nonparametric component is partially approximated by multivariable Fourier Series and partly approached by multivariable Kernel. Selection of smoothing parameters using the method with Generalized Cross-Validation (GCV). To see the performance of this method, it is then applied to the data drinking water quality sourced from Regional Drinking Water Company (PDAM) Surabaya by using Fourier Series with trend and Gaussian Kernel. The results showed that this method contributed a good performance in selecting the optimal smoothing parameters.

Initial Result of P Wave Tomography Model in Sunda-Banda Arc Transition using FMTOMO

Our study area is located near island Sumbawa, Sumba, Flores, West Timor, Indonesia and East Timor, popularly known as Sunda-Banda arc transition zone. The tectonic setting is mainly controlled by the movement of the oceanic lithosphere Indo-Australian plate subducting the Eurasian plate and Northward migration of Australian continental lithosphere into western Banda-arc in the region of Flores, Sumba and Timor island. We tried to image velocity structure beneath these regions using regional events and tomography inversion model. We collected 5 years of regional events from the Indonesian Agency of Meteorology, Climatology and Geophysics. In total, we reserved 3186 events recorded on 29 stations. For data processing, we used fast marching method as ray tracing between sources and receiver. We then employed subspace inversion as the tomography procedure to estimate the best velocity model representing the tectonic model in the region. Hypocenter data distribution is concentrated on shallow parts of the region and along the Benioff zone down to a maximum depth of 400 km. One of challenge of this study is that although events are abundance, the stations used are mostly located onshore and does not extend in the south-north direction that leads us to under determined problem in the inversion process. However, checker-board models show most our target area can be retrieved to its initial model with sign of smearing effects shown start from a depth of 50 km. After six iteration and optimized selection of damping and smoothing parameters, we observed low velocity anomaly under Bali, Lombok, Sumba, East Nusa Tenggara at shallow depth that may be related with volcanic activity. Deeper low anomaly can also be seen that may be related with partial melting process. A band of fast velocity is clearly seen that goes deepen to the north depicting subducting slabs own to a depth of 300 km. We also observed a possible of fast velocity in the northern part of our stations at shallow depth that we believe may represent the back arc thrust.

Cross-Validation, Information Theory, or Maximum Likelihood? A Comparison of Tuning Methods for Penalized Splines

Functional data analysis techniques, such as penalized splines, have become common tools used in a variety of applied research settings. Penalized spline estimators are frequently used in applied research to estimate unknown functions from noisy data. The success of these estimators depends on choosing a tuning parameter that provides the correct balance between fitting and smoothing the data. Several different smoothing parameter selection methods have been proposed for choosing a reasonable tuning parameter. The proposed methods generally fall into one of three categories: cross-validation methods, information theoretic methods, or maximum likelihood methods. Despite the well-known importance of selecting an ideal smoothing parameter, there is little agreement in the literature regarding which method(s) should be considered when analyzing real data. In this paper, we address this issue by exploring the practical performance of six popular tuning methods under a variety of simulated and real data situations. Our results reveal that maximum likelihood methods outperform the popular cross-validation methods in most situations—especially in the presence of correlated errors. Furthermore, our results reveal that the maximum likelihood methods perform well even when the errors are non-Gaussian and/or heteroscedastic. For real data applications, we recommend comparing results using cross-validation and maximum likelihood tuning methods, given that these methods tend to perform similarly (differently) when the model is correctly (incorrectly) specified.

A theoretical and experimental study of transient characteristics of the heat exchange in a thermal control system

Introduction. The “Heater-Blower-Room” thermal control system represents three different interconnected subsystems. It is necessary to study the transient characteristics of the heat exchange process, that is underway in the subsystems, including informative impulse responses, to stabilize the system operation. It is a non-parametric problem, and its solution requires identification algorithms. Materials and methods. Mathematical models of the subsystems represent the Volterra integral equation of the first kind with an undetermined difference kernel, or an impulse response. An impulse response evaluation is a solution to this equation in respect of registered noisy input and output values. The problem is to evaluate unknown impulse responses for the subsystems where the output of one subsystem is the input of another one. This problem is ill-posed, and features of identification-focused experiments do not allow to apply computational methods of classical regularization algorithms. The co-authors propose two specific non-parametric identification algorithms where impulse responses are evaluated using stable first derivatives by means of smoothing cubic splines through the optimal smoothing parameter selection on the basis of the statistical optimality criterion. Results. The co-authors solve inverse problems of impulse response identification and direct problems of heat flux reaction prediction. The research results demonstrate a high level of convergence between the evaluated data and observation findings. Both experimental and theoretical results represent the findings of the research performed by the co-authors. Conclusions. The results have proven the efficiency of the algorithms proposed for the identification of solutions to the problems of complex technical systems.

Bandwidth Selection for Kernel Generalized Regression Neural Networks in Identification of Hammerstein Systems

Abstract This paper addresses the issue of data-driven smoothing parameter (bandwidth) selection in the context of nonparametric system identification of dynamic systems. In particular, we examine the identification problem of the block-oriented Hammerstein cascade system. A class of kernel-type Generalized Regression Neural Networks (GRNN) is employed as the identification algorithm. The statistical accuracy of the kernel GRNN estimate is critically influenced by the choice of the bandwidth. Given the need of data-driven bandwidth specification we propose several automatic selection methods that are compared by means of simulation studies. Our experiments reveal that the method referred to as the partitioned cross-validation algorithm can be recommended as the practical procedure for the bandwidth choice for the kernel GRNN estimate in terms of its statistical accuracy and implementation aspects.

Nonparametric regression with ordinal responses

In this article, we explore a nonparametric approach to proportional odds regression with ordinal responses. The setting fits in the general framework of penalized likelihood non‐Gaussian regression as developed earlier in the smoothing spline ANOVA literature. Existing results in the general setting are readily applied or adapted, covering asymptotic convergence, computation, smoothing parameter selection, and Kullback–Leibler projection. Simulation studies of limited scales are conducted to assess various practical issues, and open‐source software tools are demonstrated in the analysis of some diabetic retinopathy data.

The package: nonparametric regression using local rotation matrices in

ABSTRACT The package implements nonparametric (smooth) regression for spherical data in , and is freely available from the Comprehensive Archive Network (CRAN), licensed under the MIT License. It can be used for regression when both the response and explanatory variables lie on the unit sphere. The model uses a flexible kernel-type regression determined by a rotation which depends on a smoothing parameter as well as the prediction point. A particular kernel is proposed and a smoothing parameter selection procedure is also provided. Finally, some examples are included in the package.