Generalized Cross-validation Method Research Articles

KNOT OPTIMIZATION FOR BI-RESPONSE SPLINE NONPARAMETRIC REGRESSION WITH GENERALIZED CROSS-VALIDATION (GCV)

Nonparametric regression is a statistical method used to model relationships between variables without making strong assumptions about the functional form of the relationship. Nonparametric regression models are flexible and can capture complex relationships that may not be adequately represented by simple parametric forms. Spline is one of the approaches used in nonparametric regression. Splines have the disadvantage of having to use optimal nodes in the data. Therefore, this article discusses the retrieval of optimal knot points using the generalized cross-validation method in the nonparametric bi-response spline regression model. The research results showed that the generalized-cross validation method is the best method for selecting nodes from other methods such as CV, AIC, BIC, RSS, or a more explicit validation-based approach method because of the development of the Cross Validation (CV) method which automatically selects the optimal number of nodes based on the balance between bias and variance. The process of optimizing knot points with Generalized Cross Validation (GCV) on bi-response spline nonparametric regression is implemented using Python can provide optimization at optimal knot points. Based on the results of the generalized cross-validation model analysis, it is concluded that GCV can effectively optimize knot points for spline fitting, ensuring a balanced and efficient model in capturing data patterns without overfitting.

Damage identification method based on interval regularization theory

In the field of damage identification, traditional regularization methods neglect the impact of uncertainty factors on the selection of regularization parameters, leading to a decrease in the accuracy of damage identification. Therefore, this study proposes a damage identification based on interval truncated singular value decomposition (DI-ITSVD) method that considers the uncertainty in the selection of regularization parameter. This method treats model errors and measurement noise as interval uncertainties, and introduces the quantified uncertainties into the damage identification solutions through uncertainty propagation methods to determine the interval boundary. Uncertainty regularization parameters are selected to balance residuals and solutions using interval and generalized cross-validation methods. The key aspect of the proposed method in this paper is the integration of interval uncertainty propagation with the truncated singular value decomposition method to ensure the accuracy and stability of the damage identification equation solution. A numerical example of a 29-bar planar truss has been performed to test the effectiveness of the proposed method. The superiority of this method is verified by comparing the identification results with other improved truncated singular value decomposition methods. Finally, the practical application effect of the proposed method was also verified through an experimental work.

MODELING DHF IN CENTRAL JAVA USING HYBRID NONPARAMETRIC SPLINE TRUNCATED-FOURIER SERIES APPROACH

Regression analysis aims to determine the relationship and influence of predictor variables on response variables through regression curve. The problem with nonparametric regression research so far is that it only uses one approach, causing the estimation results to be biased, even though each data sub-pattern has its own suitability depending on the approach method used. Therefore, the hybrid method emerged as a development of nonparametric regression. Hybrid models are models that combine approach methods, with the hope of increasing accuracy in modeling analysis. This research was carried out using two non-parametric approaches, namely Spline Truncated and Fourier Series. Dengue Hemorrhagic Fever (DHF) is a disease caused by the dengue virus. DHF is endemic and occurs throughout the year, especially during the rainy season because mosquitoes reproduce optimally. The aim of this research is to estimate the Hybrid Nonparametric Spline Truncated -Fourier Series model and apply the estimation results to data on DHF cases in Central Java. The data used to apply the hybrid nonparametric Spline Truncated-Fourier series regression model is DHF in the city/districts of Central Java. Estimation smoothing parameters uses the GCV (Generalized Cross Validation) method. The best model is selected based on largest R-Square and the smallest MSE. Modeling the disease of DHF cases in Central Java using the Spline Truncated-Fourier Series hybrid estimator produced the best model from the Spline Truncated model with two knot points for each predictor and the Fourier Series model with value of 9. Based on the results obtained, it can be compared that the Truncated Spline-Fourier Series hybrid model is better than the Spline Truncated model, this can be seen from the largest R-square, namely 99.94% and the smallest MSE.

Stunting Prevalence Modeling Using Nonparametric Regression of Quadratic Splines

The nonparametric regression approach is used when the shape of the regression curve between the response variable and the predictor variable is assumed to be of unknown shape. The advantages of nonparametric regression have high flexibility. A nonparametric regression approach that is often used is truncated spline which has an excellent ability to handle data whose behavior changes at certain sub-sub intervals. The purpose of this study is to obtain the best model of multivariable nonparametric regression with linear and quadratic truncated spline approaches using the Generalized Cross Validation (GCV) and Unbiased Risk (UBR) methods and to find out the factors that influence the prevalence of stunting in Indonesia in 2021. The data used were the prevalence of stunting as a response variable and the predictor variable used was the percentage of infants receiving exclusive breastfeeding for 6 months, the percentage of households that have proper sanitation, the percentage of toddlers who get Early Initiation of Breastfeeding (IMD), the percentage of poor people, and the percentage of pregnant women at risk of SEZ. The results showed that the best quadratic truncated spline nonparametric regression model in modeling stunting prevalence was quadraic truncated spline using the GCV method with three knot points. This model has a minimum GCV value of 7.29 with an MSE value of 1.82 and a R2 value of 94.07%.

Robust residual-guided iterative reconstruction for sparse-view CT in small animal imaging

Objective. We introduce a robust image reconstruction algorithm named residual-guided Golub–Kahan iterative reconstruction technique (RGIRT) designed for sparse-view computed tomography (CT), which aims at high-fidelity image reconstruction from a limited number of projection views. Approach. RGIRT utilizes an inner-outer dual iteration framework, with a flexible least square QR (FLSQR) algorithm implemented in the inner iteration and a restarted iterative scheme applied in the outer iteration. The inner FLSQR employs a flexible Golub–Kahan bidiagonalization method to reduce the size of the inverse problem, and a weighted generalized cross-validation method to adaptively estimate the regularization hyper-parameter. The inner iteration efficiently yields the intermediate reconstruction result, while the outer iteration minimizes the residual and refines the solution by using the result obtained from the inner iteration. Main results. The reconstruction performance of RGIRT is evaluated and compared to other reference methods (FBPConvNet, SART-TV, and FLSQR) using projection data from both numerical phantoms and real experimental Micro-CT data. The experimental findings, from testing various numbers of projection views and different noise levels, underscore the robustness of RGIRT. Meanwhile, theoretical analysis confirms the convergence of residual for our approach. Significance. We propose a robust iterative reconstruction algorithm for x-ray CT scans with sparse views, thereby shortening scanning time and mitigating excessive ionizing radiation exposure to small animals.

Convex model-based regularization method for force reconstruction

In the process of reconstructing structural forces, the influence of measurement errors and inherent model inaccuracies cannot be ignored. These errors exhibit a degree of correlation, and the presence of such correlation inevitably affects the quantification of uncertainties in force reconstruction. Objectively, the inherent ill-posed nature of structural inverse problems makes it difficult to obtain the forces to be identified, subject to uncertainties and highly susceptible to perturbations. Consequently, this paper introduces a force reconstruction regularization approach that explicitly considers the correlation of uncertainty parameters based on the truncated singular value regularization method. The primary objective is to refine the influence of uncertainties on the reconstructed force bounds with greater precision. When determining robust regularization parameters, the generalized cross-validation method and convex modelling approach are introduced to consider the uncertainty and its correlation in solving inverse problems. The proposed approach is rigorously validated through a comprehensive numerical case study. Error indexes and dispersion indices are employed to analyze the impact of different levels of noise and correlation on force reconstruction results. The force bounds obtained using the proposed method are compared with Monte Carlo simulation results. Finally, the validity of the proposed method is verified by an experiment with a four-story shear frame.

Application of Regularization Methods in the Sky Map Reconstruction of the Tianlai Cylinder Pathfinder Array

The Tianlai cylinder pathfinder is a radio interferometer array to test 21 cm intensity mapping techniques in the post-reionization era. It works in passive drift scan mode to survey the sky visible in the northern hemisphere. To deal with the large instantaneous field of view and the spherical sky, we decompose the drift scan data into m-modes, which are linearly related to the sky intensity. The sky map is reconstructed by solving the linear interferometer equations. Due to incomplete uv coverage of the interferometer baselines, this inverse problem is usually ill-posed, and regularization method is needed for its solution. In this paper, we use simulation to investigate two frequently used regularization methods, the Truncated Singular Value Decomposition (TSVD), and the Tikhonov regularization techniques. Choosing the regularization parameter is very important for its application. We employ the generalized cross validation method and the L-curve method to determine the optimal value. We compare the resulting maps obtained with the different regularization methods, and for the different parameters derived using the different criteria. While both methods can yield good maps for a range of regularization parameters, in the Tikhonov method the suppression of noisy modes are more gradually applied, produce more smooth maps which avoids some visual artefacts in the maps generated with the TSVD method.

Approximation of the Tikhonov regularization parameter through Aitken's extrapolation

In the present work, we study the determination of the regularization parameter and the computation of the regularized solution in Tikhonov regularization, by the Aitken's extrapolation method. In particular, this convergence acceleration method is adjusted for the approximation of quadratic forms that appear in regularization methods, such as the generalized cross-validation method, the quasi-optimality criterion, the Gfrerer/Raus method and the Morozov's discrepancy principle. We present several numerical examples to illustrate the effectiveness of the derived estimates for approximating the regularization parameter for several linear discrete ill-posed problems and we compare the described method with further existing methods, for the determination of the regularized solution.

The Efficiency of the Proposed Smoothing Method over the Classical Cubic Smoothing Spline Regression Model with Autocorrelated Residual

Spline smoothing is a technique used to filter out noise in time series observations when predicting nonparametric regression models. Its performance depends on the choice of the smoothing parameter. Most of the existing smoothing methods applied to time series data tend to over fit in the presence of autocorrelated errors. This study aims to determine the optimum performance value, goodness of fit and model overfitting properties of the proposed Smoothing Method (PSM), Generalized Maximum Likelihood (GML), Generalized Cross-Validation (GCV), and Unbiased Risk (UBR) smoothing parameter selection methods. A Monte Carlo experiment of 1,000 trials was carried out at three different sample sizes (20, 60, and 100) and three levels of autocorrelation (0.2, 05, and 0.8). The four smoothing methods' performances were estimated and compared using the Predictive Mean Squared Error (PMSE) criterion. The findings of the study revealed that: for a time series observation with autocorrelated errors, provides the best-fit smoothing method for the model, the PSM does not over-fit data at all the autocorrelation levels considered ( the optimum value of the PSM was at the weighted value of 0.04 when there is autocorrelation in the error term, PSM performed better than the GCV, GML, and UBR smoothing methods were considered at all-time series sizes (T = 20, 60 and 100). For the real-life data employed in the study, PSM proved to be the most efficient among the GCV, GML, PSM, and UBR smoothing methods compared. The study concluded that the PSM method provides the best fit as a smoothing method, works well at autocorrelation levels (ρ=0.2, 0.5, and 0.8), and does not over fit time-series observations. The study recommended that the proposed smoothing is appropriate for time series observations with autocorrelation in the error term and econometrics real-life data. This study can be applied to; non – parametric regression, non – parametric forecasting, spatial, survival, and econometrics observations.

The Efficiency of the Proposed Smoothing Method over the Classical Cubic Smoothing Spline Regression Model with Autocorrelated Residual

Spline smoothing is a technique used to filter out noise in time series observations when predicting nonparametric regression models. Its performance depends on the choice of the smoothing parameter. Most of the existing smoothing methods applied to time series data tend to overfit in the presence of autocorrelated errors. This study aims to determine the optimum performance value, goodness of fit and model overfitting properties of the proposed Smoothing Method (PSM), Generalized Maximum Likelihood (GML), Generalized Cross-Validation (GCV), and Unbiased Risk (UBR) smoothing parameter selection methods. A Monte Carlo experiment of 1,000 trials was carried out at three different sample sizes (20, 60, and 100) and three levels of autocorrelation (0.2, 05, and 0.8). The four smoothing methods' performances were estimated and compared using the Predictive Mean Squared Error (PMSE) criterion. The findings of the study revealed that: for a time series observation with autocorrelated errors, provides the best-fit smoothing method for the model, the PSM does not over-fit data at all the autocorrelation levels considered ( the optimum value of the PSM was at the weighted value of 0.04 when there is autocorrelation in the error term, PSM performed better than the GCV, GML, and UBR smoothing methods were considered at all-time series sizes (T = 20, 60 and 100). For the real-life data employed in the study, PSM proved to be the most efficient among the GCV, GML, PSM, and UBR smoothing methods compared. The study concluded that the PSM method provides the best fit as a smoothing method, works well at autocorrelation levels (ρ=0.2, 0.5, and 0.8), and does not over fit time-series observations. The study recommended that the proposed smoothing is appropriate for time series observations with autocorrelation in the error term and econometrics real-life data. This study can be applied to; non – parametric regression, non – parametric forecasting, spatial, survival, and econometrics observations.

Interpolating log-determinant and trace of the powers of matrix textbf{A} + ttextbf{B}

We develop heuristic interpolation methods for the functions tmapsto log det left( textbf{A} + ttextbf{B} right) and tmapsto {{,textrm{trace},}}left( (textbf{A} + ttextbf{B})^{p} right) where the matrices textbf{A} and textbf{B} are Hermitian and positive (semi) definite and p and t are real variables. These functions are featured in many applications in statistics, machine learning, and computational physics. The presented interpolation functions are based on the modification of sharp bounds for these functions. We demonstrate the accuracy and performance of the proposed method with numerical examples, namely, the marginal maximum likelihood estimation for Gaussian process regression and the estimation of the regularization parameter of ridge regression with the generalized cross-validation method.

Industrial serial robot calibration considering geometric and deformation errors

The absolute accuracy of the industrial serial robot is affected by the geometric errors from machining and assembling, and the elastic deformation errors from the large payload and flexible joints. The inherent features and correlations of both geometric and deformation errors have not been thoroughly discussed, leading to the unsatisfactory calibration results. In this paper, the geometric and deformation propagations are separately deduced and then combined to form a complete model having both types of errors. Geometric errors, i.e. the joint twist errors and initial transformation errors, are described at the initial pose and remain the same during the task execution of the robot. But the deformation errors are evaluated at the current pose and change values versus the change of poses. Based on the features of both errors, it is summarized that the deflections of joints are independent from the geometric errors and would not affect the geometric error propagation. Then, the redundancy within the geometric errors is proved and the singularity between two types of errors are discussed. A Generalized Cross-Validation method is adopted to solve the ill-conditioning problem caused by the different unit of the identified parameters. The simultaneous identification of both types of errors are implemented. Finally, a step-by-step compensation is proposed for a convenient error correction. A UR3 robot is taken as an example to illustrate and verify the proposed calibration method. The mean positioning absolute accuracy is 0.4672 mm after calibration. Comparisons with the calibration of only geometric errors indicates the proposed calibration method leads to higher absolute accuracy.

MODELING OF LOCAL POLYNOMIAL KERNEL NONPARAMETRIC REGRESSION FOR COVID DAILY CASES IN SEMARANG CITY, INDONESIA

Coronavirus disease 2019 (COVID-19) is an infectious disease caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) which was recently discovered. Coronavirus disease is now a pandemic that occurs in many countries in the world, one of which is Indonesia. One of the cities in Indonesia that has found many COVID cases is Semarang city, located in Central Java. Data on cases of COVID patients in Semarang City which are measured daily do not form a certain distribution pattern. We can build a model with a flexible statistical approach without any assumptions that must be used, namely the nonparametric regression. The nonparametric regression in this research using Local Polynomial Kernel approach. Determination of the polynomial order and optimal bandwidth in Local Polynomial Kernel Regression modeling use the GCV (Generalized Cross Validation) method. The data used this research are data on the number of COVID patients daily cases in Semarang, Indonesia. Based on the results of the application of the COVID patient daily cases in Semarang City, the optimal bandwidth value is 0.86 and the polynomial order is 4 with the minimum GCV is 3179.568 so that the model estimation results the MSE is 2922.22 and the determination coefficient is 97%. The estimation results show the highest number of Corona in the Semarang City at the beginning of July 2020. After the corona case increased in July, while the corona case in August decreased.

Soil radioactivity-depth profiles from regularized inversion of borehole gamma spectrometry data

An in situ borehole gamma logging method using a LaBr3 gamma detector has been developed to characterize a137Cs contaminated site. The activity-depth distribution of 137Cs was derived by inversion of the in situ measurement data using two different least squares methods, (i) Least squares optimization (LSO) and (ii) Tikhonov regularization. The regularization parameter (λ) of the Tikhonov regularization method was estimated using three different methods i.e. the L-curve, Generalized Cross Validation (GCV) and a prior information based method (PIBM). The considered inversion method variants were first validated for a137Cs contaminated pipe, and in most of the cases, the calculated activity of 137Cs was found to be within the acceptable range. The calculated 137Cs activity-depth profiles from in situ measurements were also in good agreement with the ones obtained from soil sample analysis, with an R2 ranging from 0.76 to 0.82. The GCV method for estimating λ appeared to perform better than the two other methods in terms of R2 and root mean squared error (RMSE). The L-curve method resulted in higher RMSE than the other Tikhonov regularization methods. Instability was observed in the activity concentration depth profile obtained from the LSO method. Therefore, we recommend the Tikhonov regularization with GCV for estimating λ for estimating the activity concentration-depth profile. The site studied showed 137Cs activity concentrations above the exemption limit down to depths of 0.50–0.90 m.

Nonparametric Regression Mixed Estimators of Truncated Spline and Gaussian Kernel based on Cross-Validation (CV), Generalized Cross-Validation (GCV), and Unbiased Risk (UBR) Methods

<p class='IJASEITAbtract'>Nowadays, most nonparametric regression research involves more than one predictor variable and generally uses the same type of estimator for all predictors. In the real case, each predictor variable likely has a different form of regression curve so that if it is forced, it can produce an estimation form that does not match the data pattern. Thus, it is necessary to develop a regression curve estimation model under the data pattern, namely the mixed estimator. The focus of this study is an additive nonparametric regression model, a mix of the Truncated Spline and Gaussian Kernel. There is a knot point in the Truncated Spline, while in the Gaussian Kernel, there is bandwidth. To choose the optimal knot point and bandwidth in a mixed estimator model, various methods can be used, including Cross-Validation (CV), Generalized Cross-Validation (GCV), and Unbiased Risk (UBR). This research proposes the optimal knot point and bandwidth estimation on the mixed estimator Truncated Spline and Gaussian Kernel model. Furthermore, the comparison between CV, GCV, and UBR is used to validate the proposed method. The simulation study was carried out by generating the Truncated Spline function and the Gaussian Kernel on a combination of sample size variations and variances. The simulation results show that the GCV method provides a higher coefficient of determination (R²) value and better accuracy for each combination of sample sizes and variance variations.

Tensorial conditional gradient method for solving multidimensional ill-posed problems

We consider solving a class of tensorial ill-conditioned problems. This problem is treated as a convex constrained minimization problem. This kind of ill-conditioned problems appears in several applications as color images and video restoration. A new tensor degradation model to recover color image and video from blur and noise is given. Tikhonov regularization approach is used to reduce the effect of noise in the computed solution. This approach leads to a convex tensor minimization problem, that can be solved basing on the conditional gradient method. We adapted the Generalized Cross Validation method to the tensorial model for appropriate choice of the regularization parameter. Numerical tests for image and video restoration are given to illustrate the effectiveness of the proposed approach compared with the classical method.

Integral Estimator with Kernel Approach for Estimating Nonparametric Regression Functions

Derivatives are measurements of how a function change as the input value changes, or in general a derivative shows how one quantity changes due to a change in another quantity. The concept of universal or comprehensive function derivatives is widely used in various scientific fields. For example, in economics, people are interested in studying the condition of the derivative of an objective function as the result of an optimization problem. In this study, nonparametric procedures are used to estimate a function where the form of the function does not lead to a particular function model. Suppose we are given a nonparametric regression model where f is an unknown function. The main problem of regression analysis is to determine the form of estimation f. To determine the estimation of f, one approach that can be used is the integral estimator with the Gaussian Kernel approach. Furthermore, as an application, the Labour Force Participation Rate (y) data is used with the predictor variable, namely the Average Length of Schooling (x). By using the GCV (Generalized Cross Validation) method, the optimal bandwidth is obtained at h = 80 with a GCV value of 0.243 with an MSE value of 32.1864.

Optimal Regularization Method Based on the L-Curve for Solving Rational Function Model Parameters

Rational polynomial coefficients in a rational function model (&lt;small&gt;RFM&lt;/small&gt;) have high correlation and redundancy, especially in high-order &lt;small&gt;RFMs&lt;/small&gt;, which results in ill-posed problems of the normal equation. For this reason, this article presents an optimal regularization method with the L-curve for solving rational polynomial coefficients. This method estimates the rational polynomial coefficients of an &lt;small&gt;RFM&lt;/small&gt; using the L-curve and finds the optimal regularization parameter with the minimum mean square error, then solves the parameters of the &lt;small&gt;RFM&lt;/small&gt; by the Tikhonov method based on the optimal regularization parameter. The proposed method is validated in both terrain-dependent and terrain-independent cases using Gaofen-1 and aerial images, respectively, and compared with the least-squares method, L-curve method, and generalized cross-validation method. The experimental results demonstrate that the proposed method can solve the &lt;small&gt; RFM&lt;/small&gt; parameters effectively, and their accuracy is increased by more than 85% on average relative to the other methods.

Improved Generalized Cross-Validation and Unbiased Predictive Risk Estimator Methods Using the RGSVD: Application to Inversion of Potential Field Data

The inversion of potential field data has widely utilized the generalized cross-validation (GCV) and the unbiased predictive risk estimator (UPRE) methods to determine the regularization parameter. However, these two methods are time-consuming and it is difficult for them to determine the optimal linear search range including the optimal regularization. To solve these problems, this article improves the GCV and UPRE methods using the RGSVD (randomized generalized singular value decomposition) algorithm. The improved methods first use the randomized algorithm to compute an approximate generalized singular value decomposition (GSVD) with less computational time. Then, the optimal linear search range is determined based on the generalized singular values. Finally, the GCV and the UPRE functions are efficiently computed on the basis of the results from the RGSVD algorithm. In this way, the GCV and UPRE methods using the RGSVD algorithm are able to determine the optimal regularization parameter fast and effectively. One comparative test shows the effectiveness and efficiency of the GCV and the UPRE methods using the RGSVD algorithm.

Dynamics of interaction between an Euler-Bernoulli beam and a moving damped sprung mass: Effect of beam surface roughness

This work presents analytical expressions that identify the dynamics of the interaction between a simply supported Euler-Bernoulli beam and a moving sprung damped mass on the beam when the roughness on the beam surface is taken into consideration. The roughness load can have significant effects on the dynamics of the beam and its loading capacity, especially when the beam has poor surface conditions. Using the concept of perturbation, the solutions of the beam and mass are presented and validated by comparing the results with the finite element model. Because the determination of the roughness load requires the solution of an ill-posed inverse problem, the Tikhonov regularization and generalized cross validation methods are used to identify the moving load caused by the beam surface roughness. The performances of different regularization matrices (L matrices) under four noise levels (1%, 5%, 10%, and 20%) and three road classes are investigated in terms of their effectiveness in reducing errors in the prediction of the roughness load and mass motion. The results demonstrate that there is no optimal selection of regularization matrices (L matrices) crossing all the beam surface roughness conditions under the noisy disturbance. Under the lower noise levels (1% and 5%), the L1 matrix provided less error in the solution. However, when the noise level increased to 10% and 20%, the L0 matrix, the lower order regularization matrix, provided an acceptable solution. The results also indicated that beam surface roughness has an important impact on the roughness load identification and mass motion prediction.