Generalized Cross Validation Research Articles

A machine learning approach to optimal Tikhonov regularization I: Affine manifolds

Despite a variety of available techniques, such as discrepancy principle, generalized cross validation, and balancing principle, the issue of the proper regularization parameter choice for inverse problems still remains one of the relevant challenges in the field. The main difficulty lies in constructing an efficient rule, allowing to compute the parameter from given noisy data without relying either on any a priori knowledge of the solution, noise level or on the manual input. In this paper, we propose a novel method based on a statistical learning theory framework to approximate the high-dimensional function, which maps noisy data to the optimal Tikhonov regularization parameter. After an offline phase where we observe samples of the noisy data-to-optimal parameter mapping, an estimate of the optimal regularization parameter is computed directly from noisy data. Our assumptions are that ground truth solutions of the inverse problem are statistically distributed in a concentrated manner on (lower-dimensional) linear subspaces and the noise is sub-gaussian. We show that for our method to be efficient, the number of previously observed samples of the noisy data-to-optimal parameter mapping needs to scale at most linearly with the dimension of the solution subspace. We provide explicit error bounds on the approximation accuracy from noisy data of unobserved optimal regularization parameters and ground truth solutions. Even though the results are more of theoretical nature, we present a recipe for the practical implementation of the approach. We conclude with presenting numerical experiments verifying our theoretical results and illustrating the superiority of our method with respect to several state-of-the-art approaches in terms of accuracy or computational time for solving inverse problems of various types.

Estimator Nadaraya-Watson dengan Pendekatan Cross Validation dan Generalized Cross Validation untuk Mengestimasi Produksi Jagung

<p>Nadaraya-Watson Estimator with kernel approach depends on two-parameter, those are kernel function and bandwidth choice. However, between the two of them, bandwidth choice gave a huge impact on the result of the estimation. By minimizing the value of Mean Square Error (MSE), Cross-Validation (CV) and Generalized Cross-Validation (GCV) gave the optimal bandwidth value. In this research, corn production was considered as the dependent variable, while the planted area, harvested area, and the fertilizer as the independent variable. The result of this research showed that Nadaraya-Watson Estimator with Generalized Cross-Validation gives a better corn production estimation with optimal bandwidth value 742392,2, with and with MSE 202583,9.</p><p><strong>Keywords</strong>: kernel, estimator Nadaraya-Watson, cross validation, generalized cross validation.</p>

Penerapan Model Geographically Weighted Logistic Regression Pada Data Status Kesejahteraan Masyarakat di Kalimantan Tahun 2017

Geographically Weighted Logistic Regression (GWLR) model is a regression model developed from logistic regression which is applied to spatial data. The aims of research is a GWLR model determination on dichotomous data of community welfare status based on the Human Development Index (HDI) and to find the factors influencing the probability of high welfare status of each Regency/City on Kalimantan Island in 2017. Parameters estimation of the GWLR model was done at each observation location using a weighted Maximum Likelihood Estimation (MLE) method and maximum likelihood estimator was obtained by Newton Raphson iterative method. Spatial weighting on parameter estimation was determined using Adaptive Gaussian weighting function and optimum bandwidth was determined using Generalized Cross-Validation (GCV) criterion. Based on the result of GWLR parameter testing, it was concluded that the factors influencing the probability of high welfare status of Regency/City on Kalimantan Island in 2017 were school enrollment rates (senior high school), the number of health workers, real per capita income and the open unemployment rate.

Pengujian Hipotesis Parameter Model Mixed Geographically Weighted Regression Data Indeks Pembangunan Manusia di Kalimantan Tahun 2016

Mixed Geographically Weighted Regression (MGWR) model is a Geographically Weighted Regression (GWR) model with some parameters are global (have the same value) and several other parameters are local (have different values) for each observation location. The purpose of this study is to obtain a MGWR model on the Human Development Index (HDI) data and find out the factors that influence the HDI of each district (city) in the provinces of East Kalimantan, Central Kalimantan and South Kalimantan in 2016. The parameter estimation method is carried out through two stages (backshift), namely local parameter estimation by using the Weighted Least Square (WLS) method and global parameter estimation by using the Ordinary Least Square (OLS) method. Spatial weighting on local parameter estimation is obtained by using an adaptive Bisquare weighting functions, where optimum bandwidth determination uses Generalized Cross-Validation (GCV) criterion. Based on the result of MGWR parameter testing, it was concluded that the school enrollment rates (SMP) affected the HDI of all districts (cities) in East Kalimantan, Central Kalimantan and South Kalimantan, while the population density affects the HDI only in a few districts (cities), namely East Kutai, Balikpapan, Samarinda and Bontang.

Multi-parameter Tikhonov regularization-based OTPA with application to ship-radiated noise evaluation

The purpose of this study is to reduce the errors caused by the inversion of the transfer function (TF) matrix when evaluating ship-radiated noise by operational transfer path analysis. The singular value decomposition (SVD), generalized cross validation (GCV), and L-curve methods are separately introduced to evaluate the TF matrix, and the performances are compared. In order to overcome the shortcomings of the aforementioned methods and further reduce the errors, the optimized multi-parameter (M-P) Tikhonov regularization method based on the criterion of condition number is proposed to create an optimal regularization parameter to evaluate the TF matrix herein. The feasibility is verified with a double-layer cylindrical shell model experiment in Thousand Islets Lake. The obtained results indicate that the average error of M-P Tikhonov regularization is reduced by up to 0.38 dB compared with that of the L-curve, 0.68 dB compared with that of the GCV, and 1.34 dB compared with that of the SVD under various combinations of noise levels, which can provide guidance for ship-radiated noise evaluation in engineering applications.

Regularization of a nonlinear inverse problem by discrete mollification method

‎In this article, the application of discrete mollification as a regularization procedure for solving a nonlinear inverse problem in one dimensional space is considered. Illposedness is identified as one of the main characteristics of inverse problems. It is clear that if we have a noisy data, the inverse problem becomes unstable. As such, a numerical procedure based on discrete mollification and space marching method is applied to address the ill-posedness of the mentioned problem. The regularization parameter is selected by generalized cross validation (GCV) method. The numerical stability and convergence of the proposed method are investigated. Finally, some test problems, whose exact solutions are known, are solved using this method to show the efficiency.

Inverse Load Identification in Stiffened Plate Structure Based on in situ Strain Measurement

For practical engineering structures, it is usually difficult to measure external load distribution in a direct manner, which makes inverse load identification important. Specifically, load identification is a typical inverse problem, for which the models (e.g., response matrix) are often ill-posed, resulting in degraded accuracy and impaired noise immunity of load identification. This study aims at identifying external loads in a stiffened plate structure, through comparing the effectiveness of different methods for parameter selection in regulation problems, including the Generalized Cross Validation (GCV) method, the Ordinary Cross Validation method and the truncated singular value decomposition method. With demonstrated high accuracy, the GCV method is used to identify concentrated loads in three different directions (e.g., vertical, lateral and longitudinal) exerted on a stiffened plate. The results show that the GCV method is able to effectively identify multi-source static loads, with relative errors less than 5%. Moreover, under the situation of swept frequency excitation, when the excitation frequency is near the natural frequency of the structure, the GCV method can achieve much higher accuracy compared with direct inversion. At other excitation frequencies, the average recognition error of the GCV method load identification less than 10%.

A Doubly Regularized Linear Discriminant Analysis Classifier With Automatic Parameter Selection

Linear discriminant analysis (LDA) based classifiers tend to falter in many practical settings where the training data size is smaller than, or comparable to, the number of features. As a remedy, different regularized LDA (RLDA) methods have been proposed. These methods may still perform poorly depending on the size and quality of the available training data. In particular, the test data deviation from the training data model, for example, due to noise contamination, can cause severe performance degradation. Moreover, these methods commit further to the Gaussian assumption (upon which LDA is established) to tune their regularization parameters, which may compromise accuracy when dealing with real data. To address these issues, we propose a doubly regularized LDA classifier that we denote as R2LDA. In the proposed R2LDA approach, the RLDA score function is converted into an inner product of two vectors. By substituting the expressions of the regularized estimators of these vectors, we obtain the R2LDA score function that involves two regularization parameters. To set the values of these parameters, we adopt three existing regularization techniques; the constrained perturbation regularization approach (COPRA), the bounded perturbation regularization (BPR) algorithm, and the generalized cross-validation (GCV) method. These methods are used to tune the regularization parameters based on linear estimation models, with the sample covariance matrix's square root being the linear operator. Results obtained from both synthetic and real data demonstrate the consistency and effectiveness of the proposed R2LDA approach, especially in scenarios involving test data contaminated with noise that is not observed during the training phase.

On the excess of average squared error for data-driven bandwidths in nonparametric trend estimation

We consider the problem of the optimal selection of the smoothing parameter $$${h}$$$ in kernel estimation of a trend in nonparametric regression models with strictly stationary errors. We suppose that the errors are stochastic volatility sequences. Three types of volatility sequences are studied: the log-normal volatility, the Gamma volatility and the log-linear volatility with Bernoulli innovations. We take the weighted average squared error (ASE) as the global measure of performance of the trend estimation using $$${h}$$$ and we study two classical criteria for selecting $$${h}$$$ from the data, namely the adjusted generalized cross validation and Mallows-type criteria. We establish the asymptotic distribution of the gap between the ASE evaluated at one of these selectors and the smallest possible ASE. A Monte-Carlo simulation for a log-normal stochastic volatility model illustrates that this asymptotic approximation can be accurate even for small sample sizes.

Regresi Nonparametrik Kernel Gaussian pada Pemodelan Angka Kelahiran Kasar di Provinsi Nusa Tenggara Barat

This study aims to model Crude Birth Rates (CBR) in West Nusa Tenggara Province. The nonparametric regression method was used in this research by considering data distribution patterns that do not show a linear relationship between variables. In this case, the kernel nonparametric regression using the Gaussian function and the Nadaraya-Watson estimator. The results showed optimal bandwidths of 0.55542837, 1.29042927, 0.94706041, and 0.92278896 with a value of minimum Generalized Cross-Validation (GCV) of 0.000000000432613511, which was minimized by the simulated annealing algorithm. The resulting model's accuracy can be seen from the coefficient of determination (R2) of 99.23% and the Mean Absolute Percentage Error (MAPE) of 0.007049%.

Truncated Singular Value Decomposition Regularization for Estimating Terrestrial Water Storage Changes Using GPS: A Case Study over Taiwan

It is a typical ill-conditioned problem to invert GPS-measured loading deformations for terrestrial water storage (TWS) changes. While previous studies commonly applied the 2nd-order Tikhonov regularization, we demonstrate the truncated singular value decomposition (TSVD) regularization can also be applied to solve the inversion problem. Given the fact that a regularized estimate is always biased, it is valuable to obtain estimates with different methods for better assessing the uncertainty in the solution. We also show the general cross validation (GCV) can be applied to select the truncation term for the TSVD regularization, producing a solution that minimizes predictive mean-square errors. Analyzing decade-long GPS position time series over Taiwan, we apply the TSVD regularization to estimate mean annual TWS variations for Taiwan. Our results show that the TSVD estimates can sufficiently fit the GPS-measured annual displacements, resulting in randomly distributed displacement residuals with a zero mean and small standard deviation (around 0.1 cm). On the island-wide scale, the GPS-inferred annual TWS variation is consistent with the general seasonal cycle of precipitations. However, on smaller spatial scales, we observe significant differences between the TWS changes estimated by GPS and simulated by GLDAS land surface models in terms of spatiotemporal pattern and magnitude. Based on the results, we discuss some challenges in the characterization of TWS variations using GPS observations over Taiwan.

North American historical monthly spatial climate dataset, 1901\u20132016

We present historical monthly spatial models of temperature and precipitation generated from the North American dataset version “j” from the National Oceanic and Atmospheric Administration’s (NOAA’s) National Centres for Environmental Information (NCEI). Monthly values of minimum/maximum temperature and precipitation for 1901–2016 were modelled for continental United States and Canada. Compared to similar spatial models published in 2006 by Natural Resources Canada (NRCAN), the current models show less error. The Root Generalized Cross Validation (RTGCV), a measure of the predictive error of the surfaces akin to a spatially averaged standard predictive error estimate, averaged 0.94 °C for maximum temperature models, 1.3 °C for minimum temperature and 25.2% for total precipitation. Mean prediction errors for the temperature variables were less than 0.01 °C, using all stations. In comparison, precipitation models showed a dry bias (compared to recorded values) of 0.5 mm or 0.7% of the surface mean. Mean absolute predictive errors for all stations were 0.7 °C for maximum temperature, 1.02 °C for minimum temperature, and 13.3 mm (19.3% of the surface mean) for monthly precipitation.

Exploring the Link Between Additive Heritability and Prediction Accuracy From a Ridge Regression Perspective.

Genome-Wide Association Studies (GWAS) explain only a small fraction of heritability for most complex human phenotypes. Genomic heritability estimates the variance explained by the SNPs on the whole genome using mixed models and accounts for the many small contributions of SNPs in the explanation of a phenotype. This paper approaches heritability from a machine learning perspective, and examines the close link between mixed models and ridge regression. Our contribution is two-fold. First, we propose estimating genomic heritability using a predictive approach via ridge regression and Generalized Cross Validation (GCV). We show that this is consistent with classical mixed model based estimation. Second, we derive simple formulae that express prediction accuracy as a function of the ratio , where n is the population size and p the total number of SNPs. These formulae clearly show that a high heritability does not imply an accurate prediction when p > n. Both the estimation of heritability via GCV and the prediction accuracy formulae are validated using simulated data and real data from UK Biobank.

REGRESI NONPARAMAETRIK SPLINE PADA DATA LAJU PERTUMBUHAN EKONOMI DI KALIMANTAN

The Economic Growth Rate (EGR) is an important indicator to measure the success of economic development.The welfare and progress of an economy is determined by the amount of growth shown by changes in the quantity of goods and services produced nationally. High economic growth is a goal that is expected to be achieved in a developing country. Many factors affect EGR in Kalimantan, so it is necessary to do modeling to find out the factors that significantly affect EGR. This study uses 6 factors that are suspected to influence EGR, namely the labor force participation rate, the number of large and medium industries, the average length of schooling, regional income and expenditure budgets, general allocation funds and rice productivity. The data is 2017 data obtained from the Central Statistical Agency (CSA) in 5 provinces in Kalimantan. The method used to model EGR is spline nonparametric regression and obtained the optimal knot point with the smallest GCV (Generalized Cross Validation) value of three knots. Based on the research results obtained an R2 of 82.15 percent which shows that the model formed is suitable to be used to model data patterns and there are 6 variables that have a significant effect on EGR namely labor force participation rates, number of large and medium industries, average length of schooling, regional income and expenditure budgets, general allocation funds and rice productivity.

Kernel-based spatial error model for analyzing spatial panel data

Spatial panel data model captures spatial interactions across spatial units and over time. Lots of effort have been devoted to develop effective estimation methods for parametric and nonparametric spatial panel data models. Varying coefficient model has received a great deal of attention as an important tool for modeling panel data. In this paper we propose a kernel-based spatial error model for the purpose of analyzing spatial panel data. This model is based on the idea of fixed effect time-varying coefficient model and the kernel technique of support vector machine along with the technique of regularization. A generalized cross validation method is also considered for choosing the hyperparameters which affect the performance of the proposed model. The proposed model is evaluated through numerical studies.

A new method for gridding passive microwave data with mixed measurements and spatial correlation

In this article we develop a new method to grid passive microwave data in the presence of spatial correlation patterns. Our proposal combines a Tychonov inverse method with a generalized cross validation procedure to grid the observations over a discrete retrieval grid. To build this grid, the study region is partitioned into objects following an object-based image analysis procedure. Then, this partition is refined into small cells of similar size. Two cells from the same object are considered of the same type. The procedure to estimate the brightness temperature of each cell is based on a least-squares estimation with a cell-type aware Tychonov regularization method. This method assumes that the brightness temperature heterogeneity within each cell can be neglected and that adjacent cells of the same type have similar brightness temperature. In other words, spatial correlations are considered within each object in the scene. The Tychonov regularization parameter is found using a fast generalized cross validation procedure that makes it possible to solve the inverse problem when the observational error variance is not known. We evaluate the proposed method using different synthetic scenarios and compare it with other methods. The evaluation shows an excellent performance of the proposed method when the brightness temperature field varies smoothly over each object in the scene. But it also shows that the method is competitive when the brightness temperature field does not present spatial correlations. We conclude that the proposed algorithm provides a fast and robust method to solve the original inverse problem.

Moving load identification on Euler-Bernoulli beams with viscoelastic boundary conditions by Tikhonov regularization

This work presents moving load identification on Euler-Bernoulli beams with viscoelastic boundary conditions based on beam acceleration responses. The Tikhonov regularization and generalized cross validation (GCV) methods are used to investigate the performances of different regularization matrices (L matrices) in terms of their effectiveness in reducing errors in load identification. The effects from noises, inaccurate parameters, arrangements of measurement locations, velocities, and spaces of moving loads are included in the investigations. Simulation results demonstrated that the viscoelastic boundary conditions could play an important role in the performance of these time domain moving load identifications. The use of a higher-order regularization matrix (L matrix) can efficiently reduce the relative errors of the identified loads. In these L matrices, the L 1 matrix could provide a compromised approach to reducing the relative errors more efficiently than higher-order L matrices such as L 2, L 3, and L 4.

PEMODELAN KASUS PNEUMONIA PADA BALITA DI PROVINSI BALI MENGGUNAKAN METODE REGRESI NONPARAMETRIK B-SPLINE

Pneumonia is an inflammatory lung disease caused by bacterium Streptococcus pneumonia, Chlamydophila pneumonia bacteria, influenza virus, and fungi. Bali Provincial Health Service data in 2018 shows that one of the diseases that causes many deaths in children under five is pneumonia. This study aims to model the number of pneumonia cases in children under five in Bali Province in 2018 with six research variables, namely percentage of low birth weight, percentage of coverage of infants who receive vitamin A, percentage of infants who receive exclusive breastfeeding, percentage of under-five health services, percentage of villages that implement community-based total sanitation, and percentage of neonatal complications management. In the B-Spline function there are connecting points called knots. The best estimation of the B-Spline regression model is obtained from selecting the optimum knot point with the criteria for the value of generalized cross validation (GCV) and the selected mean square error (MSE) having the minimum value. The B-Spline regression model that is formed is the quadratic B-Spline model (order 3) with five knots resulting in an value of 87.79%.

KOMPUTASI GUI-R UNTUK PEMODELAN REGRESI NONPARAMETRIK BIRESPON POLINOMIAL LOKAL PADA PENGARUH SUKU BUNGA BI TERHADAP INDEKS HARGA SAHAM GABUNGAN DAN KURS USD

Economy is one of important indicator of development country. Capital market is one of important tool in economy. The development of the capital market in Indonesian can be seen based on the composite stock price index (CSPI). Other than capital market, international trade is an important tool in the economy. Existence of the international trade generates exchange rate, one of which is USD exchange rate. Exchange rate can be increased and weakened, so it’s stability needs to be maintained. One of the factor that can influence CSPI and USD exchange rate is the BI interest rate. To be able to predict the value of CSPI and USD exchange rate then do the birespon regression modelling because between CSPI and USD exchange rate there are relationship. The regression model approach which used in this research is local polynomial. This approach has high adaptability with data. To make the modelling easier so this research arrange Graphycal User Interface (GUI) by using R software. The local polynomial birespon regression is applied to CSPI and USD exchange rate data based on BI interest rate by using GUI. The optimal modal is obtained by General Cross Validation (GCV) optimation. The optimal model is model by combination of sequences two and three, bandwidths 6 and 2,7, and local points 5,75 and 6. The value of R Square is 66,68% and the mean absolute percentage error (MAPE) is 4,0798%. This MAPE shows that the optimal model has very high accuration in prediction the data because this value of MAPE less than 10%.Keywords: CSPI, USD exchange rate, BI interest rate, birespon, local polynomial, GUI.

Current distribution reconstruction in switching arcs by means of regularization based on GSVD

The knowledge of the variation of arc current is helpful in improving the breaking performance of a vacuum circuit breaker. As a novel non-intrusive method, arc magnetic performance testing technology has attracted attention. In view of the widely used regularization method of the standard form in arc current reconstructions cannot fully overcome the ill-posed nature of the Biot-Savart equation, the regularization approach based on generalized singular value decomposition (GSVD) is proposed to improve the situation. Moreover, a discrete 3-D arc model with the help of the finite element technique incorporating the current divergence-free and boundary conditions is introduced to improve the authenticity of the model. The performance of the truncated generalized singular value decomposition (TGSVD) and general-form Tikhonov regularization with derivative operators L 1 and L 2 are studied in numerical tests. In addition, the effectiveness of generalized cross validation criterion, quasi optimality criterion and normalized cumulative periodic graph method in selecting regularization parameters are compared. The methodology can help improve arc control and guide the design of vacuum circuit breakers with improved performance parameters.