Generalized Cross-validation Function Research Articles

Kernel multivariable semiparametric regression model in estimating the level of open unemption in East Java Province

Semiparametric regression is a combination of parametric regression and nonparametric regression. Parametric regression curve components are approximated by multivariable linear functions, nonparametric regression curve components are approximated by Gaussian Kernel function. The purpose of this study is to obtain an estimate of the shape in semiparametric regression using the Kernel estimator and to model the Open Unemployment Rate (TPT) in East Java Province using the semiparametric regression model. This semiparametric regression model estimates on bandwidth. Semiparametric regression model is obtained by minimizing the Generalized Cross Validation function. The semiparametric regression model used to model TPT case data in East Java Province.

Ridge regression and its applications in genetic studies.

With the advancement of technology, analysis of large-scale data of gene expression is feasible and has become very popular in the era of machine learning. This paper develops an improved ridge approach for the genome regression modeling. When multicollinearity exists in the data set with outliers, we consider a robust ridge estimator, namely the rank ridge regression estimator, for parameter estimation and prediction. On the other hand, the efficiency of the rank ridge regression estimator is highly dependent on the ridge parameter. In general, it is difficult to provide a satisfactory answer about the selection for the ridge parameter. Because of the good properties of generalized cross validation (GCV) and its simplicity, we use it to choose the optimum value of the ridge parameter. The GCV function creates a balance between the precision of the estimators and the bias caused by the ridge estimation. It behaves like an improved estimator of risk and can be used when the number of explanatory variables is larger than the sample size in high-dimensional problems. Finally, some numerical illustrations are given to support our findings.

Numerical methods for estimating the tuning parameter in penalized least squares problems

The solution of the penalized least squares problems depends on a tuning parameter. A popular tool for specifying the tuning parameter is the generalized cross-validation (GCV). In this work, we utilize estimates for the GCV function whose minimizers can lead to the determination of the tuning parameter. The selection of an efficient estimate depends on an appropriately defined index of proximity. Bounds and specific values are derived for this index, and a thorough study proves that the proposed one-term estimate suits perfectly to statistical models with high correlated variables. This is confirmed through simulation tests for several datasets.

Efficient estimates in regression models with highly correlated covariates

The specification of accurate ridge estimates in penalized regression models strongly depends on the appropriate choice of the tuning parameter which monitors the regularization process. In this work, we propose the selection of this parameter via the minimization of an extrapolation estimate of the generalized cross-validation function. The efficiency of the estimate is characterized by an appropriately defined index of proximity; in case that its value approaches one, the estimation becomes optimal. We consider regression models with highly correlated covariates and prove that the probability of the index of proximity being close to one is high. This result is confirmed through several simulation tests.

The Effect of the Outliers and Leverage Points in the Construction of the Bayesian and Bootstrap Confidence Intervals

The aim of this research is to compare the bootstrap confidence intervals with the Bayesian confidence intervals for smoothing splines as well as the traditional confidence intervals to determine which of these limits are best in the presence of Outliers and Leverage points in data. The simulation experiments were conducted on two models: the first was linear in the presence of data that was contaminated with outliers and the other with the Leverage points: The second model was nonlinear in the presence of data contaminated with outlying observations Simulation experiments were carried out on different samples. The Penalized Least Squares method was used to fit the Nonparametric regression. The Generalized Cross Validation function (GCV) was used to select the amount of smoothing.

An Eigenvalue Approach For Estimating The Generalized Cross Validation Function For Correlated Matrices

This work proposes a fast estimate for the generalized cross-validation function when the design matrix of an experiment has correlated columns. The eigenvalue structure of this matrix is used to derive probability bounds satisfied by an appropriate index of proximity, which provides a simple and accurate estimate for the numerator of the generalized cross-validation function. The denominator of the function is evaluated by an analytical formula. Several simulation tests performed in statistical models having correlated design matrix with intercept confirm the reliability of the proposed probabilistic bounds and indicate the applicability of the proposed estimate for these models.

Automatic blind deconvolution with Toeplitz-structured sparse total least squares

Given the noise-corrupted seismic recordings, blind deconvolution simultaneously solves for the reflectivity series and the wavelet. Blind deconvolution can be formulated as a fully perturbed linear regression model and solved by the total least-squares (TLS) algorithm. However, this algorithm performs poorly when the data matrix is a structured matrix and ill-conditioned. In blind deconvolution, the data matrix has a Toeplitz structure and is ill-conditioned. Accordingly, we develop a fully automatic single-channel blind-deconvolution algorithm to improve the performance of the TLS method. The proposed algorithm, called Toeplitz-structured sparse TLS, has no assumptions about the phase of the wavelet. However, it assumes that the reflectivity series is sparse. In addition, to reduce the model space and the number of unknowns, the algorithm benefits from the structural constraints on the data matrix. Our algorithm is an alternating minimization method and uses a generalized cross validation function to define the optimum regularization parameter automatically. Because the generalized cross validation function does not require any prior information about the noise level of the data, our approach is suitable for real-world applications. We validate the proposed technique using synthetic examples. In noise-free data, we achieve a near-optimal recovery of the wavelet and the reflectivity series. For noise-corrupted data with a moderate signal-to-noise ratio (S/N), we found that the algorithm successfully accounts for the noise in its model, resulting in a satisfactory performance. However, the results deteriorate as the S/N and the sparsity level of the data are decreased. We also successfully apply the algorithm to real data. The real-data examples come from 2D and 3D data sets of the Teapot Dome seismic survey.

Estimates for the generalized cross-validation function via an extrapolation and statistical approach

Generalized cross-validation (GCV) is a popular tool for specifying the tuning parameter in linear regression model or equivalently the regularization parameter in Tikhonov regularization. In this work, we are concerned with the estimation and minimization of the GCV function by using a combination of an extrapolation procedure and a statistical approach. In particular, we derive families of estimates for the GCV function. By minimizing the estimated GCV function over a grid of values, a GCV estimate of the regularization parameter is achieved. We present several numerical examples to illustrate the effectiveness of the derived families of estimates for approximating the regularization parameter for several linear discrete ill-posed problems.

Using generalized cross validation to select regularization parameter for total variation regularization problems

The regularization approach is used widely in image restoration problems. The visual quality of the restored image depends highly on the regularization parameter. In this paper, we develop an automatic way to choose a good regularization parameter for total variation (TV) image restoration problems. It is based on the generalized cross validation (GCV) approach and hence no knowledge of noise variance is required. Due to the lack of the closed-form solution of the TV regularization problem, difficulty arises in finding the minimizer of the GCV function directly. We reformulate the TV regularization problem as a minimax problem and then apply a first-order primal-dual method to solve it. The primal subproblem is rearranged so that it becomes a special Tikhonov regularization problem for which the minimizer of the GCV function is readily computable. Hence we can determine the best regularization parameter in each iteration of the primal-dual method. The regularization parameter for the original TV regularization problem is then obtained by an averaging scheme. In essence, our method needs only to solve the TV regulation problem twice: one to determine the regularization parameter and one to restore the image with that parameter. Numerical results show that our method gives near optimal parameter, and excellent performance when compared with other state-of-the-art adaptive image restoration algorithms.

Semivarying coefficient least-squares support vector regression for analyzing high-dimensional gene-environmental data

ABSTRACTIn the context of genetics and genomic medicine, gene-environment (G×E) interactions have a great impact on the risk of human diseases. Some existing methods for identifying G×E interactions are considered to be limited, since they analyze one or a few number of G factors at a time, assume linear effects of E factors, and use inefficient selection methods. In this paper, we propose a new method to identify significant main effects and G×E interactions. This is based on a semivarying coefficient least-squares support vector regression (LS-SVR) technique, which is devised by utilizing flexible semiparametric LS-SVR approach for censored survival data. This semivarying coefficient model is used to deal with the nonlinear effects of E factors. We also derive a generalized cross validation (GCV) function for determining the optimal values of hyperparameters of the proposed method. This GCV function is also used to identify significant main effects and G×E interactions. The proposed method is evaluated through numerical studies.

GCV for Tikhonov regularization by partial SVD

Tikhonov regularization is commonly used for the solution of linear discrete ill-posed problems with error-contaminated data. A regularization parameter that determines the quality of the computed solution has to be chosen. One of the most popular approaches to choosing this parameter is to minimize the Generalized Cross Validation (GCV) function. The minimum can be determined quite inexpensively when the matrix A that defines the linear discrete ill-posed problem is small enough to rapidly compute its singular value decomposition (SVD). We are interested in the solution of linear discrete ill-posed problems with a matrix A that is too large to make the computation of its complete SVD feasible, and show how upper and lower bounds for the numerator and denominator of the GCV function can be determined fairly inexpensively for large matrices A by computing only a few of the largest singular values and associated singular vectors of A. These bounds are used to determine a suitable value of the regularization parameter. Computed examples illustrate the performance of the proposed method.

Radial basis function method modelling borehole heat transfer: the practical application

In this work, the method of radial basis function (RBF) is used for the borehole heat exchanger transfers (BHET) problems. The RBF method is an efficient mesh free technique for the numerical solution of partial differential equations. The condition number of the system of linear equations (matrix A) is large it means that the ill-conditioning of matrix A makes the numerical solution unstable. Tikhonov regularization (TR) method are presented to solve such ill-conditioned systems. In this work, generalized cross-validation (GCV) method is carried out to determine the regularization parameter ksi for the TR method that minimizes the GCV function. The some practical results of numerical experiments are presented.

Estimation and variable selection in censored regression model with smoothly clipped absolute deviation penalty

Smoothly clipped absolute deviation (SCAD) penalty is known to satisfy the desirable properties for penalty functions like as unbiasedness, sparsity and continuity. In this paper, we deal with the regression function estimation and variable selection based on SCAD penalized censored regression model. We use the local linear approximation and the iteratively reweighted least squares algorithm to solve SCAD penalized log likelihood function. The proposed method provides an efficient method for variable selection and regression function estimation. The generalized cross validation function is presented for the model selection. Applications of the proposed method are illustrated through the simulated and a real example.

Robust varying coefficient model using L1 regularization

In this paper we propose a robust version of varying coefficient models, which is based on the regularized regression with L1 regularization. We use the iteratively reweighted least squares procedure to solve L1 regularized objective function of varying coefficient model in locally weighted regression form. It provides the efficient computation of coefficient function estimates and the variable selection for given value of smoothing variable. We present the generalized cross validation function and Akaike information type criterion for the model selection. Applications of the proposed model are illustrated through the artificial examples and the real example of predicting the effect of the input variables and the smoothing variable on the output.

LS-SVM for large data sets

In this paper we propose multiclassification method for large data sets by ensembling least squares support vector machines (LS-SVM) with principal components instead of raw input vector. We use the revised one-vs-all method for multiclassification, which is one of voting scheme based on combining several binary classifications. The revised one-vs-all method is performed by using the hat matrix of LS-SVM ensemble, which is obtained by ensembling LS-SVMs trained using each random sample from the whole large training data. The leave-one-out cross validation (CV) function is used for the optimal values of hyper-parameters which affect the performance of multiclass LS-SVM ensemble. We present the generalized cross validation function to reduce computational burden of leave-one-out CV functions. Experimental results from real data sets are then obtained to illustrate the performance of the proposed multiclass LS-SVM ensemble.

Multiclass LS-SVM ensemble for large data

Multiclass classification is typically performed using the voting scheme method based on combining binary classifications. In this paper we propose multiclass classification method for large data, which can be regarded as the revised one-vs-all method. The multiclass classification is performed by using the hat matrix of least squares support vector machine (LS-SVM) ensemble, which is obtained by aggregating individual LS-SVM trained on each subset of whole large data. The cross validation function is defined to select the optimal values of hyperparameters which affect the performance of multiclass LS-SVM proposed. We obtain the generalized cross validation function to reduce computational burden of cross validation function. Experimental results are then presented which indicate the performance of the proposed method.

SVQR with asymmetric quadratic loss function

Support vector quantile regression (SVQR) can be obtained by applying support vector machine with a check function instead of an e-insensitive loss function into the quantile regression, which still requires to solve a quadratic program (QP) problem which is time and memory expensive. In this paper we propose an SVQR whose objective function is composed of an asymmetric quadratic loss function. The proposed method overcomes the weak point of the SVQR with the check function. We use the iterative procedure to solve the objective problem. Furthermore, we introduce the generalized cross validation function to select the hyper-parameters which affect the performance of SVQR. Experimental results are then presented, which illustrate the performance of proposed SVQR.

On generalized cross validation for stable parameter selection in disease models

Abstract In this paper we study advantages and limitations of the Generalized Cross Validation (GCV) approach for selecting a regularization parameter in the case of a partially stochastic linear irregular operator equation. The research has been motivated by an inverse problem in epidemiology, where the goal was to reconstruct a time dependent treatment recovery rate for plasmodium falciparum, the most dangerous form of malaria. Initial numerical simulations gave rise to a theoretical analysis of the expected value of the GCV function and the efficiency of the GCV method for different noise levels. It was shown that, as opposed to L-curve, the GCV does not necessarily generate a systematic error in the value of the regularization parameter for Tikhonov's stabilizing algorithm.

Support vector expectile regression using IRWLS procedure

In this paper we propose the iteratively reweighted least squares procedure to solve the quadratic programming problem of support vector expectile regression with an asymmetrically weighted squares loss function. The proposed procedure enables us to select the appropriate hyperparameters easily by using the generalized cross validation function. Through numerical studies on the artificial and the real data sets we show the effectiveness of the proposed method on the estimation performances.

Composite support vector quantile regression estimation

In this paper we propose a new nonparametric regression method called composite support vector quantile regression (CSVQR) that combines the formulations of support vector regression and composite quantile regression. First the CSVQR using the quadratic programming (QP) is proposed and then the CSVQR utilizing the iteratively reweighted least squares (IRWLS) procedure is proposed to overcome weakness of the QP based method in terms of computation time. The IRWLS procedure based method enables us to derive a generalized cross validation (GCV) function that is easier and faster than the conventional cross validation function. The GCV function facilitates choosing the hyperparameters that affect the performance of the CSVQR and saving computation time. Numerical experiment results are presented to illustrate the performance of the proposed method