Generalized Cross Validation Research Articles

Pendekatan Model Nonparametrik untuk Memodelkan Hubungan Antara Jumlah Uang Beredar dan Indeks Harga Konsumen di Indonesia Tahun 1969-2017

<p class="Abstract-Title">Inflation is one of the macroeconomic variables of concern to the government in addition to economic growth, unemployment and poverty. Inflation is measured by the Consumer Price Index (CPI). According to the quantity theory of the classics, argues that the price level is determined by the amount of money in circulation, prices will rise if there is an increase in the money supply, assuming the amount of goods offered is fixed, while the amount of money is doubled, sooner or later the price will doubled. Often the relationship between macroeconomic variables is not always linear, it can be exponential, logarithmic, or highly fluctuating patterns. This nonlinear relationship cannot be forced using parametric regression which generally uses the Ordinary Least Square (OLS) or Maximum Likelihood Estimation (MLE) which often implies the existence of certain distributions and linear data patterns. In some literatures, researches using a linear model with OLS, for describing the relationship between CPI and money supply. This research uses several non parametric approaches, namely kernel and <em>spline</em> functions. The results obtained are a strong positive relationship between money supply and CPI, where money supply has a significantly positive effect on CPI. The most suitable non parametric method to describe the relationship pattern between CPI and money supply is the smoothing <em>spline</em> method with Generalized Cross Validation (GCV) parameter optimization method with the smallest RMSE and MAPE criteria and functions that can follow data patterns smoothly.</p><p class="Abstract-Title"><strong>Keywords</strong>: CPI, money supply, non parametric, kernel, <em>spline</em>.</p>

The potential of generalized additive modelling for the prediction of radial growth of Norway spruce from Central Germany

During the past decades managed forest ecosystems in Central Europe underwent vast changes, induced by extreme climate conditions and occasionally adverse forest management. Tree ring width patterns mirror these changes and thus have been widely examined as environmental archives and reliable empirical data sources in ‘tree growth modelling’. Dendrochronologists often suppose linear co-variation among the covariates, variable independence and homoscedasticity. Conventionally, these assumptions were achieved by eliminating biological age trends (detrending) and removing the autocorrelation from the time series (pre-whitening). Particularly detrending might be biased according to the scientific problem and sometimes inflexible age models. In this study, we tackle these issues and examine the suitability of a flexible Generalized Additive Model (GAM) on recently developed tree ring width time series of 30 Norway spruce stands (Picea abies [L.] H. Karst) from Central Germany.The model was established to simultaneously cope with the mentioned detrending issue, to unravel nonlinear climate-growth relationships and to predict mean ring width time series for spruce stands in the region. Particularly the latter was of primary interest, since recent forest planning relies on static yield tables that often underestimate the actual growth.The model reliably captured the empirical data, indicated by a small Generalized Cross Validation criterion (GCV = 0.045) and a deviance explained of 88.6 %. The flexible additive smoothers accounted for the social status of individual trees, captured low frequency variations of changing growth conditions adequately and displayed a rather flat biological age trend. The radial increment responded positively to summer season precipitation of the current and previous year. Positive temperature responses were found during the early vegetation period, whereas high summer season temperatures negatively affected the radial growth. The seasonal transition from spring to summer in June induced a shift in the climate response of the linear predictor, leading to a distinct negative effect of temperature and a no-role of precipitation on the linear predictor.Most important, utilizing the calibrated GAM for the purely climate-driven prediction of mean ring width time series from five independent spruce sites revealed proper coherencies. Herein, the mean ring width for sites located within the climatic-optimum for spruce growth were more exactly predicted than for sites with adverse spruce growth conditions. In addition, large mean ring widths were systematically underestimated, whereas small mean ring widths were precisely predicted. Overall, we strongly recommend GAMs as a powerful tool for the investigation of nonlinear climate-growth relationships and for the prediction of radial growth in managed forest ecosystems.

Pemodelan Data Time Series dengan Pendekatan Regresi Nonparametrik B-Spline

Spline is one of the nonparametric approach, to adjust data so the final model has good flexibility. The purpose of this research is to model the time series data in the form of currency exchange rates by using the nonparametric B-spline approach. In B-spline modelling, determination of the order for the model, and the number and the placement of the knot are the criteria that must be considered. The best B-spline model obtained based on the selection of the optimal knot points with minimum Generalized Cross Validation (GCV) criteria. The modelling in this research use data on the exchange rate of the rupiah toward the US dollar in the period January 2014 - December 2018. The best B-spline model obtained by the 2 point knot approach, at points 11935.10 and 12438.29, with GCV valueequals to 55683.09.Keywords: Nonparametric Regression; B-Spline; Generalized Cross Validation

Combination Estimation of Smoothing Spline and Fourier Series in Nonparametric Regression

So far, most of the researchers developed one type of estimator in nonparametric regression. But in reality, in daily life, data with mixed patterns were often encountered, especially data patterns which partly changed at certain subintervals, and some others followed a recurring pattern in a certain trend. The estimator method used for the data pattern was a mixed estimator method of smoothing spline and Fourier series. This regression model was approached by the component smoothing spline and Fourier series. From this process, the mixed estimator was completed using two estimation stages. The first stage was the estimation with penalized least squares (PLS), and the second stage was the estimation with least squares (LS). Those estimators were then implemented using simulated data. The simulated data were gained by generating two different functions, namely, polynomial and trigonometric functions with the size of the sample being 100. The whole process was then repeated 50 times. The experiment of the two functions was modeled using a mixture of the smoothing spline and Fourier series estimators with various smoothing and oscillation parameters. The generalized cross validation (GCV) minimum was selected as the best model. The simulation results showed that the mixed estimators gave a minimum (GCV) value of 11.98. From the minimum GCV results, it was obtained that the mean square error (MSE) was 0.71 and R2 was 99.48%. So, the results obtained indicated that the model was good for a mixture estimator of smoothing spline and Fourier series.

ESTIMASI SELANG KEPERCAYAAN NILAI UJIAN NASIONAL BERBASIS KOMPETENSI BERDASARKAN MODEL REGRESI SEMIPARAMETRIK MULTIRESPON TRUNCATED SPLINE

Confidence interval estimation is important in statistical inference for the parameters of the regression model, but the theory of confidence interval estimation for multi-response semiparametric regression model parameters based on the truncated spline estimator has not been examined. In this study, we estimate the confidence interval of the multi-response semiparametric regression model based on the truncated spline estimator by using pivotal quantity method with the central limit theorem approach. This confidence interval theory is applied to data of competency-based national exam (UNBK) scores in West Nusa Tenggara Province where its UNBK in the lowest position among other provinces in Indonesia. The method used for estimating parameters is weighted least square. The best model is determined based on the Generalized Cross Validation (GCV) minimum value. Based on the estimated 95% confidence interval of parameters of the multi-response truncated spline semiparametric regression model, the results showed that the insignificant factors affecting the UNBK scores were gender and parental education duration while the report card of scores and USBK scores had a positive effect on the UNBK scores but only the UNBK scores of mathematics that report card of scores factor has a negative effect on it.

Boundary-consistent B-spline filtering schemes and application to high-fidelity simulations of turbulence

A filtering operation, based on B-spline discretizations, is introduced to target weakly growing mesh-scale oscillations that can arise in high-fidelity turbulence simulations. This is a spectral regularization that can be described using the singular values of a banded matrix operator, with the filtering strength set by a scalar- or vector-valued penalty parameter. The penalty parameter can be specified though it can also be advantageously selected to minimize the generalized cross validation (GCV) measure of distance between the pre- and post-filtered solutions. Efficient algorithms are developed to compute both the scalar and vector penalty parameters. The B-spline filter has a sharper localization to high-wavenumber than compact or explicit filters of the same stencil width and is demonstrated for solutions of the Burgers' equation, decaying Burgers' turbulence, and compressible Navier–Stokes turbulent channel flow. These simulations confirm the scheme's numerical stability and ability to narrowly target the high wavenumber components of numerical solutions. An advantage over finite-difference filters is that these B-spline filters are stable on bounded domains and even preserve formal order of accuracy.

Comparative studies on the criteria for regularization parameter selection based on moving force identification

The studies on inverse problems exist extensively in aerospace, mechanical, identification, detection, scanning imaging and other fields. Its ill-posed characteristics often lead to large oscillations in the solution of the inverse problem. In this study, the truncated generalized singular value decomposition (TGSVD) method is introduced to identify two kinds of moving forces, single and multi-axial forces. The truncating point is the most influential regularization parameter of TGSVD, which is initially selected by two classic regularization parameter selection criteria, namely, the L-curve criterion and the generalized cross-validation (GCV) criterion. Due to numerical non-uniqueness and noise disturbance in moving force identification (MFI), numerical simulation results show that neither of the two criteria can effectively help select the optimal truncating point of TGSVD. Hence, a relative percentage error (RPE) criterion is proposed for selecting the truncating point of TGSVD. Comparative studies show that the RPE criterion can be used to select the optimal truncating point of TGSVD more accurately against the GCV criterion and L-curve criterion. Moreover, the RPE criterion can be used to reflect the connections between certain properties and the ill-posedness problem existing in MFI, which should be adopted priority for the optimal truncating point selection of TGSVD.

Modelling of SDG’s Achievement in East Java Using Bi-responses Nonparametric Regression with Mixed Estimator Spline Truncated and Kernel

The Government of Indonesia has grouped 17 objectives of SDG’s into 4 pillars, among others: social pillars, economic pillars, the pillars of environmental development and the pillars of legal development and Governance. Some strategic indicators are used to know the achievement of SDG’S especially achievement in social and economic pillars such as the Human Development Index (HDI) and Gini Ratio. There are several variables affecting then. A method used to model the effects of its variables is regression analysis. The results of the initial exploration through the scatter plot shows a pattern that changes and disconnects at a certain point for a variable IPM with a variable that is allegedly influencing it. With a scatter plot, it can be found that between the Gini ratio variables with some of its predictive variables, does not have a specific pattern. Bi-responses nonparametric regression is an analysis that appropriates with that case. Bi-responses nonparametric regression analysis is a type of nonparametric regression analysis used when the pattern between two response variables and some of the predictor variables is unknown or if there is no prior information regarding the pattern of them. Most of researchers use bi-responses nonparametric regression approached by spline truncated, kernel and etc. Most of the research, use kernel and spline truncated because of their advantages such as: have a good capability to develop model from unknown pattern data and have a good visual representation. The first aim of this paper is obtaining the estimator of mixed estimator spline truncated and kernel in bi-responses nonparametric regression with weighted least square. The second aim is gaining a best model to explain the relationship indicators of SDG’s achievement with some variables affecting them. To obtain the best model, it is necessary to select the optimal of knots point and bandwidth parameters using a smallest Generalized Cross Validation (GCV).

PEMODELAN JUB DAN BI RATE TERHADAP INFLASI DAN KURS RUPIAH MENGGUNAKAN REGRESI SEMIPARAMETRIK BIRESPON BERDASARKAN ESTIMATOR PENALIZED SPLINE

Some indicators of the Indonesian economy are inflation and the exchange rate of rupiah against US dollar. Inflation and the rupiah exchange rate are thought to be influenced by the money supply (JUB) and the BI Rate. The money supply has a nonparametric relationship pattern to inflation and the rupiah exchange rate, while the BI Rate has a parametric relationship pattern to inflation and the rupiah exchange rate. The right method for detecting the relationship between inflation and the exchange rate with JUB and BI Rate is birespon semiparametric regression with a splined penalized estimator. The semiparametric regression coefficient of birespon spline penalized is estimated using the Weighted Least square (WLS) method which is determined based on the degree of polynomials, the number and location of the optimal knot points, and the optimal lambda determined based on the minimum of Generalized Cross Validation (GCV). This research uses the R Program. Based on the results of the analysis, the best spline penalized birespon semiparametric regression model is located in the number of knots is 5 at the knot points of 5257,783; 6649,469; 8976,871; 11099,19 and 13535,51 found in the first degree of response is 1 and the second degree of response is 2 with an optimal lambda of 99,99. The results of the performance evaluation of the model produce value of is 99,9007%, meaning that the model's performance is very good for out samples of the data and the MAPE value of 2.89169% is less than 10% which means the model's performance is very good.

A parameter choice rule for Tikhonov regularization based on predictive risk

In this work, we propose a new criterion for choosing the regularization parameter in Tikhonov regularization when the noise is white Gaussian. The criterion minimizes a lower bound of the predictive risk, when both data norm and noise variance are known, and the parameter choice involves minimizing a function whose solution depends only on the signal-to-noise ratio. Moreover, when neither noise variance nor data norm is given, we propose an iterative algorithm which alternates between a minimization step of finding the regularization parameter and an estimation step of estimating signal-to-noise ratio. Simulation studies on both small- and large-scale datasets suggest that the approach can provide very accurate and stable regularized inverse solutions and, for small sized samples, it outperforms discrepancy principle, balancing principle, unbiased predictive risk estimator, L-curve method, generalized cross validation, and quasi-optimality criterion, and achieves excellent stability hitherto unavailable.

Uncertain stochastic ridge estimation in partially linear regression models with elliptically distributed errors

ABSTRACTIn fitting a regression model to survey data, using additional information or prior knowledge, stochastic uncertainty occurs in specifying linear programming due to economic and financial studies. These stochastic constraints, definitely cause some changes in the classic estimators and their efficiencies. In this paper, stochastic shrinkage estimators and their positive parts are defined in the partially linear regression models when the explanatory variables are multicollinear. Also, it is assumed that the errors are dependent and follow the elliptically contoured distribution. The exact risk expressions are derived to determine the relative dominance properties of the proposed estimators. We used generalized cross validation (GCV) criterion for selecting the bandwidth of the kernel smoother and optimal shrinkage parameter. Finally, the Monté-Carlo simulation studies and an application to real world data set are illustrated to support our theoretical findings.

Semiparametric regression curve estimation for longitudinal data using mixed spline truncated and fourier series estimator

Semiparametric regression is a regression approach that is used when the form of regression curve is assumed to be partly known and partly unknown. In the semiparametric regression model, the same type of estimation method is generally used for some or all of the predictor variables. There are many examples of data cases that have different patterns on each predictor variable, so if the data is forced to use only one form of estimator to estimate the regression curve, it will produce estimation that will not be appropriate to the data pattern. Therefore, a mixed estimator needs to be developed. However, mixed estimator in previous studies was only able to handle cross-sectional data. So this study uses longitudinal data that has advantages, one of which is estimation of each individual characteristic and time (period) characteristic are obtained separately in longitudinal data. Therefore, this study develops a mixed Spline Truncated and Fourier Series estimator in semiparametric regression for longitudinal data using the Weighted Least Square method. Estimation of the semiparametric regression model for longitudinal data using mixed estimator of Spline Truncated and Fourier Series is where D (K,m) = XA (K,m) + TB (K,m) + ZC (K,m). The selection of the best model is based on knot points and optimum oscillation parameters that should be selected optimally using minimum Generalized Cross Validation (GCV) value.

Three form fourier series estimator semiparametric regression for longitudinal data

Analysis of regressionis one technique that is often used in statistical analysis. There are three regression analysis approaches, such as parametric regression, nonparametric regression and semiparametric regression. Semiparametric regression consists of parametric components and nonparametric components. Parametric component that used such as linear estimator and nonparametric component by using a Fourier series estimator. Semiparametric regression approach that use Fourier series, have an advantages which is can resolve oscillation data pattern. This study compares the three Fourier series estimators such as sine, cosine, and combination between cosine and sine or complete estimator for longitudinal data. Longitudinal data can explain more complete information than cross section data or time series data. The purpose of this study is to introduce another Fourier series for the application of electricity consumption in Madura island. The results of this study indicated the optimal model in predicting electricity consumption in Madura island. The best estimator is the Fourier series estimator with the smallest Generalized Cross Validation (GCV) and Mean Square Error (MSE), and the biggest determination coefficient values by considering the parsimony of the model.

A Conversation with Grace Wahba

Grace Wahba (née Goldsmith, born August 3, 1934), I. J. Schoenberg-Hilldale Professor of Statistics at the University of Wisconsin-Madison (Emerita), is a pioneer in methods for smoothing noisy data. Her research combines theoretical analysis, computation and methodology motivated by innovative scientific applications. Best known for the development of generalized cross-validation (GCV), the connection between splines and Bayesian posterior estimates, and “Wahba’s problem,” she has developed methods with applications in demographic studies, machine learning, DNA microarrays, risk modeling, medical imaging and climate prediction. Grace grew up in the Washington, DC area and New Jersey, and graduated from Montclair High School. She was educated at Cornell (B.A. 1956), University of Maryland, College Park (M.A. 1962) and Stanford (Ph.D. 1966), and worked in industry for several years before receiving her doctorate in 1966 and settling in Madison in 1967. Although holding several visiting appointments, she has made Madison her home for over 50 years. She is the author of Spline Models for Observational Data which has garnered more than 8000 citations. Grace is treasured as an academic advisor and has mentored 39 Ph.D. students that have resulted in more than 330 academic descendants. She was elected to the United States National Academy of Sciences in 2000 and received an honorary degree of Doctor of Science from the University of Chicago in 2007. Wahba is a Fellow of several academic societies including the American Academy of Arts and Sciences, the American Association for the Advancement of Science, the American Statistical Association and the Institute of Mathematical Statistics. Over the years, she has received a selection of notable awards in the statistics community: R. A. Fisher Lectureship (2014), Gottfried E. Noether Senior Researcher Award (2009), Committee of Presidents of Statistical Societies Elizabeth Scott Award (1996) and the first Emanuel and Carol Parzen Prize for Statistical Innovation (1994).

Point data reconstruction and smoothing using cubic splines and clusterization

An algorithm to smooth a sequence of noisy data in Rd with cubic polynomials is herein presented. The data points are assumed to be sequentially ordered, with the idea that they are sampled in time/space and represent an unknown curve to be reconstructed. An example is the reconstruction of left and right borders of a road from GPS and/or lidars data, that is a common problem encountered in applications for autonomous vehicles or for the generation of high definition digital maps. The result is a sequence of denoised points, that ideally, belong to the original unknown curve, which is the basis for an interpolation process that aims at building the best approximating curve. The problem is solved employing a least squares approach, with quasi-orthogonal projections. The possibility to weight the samples is provided, as well as a Tikhonov regularisation term to penalise the magnitude of the derivative of the cubic curve used to smooth the data. The Tikhonov term is weighted with a parameter that is determined applying the Generalised Cross-Validation (GCV), which, in particular cases, can be written in closed form. We show and validate the algorithm on a real application example reconstructing the road borders of the circuit track of Doha, Qatar.

Regularized supervised Bayesian approach for image deconvolution with regularization parameter estimation

Image deconvolution consists in restoring a blurred and noisy image knowing its point spread function (PSF). This inverse problem is ill-posed and needs prior information to obtain a satisfactory solution. Bayesian inference approach with appropriate prior on the image, in particular with a Gaussian prior, has been used successfully. Supervised Bayesian approach with maximum a posteriori (MAP) estimation, a method that has been considered recently, is unstable and suffers from serious ringing artifacts in many applications. To overcome these drawbacks, we propose a regularized version where we minimize an energy functional combined by the mean square error with H1 regularization term, and we consider the generalized cross validation (GCV) method, a widely used and very successful predictive approach, for choosing the smoothing parameter. Theoretically, we study the convergence behavior of the method and we give numerical tests to show its effectiveness.

Smoothing and Differentiation of Kinematic Data Using Functional Data Analysis Approach: An Application of Automatic and Subjective Methods

Smoothing is one of the fundamental procedures in functional data analysis (FDA). The smoothing parameter λ influences data smoothness and fitting, which is governed by selecting automatic methods, namely, cross-validation (CV) and generalized cross-validation (GCV) or subjective assessment. However, previous biomechanics research has only applied subjective assessment in choosing optimal λ without using any automatic methods beforehand. None of that research demonstrated how the subjective assessment was made. Thus, the goal of this research was to apply the FDA method to smoothing and differentiating kinematic data, specifically right hip flexion/extension (F/E) angle during the American kettlebell swing (AKS) and determine the optimal λ . CV and GCV were applied prior to the subjective assessment with various values of λ together with cubic and quintic spline (B-spline) bases using the FDA approach. The selection of optimal λ was based on smoothed and well-fitted first and second derivatives. The chosen optimal λ was 1 × 10 − 12 with a quintic spline (B-spline) basis and penalized fourth-order derivative. Quintic spline is a better smoothing and differentiation method compared to cubic spline, as it does not produce zero acceleration at endpoints. CV and GCV did not give optimal λ , forcing subjective assessment to be employed instead.

Precise instantaneous velocimetry and accelerometry with a stand-alone GNSS receiver based on sparse kernel learning

Abstract A new method is proposed to determine instantaneous velocities and accelerations with a stand-alone GNSS receiver. It is based on sparse kernel learning theory. Kernel trick is employed to represent the kinematics, and L1 norm regularization is used to get sparse solution. Analytical models rather than data series are provided, with which velocity and acceleration at any instant could be calculated. Between-epoch displacements, provided by GNSS time difference carrier phase technique, are used as training data. Efficient numerical algorithms, such as the tri-diagonal matrix algorithm and the fast iterative shrinkage thresholding algorithm, are employed to deal with the between-epoch correlations in the data and the L1 norm regularization, respectively. The hyperparameters are optimized using general cross validation or Akaike information criterion, both tailored for the L1 norm regularization problem. Both simulation and real-data results show the superior performance of the proposed method compared to the conventional finite difference method. Especially for dynamic cases, the proposed method can better represent the real dynamics and provide high-accuracy velocimetry and accelerometry results.

Generalized Cross-Validation for Simultaneous Optimization of Tuning Parameters in Ridge Regression

When multicollinearity exists in the context of robust regression, ridge rank regression estimator can be used as an alternative to the rank estimator. Performance of the ridge rank regression estimator is highly dependent on the ridge parameter, here the tuning parameter. On the other hand, suppose we are provided with some non-sample uncertain prior information (UPI) about regression coefficients. Shrinkage estimation is a well-known strategy to improve estimation, under the UPI, where the amount of shrinkage is controlled by a tuning parameter. Hence, optimization of both tuning parameters can be a problem of interest. In this study, theoretical development of the generalized cross-validation (GCV) is considered and some numerical illustrations are given to validate the theoretical findings. Our results demonstrated that using the proposed GCV criterion, the shrinkage ridge rank regression estimator behaves well in the sense of minimum risk function.

Use of Two Smoothing Parameters in Penalized Spline Estimator for Bi-variate Predictor Non-parametric Regression Model

Penalized spline criteria involve the function of goodness of fit and penalty, which in the penalty function contains smoothing parameters. It serves to control the smoothness of the curve that works simultaneously with point knots and spline degree. The regression function with two predictors in the non-parametric model will have two different non-parametric regression functions. Therefore, we propose the use of two smoothing parameters in the bi-variate predictor non-parametric regression model. We demonstrated its ability through longitudinal data simulation studies with a comparison of one smoothing parameter. It was done on several numbers of subjects with repeated measurements. The generalized cross validation value which is a measure of the model's ability is poured through the box plot. The results show that the use of two smoothing parameters is more optimal than one smoothing parameter. It was seen through a smaller generalized cross validation value on the use of two smoothing parameters. Application of blood sugar level data for patients with two smoothing parameters produced a penalized spline bi-variate predictor regression model with several segments of change patterns. There are five patterns at the time of treatment and blood pressure with the number of smoothing parameters is two, namely 0.39 and 0.73.