Maximum Likelihood Cross Validation Research Articles

Robust prediction interval estimation for Gaussian processes by cross-validation method

Probabilistic regression models typically use the Maximum Likelihood Estimation or Cross-Validation to fit parameters. These methods can give an advantage to the solutions that fit observations on average, but they do not pay attention to the coverage and the width of Prediction Intervals. A robust two-step approach is used to address the problem of adjusting and calibrating Prediction Intervals for Gaussian Processes Regression. First, the covariance hyperparameters are determined by a standard Cross-Validation or Maximum Likelihood Estimation method. A Leave-One-Out Coverage Probability is introduced as a metric to adjust the covariance hyperparameters and assess the optimal type II Coverage Probability to a nominal level. Then a relaxation method is applied to choose the hyperparameters that minimize the Wasserstein distance between the Gaussian distribution with the initial hyperparameters (obtained by Cross-Validation or Maximum Likelihood Estimation) and the proposed Gaussian distribution with the hyperparameters that achieve the desired Coverage Probability. The method gives Prediction Intervals with appropriate coverage probabilities and small widths.

Kernel-based estimation of individual location densities from smartphone data

Localizing people across space and over time is a relevant and challenging problem in many modern applications. Smartphone ubiquity gives the opportunity to collect useful individual data as never before. In this work, the focus is on location data collected by smartphone applications. We propose a kernel-based density estimation approach that exploits cyclical spatio-temporal patterns of people to estimate the individual location density at any time, uncertainty included. Model parameters are estimated by maximum likelihood cross-validation. Unlike classic tracking methods designed for high spatio-temporal resolution data, the approach is suitable when location data are sparse in time and are affected by non-negligible errors. The approach is applied to location data collected by the Earthquake Network citizen science project which carries out a worldwide earthquake early warning system based on smartphones. The approach is parsimonious and is suitable to model location data gathered by any location-aware smartphone application.

Genetic algorithm-based parameter optimization for EO-1 Hyperion remote sensing image classification

ABSTRACT Because hyperspectral remote sensing images have high band dimension and limited training samples, it is hard to achieve acceptable classification results using conventional statistical pattern recognition methods. Nevertheless, support vector machine (SVM) is suitable for hyperspectral image classification owing to its good generalization ability and minimal structural risk. Aimed to address the problems of large subjectivity, excessive time consumption, and low optimization accuracy in traditional SVM parameter selection, we applied genetic algorithm to optimize the model parameters. Using EO-1 Hyperion data as an example, an SVM object classification algorithm was proposed based on genetic algorithm optimization. To determine the effectiveness and superiority of this algorithm, a comprehensive evaluation and comparative analysis were performed with the classification effects of the cross-validation and maximum likelihood methods, respectively. Experimental results demonstrated that the proposed algorithm can effectively avert the aimlessness of artificial parameter selection and automatically optimize the SVM model parameters. It achieved relatively high overall classification accuracy (91.23%), which was 16.42% and 4.48% higher than the maximum likelihood method and cross-validation method, respectively.

교차타당성을 이용한 확률밀도함수의 불연속점 추정의 띠폭 선택

교차타당성은 커널추정량의 평활모수인 띠폭의 선택 방법으로 흔히 활용되고 있다. 연속인 확률밀도함수의 커널추정량의 띠폭 선택으로 널리 쓰이는 교차타당성 방법으로는 최대가능도교차타당성과 더불어 최소제곱교차타당성과 편의교차타당성이 있다. 확률밀도함수가 하나의 불연속점을 가질 때, Huh (2012)는 불연속점 추정을 위한 커널추정량의 띠폭 선택으로 최대가능도교차타당성을 이용한 방법을 제시하였다. 본 연구에서는 Huh (2012)에 의해 최대가능도교차타당성으로 제안된 띠폭선택의 방법과 같이 한쪽방향커널함수를 이용한 최소제곱교차타당성과 편의교차타당성으로 띠폭 선택 방법을 제시하고, 이들 띠폭 선택 방법들과 Huh (2012)의 최대가능도교차타당성을 이용한 띠폭 선택 방법을 모의실험을 통하여 비교연구 하고자 한다. The cross-validation is a popular method to select bandwidth in all types of kernel estimation. The maximum likelihood cross-validation, the least squares cross-validation and biased cross-validation have been proposed for bandwidth selection in kernel density estimation. In the case that the probability density function has a discontinuity point, Huh (2012) proposed a method of bandwidth selection using the maximum likelihood cross-validation. In this paper, two forms of cross-validation with the one-sided kernel function are proposed for bandwidth selection to estimate the location and jump size of the discontinuity point of density. These methods are motivated by the least squares cross-validation and the biased cross-validation. By simulated examples, the finite sample performances of two proposed methods with the one of Huh (2012) are compared.

Comparison of inference methods for estimating semivariogram model parameters and their uncertainty: The case of small data sets

The semivariogram model is the fundamental component in all geostatistical applications and its inference is an issue of significant practical interest. The semivariogram model is defined by a mathematical function, the parameters of which are usually estimated from the experimental data. There are important application areas in which small data sets are the norm; rainfall estimation from rain gauge data and transmissivity estimation from pumping test data are two examples from, respectively, surface and subsurface hydrology. Thus a benchmark problem in geostatistics is deciding on the most appropriate method for the inference of the semivariogram model.The various methods for semivariogram inference can be classified as indirect methods, in which there is an intermediate step of calculating the experimental semivariogram, and direct approaches that obtain the model parameter values directly as the values that minimize some objective function.To avoid subjectivity in fitting models to experimental semivariograms, ordinary least squares (OLS), weighted least squares (WLS) and generalized least squares (GLS) are often used. Uncertainty evaluation in indirect methods is done using computationally intensive resampling procedures such as the bootstrap method.Direct methods include parametric methods, such as maximum likelihood (ML) and maximum likelihood cross-validation (MLCV), and non-parametric methods, such as minimization of cross-validation statistics (CV).The bases for comparing the previous methods are the sampling distribution of the various parameters and the “goodness” of the uncertainty evaluation in a sense that we define. The final questions to be answered are (1) which is the best method for estimating each of the semivariogram parameters? (2) Which is the best method for assessing the uncertainty of each of the parameters? (3) Which method best selects the functional form of the semivariogram from among a set of options? and (4) which is the best method that jointly addresses all the previous questions?

PENAKSIRAN FUNGSI DENSITAS TIPE KERNEL DENGAN METODE CROSS-VALIDATION (C-V)

The choice of the bandwidth h is the main problem of kernel density function. In some situations it might be quite useful to have a set of retes corresponding to different bandwidth.It is necessary to agree which bandwidth is an appropriate one. Cross-Validation (C-V) is a well-known method to optimize the smoothing parameter h. In this research, we will analysis about three methods of Cross-Validation: Maximum Likelihood Cross-Validation, Least-Square Cross-Validation (Unbiased Cross-Validation) and Biased Cross-Validation with bandwidth choice optimal.

Use of Kriging Models to Approximate Deterministic Computer Models

The use of kriging models for approximation and metamodel-based design and optimization has been steadily on the rise in the past decade. The widespread usage of kriging models appears to be hampered by (1) the lack of guidance in selecting the appropriate form of the kriging model, (2) computationally efficient algorithms for estimating the model’s parameters, and (3) an effective method to assess the resulting model’s quality. In this paper, we compare (1) Maximum Likelihood Estimation (MLE) and Cross-Validation (CV) parameter estimation methods for selecting a kriging model’s parameters given its form and (2) and an R 2 of prediction and the corrected Akaike Information Criterion for assessing the quality of the created kriging model, permitting the comparison of different forms of a kriging model. These methods are demonstrated with six test problems. Finally, different forms of kriging models are examined to determine if more complex forms are more accurate and easier to fit than simple forms of kriging models for approximating computer models.

Maximum likelihood Bayesian averaging of spatial variability models in unsaturated fractured tuff

Hydrologic analyses typically rely on a single conceptual‐mathematical model. Yet hydrologic environments are open and complex, rendering them prone to multiple interpretations and mathematical descriptions. Adopting only one of these may lead to statistical bias and underestimation of uncertainty. Bayesian model averaging (BMA) [Hoeting et al., 1999] provides an optimal way to combine the predictions of several competing models and to assess their joint predictive uncertainty. However, it tends to be computationally demanding and relies heavily on prior information about model parameters. Neuman [2002, 2003] proposed a maximum likelihood version (MLBMA) of BMA to render it computationally feasible and to allow dealing with cases where reliable prior information is lacking. We apply MLBMA to seven alternative variogram models of log air permeability data from single‐hole pneumatic injection tests in six boreholes at the Apache Leap Research Site (ALRS) in central Arizona. Unbiased ML estimates of variogram and drift parameters are obtained using adjoint state maximum likelihood cross validation [Samper and Neuman, 1989a] in conjunction with universal kriging and generalized least squares. Standard information criteria provide an ambiguous ranking of the models, which does not justify selecting one of them and discarding all others as is commonly done in practice. Instead, we eliminate some of the models based on their negligibly small posterior probabilities and use the rest to project the measured log permeabilities by kriging onto a rock volume containing the six boreholes. We then average these four projections and associated kriging variances, using the posterior probability of each model as weight. Finally, we cross validate the results by eliminating from consideration all data from one borehole at a time, repeating the above process and comparing the predictive capability of MLBMA with that of each individual model. We find that MLBMA is superior to any individual geostatistical model of log permeability among those we consider at the ALRS.

Kernel bandwidth selection for a first order nonparametric streamflow simulation model

A new approach for streamflow simulation using nonparametric methods was described in a recent publication (Sharma et al. 1997). Use of nonparametric methods has the advantage that they avoid the issue of selecting a probability distribution and can represent nonlinear features, such as asymmetry and bimodality that hitherto were difficult to represent, in the probability structure of hydrologic variables such as streamflow and precipitation. The nonparametric method used was kernel density estimation, which requires the selection of bandwidth (smoothing) parameters. This study documents some of the tests that were conduced to evaluate the performance of bandwidth estimation methods for kernel density estimation. Issues related to selection of optimal smoothing parameters for kernel density estimation with small samples (200 or fewer data points) are examined. Both reference to a Gaussian density and data based specifications are applied to estimate bandwidths for samples from bivariate normal mixture densities. The three data based methods studied are Maximum Likelihood Cross Validation (MLCV), Least Square Cross Validation (LSCV) and Biased Cross Validation (BCV2). Modifications for estimating optimal local bandwidths using MLCV and LSCV are also examined. We found that the use of local bandwidths does not necessarily improve the density estimate with small samples. Of the global bandwidth estimators compared, we found that MLCV and LSCV are better because they show lower variability and higher accuracy while Biased Cross Validation suffers from multiple optimal bandwidths for samples from strongly bimodal densities. These results, of particular interest in stochastic hydrology where small samples are common, may have importance in other applications of nonparametric density estimation methods with similar sample sizes and distribution shapes.

Estimation of spatial covariance structures with application to hydrological, hydrochemical, and isotopic data from aquifers: state-of-the-art and adjoint state maximum likelihood cross-validation methods : Samper, F J; Neuman, S P Proc NATO Advanced Research Workshop on Advances in Analytical and Numerical Groundwater Flow and Quality Modelling, Lisbon, 2–6 June 1987P585–620. Publ Dordrecht: Reidel, 1987

Estimation of spatial covariance structures with application to hydrological, hydrochemical, and isotopic data from aquifers: state-of-the-art and adjoint state maximum likelihood cross-validation methods : Samper, F J; Neuman, S P Proc NATO Advanced Research Workshop on Advances in Analytical and Numerical Groundwater Flow and Quality Modelling, Lisbon, 2–6 June 1987P585–620. Publ Dordrecht: Reidel, 1987

Estimation of spatial covariance structures by adjoint state maximum likelihood cross validation: 1. Theory

This series of three papers describes a cross‐validation method to estimate the spatial covariance structure of intrinsic or nonintrinsic random functions from point or spatially averaged data that may be corrupted by noise. Any number of relevant parameters, including nugget effect, can be estimated. The theory, described in this paper, is based on a maximum likelihood approach which treats the cross‐validation errors as Gaussian. Various a posteriori statistical tests are used to verify this hypothesis and to show that in many cases, correlation between these errors is weak. The log likelihood criterion is optimized through a combination of conjugate gradient algorithms. An adjoint state theory is used to efficiently calculate the gradient of the estimation criterion, optimize the step size downgradient, and compute a lower bound for the covariance matrix of the estimation errors. Issues related to the identifiability, stability, and uniqueness of the resulting adjoint state maximum likelihood cross‐validation (ASMLCV) method are discussed. This paper also describes the manner in which ASMLCV allows one to use model structure identification criteria to select the best covariance model among a given set of alternatives. Practical aspects of ASMLCV and its application to synthetic data are presented in paper 2 (Samper and Neuman, this issue (a)). Applications to real hydrogeological data (transmissivities and groundwater levels) have been presented elsewhere, while hydrochemical and isotopic data are analyzed by ASMLCV in paper 3 (Samper and Neuman, this issue (b)).

Estimation of spatial covariance structures by adjoint state maximum likelihood cross validation: 2. Synthetic experiments

Paper 2 of this three‐part series uses synthetic data to investigate the properties of the adjoint state maximum likelihood cross‐validation (ASMLCV) method presented in paper 1 (Samper and Neuman, this issue (a)). More than 40 synthetic experiments are performed to compare various conjugate gradient algorithms; investigate the manner in which computer time varies with ASMLCV parameters; study the effect of sample size and choice of kriging points on ASMLCV estimates ; evaluate the ability of various model structure identification criteria to help select the most appropriate semivariogram model among given alternatives; study the conditions required for parameter identifiability, uniqueness, and stability; quantify the statistics of cross‐validation errors; test hypotheses concerning the distribution and autocorrelation of these errors; and illustrate the computation of approximate quality indicators for ASMLCV parameter estimates.

Estimation of spatial covariance structures by adjoint state maximum likelihood cross validation: 3. Application to hydrochemical and isotopic data

Paper 3 of this three‐part series presents applications of our adjoint state maximum likelihood cross‐validation (ASMLCV) method to real data from aquifers. The Madrid basin in Spain serves as the source of information about 11 hydrochemical variables (pH, electrical conductivity, silica content, and the concentration of major ions) and two isotopes (oxygen 18 and carbon 14). Due to a lack of sufficient vertical resolution, our analysis is restricted to the horizontal plane. With the exception of oxygen 18 and silica, the variables appear to be free of a horizontal drift. No discernible directional effects are seen. All variables exhibit a large nugget effect which is indicative of background noise. We conclude that more detailed and careful sampling in three dimensions is required if groundwater quality information is to become less prone to such noise and thereby more useful in the context of quantitative hydrogeological analyses. Despite the existing noise, we are able to confirm geostatistically some (though not all) of the hypotheses advanced by others about hydrochemical evolution and isotope changes in the basin. The ability of ASMLCV to filter out spatial variations from part of the measurement noise is illustrated on carbon 14 data. The same data are also used to investigate the utility of model structure identification criteria in selecting the best among a set of alternative semivariogram models.