Estimation Of Covariance Parameters Research Articles

Covariance Clustering: Modelling Covariance in Designed Experiments When the Number of Variables is Greater than Experimental Units

The size and complexity of datasets resulting from comparative research experiments in the agricultural domain is constantly increasing. Often the number of variables measured in an experiment exceeds the number of experimental units composing the experiment. When there is a necessity to model the covariance relationships that exist between variables in these experiments, estimation difficulties can arise due to the resulting covariance structure being of reduced rank. A statistical method, based in a linear mixed model framework, is presented for the analysis of designed experiments where datasets are characterised by a greater number of variables than experimental units, and for which the modelling of complex covariance structures between variables is desired. Aided by a clustering algorithm, the method enables the estimation of covariance through the introduction of covariance clusters as random effects into the modelling framework, providing an extension of the traditional variance components model for building covariance structures. The method was applied to a multi-phase mass spectrometry-based proteomics experiment, with the aim of exploring changes in the proteome of barley grain over time during the malting process. The modelling approach provides a new linear mixed model-based method for the estimation of covariance structures between variables measured from designed experiments, when there are a small number of experimental units, or observations, informing covariance parameter estimates.

Fixed-domain Posterior Contraction Rates for Spatial Gaussian Process Model with Nugget

Spatial Gaussian process regression models typically contain finite dimensional covariance parameters that need to be estimated from the data. We study the Bayesian estimation of covariance parameters including the nugget parameter in a general class of stationary covariance functions under fixed-domain asymptotics, which is theoretically challenging due to the increasingly strong dependence among spatial observations. We propose a novel adaptation of the Schwartz’s consistency theorem for showing posterior contraction rates of the covariance parameters including the nugget. We derive a new polynomial evidence lower bound, and propose consistent higher-order quadratic variation estimators that satisfy concentration inequalities with exponentially small tails. Our Bayesian fixed-domain asymptotics theory leads to explicit posterior contraction rates for the microergodic and nugget parameters in the isotropic Matérn covariance function under a general stratified sampling design. We verify our theory and the Bayesian predictive performance in simulation studies and an application to sea surface temperature data.

Analyzing dynamic species abundance distributions using generalized linear mixed models.

Understanding the mechanisms of ecological community dynamics and how they could be affected by environmental changes is important. Population dynamic models have well known ecological parameters that describe key characteristics of species such as the effect of environmental noise and demographic variance on the dynamics, the long‐term growth rate, and strength of density regulation. These parameters are also central for detecting and understanding changes in communities of species; however, incorporating such vital parameters into models of community dynamics is challenging. In this paper, we demonstrate how generalized linear mixed models specified as intercept‐only models with different random effects can be used to fit dynamic species abundance distributions. Each random effect has an ecologically meaningful interpretation either describing general and species‐specific responses to environmental stochasticity in time or space, or variation in growth rate and carrying capacity among species. We use simulations to show that the accuracy of the estimation depends on the strength of density regulation in discrete population dynamics. The estimation of different covariance and population dynamic parameters, with corresponding statistical uncertainties, is demonstrated for case studies of fish and bat communities. We find that species heterogeneity is the main factor of spatial and temporal community similarity for both case studies.

Designs Robust Against Presence of an Outlier in an Analysis of Covariance Model

Presence of one or more aberration in the observations affects inference procedure in statistical analysis. Block designs robust against presence of aberrations can be found in the literature for both regression and block design set-ups. In this paper, an attempt has been made to find robust designs in a block design set-up with covariates, when there is one single wild observation in the study variable. Specifically, effects on the estimation of a full set of orthonormal treatment contrasts and that on the estimation of covariate parameters have been considered.

Asymptotic properties of the maximum likelihood and cross validation estimators for transformed Gaussian processes

The asymptotic analysis of covariance parameter estimation of Gaussian processes has been subject to intensive investigation. However, this asymptotic analysis is very scarce for non-Gaussian processes. In this paper, we study a class of non-Gaussian processes obtained by regular non-linear transformations of Gaussian processes. We provide the increasing-domain asymptotic properties of the (Gaussian) maximum likelihood and cross validation estimators of the covariance parameters of a non-Gaussian process of this class. We show that these estimators are consistent and asymptotically normal, although they are defined as if the process was Gaussian. They do not need to model or estimate the non-linear transformation. Our results can thus be interpreted as a robustness of (Gaussian) maximum likelihood and cross validation towards non-Gaussianity. Our proofs rely on two technical results that are of independent interest for the increasing-domain asymptotic literature of spatial processes. First, we show that, under mild assumptions, coefficients of inverses of large covariance matrices decay at an inverse polynomial rate as a function of the corresponding observation location distances. Second, we provide a general central limit theorem for quadratic forms obtained from transformed Gaussian processes. Finally, our asymptotic results are illustrated by numerical simulations.

Local inversion-free estimation of spatial Gaussian processes

Maximizing the likelihood has been widely used for estimating the unknown covariance parameters of spatial Gaussian processes. However, evaluating and optimizing the likelihood function can be computationally intractable, particularly for large number of (possibly) irregularly spaced observations, due to the need to handle the inverse of ill-conditioned and large covariance matrices. Extending the “inversion-free” method of Anitescu, Chen and Stein [1], we investigate a broad class of covariance parameter estimation based on inversion-free surrogate losses and block diagonal approximation schemes of the covariance structure. This class of estimators yields a spectrum for negotiating the trade-off between statistical accuracy and computational cost. We present fixed-domain asymptotic properties of our proposed method, establishing $\sqrt{n}$-consistency and asymptotic normality results for isotropic Matern Gaussian processes observed on a multi-dimensional and irregular lattice. Simulation studies are also presented for assessing the scalability and statistical efficiency of the proposed algorithm for large data sets.

Locally stationary spatio-temporal interpolation of Argo profiling float data.

Argo floats measure seawater temperature and salinity in the upper 2000 m of the global ocean. Statistical analysis of the resulting spatio-temporal dataset is challenging owing to its non-stationary structure and large size. We propose mapping these data using locally stationary Gaussian process regression where covariance parameter estimation and spatio-temporal prediction are carried out in a moving-window fashion. This yields computationally tractable non-stationary anomaly fields without the need to explicitly model the non-stationary covariance structure. We also investigate Student t-distributed fine-scale variation as a means to account for non-Gaussian heavy tails in ocean temperature data. Cross-validation studies comparing the proposed approach with the existing state of the art demonstrate clear improvements in point predictions and show that accounting for the non-stationarity and non-Gaussianity is crucial for obtaining well-calibrated uncertainties. This approach also provides data-driven local estimates of the spatial and temporal dependence scales for the global ocean, which are of scientific interest in their own right.

Maximum likelihood estimation for Gaussian processes under inequality constraints

We consider covariance parameter estimation for a Gaussian process under inequality constraints (boundedness, monotonicity or convexity) in fixed-domain asymptotics. We address the estimation of the variance parameter and the estimation of the microergodic parameter of the Mat\'ern and Wendland covariance functions. First, we show that the (unconstrained) maximum likelihood estimator has the same asymptotic distribution, unconditionally and conditionally to the fact that the Gaussian process satisfies the inequality constraints. Then, we study the recently suggested constrained maximum likelihood estimator. We show that it has the same asymptotic distribution as the (unconstrained) maximum likelihood estimator. In addition, we show in simulations that the constrained maximum likelihood estimator is generally more accurate on finite samples. Finally, we provide extensions to prediction and to noisy observations.

Free-coordinate estimation for doubly multivariate data

The article addresses the best unbiased estimators of the block compound symmetric covariance structure for m−variate observations with equal mean vector over u sites under the assumption of multivariate quasi-normality. The free-coordinate approach is used to prove that the quadratic estimation of covariance parameters is equivalent to linear estimation with a properly defined inner product in the space of symmetric matrices. Without assumption of normality but quasi-normality, meaning that up to fourth moments are the same as in the normal case, the estimators are best linear and best quadratic for mean and covariance parameters, respectively. Finally, strong consistency is proven. The properties of the estimators in the proposed model are compared against a similar model available in the literature. An application of the proposed approach to a clinical study data is presented.

Using empirical Bayes predictors from generalized linear mixed models to test and visualize associations among longitudinal outcomes.

Medical research is often designed to investigate changes in a collection of response variables that are measured repeatedly on the same subjects. The multivariate generalized linear mixed model (MGLMM) can be used to evaluate random coefficient associations (e.g. simple correlations, partial regression coefficients) among outcomes that may be non-normal and differently distributed by specifying a multivariate normal distribution for their random effects and then evaluating the latent relationship between them. Empirical Bayes predictors are readily available for each subject from any mixed model and are observable and hence, plotable. Here, we evaluate whether second-stage association analyses of empirical Bayes predictors from a MGLMM, provide a good approximation and visual representation of these latent association analyses using medical examples and simulations. Additionally, we compare these results with association analyses of empirical Bayes predictors generated from separate mixed models for each outcome, a procedure that could circumvent computational problems that arise when the dimension of the joint covariance matrix of random effects is large and prohibits estimation of latent associations. As has been shown in other analytic contexts, the p-values for all second-stage coefficients that were determined by naively assuming normality of empirical Bayes predictors provide a good approximation to p-values determined via permutation analysis. Analyzing outcomes that are interrelated with separate models in the first stage and then associating the resulting empirical Bayes predictors in a second stage results in different mean and covariance parameter estimates from the maximum likelihood estimates generated by a MGLMM. The potential for erroneous inference from using results from these separate models increases as the magnitude of the association among the outcomes increases. Thus if computable, scatterplots of the conditionally independent empirical Bayes predictors from a MGLMM are always preferable to scatterplots of empirical Bayes predictors generated by separate models, unless the true association between outcomes is zero.

How should a spatial-coverage sample design for a geostatistical soil survey be supplemented to support estimation of spatial covariance parameters?

We use an expression for the error variance of geostatistical predictions, which includes the effect of uncertainty in the spatial covariance parameters, to examine the performance of sample designs in which a proportion of the total number of observations are distributed according to a spatial coverage design, and the remaining observations are added at supplementary close locations. This expression has been used in previous studies on numerical optimization of spatial sampling, the objective of this study was to use it to discover simple rules of thumb for practical geostatistical sampling. Results for a range of sample sizes and contrasting properties of the underlying random variables show that there is an improvement on adding just a few sample points and close pairs, and a rather slower increase in the prediction error variance as the proportion of sample points allocated in this way is increased above 10 to 20% of the total sample size. One may therefore propose a rule of thumb that, for a fixed sample size, 90% of sample sites are distributed according to a spatial coverage design, and 10% are then added at short distances from sites in the larger subset to support estimation of spatial covariance parameters.

On Geostatistical Methods for Radio Environment Maps Generation under Location Uncertainty

Radio environment map (REM) can provide important information for designing and optimizing the performance of wireless communication networks. However, the location uncertainty related to the measurements used to build the REM can considerably deteriorate the accuracy of such map. This paper addresses this problem by proposing a modified approach of a classical geostatistical prediction tool, named Kriging method, which incorporates the location uncertainty and is able to improve the REM accuracy without adding significant complexity. Finally, we also show through simulation results that the average path loss and covariance parameter estimation play an important role and should be considered when the location errors occur in the wireless communication systems.

Finite-Dimensional Gaussian Approximation with Linear Inequality Constraints

Introducing inequality constraints in Gaussian process (GP) models can lead to more realistic uncertainties in learning a great variety of real-world problems. We consider the finite-dimensional Gaussian approach from Maatouk and Bay (2017) which can satisfy inequality conditions everywhere (either boundedness, monotonicity or convexity). Our contributions are threefold. First, we extend their approach in order to deal with general sets of linear inequalities. Second, we explore several Markov Chain Monte Carlo (MCMC) techniques to approximate the posterior distribution. Third, we investigate theoretical and numerical properties of the constrained likelihood for covariance parameter estimation. According to experiments on both artificial and real data, our full framework together with a Hamiltonian Monte Carlo-based sampler provides efficient results on both data fitting and uncertainty quantification.

An example of inconsistent MLE of spatial covariance parameters under increasing domain asymptotics

Asymptotic properties of estimators of covariance parameters in spatial statistics are commonly considered under the frameworks of increasing domain and fixed domain asymptotics, respectively. Although inconsistency is a general conclusion under the framework of fixed domain asymptotics, it is generally believed that consistency should generally hold under the framework of increasing domain asymptotics. This article provides an example in which the maximum likelihood estimator (MLE) of covariance parameters is still inconsistent under the framework of increasing domain asymptotics. Therefore, consistency may still be a problem under the framework of increasing domain asymptotics.

Conditional versus Marginal Covariance Representation for Linear and Nonlinear Models

Grouped data, such as repeated measures and longitudinal data, are increasingly collected in different areas of application, as varied as clinical trials, epidemiological studies, and educational testing. It is often of interest, for these data, to explore possible relationships between one or more response variables and available covariates. Because of the within-group correlation typically present with this type of data, special regression models that allow the joint estimation of mean and covariance parameters need to be used. Two main approaches have been proposed to represent the covariance structure of the data with these models: (i) via the use of random effects, the so-called conditional model and (ii) through direct representation of the covariance structure of the responses, known as the marginal approach. Here we discuss and compare these two approaches in the context of linear and non-linear regression models with additive Gaussian errors, using a real data example to motivate and illustrate the discussion.

Estimation of gravity noise variance and signal covariance parameters in least squares collocation with considering data resolution

<p>The article describes an implementation of the negative log-likelihood function in the determination of uncorrelated noise standard deviation together with the parameters of spherical signal covariance model in least squares collocation (LSC) of gravity anomalies. The correctness and effectiveness of restricted maximum likelihood (REML) estimates are fully validated by leave-one-out validation (LOO). These two complementary methods give an opportunity to inspect the parametrization of the signal and uncorrelated noise in details and can provide some guidance related to the estimation of individual parameters. The study provides the practical proof that noise variance is related with the data resolution, which is often neglected and the information on a priori noise variance is based on the measurement error. The data have been downloaded from U.S. terrestrial gravity database and resampled to enable an analysis with four different horizontal resolutions. These data are intentionally the same, as in the previous study of the same author, with the application of the planar covariance model. The aim is to compare the results from two different covariance models, which have different covariance approximation at larger distances. The most interesting outputs from this study confirm previous observations on the relations of the data resolution, a priori noise variance, signal spectrum and LSC accuracy.</p>

Effect of Smoothing in Generalized Linear Mixed Models on the Estimation of Covariance Parameters for Longitudinal Data.

Besides being mainly used for analyzing clustered or longitudinal data, generalized linear mixed models can also be used for smoothing via restricting changes in the fit at the knots in regression splines. The resulting models are usually called semiparametric mixed models (SPMMs). We investigate the effect of smoothing using SPMMs on the correlation and variance parameter estimates for serially correlated longitudinal normal, Poisson and binary data. Through simulations, we compare the performance of SPMMs to other simpler methods for estimating the nonlinear association such as fractional polynomials, and using a parametric nonlinear function. Simulation results suggest that, in general, the SPMMs recover the true curves very well and yield reasonable estimates of the correlation and variance parameters. However, for binary outcomes, SPMMs produce biased estimates of the variance parameters for high serially correlated data. We apply these methods to a dataset investigating the association between CD4 cell count and time since seroconversion for HIV infected men enrolled in the Multicenter AIDS Cohort Study.

Optimal estimation for doubly multivariate data in blocked compound symmetric covariance structure

The paper deals with the best unbiased estimators of the blocked compound symmetric covariance structure for m-variate observations over u sites under the assumption of multivariate normality. The free-coordinate approach is used to prove that the quadratic estimation of covariance parameters is equivalent to linear estimation with a properly defined inner product in the space of symmetric matrices. Complete statistics are then derived to prove that the estimators are best unbiased. Finally, strong consistency is proven. The proposed method is implemented with a real data set.

Model-Based Sampling Design for Multivariate Geostatistics

The quality of inferences made from geostatistical data is affected significantly by the spatial locations, or design, of the sites that are sampled. A large body of published work exists on sampling design for univariate geostatistics, but not for multivariate geostatistics. This article considers multivariate spatial sampling design based on criteria targeted at classical co-kriging (prediction with known covariance parameters), estimation of covariance (including cross-covariance) parameters, and empirical co-kriging (prediction with estimated covariance parameters). Through a combination of analytical results and examples, we investigate the characteristics of optimal designs with respect to each criterion, addressing in particular the design’s degree of collocation. We also consider the robustness of the optimal design to the strength of spatial correlation and cross-correlation; the effects of smoothness and/or separability of the sampled process on the optimal design; the relationship between optimal designs for the multivariate problems considered here and univariate problems considered previously; and the efficiency of optimal collocated designs. One key finding is that optimal collocated designs are highly efficient in many cases. Supplementary materials are available online.

Precise estimation of covariance parameters in least-squares collocation by restricted maximum likelihood

Precise estimates of the covariance parameters are essential in least-squares collocation (LSC) in the case of increased accuracy requirements. This paper implements restricted maximum likelihood (REML) method for the estimation of three covariance parameters in LSC with the Gauss-Markov second-order function (GM2), which is often used in interpolation of gravity anomalies. The estimates are then validated with the use of an independent technique, which has been often omitted in the previous works that are confined to covariance parameters errors based on the information matrix. The crossvalidation of REML estimates with the use of hold-out method (HO) helps in understanding of REML estimation errors. We analyzed in detail the global minimum of negative log-likelihood function (NLLF) in the estimation of covariance parameters, as well, as the accuracy of the estimates. We found that the correlation between covariance parameters may critically contribute to the errors of their estimation. It was also found that knowing some intrinsic properties of the covariance function may help in the scoring process.