Unknown Error Variance Research Articles

Estimation of the density for censored and contaminated data

AbstractConsider a situation where one is interested in estimating the density of a survival time that is subject to random right censoring and measurement errors. This happens often in practice, like in public health (pregnancy length), medicine (duration of infection), ecology (duration of forest fire), among others. We assume a classical additive measurement error model with Gaussian noise and unknown error variance and a random right censoring scheme. Under this setup, we develop minimal conditions under which the assumed model is identifiable when no auxiliary variables or validation data are available, and we offer a flexible estimation strategy using Laguerre polynomials for the estimation of the error variance and the density of the survival time. The asymptotic normality of the proposed estimators is established, and the numerical performance of the methodology is investigated on both simulated and real data on gestational age.

High-dimensional properties for empirical priors in linear regression with unknown error variance

We study full Bayesian procedures for high-dimensional linear regression. We adopt data-dependent empirical priors introduced in Martin et al. (Bernoulli 23(3):1822–1847, 2017). In their paper, these priors have nice posterior contraction properties and are easy to compute. Our paper extend their theoretical results to the case of unknown error variance . Under proper sparsity assumption, we achieve model selection consistency, posterior contraction rates as well as Bernstein von-Mises theorem by analyzing multivariate t-distribution.

Optimal minimax rates against nonsmooth alternatives

Summary This study investigates optimal minimax rates for specification testing when the alternative hypothesis is built on a set of nonsmooth functions. The set consists of bounded functions that are not necessarily differentiable with no smoothness constraints imposed on their derivatives. In the instrumental variable regression set up with an unknown error variance structure, we find that the optimal minimax rate is $n^{-1/4}$, where n is the sample size. The rate is achieved by a simple test based on the difference between nonparametric and parametric variance estimators. Simulation studies illustrate that the test has reasonable power against various nonsmooth alternatives. The empirical application to Engel curves specification emphasizes the good applicability of the test.

CONFIDENCE INTERVALS CONSTRUCTED BY MODEL AVERAGING AND BOOTSTRAP SMOOTHING

We assess the performance of confidence intervals constructed by model averaging and bootstrap smoothing, using a simple testbed situation of two nested linear regression models with either known error variance or unknown error variance. The assessment is carried out by deriving computationally convenient exact expressions for the coverage probability and scaled expected length where the scaling is with respect to the usual confidence interval, with the same minimum coverage, based on the full model. We assessed the properties of the confidence interval centered on the model averaged estimator proposed by Buckland et al. (1997) with half-width proportional to the standard error of the estimate put forward by Buckland et al. (1997) of their formula (9). We considered both the finite sample case and the asymptotic case. Our results show that this model averaged confidence interval cannot be generally recommended, although it performs well when the error degrees of freedom is 1. We also assessed the properties of the confidence intervals centered on the bootstrap smoothed estimator with half-width proportional to the estimate of (a) the standard deviation of this estimator and (b) the delta method approximation, derived by Efron (2014), to the standard deviation of this estimator. We show that, except for the case of unknown error variance and small error degrees of freedom, the confidence interval centered on the bootstrap smoothed estimator does not have scaled expected length that is substantially less than 1, when the simpler model is correct. For this reason, we sought a formula for the data-based width that leads to improved performance of the confidence interval centered on the bootstrap smoothed estimator. Since this search failed, it raises the following question. Is there any formula for the data-based width of such a confidence interval so that it performs well? To answer this question, we compute the performance bound due to Kabaila & Kong (2016).

Learning high-dimensional Gaussian linear structural equation models with heterogeneous error variances

A new approach is presented for learning high-dimensional Gaussian linear structural equation models from only observational data when unknown error variances are heterogeneous. The proposed method consists of three steps: inferring (1) the moralized graph using the inverse covariance matrix, (2) the ordering using conditional variances, and (3) the directed edges using conditional independence relationships. These three problems can be efficiently addressed using inversion of parts of the covariance matrix. It is proved that a sample size of n=Ω(dm2logp) is sufficient for the proposed algorithm to recover the true directed graph, where p is the number of nodes and dm is the maximum degree. It is also shown that the proposed algorithm requires O(p3+pdm4) operations in the worst-case, and hence, it is computationally feasible for recovering large-scale graphs. It is verified through simulations that the proposed algorithm is statistically consistent and computationally feasible in high-dimensional and large-scale graph settings, and performs well compared to the state-of-the-art structural learning algorithms. It is also demonstrated through protein signaling data that our algorithm is well-suited to the estimation of directed acyclic graphical models for multivariate data in comparison to other methods used for normally distributed data.

Global-Local Mixtures: A Unifying Framework

Global-local mixtures, including Gaussian scale mixtures, have gained prominence in recent times, both as a sparsity inducing prior in p ≫ n problems as well as default priors for non-linear many-to-one functionals of high-dimensional parameters. Here we propose a unifying framework for global-local scale mixtures using the Cauchy-Schlomilch and Liouville integral transformation identities, and use the framework to build a new Bayesian sparse signal recovery method. This new method is a Bayesian counterpart of the $\sqrt {\text {Lasso}}$ (Belloni et al., Biometrika 98, 4, 791–806, 2011) that adapts to unknown error variance. Our framework also characterizes well-known scale mixture distributions including the Laplace density used in Bayesian Lasso, logit and quantile via a single integral identity. Finally, we derive a few convolutions that commonly arise in Bayesian inference and posit a conjecture concerning bridge and uniform correlation mixtures.

Identifiability of Gaussian linear structural equation models with homogeneous and heterogeneous error variances

In this work, we consider the identifiability assumption of Gaussian linear structural equation models (SEMs) in which each variable is determined by a linear function of its parents plus normally distributed error. It has been shown that linear Gaussian structural equation models are fully identifiable if all error variances are the same or known. Hence, this work proves the identifiability of Gaussian SEMs with both homogeneous and heterogeneous unknown error variances. Our new identifiability assumption exploits not only error variances, but edge weights; hence, it is strictly milder than prior work on the identifiability result. We further provide a structure learning algorithm that is statistically consistent and computationally feasible, based on our new assumption. The proposed algorithm assumes that all relevant variables are observed, while it does not assume causal minimality and faithfulness. We verify our theoretical findings through simulations and real multivariate data, and compare our algorithm to state-of-the-art PC, GES and GDS algorithms.

Variance Prior Forms for High-Dimensional Bayesian Variable Selection

Consider the problem of high dimensional variable selection for the Gaussian linear model when the unknown error variance is also of interest. In this paper, we show that the use of conjugate shrinkage priors for Bayesian variable selection can have detrimental consequences for such variance estimation. Such priors are often motivated by the invariance argument of Jeffreys (1961). Revisiting this work, however, we highlight a caveat that Jeffreys himself noticed; namely that biased estimators can result from inducing dependence between parameters a priori. In a similar way, we show that conjugate priors for linear regression, which induce prior dependence, can lead to such underestimation in the Bayesian high-dimensional regression setting. Following Jeffreys, we recommend as a remedy to treat regression coefficients and the error variance as independent a priori. Using such an independence prior framework, we extend the Spike-and-Slab Lasso of Rockova and George (2018) to the unknown variance case. This extended procedure outperforms both the fixed variance approach and alternative penalized likelihood methods on simulated data. On the protein activity dataset of Clyde and Parmigiani (1998), the Spike-and-Slab Lasso with unknown variance achieves lower cross-validation error than alternative penalized likelihood methods, demonstrating the gains in predictive accuracy afforded by simultaneous error variance estimation. The unknown variance implementation of the Spike-and-Slab Lasso is provided in the publicly available R package SSLASSO (Rockova and Moran, 2017).

On confidence intervals centred on bootstrap smoothed estimators

Bootstrap smoothed (bagged) estimators have been proposed as an improvement on estimators found after preliminary data‐based model selection. Efron derived a widely applicable formula for a delta method approximation to the standard deviation of the bootstrap smoothed estimator. He also considered a confidence interval centred on the bootstrap smoothed estimator, with width proportional to the estimate of this standard deviation. Recently, Kabaila and Wijethunga assessed the performance of this confidence interval in the scenario of two nested linear regression models, the full model and the simpler model, for the case of known error variance and preliminary model selection using a hypothesis test. They found that the performance of this confidence interval was not substantially better than the usual confidence interval based on the full model, with the same minimum coverage. We extend this assessment to the case of unknown error variance by deriving a computationally convenient exact formula for the ideal (i.e., in the limit as the number of bootstrap replications diverges to infinity) delta method approximation to the standard deviation of the bootstrap smoothed estimator. Our results show that, unlike the known error variance case, there are circumstances in which this confidence interval has attractive properties.

Uniformly valid confidence sets based on the Lasso

In a linear regression model of fixed dimension $p \leq n$, we construct confidence regions for the unknown parameter vector based on the Lasso estimator that uniformly and exactly hold the prescribed in finite samples as well as in an asymptotic setup. We thereby quantify estimation uncertainty as well as the "post-model selection error" of this estimator. More concretely, in finite samples with Gaussian errors and asymptotically in the case where the Lasso estimator is tuned to perform conservative model selection, we derive exact formulas for computing the minimal coverage probability over the entire parameter space for a large class of shapes for the confidence sets, thus enabling the construction of valid confidence regions based on the Lasso estimator in these settings. The choice of shape for the confidence sets and comparison with the confidence ellipse based on the least-squares estimator is also discussed. Moreover, in the case where the Lasso estimator is tuned to enable consistent model selection, we give a simple confidence region with minimal coverage probability converging to one. Finally, we also treat the case of unknown error variance and present some ideas for extensions.

Passive acoustic source localization with multiple horizontal arrays in shallow water

This paper considers concurrent matched-field processing of data from multiple, spatially separated acoustic arrays with application to towed-source data received on two bottom-moored horizontal line arrays from the SWellEx-96 shallow water experiment. Matched-field processors are derived for multiple arrays and multiple-snapshot data using maximum-likelihood estimates for unknown complex-valued source strengths and unknown error variances. Starting from a coherent processor where phase and amplitude is known between all arrays, likelihood expressions are derived for various assumptions on relative source spectral information (amplitude and phase at different frequencies) between arrays and from snapshot to snapshot. Processing the two arrays with a coherent-array processor (with inter-array amplitude and phase known) or with an incoherent-array processor (no inter-array spectral information) both yield improvements in localization over processing the arrays individually. The best results with this data set were obtained with a processor that exploits relative amplitude information but not relative phase between arrays. The localization performance improvement is retained when the multiple-array processors are applied to short arrays that individually yield poor performance.

Multiple-array passive acoustic source localization in shallow water.

This paper considers concurrent matched-field processing of data from multiple, spatially-separated acoustic arrays with application to towed-source data received on two bottom-moored horizontal line arrays from the SWellEx-96 shallow water experiment. Matched-field processors are derived for multiple arrays and multiple-snapshot data using maximum-likelihood estimates for unknown complex-valued source strengths and unknown error variances. Starting from a coherent processor where phase and amplitude is known between all arrays, likelihood expressions are derived for various assumptions on relative source spectral information (amplitude and phase at different frequencies) between arrays and from snapshot to snapshot. Processing the two arrays with a coherent-array processor (with inter-array amplitude and phase known) or with an incoherent-array processor (no inter-array spectral information) both yield improvements in localization over processing the arrays individually. The best results with this data set were obtained with a processor that exploits relative amplitude information but not relative phase between arrays. The localization performance improvement is retained when the multiple-array processors are applied to short arrays that individually yield poor performance.

Supremum norm posterior contraction and credible sets for nonparametric multivariate regression

In the setting of nonparametric multivariate regression with unknown error variance $\sigma^{2}$, we study asymptotic properties of a Bayesian method for estimating a regression function $f$ and its mixed partial derivatives. We use a random series of tensor product of B-splines with normal basis coefficients as a prior for $f$, and $\sigma$ is either estimated using the empirical Bayes approach or is endowed with a suitable prior in a hierarchical Bayes approach. We establish pointwise, $L_{2}$ and $L_{\infty}$-posterior contraction rates for $f$ and its mixed partial derivatives, and show that they coincide with the minimax rates. Our results cover even the anisotropic situation, where the true regression function may have different smoothness in different directions. Using the convergence bounds, we show that pointwise, $L_{2}$ and $L_{\infty}$-credible sets for $f$ and its mixed partial derivatives have guaranteed frequentist coverage with optimal size. New results on tensor products of B-splines are also obtained in the course.

Identification of Linear Dynamic Systems using Dynamic Iterative Principal Component Analysis

The paper is concerned with identifying models from data that have errors in both outputs and inputs, popularly known as the errors-in-variables (EIV) problem. The total least squares formulation of the problem is known to offer a few well-known solutions. In this work, we present a novel and systematic approach to the identification of linear dynamic models for the EIV case in the principal component analysis (PCA) framework. A methodology for the systematic recovery of the process model, including the determination of order and delay, using what we term as dynamic, iterative PCA is presented. The core step consists of determining the structure of the constraint matrix by a systematic exploitation of the stacking and PCA order, input-output partitioning of the constraint matrix and an appropriate rotation. Optimal estimates of the (input-output) noise covariance matrices are also obtained. The proposed method can be applied to a broad class of linear processes including the case of unequal and unknown error variances. Simulation results are presented to demonstrate the effectiveness and consistency of the proposed method.

A method for testing interaction in unreplicated two-way tables: using all pairwise interaction contrasts

Several methods exist for testing interaction in unreplicated two-way layouts. Some are based on specifying a functional form for the interaction term and perform well provided that the functional form is appropriate. Other methods do not require such a functional form to be specified but only test for the presence of non-additivity and do not provide a suitable estimate of error variance for a non-additive model. This paper presents a method for testing for interaction in unreplicated two-way tables that is based on testing all pairwise interaction contrasts. This method (i) is easy to implement, (ii) does not assume a functional form for the interaction term, (iii) can find a sub-table of data which may be free from interaction and to base the estimate of unknown error variance, and (iv) can be used for incomplete two-way layouts. The proposed method is illustrated using examples and its power is investigated via simulation studies. Simulation results show that the proposed method is competitive with existing methods for testing for interaction in unreplicated two-way layouts.

Empirical likelihood for heteroscedastic partially linear errors-in-variables model with α-mixing errors

In this paper, we apply the empirical likelihood method to heteroscedastic partially linear errors-in-variables model. For the cases of known and unknown error variances, the two different empirical log-likelihood ratios for the parameter of interest are constructed. If the error variances are known, the empirical log-likelihood ratio is proved to be asymptotic chi-square distribution under the assumption that the errors are given by a sequence of stationary α-mixing random variables. Furthermore, if the error variances are unknown, we show that the proposed statistic is asymptotically standard chi-square distribution when the errors are independent. Simulations are carried out to assess the performance of the proposed method.

Nonlinear regression models with increasing numbers of unknown parameters

Regression models with an increasing number of unknown parameters and with unknown and different observation error variances are of interest in important applications. The reason is that, with an increased number of unknown parameters, the unknown function can be approximated more accurately in experiments. Moreover, in some applications, repeated tests at a single point are costly (financially and technically), which hampers the estimation of the unknown error variance, which is different at different observation points. Regression models have been addressed in numerous publications [1‐3], but models with an increasing number of unknown parameters have received little attention, which motivates our interest in this subject. In [4, 5] various approaches were suggested for estimating the covariance matrix of the deviation vector of the unknown parameter for linear models of finite dimension. Linear regression models of an increasing dimension were studied in [6], where necessary and sufficient conditions were found for the consistency of least squares estimators and the covariance matrix of the deviation vector. In this paper, an approach is suggested for determining least squares estimators and covariance matrix elements. The results can be used to construct a confidence interval for the unknown function in nonlinear regression models with an increasing number of unknown parameters.

Precise asymptotics for the first moment of the error variance estimator in linear models

Let σ 2 be the unknown error variance of a linear model and let σ ˆ 2 be the estimator of σ 2 based on the residual sum of squares. In this work, we show the precise asymptotics in the law of the logarithm for the first moment of the error variance estimator.

Iterative Weighted Semiparametric Least Squares Estimation in Repeated Measurement Partially Linear Regression Models

Consider a repeated measurement partially linear regression model with an unknown vector parameter β 1, an unknown function g(·), and unknown heteroscedastic error variances. In order to improve the semiparametric generalized least squares estimator (SGLSE) of β, we propose an iterative weighted semiparametric least squares estimator (IWSLSE) and show that it improves upon the SGLSE in terms of asymptotic covariance matrix. An adaptive procedure is given to determine the number of iterations. We also show that when the number of replicates is less than or equal to two, the IWSLSE can not improve upon the SGLSE. These results are generalizations of those in [2] to the case of semiparametric regressions.

An asymptotic theory for semiparametric generalized least squares estimation in partially linear regression models

Consider a partially linear regression model with an unknown vector parameter β, an unknown functiong(·), and unknown heteroscedastic error variances. In this paper we develop an asymptotic semiparametric generalized least squares estimation theory under some weak moment conditions. These moment conditions are satisfied by many of the error distributions encountered in practice, and our theory does not require the number of replications to go to infinity.