Arbitrary Covariance Research Articles

Improved Hölder and reverse Hölder inequalities for Gaussian random vectors

We propose algebraic criteria that yield sharp Hölder types of inequalities for the product of functions of Gaussian random vectors with arbitrary covariance structure. While our lower inequality appears to be new, we prove that the upper inequality gives an equivalent formulation for the geometric Brascamp–Lieb inequality for Gaussian measures. As an application, we retrieve the Gaussian hypercontractivity as well as its reverse and we present a generalization of the sharp Young and reverse Young inequalities. From the latter, we recover several known inequalities in the literature including the Prékopa–Leindler and Barthe inequalities.

Multivariate Mixtures of Normal Distributions: Properties, Random Vector Generation, Fitting, and as Models of Market Daily Changes

Mixtures of normal distributions provide a useful modeling extension of the normal distribution—both univariate and multivariate. Unlike the normal distribution, mixtures of normals can capture the kurtosis (fat tails) and nonzero skewness often necessary for accurately modeling a variety of real-world variables. An efficient analytical Monte Carlo method is proposed for considering multivariate mixtures of normal distributions having arbitrary covariance matrices. The method consists of a linear transformation of a multivariate normal having a computed covariance matrix into the desired multivariate mixture of normal distributions. The computed covariance matrix is derived analytically. Among the properties of the multivariate mixture of normals that we demonstrate is that any linear combination of mixtures of normal distributions is also a mixture of normal distributions. Methods of fitting mixtures of normal distributions are briefly discussed. A motivating example carried throughout this paper is the use of multivariate mixtures of normals for modeling daily changes in market variables.

Simulation of stationary Gaussian noise with regard to the Langevin equation with memory effect.

We present an efficient method for simulating a stationary Gaussian noise with an arbitrary covariance function, and then we study numerically the impact of time-correlated noise on the time evolution of a (1+1)-dimensional generalized Langevin equation by comparing also to analytical results. Finally, we apply our method to the generalized Langevin equation with an external harmonic and double-well potential.

BAYESIAN INFERENCE OF HIDDEN GAMMA WEAR PROCESS MODEL FOR SURVIVAL DATA WITH TIES.

Survival data often contain tied event times. Inference without careful treatment of the ties can lead to biased estimates. This paper develops the Bayesian analysis of a stochastic wear process model to fit survival data that might have a large number of ties. Under a general wear process model, we derive the likelihood of parameters. When the wear process is a Gamma process, the likelihood has a semi-closed form that allows posterior sampling to be carried out for the parameters, hence achieving model selection using Bayesian deviance information criterion. An innovative simulation algorithm via direct forward sampling and Gibbs sampling is developed to sample event times that may have ties in the presence of arbitrary covariates; this provides a tool to assess the precision of inference. An extensive simulation study is reported and a data set is used to further illustrate the proposed methodology.

Generalized Fisher matrices

The Fisher Information Matrix formalism is extended to cases where the data is divided into two parts (X,Y), where the expectation value of Y depends on X according to some theoretical model, and X and Y both have errors with arbitrary covariance. In the simplest case, (X,Y) represent data pairs of abscissa and ordinate, in which case the analysis deals with the case of data pairs with errors in both coordinates, but X can be any measured quantities on which Y depends. The analysis applies for arbitrary covariance, provided all errors are gaussian, and provided the errors in X are small, both in comparison with the scale over which the expected signal Y changes, and with the width of the prior distribution. This generalises the Fisher Matrix approach, which normally only considers errors in the `ordinate' Y. In this work, we include errors in X by marginalising over latent variables, effectively employing a Bayesian hierarchical model, and deriving the Fisher Matrix for this more general case. The methods here also extend to likelihood surfaces which are not gaussian in the parameter space, and so techniques such as DALI (Derivative Approximation for Likelihoods) can be generalised straightforwardly to include arbitrary gaussian data error covariances. For simple mock data and theoretical models, we compare to Markov Chain Monte Carlo experiments, illustrating the method with cosmological supernova data. We also include the new method in the Fisher4Cast software.

Comparison and anti-concentration bounds for maxima of Gaussian random vectors

Slepian and Sudakov–Fernique type inequalities, which compare expectations of maxima of Gaussian random vectors under certain restrictions on the covariance matrices, play an important role in probability theory, especially in empirical process and extreme value theories. Here we give explicit comparisons of expectations of smooth functions and distribution functions of maxima of Gaussian random vectors without any restriction on the covariance matrices. We also establish an anti-concentration inequality for the maximum of a Gaussian random vector, which derives a useful upper bound on the Lévy concentration function for the Gaussian maximum. The bound is dimension-free and applies to vectors with arbitrary covariance matrices. This anti-concentration inequality plays a crucial role in establishing bounds on the Kolmogorov distance between maxima of Gaussian random vectors. These results have immediate applications in mathematical statistics. As an example of application, we establish a conditional multiplier central limit theorem for maxima of sums of independent random vectors where the dimension of the vectors is possibly much larger than the sample size.

THE BETA-MANOVA ENSEMBLE WITH GENERAL COVARIANCE

We consider adding arbitrary covariance to the β-Jacobi random matrix model. We recall that for β = 1 the Jacobi random matrix model may be thought of as the eigenvalues, λi, of YtY(XTX + YtY)-1 where X and Y are matrices whose elements are i.i.d. standard normals. Equivalently we can take the generalized cosine singular values of (Y, X), ci, and use [Formula: see text]. When β = 1 we add covariance by considering YtY(YtY + ΩXtXΩ)-1, for a positive definite diagonal matrix Ω. Equivalently, and preferably, we consider the generalized singular value decomposition (gsvd) of (Y, XΩ). We refer to Ω = I as the Jacobi case and the general Ω case as the MANOVA case. In this paper, we provide a matrix model for the general β-MANOVA ensemble. In particular, we provide an algorithm for the numerical sampling of eigenvalues or generalized cosine singular values. The β-MANOVA algorithm uses the β-Wishart algorithm of Forrester and Dubbs–Edelman–Koev–Venkataramana as a subroutine, perhaps making β-MANOVA the first "second-order" continuous-β random matrix algorithm. Our proofs make use of a conjecture of MacDonald (proven by Baker and Forrester), a theorem of Kaneko, and many identities from Forrester's Log-Gases and Random Matrices. We supply numerical evidence that our theorems are correct.

Linear Transceiver Design for Downlink Multiuser MIMO Systems: Downlink-Interference Duality Approach

This paper considers linear transceiver design for downlink multiuser multiple-input multiple-output (MIMO) systems. We examine different transceiver design problems. We focus on two groups of design problems. The first group is the weighted sum mean-square-error (WSMSE) (i.e., symbol-wise or user-wise WSMSE) minimization problems and the second group is the minimization of the maximum weighted mean-square-error (WMSE) (symbol-wise or user-wise WMSE) problems. The problems are examined for the practically relevant scenario where the power constraint is a combination of per base station (BS) antenna and per symbol (user), and the noise vector of each mobile station is a zero-mean circularly symmetric complex Gaussian random variable with arbitrary covariance matrix. For each of these problems, we propose a novel downlink-interference duality based iterative solution. Each of these problems is solved as follows. First, we establish a new mean-square-error (MSE) downlink-interference duality. Second, we formulate the power allocation part of the problem in the downlink channel as a Geometric Program (GP). Third, using the duality result and the solution of GP, we utilize alternating optimization technique to solve the original downlink problem. For the first group of problems, we have established symbol-wise and user-wise WSMSE downlink-interference duality. These duality are established by formulating the noise covariance matrices of the interference channels as fixed point functions. On the other hand, for the second group of problems, we have established symbol-wise and user-wise MSE downlink-interference duality. These duality are established by formulating the noise covariance matrices of the interference channels as marginally stable (convergent) discrete-time-switched systems. The proposed duality based iterative solutions can be extended straightforwardly to solve many other linear transceiver design problems. We also show that our MSE downlink-interference duality unify all existing MSE duality. In our simulation results, we have observed that the proposed duality based iterative algorithms utilize less total BS power than that of the existing algorithms.

LÉVY-BASED ERROR PREDICTION IN CIRCULAR SYSTEMATIC SAMPLING

In the present paper, Lévy-based error prediction in circular systematic sampling is developed. A model-based statistical setting as in Hobolth and Jensen (2002) is used, but the assumption that the measurement function is Gaussian is relaxed. The measurement function is represented as a periodic stationary stochastic process X obtained by a kernel smoothing of a Lévy basis. The process X may have an arbitrary covariance function. The distribution of the error predictor, based on measurements in n systematic directions is derived. Statistical inference is developed for the model parameters in the case where the covariance function follows the celebrated p-order covariance model.

Covariance Matrices under Bell-like Detections

We derive a simple formula for the transformation of an arbitrary covariance matrix of (n + 2) bosonic modes under general Bell-like detections, where the last two modes are combined in an arbitrary beam splitter (i.e., with arbitrary transmissivity) and then homodyned. In particular, we consider the realistic condition of non-unit quantum efficiency for the homodyne detectors. This formula can easily be specialized to describe the standard Bell measurement and the heterodyne detection, which are exploited in many contexts, including protocols of quantum teleportation, entanglement swapping and quantum cryptography. In its general form, our formula can be adopted to study quantum information protocols in the presence of experimental imperfections and asymmetric setups, e.g., deriving from the use of unbalanced beam splitters.

GENETIC CONSTRAINTS ON ADAPTATION TO A CHANGING ENVIRONMENT

Genetic correlations between traits can constrain responses to natural selection. To what extent such correlations limit adaptation depends on patterns of directional selection. I derive the expected rate of adaptation (or evolvability) under randomly changing selection gradients. When directional selection gradients have an arbitrary covariance matrix, the average rate of adaptation depends on genetic correlations between traits, contrary to the isotropic case investigated in previous studies. Adaptation may be faster on average with more genetic correlation between traits, if these traits are selected to change jointly more often than the average pair of traits. However, natural selection maximizes the long-term fitness of a population, not necessarily its rate of adaptation. I therefore derive the average lag load caused by deviations of the mean phenotype from an optimum, under several forms of environmental changes typically experienced by natural populations, both stochastic and deterministic. Simple formulas are produced for how the G matrix affects long-term fitness in these contexts, and I discuss how their parameters can be estimated empirically.

On the Conjugate Gradient Matched Filter

The conjugate gradient (CG) algorithm is an efficient method for the calculation of the weight vector of the matched filter (MF). As an iterative algorithm, it produces a series of approximations to the MF weight vector, each of which can be used to filter the test signal and form a test statistic. This effectively leads to a family of detectors, referred to as the CG-MF detectors, which are indexed by the number of iterations incurred. We first consider a general case involving an arbitrary covariance matrix of the disturbance (including interference, noise, etc.) and show that all CG-MF detectors attain constant false alarm rate (CFAR) and, furthermore, are optimum in the sense that the th CG-MF detector yields the highest output signal-to-interference-and-noise ratio (SINR) among all linear detectors within the th Krylov subspace. We then consider a structured case frequently encountered in practice, where the covariance matrix of the disturbance contains a low-rank component (rank- ) due to dominant interference sources, a scaled identity due to the presence of a white noise, and a perturbation component containing the residual interference. We show that the st CG-MF detector achieves CFAR and an output SINR nearly identical to that of the MF detector which requires complete iterations of the CG algorithm till reaching convergence. Hence, the st CG-MF detector can be used in place of the MF detector for significant computational saving when is small. Numerical results are presented to verify the accuracy of our analysis for the CG-MF detectors.

Robust Sum MSE Optimization for Downlink Multiuser MIMO Systems With Arbitrary Power Constraint: Generalized Duality Approach

This paper considers linear minimum mean-square-error (MMSE) transceiver design problems for downlink multiuser multiple-input multiple-output (MIMO) systems where imperfect channel state information is available at the base station (BS) and mobile stations (MSs). We examine robust sum mean-square-error (MSE) minimization problems. The problems are examined for the generalized scenario where the power constraint is per BS, per BS antenna, per user or per symbol, and the noise vector of each MS is a zero-mean circularly symmetric complex Gaussian random variable with arbitrary covariance matrix. For each of these problems, we propose a novel duality based iterative solution. Each of these problems is solved as follows. First, we establish a novel sum average mean-square-error (AMSE) duality. Second, we formulate the power allocation part of the problem in the downlink channel as a Geometric Program (GP). Third, using the duality result and the solution of GP, we utilize alternating optimization technique to solve the original downlink problem. To solve robust sum MSE minimization constrained with per BS antenna and per BS power problems, we have established novel downlink-uplink duality. On the other hand, to solve robust sum MSE minimization constrained with per user and per symbol power problems, we have established novel downlink-interference duality. For the total BS power constrained robust sum MSE minimization problem, the current duality is established by modifying the constraint function of the dual uplink channel problem. And, for the robust sum MSE minimization with per BS antenna and per user (symbol) power constraint problems, our duality are established by formulating the noise covariance matrices of the uplink and interference channels as fixed point functions, respectively. We also show that our sum AMSE duality are able to solve other sum MSE-based robust design problems. Computer simulations verify the robustness of the proposed robust designs compared to the nonrobust/naive designs.

Maximum likelihood estimation of Gaussian mixture models using stochastic search

Gaussian mixture models (GMM), commonly used in pattern recognition and machine learning, provide a flexible probabilistic model for the data. The conventional expectation-maximization (EM) algorithm for the maximum likelihood estimation of the parameters of GMMs is very sensitive to initialization and easily gets trapped in local maxima. Stochastic search algorithms have been popular alternatives for global optimization but their uses for GMM estimation have been limited to constrained models using identity or diagonal covariance matrices. Our major contributions in this paper are twofold. First, we present a novel parametrization for arbitrary covariance matrices that allow independent updating of individual parameters while retaining validity of the resultant matrices. Second, we propose an effective parameter matching technique to mitigate the issues related with the existence of multiple candidate solutions that are equivalent under permutations of the GMM components. Experiments on synthetic and real data sets show that the proposed framework has a robust performance and achieves significantly higher likelihood values than the EM algorithm.

A General Framework for Detecting Disease Associations with Rare Variants in Sequencing Studies

Biological and empirical evidence suggests that rare variants account for a large proportion of the genetic contributions to complex human diseases. Recent technological advances in high-throughput sequencing platforms have made it possible for researchers to generate comprehensive information on rare variants in large samples. We provide a general framework for association testing with rare variants by combining mutation information across multiple variant sites within a gene and relating the enriched genetic information to disease phenotypes through appropriate regression models. Our framework covers all major study designs (i.e., case-control, cross-sectional, cohort and family studies) and all common phenotypes (e.g., binary, quantitative, and age at onset), and it allows arbitrary covariates (e.g., environmental factors and ancestry variables). We derive theoretically optimal procedures for combining rare mutations and construct suitable test statistics for various biological scenarios. The allele-frequency threshold can be fixed or variable. The effects of the combined rare mutations on the phenotype can be in the same direction or different directions. The proposed methods are statistically more powerful and computationally more efficient than existing ones. An application to a deep-resequencing study of drug targets led to a discovery of rare variants associated with total cholesterol. The relevant software is freely available.

Volatility Forecasting and Explanatory Variables: A Tractable Bayesian Approach to Stochastic Volatility

We provide a formulation of stochastic volatility (SV) based on Gaussian process regression (GPR). Forecasting volatility out-of-sample, both simulation and empirical analyses show that our GPR-based stochastic volatility (GPSV) model clearly outperforms SV and GARCH benchmarks, especially at long horizons. Most importantly, our approach enables the straightforward incorporation of arbitrary covariates without requiring the specification of functional forms a priori. Augmenting the GPSV model with exogenous variables increases its performance even further. In particular, a simple set of covariates reduces the error rate on one-year out-of-sample forecasting during the 2007-09 recession by 26% relative to a benchmark range-based SV model.

Bayesian Minimum Mean-Square Error Estimation for Classification Error—Part II: Linear Classification of Gaussian Models

In this paper, Part II of a two-part study, we derive a closed-form analytic representation of the Bayesian minimum mean-square error (MMSE) error estimator for linear classification assuming Gaussian models. This is presented in a general framework permitting a structure on the covariance matrices and a very flexible class of prior parameter distributions with four free parameters. Closed-form solutions are provided for known, scaled identity, and arbitrary covariance matrices. We examine performance in small sample settings via simulations on both synthetic and real genomic data, and demonstrate the robustness of these error estimators to false Gaussian modeling assumptions by applying them to Johnson distributions.

Blind Separation of Gaussian Sources With General Covariance Structures: Bounds and Optimal Estimation

We consider the separation of Gaussian sources exhibiting general, arbitrary (not necessarily stationary) covariance structures. First, assuming a semi-blind scenario, in which the sources' covariance structures are known, we derive the maximum likelihood estimate of the separation matrix, as well as the induced Cramér-Rao lower bound (iCRLB) on the attainable Interference to Source Ratio (ISR). We then extend our results to the fully blind scenario, in which the covariance structures are unknown. We show that (under a scaling convention) the Fisher information matrix in this case is block-diagonal, implying that the same iCRLB (as in the semi-blind scenario) applies in this case as well. Subsequently, we demonstrate that the same “semi-blind” optimal performance can be approached asymptotically in the “fully blind” scenario if the sources are sufficiently ergodic, or if multiple snapshots are available.

Urn sampling, interval censoring and proportional hazard models

This paper proposes a new distribution-free statistical method for testing hypotheses about covariates for survival data having simultaneously right-, left- and interval-censored survival times. The new test is motivated by the analogue between urn sampling and the Cox’s proportional hazard models. Investigations of the significance levels and power as a function of the proportion of interval-censored observations and interval length show that the test performs well for most censoring situations encountered in practice. Simulation results also suggest that there is negligible loss of power in the practical situation in which the mean interval length for interval-censored observations is less than the mean survival time. This holds even with heavy interval censoring. Comparison with the widely used Mantel’s method for comparing two groups shows that the power of the new method appears to be superior. Furthermore, the test is relatively simple to carry out and generalizes to comparing k populations as well as the testing of general linear hypothesis for arbitrary covariates.

A trivariate nakagami-m distribution with arbitrary covariance matrix and applications to generalized-selection diversity receivers

This paper deals with a trivariate Nakagami-m distribution derived from the diagonal elements of a Wishart matrix. For this distribution, infinite series representations for its probability density and cumulative distribution functions are derived having an arbitrary covariance matrix and integer-order fading parameters. Moreover, upper bounds on the error resulting from truncating the infinite series are obtained. Based on the derived formulas, the performance of triple-branch generalized selection combining (GSC) receivers is analyzed. For this type of receivers, the outage and the average bit error probability for a variety of modulation schemes are analytically obtained. The performance of GSC receivers is compared to that of conventional selection and maximal-ratio diversity schemes. In order to check the accuracy and convergence of the derived formulas, various performance evaluation results are presented and compared to equivalent simulation ones.