Structure Of Covariance Matrices Research Articles

A mixture of logistic skew-normal multinomial models

The logistic normal multinomial distribution is gaining interest in modelling microbiome data. It utilizes a hierarchical structure such that the observed counts conditional on the compositions are assumed to be multinomial random variables and the log-ratio transformed compositions are assumed to be from a Gaussian distribution. While multinomial distribution accounts for the compositional nature of the data, and a Gaussian prior offers flexibility in the structure of covariance matrices, the log-ratio transformed compositions of the microbiome data can be highly skewed, especially at a lower taxonomic level. Thus, a Gaussian distribution may not be an ideal prior for the log-ratio transformed compositions. A novel mixture of logistic skew-normal multinomial (LSNM) distribution is proposed in which a multivariate skew-normal distribution is utilized as a prior for the log-ratio transformed compositions. A variational Gaussian approximation in conjunction with the EM algorithm is utilized for parameter estimation.

Adaptive Tests for Bandedness of High-dimensional Covariance Matrices

Estimation of the high-dimensional banded covariance matrix is widely used in multivariate statistical analysis. To ensure the validity of estimation, we aim to test the hypothesis that the covariance matrix is banded with a certain bandwidth under the high-dimensional framework. Though several testing methods have been proposed in the literature, the existing tests are only powerful for some alternatives with certain sparsity levels, whereas they may not be powerful for alternatives with other sparsity structures. The goal of this paper is to propose a new test for the bandedness of high-dimensional covariance matrix, which is powerful for alternatives with various sparsity levels. The proposed new test also be used for testing the banded structure of covariance matrices of error vectors in high-dimensional factor models. Based on these statistics, a consistent bandwidth estimator is also introduced for a banded high dimensional covariance matrix. Extensive simulation studies and an application to a prostate cancer dataset from protein mass spectroscopy are conducted for evaluating the effectiveness of the proposed adaptive tests blue and bandwidth estimator for the banded covariance matrix.

Long-time adaptive coherent detection of small targets in sea clutter by fast inversion algorithm of block tridiagonal speckle covariance matrices

Low-velocity small target detection in maritime surveillance radars is always a challenging task. Low signal-to-clutter ratio requires long-time coherent integration to obtain enough gain of target returns. However, long-time coherent integration encounters insufficient secondary data due to the spatial inhomogeneity of sea clutter. In this paper, considering the decorrelation time of speckle component of sea clutter short up to a dozen of milliseconds, the spherical invariant random vector (SIRV) model with block tridiagonal speckle covariance matrix and the inverse Gamma distributed texture is proposed to model sea clutter sequences in several tenths of a second. In this model, a long-time adaptive generalized likelihood ratio test with linear threshold detector (GLRT-LTD) is constructed. Owing to the block tridiagonal structure of speckle covariance matrices, the adaptive detection requires much less reference cells for speckle covariance matrix estimation and much lower computational cost for its inversion. The proposed detector is verified by an X-band high-resolution island-based radar data with an unmanned aerial vehicle (UAV) as test target. The experimental result shows that it obtains competitive detection performance in comparison with the state-of-the-art detectors.

A KKT Conditions Based Transceiver Optimization Framework for RIS-Aided Multiuser MIMO Networks

In many core problems of signal processing and wireless communications, Karush-Kuhn-Tucker (KKT) conditions based optimization plays a fundamental role. Hence we investigate the KKT conditions in the context of optimizing positive semidefinite matrix variables under nonconvex rank constraints. More explicitly, based on the properties of KKT conditions, we optimize a reconfigurable intelligent surface (RIS) aided multi-user multi-input multi-output (MU-MIMO) network. Specifically, we consider the capacity maximization and sum mean square error (MSE) minimization problems of both the RIS-aided MU-MIMO uplink (UL) and downlink (DL) under multiple weighted power constraints and rank constraints. As for the RIS-aided MU-MIMO UL, the optimal structures of the signal covariance matrices are derived based on the KKT conditions. Furthermore, an efficient procedure is designed for solving the capacity maximization and sum mean square error (MSE) minimization problems. Then the UL-DL dualities are exploited for solving the capacity maximization and MSE minimization problems of the RIS-aided MU-MIMO DL based on the results of the UL optimization. Hence in the proposed framework, the phase shifting matrix of the RIS is jointly optimized with the signal covariance matrices for both the UL and DL. Our simulation results demonstrate the performance advantages of the proposed framework.

New collectivity measures for financial covariances and correlations

Complex systems are usually non-stationary and their dynamics is often dominated by collective effects. Collectivity, defined as coherent motion of the whole system or of some of its parts, manifests itself in the time-dependent structures of covariance and correlation matrices. The largest eigenvalue corresponds to the collective motion of the system as a whole, while the other large, isolated, eigenvalues indicate collectivity in parts of the system. In the case of finance, these are industrial sectors. By removing the collective motion of the system as a whole, the latter effects are much better revealed. We measure a remaining collectivity to which we refer as average sector collectivity. We identify collective signals around the Lehman Brothers crash and after the dot-com bubble burst. For the Lehman Brothers crash, we find a potential precursor. We analyze 213 US stocks over a period of more than 30 years from 1990 to 2021. We plot the average sector collectivity versus the collectivity corresponding to the largest eigenvalue to study the whole market trajectory in a two dimensional space spanned by both collectivities. Therefore, we capture the average sector collectivity in a much more precise way. Additionally, we observe that larger values in the average sector collectivity are often accompanied by trend shifts in the mean covariances and mean correlations. As of 2015/2016 the collectivity in the US stock markets changed fundamentally.

Casting vector time series: algorithms for forecasting, imputation, and signal extraction

Recursive algorithms, based upon the nested structure of Toeplitz covariance matrices arising from stationary processes, are presented for the efficient computation of multi-step ahead forecast error covariances for nonstationary vector time series. Further, we discuss time reversal to forecast the past, and a filtering algorithm for imputation of missing values. These quantities are required to quantify multi-step ahead forecast error and signal extraction error. An information filter is presented, which provides imputations for arbitrary patterns of missing values (such as ragged edge patterns occurring in mixed frequency data). The methods are applied to multivariate retail data exhibiting trend dynamics and seasonality.

Stepwise Covariance-Free Common Principal Components (CF-CPC) With an Application to Neuroscience.

Finding the common principal component (CPC) for ultra-high dimensional data is a multivariate technique used to discover the latent structure of covariance matrices of shared variables measured in two or more k conditions. Common eigenvectors are assumed for the covariance matrix of all conditions, only the eigenvalues being specific to each condition. Stepwise CPC computes a limited number of these CPCs, as the name indicates, sequentially and is, therefore, less time-consuming. This method becomes unfeasible when the number of variables p is ultra-high since storing k covariance matrices requires O(kp2) memory. Many dimensionality reduction algorithms have been improved to avoid explicit covariance calculation and storage (covariance-free). Here we propose a covariance-free stepwise CPC, which only requires O(kn) memory, where n is the total number of examples. Thus for n < < p, the new algorithm shows apparent advantages. It computes components quickly, with low consumption of machine resources. We validate our method CFCPC with the classical Iris data. We then show that CFCPC allows extracting the shared anatomical structure of EEG and MEG source spectra across a frequency range of 0.01–40 Hz.

Exploitation of Kronecker Structure in Gaussian Process Regression for Efficient Biomedical Signal Processing

Abstract Gaussian processes are a versatile tool for data processing. Unfortunately, due to storage and runtime requirements, standard Gaussian process (GP) methods are limited to a few thousand data points. Thus, they are infeasible in most biomedical, spatio-temporal problems. The methods treated in this work cover GP inference and hyperparameter optimization, exploiting the Kronecker structure of covariance matrices. To solve regression and source separation problems, two different approaches are presented. The first approach uses efficient matrix-vector-products, whilst the second approach is based on efficient solutions to the eigendecomposition. The latter also enables efficient hyperparameter optimization. In comparison to standard GP methods, the proposed methods can be applied to very large biomedical datasets without any further performance loss and perform substantially faster. The performance is demonstrated on esophageal manometry data, where the cardiac and respiratory signal components are to be inferred by source separation.

A Graph Clustering Approach to Localization for Adaptive Covariance Tuning in Data Assimilation Based on State-Observation Mapping.

An original graph clustering approach for the efficient localization of error covariances is proposed within an ensemble-variational data assimilation framework. Here, the localization term is very generic and refers to the idea of breaking up a global assimilation into subproblems. This unsupervised localization technique based on a linearized state-observation measure is general and does not rely on any prior information such as relevant spatial scales, empirical cutoff radii or homogeneity assumptions. Localization is performed via graph theory, a branch of mathematics emerging as a powerful approach to capturing complex and highly interconnected Earth and environmental systems in computational geosciences. The novel approach automatically segregates the state and observation variables in an optimal number of clusters, and it is more amenable to scalable data assimilation. The application of this method does not require underlying block-diagonal structures of prior covariance matrices. To address intercluster connectivity, two alternative data adaptations are proposed. Once the localization is completed, a covariance diagnosis and tuning are performed within each cluster, whose contribution is sequentially integrated into the entire covariance matrix. Numerical twin experiment tests show the reduced cost and added flexibility of this approach compared to global covariance tuning, and more accurate results yielded for both observation- and background-error parameter tuning.

Modified Pillai’s trace statistics for two high-dimensional sample covariance matrices

The goal of this study was to test the equality of two covariance matrices by using modified Pillai’s trace statistics under a high-dimensional framework, i.e., the dimension and sample sizes go to infinity proportionally. In this paper, we introduce two modified Pillai’s trace statistics and obtain their asymptotic distributions under the null hypothesis. The benefits of the proposed statistics include the following: (1) the sample size can be smaller than the dimensions; (2) the limiting distributions of the proposed statistics are universal; and (3) we do not restrict the structure of the population covariance matrices. The theoretical results are established under mild and practical assumptions, and their properties are demonstrated numerically by simulations and a real data analysis.

Multivariate High-Frequency-Based Factor Model

We propose a new predictive model for large-dimensional realized covariance matrices. Using high-frequency data, we construct daily realized covariance matrices for the constituents of the S\&P 500 Index and a set of observable factors. Using a standard decomposition of the joint covariance matrix we express the covariance matrix of the individual assets similar to an approximate factor model. A novel feature of the model, is the use of the hierarchical clustering algorithm to determine the structure of the idiosyncratic covariance matrices. To construct a conditional covariance model, we suggest to model the components of the covariance structure separately using autoregressive time series regressions. In an out-of-sample portfolio selection exercise, we find that the proposed model outperforms other commonly used multivariate volatility models in extant literature.

Model selection in linear mixed-effect models

Linear mixed-effects models are a class of models widely used for analyzing different types of data: longitudinal, clustered and panel data. Many fields, in which a statistical methodology is required, involve the employment of linear mixed models, such as biology, chemistry, medicine, finance and so forth. One of the most important processes, in a statistical analysis, is given by model selection. Hence, since there are a large number of linear mixed model selection procedures available in the literature, a pressing issue is how to identify the best approach to adopt in a specific case. We outline mainly all approaches focusing on the part of the model subject to selection (fixed and/or random), the dimensionality of models and the structure of variance and covariance matrices, and also, wherever possible, the existence of an implemented application of the methodologies set out.

Under-Determined Convolutive Blind Source Separation Combining Density-Based Clustering and Sparse Reconstruction in Time-Frequency Domain

Blind source separation (BSS) in time-frequency (TF) domain is a versatile framework to recover sources from the recorded mixture signals in a reverberant environment. In general, a two-stage strategy is one of the popular BSS frameworks for the underdetermined BSS case (the number of mixtures is less than the number of sources), which is a tough problem due to the mixing matrix is not invertible. In this paper, we propose a new two-stage scheme combining density-based clustering and sparse reconstruction to estimate mixing matrix and sources, respectively. At the first stage, we transform the mixing matrix estimation as an eigenvector clustering problem based on a particular local dominant assumption. The eigenvectors are first exploited from the rank-one structure of local covariance matrices of mixture TF vectors. These eigenvectors are then clustered and adjusted to give estimated mixing matrix by cooperating density-based clustering and weight clustering. At the second stage, we transform the source reconstruction as a $\ell _{p}$ norm $(0 minimization by an iterative Lagrange multiplier method. With a proper initialization, the obtained solution is a global minimum for any $p$ in (0, 1] with convergence guarantee. The proposed approach is demonstrated to be superior to the state-of-the-art baseline methods in various underdetermined experiments.

Series approximations for Rayleigh distributions of arbitrary dimensions and covariance matrices

The multivariate Rayleigh distribution is of crucial importance to many applied problems of engineering, such as in the analysis of multi-antenna wireless systems. Due to the lack of a generalised closed form of the distribution, the dependence on effective approximation methods for evaluation has created numerous numerical approaches with considerable restrictions in both dimensionality, as well as the structure of covariance matrices. In this paper we extend a previously introduced method [1] without either of these limitations. We then compare the performance of the new algorithms to recent integration methods of fixed dimension, presented by Beaulie and Zhang [2] and highlight the advantages of the new method.

Weighted covariance matrix estimation

The paper proposes a cross-validated linear shrinkage estimation for population covariance matrices. Moreover we also propose a novel weighted estimator based on the thresholding and shrinkage methods for high dimensional datasets. It is applicable to a wider scope of different structures of covariance matrices. Some theoretical results about the cross-validated shrinkage method and weighted covariance estimation methods are also developed. The finite-sample performance of the proposed methods is illustrated through extensive simulations and real data analysis.

High-performance Processing of Covariance Matrices Using GPU Computations

Practical applicability of many statistical algorithms is limited by large sizes of corresponding covariance matrices. These limitations can be significantly weakened due to effective use of the structure of covariance matrices, properties of the autocorrelation function, and advantages of the architecture of modern GPUs. This paper presents GPU implementations of the algorithms for inversion of a covariance matrix and solution of a system of linear equations whose coefficient matrix is a covariance matrix. Inversion of close to sparse covariance matrices is also considered in the work. For all the cases considered, significant accelerations were obtained in comparison with Octave mathematical software and ViennaCL computational library. For example, implemented algorithm of solution of a linear system was 6 times faster as compared with the implementation of Octave on the CPU and 3 times faster as compared with the ViennaCL implementation on the GPU for general matrices. The performance of inversion of a covariance matrix was 14 times faster than inversion algorithm of Octave on the CPU and 6 times faster than ViennaCL inversion algorithm on GPU.

RFI Mitigation for UWB Radar Via Hyperparameter-Free Sparse SPICE Methods

Radio frequency interference (RFI) causes serious problems to ultrawideband (UWB) radar operations due to severely degrading radar imaging capability and target detection performance. This paper formulates proper data models and proposes novel methods for effective RFI mitigation. We first apply the single-snapshot Sparse Iterative Covariance-based Estimation (SPICE) algorithm to data from each pulse repetition interval for RFI mitigation and discuss the connection of SPICE to the $l_{1}$ -penalized least absolute deviation ( $l_{1}$ -PLAD) approach. Then, we devise a modified group SPICE algorithm and we prove that it is equivalent to a special case of the $l_{1,2}$ -PLAD method. The modified group SPICE algorithm can be applied to data from a coherent processing interval for effective RFI mitigation. Both the single-snapshot SPICE and the modified group SPICE methods simultaneously exploit the sparsity properties of both RFI spectrum and UWB radar target echoes. Unlike the existing sparsity-based RFI suppression methods, such as the robust principal component analysis algorithm, the proposed methods are hyperparameter-free and therefore easier to use in practical applications. Furthermore, the fast implementation of the SPICE methods is considered by exploiting the special structures of both single-snapshot and multiple-snapshot covariance matrices. Finally, the results obtained from applying the SPICE methods to simulated data as well as measured data collected by the U.S. Army Research Laboratory synthetic aperture radar system are presented to demonstrate the effectiveness of the proposed methods.

Fast iterative tomographic wavefront estimation with recursive Toeplitz reconstructor structure for large-scale systems.

Tomographic wavefront reconstruction is the main computational bottleneck to realize real-time correction for turbulence-induced wavefront aberrations in future laser-assisted tomographic adaptive-optics (AO) systems for ground-based giant segmented mirror telescopes because of its unprecedented number of degrees of freedom, N, i.e., the number of measurements from wavefront sensors. In this paper, we provide an efficient implementation of the minimum-mean-square error (MMSE) tomographic wavefront reconstruction, which is mainly useful for some classes of AO systems not requiring multi-conjugation, such as laser-tomographic AO, multi-object AO, and ground-layer AO systems, but is also applicable to multi-conjugate AO systems. This work expands that by Conan [Proc. SPIE9148, 91480R (2014)PSISDG0277-786X10.1117/12.2054472] to the multi-wavefront tomographic case using natural and laser guide stars. The new implementation exploits the Toeplitz structure of covariance matrices used in an MMSE reconstructor, which leads to an overall O(N log N) real-time complexity compared with O(N2) of the original implementation using straight vector-matrix multiplication. We show that the Toeplitz-based algorithm leads to 60nm rms wavefront error improvement for the European Extremely Large Telescope laser-tomography AO system over a well-known sparse-based tomographic reconstruction; however, the number of iterations required for suitable performance is still beyond what a real-time system can accommodate to keep up with the time-varying turbulence.

Nonlinear Geospatial Frame Transformations in the Presence of Noisy Data

This paper presents an extended least squares framework for geospatial frame transformation problems with nonlinear models in the presence of noisy data. The goal is to set up an analytic scheme for obtaining transformed coordinates in the target frame with improved accuracy over the classic stepwise approach that is commonly used in practice. The proposed methodology is based on the joint spatial adjustment of all observed coordinates from the respective frames (initial and target) in conjunction with their stochastic model over all points of interest. This allows us to properly handle the cross-correlated noise in the initial coordinates between control and new points, and thus to overcome a crucial limitation of other operational approaches. In algebraic terms, the proposed approach relies on the weighted least squares adjustment of a stacked set of Gauss–Helmert models that can unequivocally describe any geospatial frame transformation over different groups of points. The analytic solution to this stacking problem is derived for the case of nonlinear parametric models, regardless of the structure of the error covariance matrices of the input data. The study focuses on computational aspects and the proper implementation of the solution algorithm via the Newton–Gauss iteration method. A numerical example is given to demonstrate the expected improvement in the statistical accuracy of the transformed coordinates under the proposed stacking approach.

Multiband tangent space mapping and feature selection for classification of EEG during motor imagery

Objective. When designing multiclass motor imagery-based brain–computer interface (MI-BCI), a so-called tangent space mapping (TSM) method utilizing the geometric structure of covariance matrices is an effective technique. This paper aims to introduce a method using TSM for finding accurate operational frequency bands related brain activities associated with MI tasks. Approach. A multichannel electroencephalogram (EEG) signal is decomposed into multiple subbands, and tangent features are then estimated on each subband. A mutual information analysis-based effective algorithm is implemented to select subbands containing features capable of improving motor imagery classification accuracy. Thus obtained features of selected subbands are combined to get feature space. A principal component analysis-based approach is employed to reduce the features dimension and then the classification is accomplished by a support vector machine (SVM). Main results. Offline analysis demonstrates the proposed multiband tangent space mapping with subband selection (MTSMS) approach outperforms state-of-the-art methods. It acheives the highest average classification accuracy for all datasets (BCI competition dataset 2a, IIIa, IIIb, and dataset JK-HH1). Significance. The increased classification accuracy of MI tasks with the proposed MTSMS approach can yield effective implementation of BCI. The mutual information-based subband selection method is implemented to tune operation frequency bands to represent actual motor imagery tasks.