Inverse Covariance Estimation Research Articles

Accessing dynamic functional connectivity using [formula omitted]-regularized sparse-smooth inverse covariance estimation from fMRI

Inferring dynamic functional connectivity (dFC) from functional magnetic resonance imaging (fMRI) is crucial to understand the time-variant functional inter-relationships among brain regions. Because of the sparse property of functional connectivity networks, sparsity-promoting dFC estimation methods, which are mainly based on l1-norm regularization, are gaining popularity. However, l1-norm regularization cannot provide the maximum sparsity solution as the most natural sparsity promoting norm, the l0-norm. But l0-norm is seldom used to infer sparse dFC because an efficient algorithm to address the non-convexity problem of l0-norm is lacking. In this work, we develop a new l0-norm regularization-based inverse covariance estimation method for estimating dFC from fMRI. This novel method employs l0-norm regularizations on both spatial and temporal scales to enhance the spatial sparsity and temporal smoothness of dFC estimates. To overcome the non-convexity of l0-norm, we further propose an effective optimization algorithm based on the coordinate descent (CD). The performance of the proposed l0-norm-based sparse-smooth regularization (L0-SSR) method is examined using a series of synthetic datasets concerning various types of network topology. We further apply the proposed L0-SSR method to real fMRI data recorded in block-design motor tasks from 45 participants for the exploration of task induced dFC. Results on synthetic and real-world fMRI data show that, the L0-SSR method can achieve more accurate and interpretable dFC estimates than conventional l1-norm-based dFC estimation methods. Hence, the proposed L0-SSR method could serve as a powerful analytical tool to infer highly complex, variable, and sparse dFC patterns.

Spectral guided sparse inverse covariance estimation of metabolic networks in Parkinson's disease

In neurodegenerative disorders, a clearer understanding of the underlying aberrant networks facilitates the search for effective therapeutic targets and potential cures. [18F]-fluorodeoxyglucose (FDG) positron emission tomography (PET) imaging data of brain metabolism reflects the distribution of glucose consumption known to be directly related to neural activity. In FDG PET resting-state metabolic data, characteristic disease-related patterns have been identified in group analysis of various neurodegenerative conditions using principal component analysis of multivariate spatial covariance. Notably, among several parkinsonian syndromes, the identified Parkinson's disease-related pattern (PDRP) has been repeatedly validated as an imaging biomarker of PD in independent groups worldwide. Although the primary nodal associations of this network are known, its connectivity is not fully understood. Here, we describe a novel approach to elucidate functional principal component (PC) network connections by performing graph theoretical sparse network derivation directly within the disease relevant PC partition layer of the whole brain data rather than by searching for associations retrospectively in whole brain sparse representations. Using sparse inverse covariance estimation of each overlapping PC partition layer separately, a single coherent network is detected for each layer in contrast to more spatially modular segmentation in whole brain data analysis. Using this approach, the major nodal hubs of the PD disease network are identified and their characteristic functional pathways are clearly distinguished within the basal ganglia, midbrain and parietal areas. Network associations are further clarified using Laplacian spectral analysis of the adjacency matrices. In addition, the innate discriminative capacity of the eigenvector centrality of the graph derived networks in differentiating PD versus healthy external data provides evidence of their validity.

Inference and Analysis of SPIEC-EASI Microbiome Networks.

Network analysis facilitates examination of the interactions between different populations in a community. It can provide a range of metrics describing the social characteristics of each population and emergent structural properties of the community, which may be used to address novel ecological questions. Using a publicly available dataset, this chapter provides point-by-point code and instructions to infer and analyze a SPIEC-EASI (SParse InversE Covariance Estimation for Ecological Association Inference) network using free, open source software (R and Gephi).

Spatiotemporal wind forecasting by learning a hierarchically sparse inverse covariance matrix using wind directions

Given the advances in online data acquisition systems, statistical learning models are increasingly used to forecast wind speed. In electricity markets, wind farm production forecasts are needed for the day-ahead, intra-day, and real-time markets. In this work, we use a spatiotemporal model that leverages wind dynamics to forecast wind speed. Using a priori knowledge of the wind direction, we propose a maximum likelihood estimate of the inverse covariance matrix regularized with a hierarchical sparsity-inducing penalty. The resulting inverse covariance estimate not only exhibits the benefits of a sparse estimator, but also enables meaningful sparse structures by considering wind direction. A proximal method is used to solve the underlying optimization problem. The proposed methodology is used to forecast six-hour-ahead wind speeds in 20-minute time intervals for a case study in Texas. We compare our method with a number of other statistical methods. Prediction performance measures and the Diebold–Mariano test show the potential of the proposed method, specifically when reasonably accurate estimates of the wind directions are available.

On the Solution of ℓ0-Constrained Sparse Inverse Covariance Estimation Problems

The sparse inverse covariance matrix is used to model conditional dependencies between variables in a graphical model to fit a multivariate Gaussian distribution. Estimating the matrix from data are well known to be computationally expensive for large-scale problems. Sparsity is employed to handle noise in the data and to promote interpretability of a learning model. Although the use of a convex ℓ1 regularizer to encourage sparsity is common practice, the combinatorial ℓ0 penalty often has more favorable statistical properties. In this paper, we directly constrain sparsity by specifying a maximally allowable number of nonzeros, in other words, by imposing an ℓ0 constraint. We introduce an efficient approximate Newton algorithm using warm starts for solving the nonconvex ℓ0-constrained inverse covariance learning problem. Numerical experiments on standard data sets show that the performance of the proposed algorithm is competitive with state-of-the-art methods. Summary of Contribution: The inverse covariance estimation problem underpins many domains, including statistics, operations research, and machine learning. We propose a scalable optimization algorithm for solving the nonconvex ℓ0-constrained problem.

Optimized Elastic Network Models With Direct Characterization of Inter-Residue Cooperativity for Protein Dynamics.

The elastic network models (ENMs)are known as representative coarse-grained models to capture essential dynamics of proteins. Due to simple designs of the force constants as a decay with spatial distances of residue pairs in many previous studies, there is still much room for the improvement of ENMs. In this article, we directly computed the force constants with the inverse covariance estimation using a ridge-type operater for the precision matrix estimation (ROPE)on a large-scale set of NMR ensembles. Distance-dependent statistical analyses on the force constants were further comprehensively performed in terms of several paired types of sequence and structural information, including secondary structure, relative solvent accessibility, sequence distance and terminal. Various distinguished distributions of the mean force constants highlight the structural and sequential characteristics coupled with the inter-residue cooperativity beyond the spatial distances. We finally integrated these structural and sequential characteristics to build novel ENM variations using the particle swarm optimization for the parameter estimation. The considerable improvements on the correlation coefficient of the mean-square fluctuation and the mode overlap were achieved by the proposed variations when compared with traditional ENMs. This study opens a novel way to develop more accurate elastic network models for protein dynamics.

Beyond Covariance: SICE and Kernel Based Visual Feature Representation

The past several years have witnessed increasing research interest on covariance-based feature representation. Originally proposed as a region descriptor, it has now been used as a general representation in various recognition tasks, demonstrating promising performance. However, covariance matrix has some inherent shortcomings such as singularity in the case of small sample, limited capability in modeling complicated feature relationship, and a single, fixed form of representation. To achieve better recognition performance, this paper argues that more capable and flexible symmetric positive definite (SPD)-matrix-based representation shall be explored, and this is attempted in this work by exploiting prior knowledge of data and nonlinear representation. Specifically, to better deal with the issues of small number of feature vectors and high feature dimensionality, we propose to exploit the structure sparsity of visual features and exemplify sparse inverse covariance estimate as a new feature representation. Furthermore, to effectively model complicated feature relationship, we propose to directly compute kernel matrix over feature dimensions, leading to a robust, flexible and open framework of SPD-matrix-based representation. Through theoretical analysis and experimental study, the proposed two representations well demonstrate their advantages over the covariance counterpart in skeletal human action recognition, image set classification and object classification tasks.

Estimating sparse networks with hubs

Graphical modelling techniques based on sparse estimation have been applied to infer complex networks in many fields, including biology and medicine, engineering, finance and social sciences. One structural feature of some of these networks that poses a challenge for statistical inference is the presence of a small number of strongly interconnected nodes, which are called hubs. For example, in microbiome research hubs or microbial taxa play a significant role in maintaining stability of the microbial community structure. In this paper, we investigate the problem of estimating sparse networks in which there are a few highly connected hub nodes. Methods based on L1-regularization have been widely used for performing sparse estimation in the graphical modelling context. However, while these methods encourage sparsity, they do not take into account structural information of the network. We introduce a new method for estimating networks with hubs that exploits the ability of (inverse) covariance estimation methods to include structural information about the underlying network. Our method is a weighted lasso approach with novel row/column sum weights, which we refer to as the hubs weighted graphical lasso. A practical advantage of the new method is that it leads to an optimization problem that is solved using the efficient graphical lasso algorithm that is already implemented in the R package glasso (Friedman et al., 2019). We establish large sample properties of the method when the number of parameters diverges with the sample size. We then show via simulations that the method outperforms competing methods and illustrate its performance with an application to microbiome data.

Microbial Interaction Network Estimation via Bias-Corrected Graphical Lasso

With the increasing availability of microbiome 16S data, network estimation has become a useful approach to studying the interactions between microbial taxa. Network estimation on a set of variables is frequently explored using graphical models, in which the relationship between two variables is modeled via their conditional dependency given the other variables. Various methods for sparse inverse covariance estimation have been proposed to estimate graphical models in the high-dimensional setting, including graphical lasso. However, current methods do not address the compositional count nature of microbiome data, where abundances of microbial taxa are not directly measured, but are reflected by the observed counts in an error-prone manner. Adding to the challenge is that the sum of the counts within each sample, termed “sequencing depth,” is an experimental technicality that carries no biological information but can vary drastically across samples. To address these issues, we develop a new approach to network estimation, called BC-GLASSO (bias-corrected graphical lasso), which models the microbiome data using a logistic normal multinomial distribution with the sequencing depths explicitly incorporated, corrects the bias of the naive empirical covariance estimator arising from the heterogeneity in sequencing depths, and builds the inverse covariance estimator via graphical lasso. We demonstrate the advantage of BC-GLASSO over current approaches to microbial interaction network estimation under a variety of simulation scenarios. We also illustrate the efficacy of our method in an application to a human microbiome data set.

Evaluating the significance of contact maps in low-homology protein modeling using contact-assisted threading

The development of improved threading algorithms for remote homology modeling is a critical step forward in template-based protein structure prediction. We have recently demonstrated the utility of contact information to boost protein threading by developing a new contact-assisted threading method. However, the nature and extent to which the quality of a predicted contact map impacts the performance of contact-assisted threading remains elusive. Here, we systematically analyze and explore this interdependence by employing our newly-developed contact-assisted threading method over a large-scale benchmark dataset using predicted contact maps from four complementary methods including direct coupling analysis (mfDCA), sparse inverse covariance estimation (PSICOV), classical neural network-based meta approach (MetaPSICOV), and state-of-the-art ultra-deep learning model (RaptorX). Experimental results demonstrate that contact-assisted threading using high-quality contacts having the Matthews Correlation Coefficient (MCC) ≥ 0.5 improves threading performance in nearly 30% cases, while low-quality contacts with MCC <0.35 degrades the performance for 50% cases. This holds true even in CASP13 dataset, where threading using high-quality contacts (MCC ≥ 0.5) significantly improves the performance of 22 instances out of 29. Collectively, our study uncovers the mutual association between the quality of predicted contacts and its possible utility in boosting threading performance for improving low-homology protein modeling.

A dual spectral projected gradient method for log-determinant semidefinite problems

We extend the result on the spectral projected gradient method by Birgin et al. in 2000 to a log-determinant semidefinite problem with linear constraints and propose a spectral projected gradient method for the dual problem. Our method is based on alternate projections on the intersection of two convex sets, which first projects onto the box constraints and then onto a set defined by a linear matrix inequality. By exploiting structures of the two projections, we show that the same convergence properties can be obtained for the proposed method as Birgin’s method where the exact orthogonal projection onto the intersection of two convex sets is performed. Using the convergence properties, we prove that the proposed algorithm attains the optimal value or terminates in a finite number of iterations. The efficiency of the proposed method is illustrated with the numerical results on randomly generated synthetic/deterministic data and gene expression data, in comparison with other methods including the inexact primal–dual path-following interior-point method, the Newton-CG primal proximal-point algorithm, the adaptive spectral projected gradient method, and the adaptive Nesterov’s smooth method. For the gene expression data, our results are compared with the quadratic approximation for sparse inverse covariance estimation method. We show that our method outperforms the other methods in obtaining a better objective value fast.

Sparse inverse covariance estimation for high-throughput microRNA sequencing data in the Poisson log-normal graphical model

ABSTRACTWe introduce a one-step EM algorithm to estimate the graphical structure in a Poisson-Log-Normal graphical model. This procedure is equivalent to a normality transformation that makes the problem of identifying relationships in high-throughput microRNA (miRNA) sequence data feasible. The Poisson-log-normal model moreover allows us to directly account for known overdispersion relationships present in this data set. We show that our EM algorithm provides a provable increase in performance in determining the network structure. The model is shown to provide an increase in performance in simulation settings over a range of network structures. The model is applied to high-throughput miRNA sequencing data from patients with breast cancer from The Cancer Genome Atlas (TCGA). By selecting the most highly connected miRNA molecules in the fitted network we find that nearly all of them are known to be involved in the regulation of breast cancer.

Variant-specific vulnerability in metabolic connectivity and resting-state networks in behavioural variant of frontotemporal dementia

Brain connectivity measures represent candidate biomarkers of neuronal dysfunction in neurodegenerative diseases. Previous findings suggest that the behavioural variant of frontotemporal dementia (bvFTD) and its variants (i.e., frontal and temporo-limbic) may be related to the vulnerability of distinct functional connectivity networks. In this study, 82 bvFTD patients were included, and two patient groups were identified as frontal and temporo-limbic bvFTD variants. Two advanced multivariate analytical approaches were applied to FDG-PET data, i.e., sparse inverse covariance estimation (SICE) method and seed-based interregional correlation analysis (IRCA). These advanced methods allowed the assessment of (i) the whole-brain metabolic connectivity, without any a priori assumption, and (ii) the main brain resting-state networks of crucial relevance for cognitive and behavioural functions.In the whole bvFTD group, we found dysfunctional connectivity patterns in frontal and limbic regions and in all major brain resting-state networks as compared to healthy controls (HC N = 82). In the two bvFTD variants, SICE and IRCA analyses identified variant-specific reconfigurations of whole-brain connectivity and resting-state networks. Specifically, the frontal bvFTD variant was characterised by metabolic connectivity alterations in orbitofrontal regions and anterior resting-state networks, while the temporo-limbic bvFTD variant was characterised by connectivity alterations in the limbic and salience networks. These results highlight different neural vulnerabilities in the two bvFTD variants, as shown by the dysfunctional connectivity patterns, with relevance for the different neuropsychological profiles. This new evidence provides further insight in the variability of bvFTD and may contribute to a more accurate classification of these patients.

Sparse Inverse Covariance Estimation: A Data Mining Technique to Unravel Holistic Patterns among Business Practices in Firms

ABSTRACTFirms are seeking ways to improve managerial decision making in order to enhance operational performance. However, the complexities underlying business processes often mean that operational performance depends on a multitude of factors. Yet, at times the number of empirical cases is rather limited. This presents the challenge of discerning meaningful patterns among a large number of variables that can then be used to derive generalized frameworks and mental models for decision making. In this article, we tackle this challenge with an extension of Sparse Inverse Covariance Estimation (SICE), a novel data mining technique, to address decisions in Operations and Supply Chain Management. We conduct a simulation study to validate the effectiveness of this extension in improving the accuracy and stability of pattern detection. We then apply it to an empirical dataset that is characterized by high dimension, low sample size, and lack of multivariate normal distribution. Our study pioneers the application of SICE in Operations and Supply Chain research. We also extend SICE with bootstrapping. The extended SICE is an effective technique for mining a complex empirical dataset and is a valuable aid for decision support.

The Generalized Ridge Estimator of the Inverse Covariance Matrix

The ridge inverse covariance estimator is generalized to allow for entry-wise penalization. An efficient algorithm for its evaluation is proposed. Its computational accuracy is benchmarked against implementations of specific cases the generalized ridge inverse covariance estimator encompasses. The proposed estimator shrinks toward a user-specified, nonrandom target matrix and is shown to be positive definite and consistent. It is pointed out how the generalized ridge inverse covariance estimator can be used to obtain a generalization of the graphical lasso estimator as well as of its elastic net counterpart. The usage of the presented estimator is illustrated in graphical modeling of omics data. Supplementary materials for this article are available online.

Functional segregation of executive control network and frontoparietal network in Alzheimer's disease

Functional connectivity pattern altered of default mode network (DMN) is gaining more attention as a potential noninvasive biomarker to diagnose incipient Alzheimer's disease. However, the changed functional connectivity except for DMN, the longitudinal changes in executive control network (ECN) and frontoparietal network (FPN) also has attracted wide interest. Moreover, AD-related functional connectivity abnormalities within the DMN are well replicated research, but the (increased/decreased and reduced) functional connectivity in ECN and FPN weren't receive adequate attention. To address the above issues, in this paper, we adopt sparse inverse covariance estimation (SICE) approach to investigate the changed functional connectivity of ECN and FPN on the ADNI2 dataset. Our experimental results indicate the left superior frontal gyrus (SFGmed.L) and left thalamus (THA.L) regions of ECN has shown increased functional connectivity, the left anterior cingulate (ACG.L) region of ECN has shown decreased functional connectivity. The Superior Parietal Gyrus (SPG) regions and left paracentral lobule (PCL.L) of FPN has shown increased functional connectivity, the left supramarginal gyrus (SMG.L) regions has shown decreased functional connectivity in AD patients. On the other hand, the ACG.L regions in ECN, SMG.L and left inferior parietal (IPL.L) in FPN have shown significantly reduced functional connectivity. These results demonstrate that increased/decreased functional connectivity and reduced functional connectivity not only within DMN, but also associated with ECN and FPN. It also suggest that AD is associated with the characteristics of large-scale functional networks, and these changed functional connectivity possibly as a potential noninvasive biomarker to diagnose incipient Alzheimer's disease.

A permutation-based Bayesian approach for inverse covariance estimation

Covariance estimation and selection for multivariate datasets in a high-dimensional regime is a fundamental problem in modern statistics. Gaussian graphical models are a popular class of models used for this purpose. Current Bayesian methods for inverse covariance matrix estimation under Gaussian graphical models require the underlying graph and hence the ordering of variables to be known. However, in practice, such information on the true underlying model is often unavailable. We therefore propose a novel permutation-based Bayesian approach to tackle the unknown variable ordering issue. In particular, we utilize multiple maximum a posteriori estimates under the DAG-Wishart prior for each permutation, and subsequently construct the final estimate of the inverse covariance matrix. The proposed estimator has smaller variability and yields order-invariant property. We establish posterior convergence rates under mild assumptions and illustrate that our method outperforms existing approaches in estimating the inverse covariance matrices via simulation studies.

Sparse Multi-task Inverse Covariance Estimation for Connectivity Analysis in EEG Source Space.

Understanding how different brain areas interact to generate complex behavior is a primary goal of neuroscience research. One approach, functional connectivity analysis, aims to characterize the connectivity patterns within brain networks. In this paper, we address the problem of discriminative connectivity, i.e. determining the differences in network structure under different experimental conditions. We introduce a novel model called Sparse Multi-task Inverse Covariance Estimation (SMICE) which is capable of estimating a common connectivity network as well as discriminative networks across different tasks. We apply the method to EEG signals after solving the inverse problem of source localization, yielding networks defined on the cortical surface. We propose an efficient algorithm based on the Alternating Direction Method of Multipliers (ADMM) to solve SMICE. We apply our newly developed framework to find common and discriminative connectivity patterns for α-oscillations during the Sleep Onset Process (SOP) and during Rapid Eye Movement (REM) sleep. Even though both stages exhibit a similar α-oscillations, we show that the underlying networks are distinct.

Investigating executive control network and default mode network dysfunction in major depressive disorder

Major depressive disorder (MDD) is a mental disorder that causes a persistent feeling of sadness and loss of interest. It affects how you feel, think and behave and can lead to a variety of emotional and physical problems. Previous investigated the MDD has primarily focused on interpreting the abnormal functional connectivity of different brain regions, rather than entire brain networks. Although recent studies have found depression-related abnormalities functional connectivity of the brain networks, it is not clear the certain brain regions or the various functional networks as a whole are involved in the emotional dysregulation of depression. Moreover, abnormal functional connectivity regions are well replicated research, however, default mode network (DMN) and executive control network (ECN) haven’t received adequately examined in MDD. To address the above issues, in this paper, we adopt sparse inverse covariance estimation (SICE) approach to investigate the functional connectivity both DMN and ECN. Our experimental results show functional connectivity of ECN have significantly changed in MDD, where the anterior cingulate cortex (ACG.R) region and the thalamus region (THA.R) have shown increased functional connectivity compared with normal control, the superior frontal gyrus (SFGmed.R) region has shown decreased functional connectivity. Moreover, the precuneus lobule (PCUN.L) region and posterior cingulate cortex (PCG.L) of DMN have shown increased functional connectivity. These results may suggest that MDD is associated with large-scale functional networks, rather than the portions of brain regions. These changed functional connectivity may provide more neuroimaging biomarker for MDD diagnosis.

Compositional data network analysis via lasso penalized D-trace loss.

With the development of high-throughput sequencing techniques for 16S-rRNA gene profiling, the analysis of microbial communities is becoming more and more attractive and reliable. Inferring the direct interaction network among microbial communities helps in the identification of mechanisms underlying community structure. However, the analysis of compositional data remains challenging by the relative information conveyed by such data, as well as its high dimensionality. In this article, we first propose a novel loss function for compositional data called CD-trace based on D-trace loss. A sparse matrix estimator for the direct interaction network is defined as the minimizer of lasso penalized CD-trace loss under positive-definite constraint. An efficient alternating direction algorithm is developed for numerical computation. Simulation results show that CD-trace compares favorably to gCoda and that it is better than sparse inverse covariance estimation for ecological association inference (SPIEC-EASI) (hereinafter S-E) in network recovery with compositional data. Finally, we test CD-trace and compare it to the other methods noted above using mouse skin microbiome data. The CD-trace is open source and freely available from https://github.com/coamo2/CD-trace under GNU LGPL v3. Supplementary data are available at Bioinformatics online.