DREAM3 Challenge Research Articles

EIEPCF: accurate inference of functional gene regulatory networks by eliminating indirect effects from confounding factors.

Reconstructing functional gene regulatory networks (GRNs) is a primary prerequisite for understanding pathogenic mechanisms and curing diseases in animals, and it also provides an important foundation for cultivating vegetable and fruit varieties that are resistant to diseases and corrosion in plants. Many computational methods have been developed to infer GRNs, but most of the regulatory relationships between genes obtained by these methods are biased. Eliminating indirect effects in GRNs remains a significant challenge for researchers. In this work, we propose a novel approach for inferring functional GRNs, named EIEPCF (eliminating indirect effects produced by confounding factors), which eliminates indirect effects caused by confounding factors. This method eliminates the influence of confounding factors on regulatory factors and target genes by measuring the similarity between their residuals. The validation results of the EIEPCF method on simulation studies, the gold-standard networks provided by the DREAM3 Challenge and the real gene networks of Escherichia coli demonstrate that it achieves significantly higher accuracy compared to other popular computational methods for inferring GRNs. As a case study, we utilized the EIEPCF method to reconstruct the cold-resistant specific GRN from gene expression data of cold-resistant in Arabidopsis thaliana. The source code and data are available at https://github.com/zhanglab-wbgcas/EIEPCF.

Neural Granger Causality.

While most classical approaches to Granger causality detection assume linear dynamics, many interactions in real-world applications, like neuroscience and genomics, are inherently nonlinear. In these cases, using linear models may lead to inconsistent estimation of Granger causal interactions. We propose a class of nonlinear methods by applying structured multilayer perceptrons (MLPs) or recurrent neural networks (RNNs) combined with sparsity-inducing penalties on the weights. By encouraging specific sets of weights to be zero-in particular, through the use of convex group-lasso penalties-we can extract the Granger causal structure. To further contrast with traditional approaches, our framework naturally enables us to efficiently capture long-range dependencies between series either via our RNNs or through an automatic lag selection in the MLP. We show that our neural Granger causality methods outperform state-of-the-art nonlinear Granger causality methods on the DREAM3 challenge data. This data consists of nonlinear gene expression and regulation time courses with only a limited number of time points. The successes we show in this challenging dataset provide a powerful example of how deep learning can be useful in cases that go beyond prediction on large datasets. We likewise illustrate our methods in detecting nonlinear interactions in a human motion capture dataset.

Abstract 5108: Somatic small variant and copy number alteration calling with the Genome Analysis Toolkit

Abstract Somatic small mutations, SNVs or Indels, and copy number alterations are the two categories of mutations with the largest impact on cancer tumors. The Broad Institute has released somatic variant calling workflows for small mutations (M2) and copy number alterations (ModelSegments) based on the Genome Analysis Toolkit (GATK). The suite of workflows can call variants in capture or whole-genome sequencing data and will include functional annotations (Funcotator), such as protein change (for small variants) and impacted gene (for all variants). Common artifacts in sequencing data, such as those arising from oxidative DNA damage, FFPE/deamination, or mapping errors, are corrected automatically. Evaluation of the workflows is standardized and repeatable, which allows tracking of performance across versions, both detection performance (e.g. sensitivity, precision), as well as runtime performance (e.g. CPU and RAM usage). A matched normal is not required for a given tumor sample, since the workflows can leverage pre-processed panels of normals (PoNs). The workflows are freely available, are portable (i.e. can be run on local, on-prem, or cloud compute), are optimized for cost reduction, and can be tuned to optimally leverage available compute.The measured sensitivity of M2 was at least 0.93 for small somatic nucleotide variants (SNVs) and 0.83 for small insertions/deletions (Indels) on DREAM1, DREAM2, and DREAM3 challenges, and on a titrated mixture of germline samples (>=100x depth, AF = 0.2). The measured precision of M2 ranged from 0.91 to 0.98 on DREAM1, DREAM2, and DREAM3 for both SNVs and Indels. The false positive rate (FPR) of M2 was between 0.03 and 0.21 FP/Mb for SNVs, and between 0.0 and 0.1 FP/Mb for indels, on twelve paired, replicate normal-normal samples. The cost of the M2 workflow is about USD$1.15 for a pair of 35x WGS matched tumor-normal samples, using Google Cloud Compute, and required about 32 hours of CPU time on a single core with 3GB RAM. The measured sensitivity of ModelSegments was at least 0.91 for deletions and amplifications across three cohorts of TCGA whole-exome samples (Stomach adenocarcinoma N=39, Thyroid carcinoma N=50, and Lung adenocarcinoma N=60). The measured specificity for the same set of cohorts was at least 0.96 for both deletions and amplifications. All results reported here were using the corresponding SNP Array results as a truth set. GATK MS cost was approximately USD$0.65 on a 30x WGS pair using Google Cloud Compute and required about 6 hours of CPU time with a single core. The RAM usage was varied automatically in the workflow to minimize cost, but was in the range of 2-13GB. Citation Format: Lee Lichtenstein, Jonn Smith, David Benjamin, Aaron Chevalier, Kristian Cibulskis, Samuel K. Lee, Eric Banks. Somatic small variant and copy number alteration calling with the Genome Analysis Toolkit [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2019; 2019 Mar 29-Apr 3; Atlanta, GA. Philadelphia (PA): AACR; Cancer Res 2019;79(13 Suppl):Abstract nr 5108.

Inference of gene regulatory networks with multi-objective cellular genetic algorithm

Reverse engineering of biochemical networks remains an important open challenge in computational systems biology. The goal of model inference is to, based on time-series gene expression data, obtain the sparse topological structure and parameters that quantitatively understand and reproduce the dynamics of biological systems. In this paper, we propose a multi-objective approach for the inference of S-System structures for Gene Regulatory Networks (GRNs) based on Pareto dominance and Pareto optimality theoretical concepts instead of the conventional single-objective evaluation of Mean Squared Error (MSE). Our motivation is that, using a multi-objective formulation for the GRN, it is possible to optimize the sparse topology of a given GRN as well as the kinetic order and rate constant parameters in a decoupled S-System, yet avoiding the use of additional penalty weights. A flexible and robust Multi-Objective Cellular Evolutionary Algorithm is adapted to perform the tasks of parameter learning and network topology inference for the proposed approach. The resulting software, called MONET, is evaluated on real-based academic and synthetic time-series of gene expression taken from the DREAM3 challenge and the IRMA in vivo datasets. The ability to reproduce biological behavior and robustness to noise is assessed and compared. The results obtained are competitive and indicate that the proposed approach offers advantages over previously used methods. In addition, MONET is able to provide experts with a set of trade-off solutions involving GRNs with different typologies and MSEs.

Reconstruction, Topological and Gene Ontology Enrichment Analysis of Cancerous Gene Regulatory Network Modules

The availability of large set of high throughput biological data needs algorithm that automatically reconstructs gene regulatory networks from these datasets. Cancerous regulatory network modules when analyzed critically may reveal the underlying mechanism of cancer, which may help in better diagnosis. Identification of cancerous genes and their regulation is an important research area in cancer systems biology. In this paper, we introduced an algorithm to infer cancerous gene regulatory network modules from gene expression profiles. The proposed algorithm has been applied to gene expression dataset of colon cancer patients and several network modules have been identified. We performed topological analysis of inferred network modules in terms of network density, degree distribution, clustering coefficient, average path length, network heterogeneity, and centrality measures. Further, GO-based enrichment analysis of the inferred network has been performed. To validate the proposed algorithm, it has been tested on benchmark dataset taken from DREAM3 challenge project. Keywords: Gene regulatory network, biological network, systems biology, GO enrichment analysis, topological analysis, cancerous network modules.

GeNOSA: inferring and experimentally supporting quantitative gene regulatory networks in prokaryotes.

The establishment of quantitative gene regulatory networks (qGRNs) through existing network component analysis (NCA) approaches suffers from shortcomings such as usage limitations of problem constraints and the instability of inferred qGRNs. The proposed GeNOSA framework uses a global optimization algorithm (OptNCA) to cope with the stringent limitations of NCA approaches in large-scale qGRNs. OptNCA performs well against existing NCA-derived algorithms in terms of utilization of connectivity information and reconstruction accuracy of inferred GRNs using synthetic and real Escherichia coli datasets. For comparisons with other non-NCA-derived algorithms, OptNCA without using known qualitative regulations is also evaluated in terms of qualitative assessments using a synthetic Saccharomyces cerevisiae dataset of the DREAM3 challenges. We successfully demonstrate GeNOSA in several applications including deducing condition-dependent regulations, establishing high-consensus qGRNs and validating a sub-network experimentally for dose-response and time-course microarray data, and discovering and experimentally confirming a novel regulation of CRP on AscG. All datasets and the GeNOSA framework are freely available from http://e045.life.nctu.edu.tw/GeNOSA. syho@mail.nctu.edu.tw Supplementary data are available at Bioinformatics online.

IPCA-CMI: an algorithm for inferring gene regulatory networks based on a combination of PCA-CMI and MIT score.

Inferring gene regulatory networks (GRNs) is a major issue in systems biology, which explicitly characterizes regulatory processes in the cell. The Path Consistency Algorithm based on Conditional Mutual Information (PCA-CMI) is a well-known method in this field. In this study, we introduce a new algorithm (IPCA-CMI) and apply it to a number of gene expression data sets in order to evaluate the accuracy of the algorithm to infer GRNs. The IPCA-CMI can be categorized as a hybrid method, using the PCA-CMI and Hill-Climbing algorithm (based on MIT score). The conditional dependence between variables is determined by the conditional mutual information test which can take into account both linear and nonlinear genes relations. IPCA-CMI uses a score and search method and defines a selected set of variables which is adjacent to one of or Y. This set is used to determine the dependency between X and Y. This method is compared with the method of evaluating dependency by PCA-CMI in which the set of variables adjacent to both X and Y, is selected. The merits of the IPCA-CMI are evaluated by applying this algorithm to the DREAM3 Challenge data sets with n variables and n samples () and to experimental data from Escherichia coil containing 9 variables and 9 samples. Results indicate that applying the IPCA-CMI improves the precision of learning the structure of the GRNs in comparison with that of the PCA-CMI.

OKVAR-Boost: a novel boosting algorithm to infer nonlinear dynamics and interactions in gene regulatory networks

Motivation: Reverse engineering of gene regulatory networks remains a central challenge in computational systems biology, despite recent advances facilitated by benchmark in silico challenges that have aided in calibrating their performance. A number of approaches using either perturbation (knock-out) or wild-type time-series data have appeared in the literature addressing this problem, with the latter using linear temporal models. Nonlinear dynamical models are particularly appropriate for this inference task, given the generation mechanism of the time-series data. In this study, we introduce a novel nonlinear autoregressive model based on operator-valued kernels that simultaneously learns the model parameters, as well as the network structure.Results: A flexible boosting algorithm (OKVAR-Boost) that shares features from L2-boosting and randomization-based algorithms is developed to perform the tasks of parameter learning and network inference for the proposed model. Specifically, at each boosting iteration, a regularized Operator-valued Kernel-based Vector AutoRegressive model (OKVAR) is trained on a random subnetwork. The final model consists of an ensemble of such models. The empirical estimation of the ensemble model’s Jacobian matrix provides an estimation of the network structure. The performance of the proposed algorithm is first evaluated on a number of benchmark datasets from the DREAM3 challenge and then on real datasets related to the In vivo Reverse-Engineering and Modeling Assessment (IRMA) and T-cell networks. The high-quality results obtained strongly indicate that it outperforms existing approaches.Availability: The OKVAR-Boost Matlab code is available as the archive: http://amis-group.fr/sourcecode-okvar-boost/OKVARBoost-v1.0.zip.Contact: florence.dalche@ibisc.univ-evry.frSupplementary information: Supplementary data are available at Bioinformatics online.

The self‐assessment trap: can we all be better than average?

Mol Syst Biol. 7: 537 Computational systems biology seems to be caught in what we call the ‘self‐assessment trap’, in which researchers wishing to publish their analytical methods are required by referees or by editorial policy (e.g., Bioinformatics, BMC Bioinformatics, Nucleic Acids Research) to compare the performance of their own algorithms against other methodologies, thus being forced to be judge, jury and executioner. The result is that the authors’ method tends to be the best in an unreasonable majority of cases (Table I). In many instances, this bias is the result of selective reporting of performance in the niche in which the method is superior. Evidence of that is that most papers reporting best performance choose only one or two metrics of performance, but when the number of performance metrics is larger than two, most methods fail to be the best in all categories assessed (Table I). Choosing many metrics can dramatically change the determination of best performance (Supplementary Table S1). Selective reporting can be inadvertent, but in some cases biases are more disingenuous, involving hiding information or quietly cutting corners in the performance evaluation (similar problems have been discussed in assessments of the performance of supercomputers, e.g., Bailey (1991)). View this table: Table 1. Break out of 57 surveyed papers in which the authors assess their own methods Even assuming that there is no selective reporting, we would like to argue that papers reporting good‐yet‐not‐the‐best methods (of which we found none in our literature survey of self‐assessed papers …

Estimating Sparse Gene Regulatory Networks Using a Bayesian Linear Regression

In this paper, we propose a gene regulatory network (GRN) estimation method, which assumes that such networks are typically sparse, using time-series microarray datasets. We represent the regulatory relationships between the genes using weights, with the "net" regulation influence on a gene's expression being the summation of the independent regulatory inputs. We estimate the weights using a Bayesian linear regression method for sparse parameter vectors. We apply our proposed method to the extraction of differential gene expression software selected genes of a human buffy-coat microarray expression profile dataset of ventilator-associated pneumonia (VAP), and compare the estimation result with the GRNs estimated using both a correlation coefficient method and a database-based method ingenuity pathway analysis. A biological analysis of the resulting consensus network that is derived using the GRNs, estimated with both our and the correlation-coefficient methods results in four biologically meaningful subnetworks. Also, our method performs either better than or competitively with the existing well-established GRN estimation methods. Moreover, it performs comparatively with respect to: 1) the ground-truth GRNs for the in silico 50- and 100-gene datasets reported recently in the DREAM3 challenge and 2) the GRN estimated using a mutual information-based method for the top-ranked Bayesian analysis of time series (a Bayesian user-friendly software for analyzing time-series microarray experiments) selected genes of the VAP dataset.

Correction: Towards a Rigorous Assessment of Systems Biology Models: The DREAM3 Challenges

In the Funding section a digit was missing from one of the grant numbers. NIH grant U54-CA11296 should read U54-{type:entrez-nucleotide,attrs:{text:CA112967,term_id:34966274,term_text:CA112967}}CA112967.

Towards a Rigorous Assessment of Systems Biology Models: The DREAM3 Challenges

BackgroundSystems biology has embraced computational modeling in response to the quantitative nature and increasing scale of contemporary data sets. The onslaught of data is accelerating as molecular profiling technology evolves. The Dialogue for Reverse Engineering Assessments and Methods (DREAM) is a community effort to catalyze discussion about the design, application, and assessment of systems biology models through annual reverse-engineering challenges.Methodology and Principal FindingsWe describe our assessments of the four challenges associated with the third DREAM conference which came to be known as the DREAM3 challenges: signaling cascade identification, signaling response prediction, gene expression prediction, and the DREAM3 in silico network challenge. The challenges, based on anonymized data sets, tested participants in network inference and prediction of measurements. Forty teams submitted 413 predicted networks and measurement test sets. Overall, a handful of best-performer teams were identified, while a majority of teams made predictions that were equivalent to random. Counterintuitively, combining the predictions of multiple teams (including the weaker teams) can in some cases improve predictive power beyond that of any single method.ConclusionsDREAM provides valuable feedback to practitioners of systems biology modeling. Lessons learned from the predictions of the community provide much-needed context for interpreting claims of efficacy of algorithms described in the scientific literature.

Improved Reconstruction of In Silico Gene Regulatory Networks by Integrating Knockout and Perturbation Data

We performed computational reconstruction of the in silico gene regulatory networks in the DREAM3 Challenges. Our task was to learn the networks from two types of data, namely gene expression profiles in deletion strains (the ‘deletion data’) and time series trajectories of gene expression after some initial perturbation (the ‘perturbation data’). In the course of developing the prediction method, we observed that the two types of data contained different and complementary information about the underlying network. In particular, deletion data allow for the detection of direct regulatory activities with strong responses upon the deletion of the regulator while perturbation data provide richer information for the identification of weaker and more complex types of regulation. We applied different techniques to learn the regulation from the two types of data. For deletion data, we learned a noise model to distinguish real signals from random fluctuations using an iterative method. For perturbation data, we used differential equations to model the change of expression levels of a gene along the trajectories due to the regulation of other genes. We tried different models, and combined their predictions. The final predictions were obtained by merging the results from the two types of data. A comparison with the actual regulatory networks suggests that our approach is effective for networks with a range of different sizes. The success of the approach demonstrates the importance of integrating heterogeneous data in network reconstruction.