Dialogue For Reverse Engineering Assessments And Methods Research Articles

From complex data to biological insight: \u2018DEKER\u2019 feature selection and network inference

Network inference is a valuable approach for gaining mechanistic insight from high-dimensional biological data. Existing methods for network inference focus on ranking all possible relations (edges) among all measured quantities such as genes, proteins, metabolites (features) observed, which yields a dense network that is challenging to interpret. Identifying a sparse, interpretable network using these methods thus requires an error-prone thresholding step which compromises their performance. In this article we propose a new method, DEKER-NET, that addresses this limitation by directly identifying a sparse, interpretable network without thresholding, improving real-world performance. DEKER-NET uses a novel machine learning method for feature selection in an iterative framework for network inference. DEKER-NET is extremely flexible, handling linear and nonlinear relations while making no assumptions about the underlying distribution of data, and is suitable for categorical or continuous variables. We test our method on the Dialogue for Reverse Engineering Assessments and Methods (DREAM) challenge data, demonstrating that it can directly identify sparse, interpretable networks without thresholding while maintaining performance comparable to the hypothetical best-case thresholded network of other methods.

Crowdsourcing a High-Risk Classifier for Multiple Myeloma Patients

Crowdsourcing a High-Risk Classifier for Multiple Myeloma Patients

Assessment of a Clinical Trial–Derived Survival Model in Patients With Metastatic Castration-Resistant Prostate Cancer

Randomized clinical trials (RCTs) are considered the criterion standard for clinical evidence. Despite their many benefits, RCTs have limitations, such as costliness, that may reduce the generalizability of their findings among diverse populations and routine care settings. To assess the performance of an RCT-derived prognostic model that predicts survival among patients with metastatic castration-resistant prostate cancer (CRPC) when the model is applied to real-world data from electronic health records (EHRs). The RCT-trained model and patient data from the RCTs were obtained from the Dialogue for Reverse Engineering Assessments and Methods (DREAM) challenge for prostate cancer, which occurred from March 16 to July 27, 2015. This challenge included 4 phase 3 clinical trials of patients with metastatic CRPC. Real-world data were obtained from the EHRs of a tertiary care academic medical center that includes a comprehensive cancer center. In this study, the DREAM challenge RCT-trained model was applied to real-world data from January 1, 2008, to December 31, 2019; the model was then retrained using EHR data with optimized feature selection. Patients with metastatic CRPC were divided into RCT and EHR cohorts based on data source. Data were analyzed from March 23, 2018, to October 22, 2020. Patients who received treatment for metastatic CRPC. The primary outcome was the performance of an RCT-derived prognostic model that predicts survival among patients with metastatic CRPC when the model is applied to real-world data. Model performance was compared using 10-fold cross-validation according to time-dependent integrated area under the curve (iAUC) statistics. Among 2113 participants with metastatic CRPC, 1600 participants were included in the RCT cohort, and 513 participants were included in the EHR cohort. The RCT cohort comprised a larger proportion of White participants (1390 patients [86.9%] vs 337 patients [65.7%]) and a smaller proportion of Hispanic participants (14 patients [0.9%] vs 42 patients [8.2%]), Asian participants (41 patients [2.6%] vs 88 patients [17.2%]), and participants older than 75 years (388 patients [24.3%] vs 191 patients [37.2%]) compared with the EHR cohort. Participants in the RCT cohort also had fewer comorbidities (mean [SD], 1.6 [1.8] comorbidities vs 2.5 [2.6] comorbidities, respectively) compared with those in the EHR cohort. Of the 101 variables used in the RCT-derived model, 10 were not available in the EHR data set, 3 of which were among the top 10 features in the DREAM challenge RCT model. The best-performing EHR-trained model included only 25 of the 101 variables included in the RCT-trained model. The performance of the RCT-trained and EHR-trained models was adequate in the EHR cohort (mean [SD] iAUC, 0.722 [0.118] and 0.762 [0.106], respectively); model optimization was associated with improved performance of the best-performing EHR model (mean [SD] iAUC, 0.792 [0.097]). The EHR-trained model classified 256 patients as having a high risk of mortality and 256 patients as having a low risk of mortality (hazard ratio, 2.7; 95% CI, 2.0-3.7; log-rank P < .001). In this study, although the RCT-trained models did not perform well when applied to real-world EHR data, retraining the models using real-world EHR data and optimizing variable selection was beneficial for model performance. As clinical evidence evolves to include more real-world data, both industry and academia will likely search for ways to balance model optimization with generalizability. This study provides a pragmatic approach to applying RCT-trained models to real-world data.

Feature Selection for Topological Proximity Prediction of Single-Cell Transcriptomic Profiles in Drosophila Embryo Using Genetic Algorithm

Single-cell transcriptomics data, when combined with in situ hybridization patterns of specific genes, can help in recovering the spatial information lost during cell isolation. Dialogue for Reverse Engineering Assessments and Methods (DREAM) consortium conducted a crowd-sourced competition known as DREAM Single Cell Transcriptomics Challenge (SCTC) to predict the masked locations of single cells from a set of 60, 40 and 20 genes out of 84 in situ gene patterns known in Drosophila embryo. We applied a genetic algorithm (GA) to predict the most important genes that carry positional and proximity information of the single-cell origins, in combination with the base distance mapping algorithm DistMap. Resulting gene selection was found to perform well and was ranked among top 10 in two of the three sub-challenges. However, the details of the method did not make it to the main challenge publication, due to an intricate aggregation ranking. In this work, we discuss the detailed implementation of GA and its post-challenge parameterization, with a view to identify potential areas where GA-based approaches of gene-set selection for topological association prediction may be improved, to be more effective. We believe this work provides additional insights into the feature-selection strategies and their relevance to single-cell similarity prediction and will form a strong addendum to the recently published work from the consortium.

Prediction of cell position using single-cell transcriptomic data: an iterative procedure.

Single-cell sequencing reveals cellular heterogeneity but not cell localization. However, by combining single-cell transcriptomic data with a reference atlas of a small set of genes, it would be possible to predict the position of individual cells and reconstruct the spatial expression profile of thousands of genes reported in the single-cell study. With the purpose of developing new algorithms, the Dialogue for Reverse Engineering Assessments and Methods (DREAM) consortium organized a crowd-sourced competition known as DREAM Single Cell Transcriptomics Challenge (SCTC). Within this context, we describe here our proposed procedures for adequate reference genes selection, and an iterative procedure to predict spatial expression profile of other genes.

Prediction of cell position using single-cell transcriptomic data: an iterative procedure

Single-cell sequencing reveals cellular heterogeneity but not cell localization. However, by combining single-cell transcriptomic data with a reference atlas of a small set of genes, it would be possible to predict the position of individual cells and reconstruct the spatial expression profile of thousands of genes reported in the single-cell study. To develop new algorithms for this purpose, the Dialogue for Reverse Engineering Assessments and Methods (DREAM) consortium organized a crowd-sourced competition known as DREAM Single Cell Transcriptomics Challenge (SCTC). In the spirit of this framework, we describe here the proposed procedures for adequate reference genes selection, and an iterative procedure to predict spatial expression profile of other genes.

A Stable, Unified Density Controlled Memetic Algorithm for Gene Regulatory Network Reconstruction Based on Sparse Fuzzy Cognitive Maps

Gene regulatory networks (GRNs) denote the interrelation among genes in the genomic level. GRNs have a sparse network structures, and as a simulation of GRNs, the density of The Dialogue for Reverse Engineering Assessments and Methods (DREAM) challenge is less than 5%. So using sparse models to represent GRNs is a meaningful task. Fuzzy cognitive maps (FCMs) have been used to reconstruct GRNs. However, the networks learned by automated derivate-free methods is much denser than those in practical applications. Moreover, the performance of current sparse FCM learning algorithms is worse than what we expect. Therefore, proposing a fast, simple and sparse FCM learning algorithm is a realistic demand. Here, we propose a new unified algorithm: Density Controlled Memetic Algorithm (DC-MA) for learning sparse FCMs. As a simple and good-performance algorithm, memetic algorithm (MA) is chosen as the framework of DC-MA. In DC-MA, a new crossover operator and a mutation operator are designed to optimize the target, control the density and ensure the stability; the local search is used to improve the accuracy and a special self-learning operator is proposed to adjust density. To test the effectiveness of our algorithm, DC-MA is performed on synthetic data with varying sizes and densities. The results show that DC-MA obtains good performance in learning sparse FCMs from time series. On the benchmark datasets DREAM3, DREAM4 and large-scale GRN reconstruction DREAM5 dataset, DC-MA shows high accuracy. The good performance in learning sparse FCMs shows the effectiveness of DC-MA, and the simplicity and scalability of the framework ensure that DC-MA can be adapted to a wide range of needs.

A predictive model of overall survival in patients with metastatic castration-resistant prostate cancer.

Metastatic castration resistant prostate cancer (mCRPC) is one of the most common cancers with a poor prognosis. To improve prognostic models of mCRPC, the Dialogue for Reverse Engineering Assessments and Methods (DREAM) Consortium organized a crowdsourced competition known as the Prostate Cancer DREAM Challenge. In the competition, data from four phase III clinical trials were utilized. A total of 1600 patients' clinical information across three of the trials was used to generate prognostic models, whereas one of the datasets (313 patients) was held out for blinded validation. The previously introduced prognostic model of overall survival of chemotherapy-naive mCRPC patients treated with docetaxel or prednisone (so called Halabi model) was used as a performance baseline. This paper presents the model developed by the team TYTDreamChallenge and its improved version to predict the prognosis of mCRPC patients within the first 30 months after starting the treatment based on available clinical features of each patient. In particular, by replacing our original larger set of eleven features with a smaller more carefully selected set of only five features the prediction performance on the independent validation cohort increased up to 5.4 percent. While the original TYTDreamChallenge model (iAUC=0.748) performed similarly as the performance-baseline model developed by Halabi et al. (iAUC=0.743), the improved post-challenge model (iAUC=0.779) showed markedly improved performance by using only PSA, ALP, AST, HB, and LESIONS as features. This highlights the importance of the selection of the clinical features when developing the predictive models.

Prediction of overall survival for patients with metastatic castration-resistant prostate cancer: development of a prognostic model through a crowdsourced challenge with open clinical trial data

Improvements to prognostic models in metastatic castration-resistant prostate cancer have the potential to augment clinical trial design and guide treatment strategies. In partnership with Project Data Sphere, a not-for-profit initiative allowing data from cancer clinical trials to be shared broadly with researchers, we designed an open-data, crowdsourced, DREAM (Dialogue for Reverse Engineering Assessments and Methods) challenge to not only identify a better prognostic model for prediction of survival in patients with metastatic castration-resistant prostate cancer but also engage a community of international data scientists to study this disease. Data from the comparator arms of four phase 3 clinical trials in first-line metastatic castration-resistant prostate cancer were obtained from Project Data Sphere, comprising 476 patients treated with docetaxel and prednisone from the ASCENT2 trial, 526 patients treated with docetaxel, prednisone, and placebo in the MAINSAIL trial, 598 patients treated with docetaxel, prednisone or prednisolone, and placebo in the VENICE trial, and 470 patients treated with docetaxel and placebo in the ENTHUSE 33 trial. Datasets consisting of more than 150 clinical variables were curated centrally, including demographics, laboratory values, medical history, lesion sites, and previous treatments. Data from ASCENT2, MAINSAIL, and VENICE were released publicly to be used as training data to predict the outcome of interest-namely, overall survival. Clinical data were also released for ENTHUSE 33, but data for outcome variables (overall survival and event status) were hidden from the challenge participants so that ENTHUSE 33 could be used for independent validation. Methods were evaluated using the integrated time-dependent area under the curve (iAUC). The reference model, based on eight clinical variables and a penalised Cox proportional-hazards model, was used to compare method performance. Further validation was done using data from a fifth trial-ENTHUSE M1-in which 266 patients with metastatic castration-resistant prostate cancer were treated with placebo alone. 50 independent methods were developed to predict overall survival and were evaluated through the DREAM challenge. The top performer was based on an ensemble of penalised Cox regression models (ePCR), which uniquely identified predictive interaction effects with immune biomarkers and markers of hepatic and renal function. Overall, ePCR outperformed all other methods (iAUC 0·791; Bayes factor >5) and surpassed the reference model (iAUC 0·743; Bayes factor >20). Both the ePCR model and reference models stratified patients in the ENTHUSE 33 trial into high-risk and low-risk groups with significantly different overall survival (ePCR: hazard ratio 3·32, 95% CI 2·39-4·62, p<0·0001; reference model: 2·56, 1·85-3·53, p<0·0001). The new model was validated further on the ENTHUSE M1 cohort with similarly high performance (iAUC 0·768). Meta-analysis across all methods confirmed previously identified predictive clinical variables and revealed aspartate aminotransferase as an important, albeit previously under-reported, prognostic biomarker. Novel prognostic factors were delineated, and the assessment of 50 methods developed by independent international teams establishes a benchmark for development of methods in the future. The results of this effort show that data-sharing, when combined with a crowdsourced challenge, is a robust and powerful framework to develop new prognostic models in advanced prostate cancer. Sanofi US Services, Project Data Sphere.

A predictive model of overall survival in patients with metastatic castration-resistant prostate cancer

Metastatic castration resistant prostate cancer (mCRPC) is one of the most common cancers with a poor prognosis. To improve prognostic models of mCRPC, the Dialogue for Reverse Engineering Assessments and Methods (DREAM) Consortium organized a crowdsourced competition known as the Prostate Cancer DREAM Challenge. In the competition, data from four phase III clinical trials were utilized. A total of 1600 patients’ clinical information across three of the trials was used to generate prognostic models, whereas one of the datasets (313 patients) was held out for blinded validation. As a performance baseline, a model presented in a recent study (so called Halabi model) was used to assess improvements of the new models. This paper presents the model developed by the team TYTDreamChallenge to predict survival risk scores for mCRPC patients at 12, 18, 24 and 30-months after trial enrollment based on clinical features of each patient, as well as an improvement of the model developed after the challenge. The TYTDreamChallenge model performed similarly as the gold-standard Halabi model, whereas the post-challenge model showed markedly improved performance. Accordingly, a main observation in this challenge was that the definition of the clinical features used plays a major role and replacing our original larger set of features with a small subset for training increased the performance in terms of integrated area under the ROC curve from 0.748 to 0.779.

Summary of the DREAM8 Parameter Estimation Challenge: Toward Parameter Identification for Whole-Cell Models.

Whole-cell models that explicitly represent all cellular components at the molecular level have the potential to predict phenotype from genotype. However, even for simple bacteria, whole-cell models will contain thousands of parameters, many of which are poorly characterized or unknown. New algorithms are needed to estimate these parameters and enable researchers to build increasingly comprehensive models. We organized the Dialogue for Reverse Engineering Assessments and Methods (DREAM) 8 Whole-Cell Parameter Estimation Challenge to develop new parameter estimation algorithms for whole-cell models. We asked participants to identify a subset of parameters of a whole-cell model given the model’s structure and in silico “experimental” data. Here we describe the challenge, the best performing methods, and new insights into the identifiability of whole-cell models. We also describe several valuable lessons we learned toward improving future challenges. Going forward, we believe that collaborative efforts supported by inexpensive cloud computing have the potential to solve whole-cell model parameter estimation.

Big Data for Cancer Research.

From the rise in data-mining and the flood of big data has emerged a new era, an informatics age that will probably be the next revolution for humankind in terms of the management of information and generation of new knowledge in diverse fields. A special issue discussing big data storage, management and analyses published by Nature opened the curtain of the big data era in 2008.1 The big data era demonstrated its utility in the successful influenza prediction via Google in 2009.2 Cancer is a heterogeneous disease per se, though through clonal expansion, it is also known that heterogeneity is present within a single patient. It is understandable that somatic mutations accumulate among different single cancer cells. Our current technology for cancer biomarker discovery is usually capable of detecting a snapshot, but not the dynamic and longitudinal changes of the cancer landscape. Big data can tackle this hurdle by gathering different molecular features at the DNA, RNA, protein and metabolite levels. Biodata-mining has led those working in the life sciences to embrace the informatics field, in order to accommodate the deluge of big data generated by next-gene ration biotechnologies, such as next-generation sequencing, proteomics and metabolomics, as well as the structured and unstructured biomedical and healthcare data from electronic health records. Life scientists are bringing together tremendous volumes of information from various high-throughput studies. The holistic approach on data informatics may facilitate the emergence of an era of precision medicine, which ultimately provides treatments tailored to each individual’s needs. Big data certainly challenges cancer research in various aspects. The trends of integrating different platforms and their findings create an intersection among different areas and dimensionalities, which is ultimately a step towards a holistic understanding of the disease. An omics approach to the study of cancer is keeping its fast pace.3 Nowadays, next-generation sequencing, proteomics and metabolomics develop rapidly in various directions. However, they also pose a difficult question regarding the existing platform of using a single dimension to measure cancer. Multi-dimensional approach to cancer research is urgently needed. On the other hand, there is a sea of data, not only huge in volume and complex in structure, but also vast in dynamic scale and depth. We foresee that a large amount of loosely connected, inherently noisy and heterogeneous data may also be gathered in the databases. Do all the data contain useful information? How can we judge whether the information is useful or junk? Thus, certain standards, guidelines or harmanizations may be needed to consolidate useful data in the same database. Some international consortia and institutions have made efforts to set rules to guarantee transparency and traceability so as to ensure the usefulness of the data.4 Electronic health records are becoming a better resource for databases. The information can be exchanged and gathered via telemedicine and mobile connectivity. Patients’ information and data security become an issue and more security measures need to be in place. We are in the precision medicine era, and meticulous decision making based on individual, detailed molecular profiles is a pressing need. Tumor molecular profiling has enabled the subclassification of cancer to redefine treatment regimes. Big data are certainly sailing in the same boat with precision medicine.5 Nowadays, more and more databases have been established, eg, The Catalogue of Somatic Mutations in Cancer (COSMIC), which is the largest database of somatic mutations and their effects on human cancer.4 Another example is the DREAM (Dialogue for Reverse Engineering Assessments and Methods) Consortium, which uses open innovation crowd sourcing to identify top-performing computational methods for inferring genetic heterogeneity from next-generation sequencing data provided by a large multi-institutional community of cancer genomics projects, including the International Cancer Genome Consortium and The Cancer Genome Atlas.7 These huge databases may lead to a paradigm shift in cancer research, researchers from any corner of the world can share and utilize the data for further research and analysis. It is envisioned that in the years to come, we will step into a revolutionized era of utilizing big data to benefit our cancer patients.

Gene regulatory network inference and validation using relative change ratio analysis and time-delayed dynamic Bayesian network.

The Dialogue for Reverse Engineering Assessments and Methods (DREAM) project was initiated in 2006 as a community-wide effort for the development of network inference challenges for rigorous assessment of reverse engineering methods for biological networks. We participated in the in silico network inference challenge of DREAM3 in 2008. Here we report the details of our approach and its performance on the synthetic challenge datasets. In our methodology, we first developed a model called relative change ratio (RCR), which took advantage of the heterozygous knockdown data and null-mutant knockout data provided by the challenge, in order to identify the potential regulators for the genes. With this information, a time-delayed dynamic Bayesian network (TDBN) approach was then used to infer gene regulatory networks from time series trajectory datasets. Our approach considerably reduced the searching space of TDBN; hence, it gained a much higher efficiency and accuracy. The networks predicted using our approach were evaluated comparatively along with 29 other submissions by two metrics (area under the ROC curve and area under the precision-recall curve). The overall performance of our approach ranked the second among all participating teams.Electronic supplementary materialThe online version of this article (doi:10.1186/s13637-014-0012-3) contains supplementary material, which is available to authorized users.

Reconstruction of gene co-expression network from microarray data using local expression patterns.

BackgroundBiological networks connect genes, gene products to one another. A network of co-regulated genes may form gene clusters that can encode proteins and take part in common biological processes. A gene co-expression network describes inter-relationships among genes. Existing techniques generally depend on proximity measures based on global similarity to draw the relationship between genes. It has been observed that expression profiles are sharing local similarity rather than global similarity. We propose an expression pattern based method called GeCON to extract Gene CO-expression Network from microarray data. Pair-wise supports are computed for each pair of genes based on changing tendencies and regulation patterns of the gene expression. Gene pairs showing negative or positive co-regulation under a given number of conditions are used to construct such gene co-expression network. We construct co-expression network with signed edges to reflect up- and down-regulation between pairs of genes. Most existing techniques do not emphasize computational efficiency. We exploit a fast correlogram matrix based technique for capturing the support of each gene pair to construct the network.ResultsWe apply GeCON to both real and synthetic gene expression data. We compare our results using the DREAM (Dialogue for Reverse Engineering Assessments and Methods) Challenge data with three well known algorithms, viz., ARACNE, CLR and MRNET. Our method outperforms other algorithms based on in silico regulatory network reconstruction. Experimental results show that GeCON can extract functionally enriched network modules from real expression data.ConclusionsIn view of the results over several in-silico and real expression datasets, the proposed GeCON shows satisfactory performance in predicting co-expression network in a computationally inexpensive way. We further establish that a simple expression pattern matching is helpful in finding biologically relevant gene network. In future, we aim to introduce an enhanced GeCON to identify Protein-Protein interaction network complexes by incorporating variable density concept.

Harnessing the Crowd

Abstract Scientists showcased results from a pair of open-source computational cancer research challenges at the 7th annual DREAM (Dialogue for Reverse Engineering Assessments and Methods) conference in San Francisco, CA.

Joint estimation of causal effects from observational and intervention gene expression data

BackgroundIn recent years, there has been great interest in using transcriptomic data to infer gene regulatory networks. For the time being, methodological development in this area has primarily made use of graphical Gaussian models for observational wild-type data, resulting in undirected graphs that are not able to accurately highlight causal relationships among genes. In the present work, we seek to improve the estimation of causal effects among genes by jointly modeling observational transcriptomic data with arbitrarily complex intervention data obtained by performing partial, single, or multiple gene knock-outs or knock-downs.ResultsUsing the framework of causal Gaussian Bayesian networks, we propose a Markov chain Monte Carlo algorithm with a Mallows proposal model and analytical likelihood maximization to sample from the posterior distribution of causal node orderings, and in turn, to estimate causal effects. The main advantage of the proposed algorithm over previously proposed methods is its flexibility to accommodate any kind of intervention design, including partial or multiple knock-out experiments. Using simulated data as well as data from the Dialogue for Reverse Engineering Assessments and Methods (DREAM) 2007 challenge, the proposed method was compared to two alternative approaches: one requiring a complete, single knock-out design, and one able to model only observational data.ConclusionsThe proposed algorithm was found to perform as well as, and in most cases better, than the alternative methods in terms of accuracy for the estimation of causal effects. In addition, multiple knock-outs proved to contribute valuable additional information compared to single knock-outs. Finally, the simulation study confirmed that it is not possible to estimate the causal ordering of genes from observational data alone. In all cases, we found that the inclusion of intervention experiments enabled more accurate estimation of causal regulatory relationships than the use of wild-type data alone.

Gene interaction networks based on kernel correlation metrics

In this paper, a Kernel correlation coefficient (KCC) method is proposed to elucidate the gene nonlinear relationships as a distance metric. To evaluate the performance of this nonlinear distance measure, a biological network of the Gaussian Kernel on a public dataset of yeast genes is constructed by using a graph theory. Specifically, the distribution and properties of this new measure are analysed and compared with the classical Pearson correlation method. The reliability and advantages of our proposed Kernel correlation metric is verified and shown formally on ten showcases of the DREAM (Dialogue for Reverse Engineering Assessments and Methods) project. Test experiment results demonstrate that the proposed Kernel correlation coefficient measure has a strong capability in identifying interaction genes, and that the proposed method can detect accurately the key genes and functional interactions (also known as the cliques) as compared to the commonly used Pearson correlation and Mutual Information measures.

GeneNetWeaver:in silicobenchmark generation and performance profiling of network inference methods

Over the last decade, numerous methods have been developed for inference of regulatory networks from gene expression data. However, accurate and systematic evaluation of these methods is hampered by the difficulty of constructing adequate benchmarks and the lack of tools for a differentiated analysis of network predictions on such benchmarks. Here, we describe a novel and comprehensive method for in silico benchmark generation and performance profiling of network inference methods available to the community as an open-source software called GeneNetWeaver (GNW). In addition to the generation of detailed dynamical models of gene regulatory networks to be used as benchmarks, GNW provides a network motif analysis that reveals systematic prediction errors, thereby indicating potential ways of improving inference methods. The accuracy of network inference methods is evaluated using standard metrics such as precision-recall and receiver operating characteristic curves. We show how GNW can be used to assess the performance and identify the strengths and weaknesses of six inference methods. Furthermore, we used GNW to provide the international Dialogue for Reverse Engineering Assessments and Methods (DREAM) competition with three network inference challenges (DREAM3, DREAM4 and DREAM5). GNW is available at http://gnw.sourceforge.net along with its Java source code, user manual and supporting data. Supplementary data are available at Bioinformatics online. dario.floreano@epfl.ch.

Inferring Regulatory Networks from Expression Data Using Tree-Based Methods

One of the pressing open problems of computational systems biology is the elucidation of the topology of genetic regulatory networks (GRNs) using high throughput genomic data, in particular microarray gene expression data. The Dialogue for Reverse Engineering Assessments and Methods (DREAM) challenge aims to evaluate the success of GRN inference algorithms on benchmarks of simulated data. In this article, we present GENIE3, a new algorithm for the inference of GRNs that was best performer in the DREAM4 In Silico Multifactorial challenge. GENIE3 decomposes the prediction of a regulatory network between p genes into p different regression problems. In each of the regression problems, the expression pattern of one of the genes (target gene) is predicted from the expression patterns of all the other genes (input genes), using tree-based ensemble methods Random Forests or Extra-Trees. The importance of an input gene in the prediction of the target gene expression pattern is taken as an indication of a putative regulatory link. Putative regulatory links are then aggregated over all genes to provide a ranking of interactions from which the whole network is reconstructed. In addition to performing well on the DREAM4 In Silico Multifactorial challenge simulated data, we show that GENIE3 compares favorably with existing algorithms to decipher the genetic regulatory network of Escherichia coli. It doesn't make any assumption about the nature of gene regulation, can deal with combinatorial and non-linear interactions, produces directed GRNs, and is fast and scalable. In conclusion, we propose a new algorithm for GRN inference that performs well on both synthetic and real gene expression data. The algorithm, based on feature selection with tree-based ensemble methods, is simple and generic, making it adaptable to other types of genomic data and interactions.

Towards a Rigorous Assessment of Systems Biology Models: The DREAM3 Challenges

BackgroundSystems biology has embraced computational modeling in response to the quantitative nature and increasing scale of contemporary data sets. The onslaught of data is accelerating as molecular profiling technology evolves. The Dialogue for Reverse Engineering Assessments and Methods (DREAM) is a community effort to catalyze discussion about the design, application, and assessment of systems biology models through annual reverse-engineering challenges.Methodology and Principal FindingsWe describe our assessments of the four challenges associated with the third DREAM conference which came to be known as the DREAM3 challenges: signaling cascade identification, signaling response prediction, gene expression prediction, and the DREAM3 in silico network challenge. The challenges, based on anonymized data sets, tested participants in network inference and prediction of measurements. Forty teams submitted 413 predicted networks and measurement test sets. Overall, a handful of best-performer teams were identified, while a majority of teams made predictions that were equivalent to random. Counterintuitively, combining the predictions of multiple teams (including the weaker teams) can in some cases improve predictive power beyond that of any single method.ConclusionsDREAM provides valuable feedback to practitioners of systems biology modeling. Lessons learned from the predictions of the community provide much-needed context for interpreting claims of efficacy of algorithms described in the scientific literature.