Dynamic Network Inference Research Articles

Alignment and stability of embeddings: Measurement and inference improvement

Representation learning (RL) methods learn objects’ latent embeddings where information is preserved by distance. Since certain distance functions are invariant to certain linear transformations, one may obtain different embeddings while preserving the same information. In dynamic systems, a temporal difference in embeddings may be explained by the stability of the system or by the misalignment of embeddings due to arbitrary transformations. This study focuses on the embedding alignment problem to distinguish structural changes inherent to a system from arbitrary changes caused by representation learning methods, and quantify their magnitudes. In order to avoid any confusion due to the naming conventions in the literature, it should be noted that embedding alignment problems are different from graph matching/network alignment problems. In the representation learning literature, although the embedding alignment issue has been acknowledged, its measurement and empirical analysis have not received sufficient interest. In this work, we explore the embedding alignment and its parts, propose novel metrics to measure alignment and stability, and show their suitability through synthetic experiments. Real-world experiments show that both static and dynamic RL methods are prone to produce misaligned embeddings and such misalignment worsens the performance of dynamic network inference tasks. By ensuring alignment, the prediction accuracy raises by up to 90% in static and up to 40% in dynamic RL methods.

Inferring links in directed complex networks through feed forward loop motifs

Complex networks are mathematical abstractions of real-world systems using sets of nodes and edges representing the entities and their interactions. Prediction of unknown interactions in such networks is a problem of interest in biology, sociology, physics, engineering, etc. Most complex networks exhibit the recurrence of subnetworks, called network motifs. Within the realm of social science, link prediction (LP) models are employed to model opinions, trust, privacy, rumor spreading in social media, academic and corporate collaborations, liaisons among lawbreakers, and human mobility resulting in contagion. We present an LP metric based on a motif in directed complex networks, called feed-forward loop (FFL). Unlike nearest neighbor-based metrics and machine learning-based techniques that gauge the likelihood of a link based on node similarity, the proposed approach leverages a known dichotomy in the motif distribution of directed networks. Complex networks are sparse, causing most nodes and their associated links to have low motif participation. Yet, due to intrinsic network motif-richness, few links participate in many distinct motif substructures. Thus, the FFL-based metric combines the presence and absence of motifs as a signature to outperform baseline metrics on ten directed social and biological network datasets. We conclude with the future of the FFL-based metric in dynamic network inference as well as its use in designing combined metrics using network motifs of varying orders as features.

Inference of dynamic interaction networks: A comparison between Lotka-Volterra and multivariate autoregressive models.

Networks are ubiquitous throughout biology, spanning the entire range from molecules to food webs and global environmental systems. Yet, despite substantial efforts by the scientific community, the inference of these networks from data still presents a problem that is unsolved in general. One frequent strategy of addressing the structure of networks is the assumption that the interactions among molecular or organismal populations are static and correlative. While often successful, these static methods are no panacea. They usually ignore the asymmetry of relationships between two species and inferences become more challenging if the network nodes represent dynamically changing quantities. Overcoming these challenges, two very different network inference approaches have been proposed in the literature: Lotka-Volterra (LV) models and Multivariate Autoregressive (MAR) models. These models are computational frameworks with different mathematical structures which, nevertheless, have both been proposed for the same purpose of inferring the interactions within coexisting population networks from observed time-series data. Here, we assess these dynamic network inference methods for the first time in a side-by-side comparison, using both synthetically generated and ecological datasets. Multivariate Autoregressive and Lotka-Volterra models are mathematically equivalent at the steady state, but the results of our comparison suggest that Lotka-Volterra models are generally superior in capturing the dynamics of networks with non-linear dynamics, whereas Multivariate Autoregressive models are better suited for analyses of networks of populations with process noise and close-to linear behavior. To the best of our knowledge, this is the first study comparing LV and MAR approaches. Both frameworks are valuable tools that address slightly different aspects of dynamic networks.

Sliding window change point detection based dynamic network model inference framework for airport ground service process

With the rapid development of civil aviation, the number of airport flights is continually increasing and the low efficiency of flight ground services has become one of the main reasons for flight delays. Obtaining a precise description of the flight ground service network is of great importance to the efficient and safe operation of airports. In this study, we constructed a dynamic network model inference framework based on sliding window change point detection. First, a change point detection algorithm was proposed to divide all time points into two parts, namely, a set of possible change points and a set of completely impossible change points, which greatly reduces the true change point search space. Then, two novel change point elimination judgment rules were designed and combined with the structure learning algorithm to achieve iterative optimization within the possible change point sets, obtain the final true change points, and realize precise dynamic network inference. Based on the experimental results, our proposed method has a higher accuracy and computational efficiency than the state-of-the-art methods, and it can be easily adapted to large-scale datasets. Finally, the proposed model was used in an airport flight ground service analysis. Each day was divided into four stages and the potential dependence relationships between different nodes in each stage were accurately determined. Thus, the model has great practical significance for the accurate scheduling and efficient operation of airport ground services.

Computational Derivation of Core, Dynamic Human Blunt Trauma Inflammatory Endotypes.

Systemic inflammation ensues following traumatic injury, driving immune dysregulation and multiple organ dysfunction (MOD). While a balanced immune/inflammatory response is ideal for promoting tissue regeneration, most trauma patients exhibit variable and either overly exuberant or overly damped responses that likely drive adverse clinical outcomes. We hypothesized that these inflammatory phenotypes occur in the context of severe injury, and therefore sought to define clinically distinct endotypes of trauma patients based on their systemic inflammatory responses. Using Patient-Specific Principal Component Analysis followed by unsupervised hierarchical clustering of circulating inflammatory mediators obtained in the first 24 h after injury, we segregated a cohort of 227 blunt trauma survivors into three core endotypes exhibiting significant differences in requirement for mechanical ventilation, duration of ventilation, and MOD over 7 days. Nine non-survivors co-segregated with survivors. Dynamic network inference, Fisher Score analysis, and correlations of IL-17A with GM-CSF, IL-10, and IL-22 in the three survivor sub-groups suggested a role for type 3 immunity, in part regulated by Th17 and γδ 17 cells, and related tissue-protective cytokines as a key feature of systemic inflammation following injury. These endotypes may represent archetypal adaptive, over-exuberant, and overly damped inflammatory responses.

Inference of a Dynamic Aging-related Biological Subnetwork via Network Propagation.

Gene expression (GE)data capture valuable condition-specific information ("condition" can mean a biological process, disease stage, age, patient, etc.)However, GE analyses ignore physical interactions between gene products, i.e., proteins. Because proteins function by interacting with each other, and because biological networks (BNs)capture these interactions, BN analyses are promising. However, current BN data fail to capture condition-specific information. Recently, GE and BN data have been integrated using network propagation (NP)to infer condition-specific BNs. However, existing NP-based studies result in a static condition-specific subnetwork, even though cellular processes are dynamic. A dynamic process of our interest is human aging. We use prominent existing NP methods in a new task of inferring a dynamic rather than static condition-specific (aging-related)subnetwork. Then, we study evolution of network structure with age - we identify proteins whose network positions significantly change with age and predict them as new aging-related candidates. We validate the predictions via e.g., functional enrichment analyses and literature search. Dynamic network inference via NP yields higher prediction quality than the only existing method for inferring a dynamic aging-related BN, which does not use NP. Our data and code are available at https://nd.edu/~cone/dynetinf.

Dynamic network inference and association computation discover gene modules regulating virulence, mycotoxin and sexual reproduction in Fusarium graminearum

BackgroundThe filamentous fungus Fusarium graminearum causes devastating crop diseases and produces harmful mycotoxins worldwide. Understanding the complex F. graminearum transcriptional regulatory networks (TRNs) is vital for effective disease management. Reconstructing F. graminearum dynamic TRNs, an NP (non-deterministic polynomial) -hard problem, remains unsolved using commonly adopted reductionist or co-expression based approaches. Multi-omic data such as fungal genomic, transcriptomic data and phenomic data are vital to but so far have been largely isolated and untapped for unraveling phenotype-specific TRNs.ResultsHere for the first time, we harnessed these resources to infer global TRNs for F. graminearum using a Bayesian network based algorithm called “Module Networks”. The inferred TRNs contain 49 regulatory modules that show condition-specific gene regulation. Through a thorough validation based on prior biological knowledge including functional annotations and TF binding site enrichment, our network prediction displayed high accuracy and concordance with existing knowledge. One regulatory module was partially validated using network perturbations caused by Tri6 and Tri10 gene disruptions, as well as using Tri6 Chip-seq data. We then developed a novel computational method to calculate the associations between modules and phenotypes, and identified major module groups regulating different phenotypes. As a result, we identified TRN subnetworks responsible for F. graminearum virulence, sexual reproduction and mycotoxin production, pinpointing phenotype-associated modules and key regulators. Finally, we found a clear compartmentalization of TRN modules in core and lineage-specific genomic regions in F. graminearum, reflecting the evolution of the TRNs in fungal speciation.ConclusionsThis system-level reconstruction of filamentous fungal TRNs provides novel insights into the intricate networks of gene regulation that underlie key processes in F. graminearum pathobiology and offers promise for the development of improved disease control strategies.

Detecting changes in transient complex systems via dynamic network inference

Graph analytics methods have evoked significant interest in recent years. Their applicability to real-world complex systems is currently limited by the challenges of inferring effective graph representations of the high-dimensional, noisy, nonlinear and transient dynamics from limited time series outputs, as well as of extracting statistical quantifiers that capture the salient structure of the inferred graphs for detecting change. In this article, we present an approach to detecting changes in complex dynamic systems that is based on spectral-graph-theory and uses a single realization of time series data collected under specific, common types of transient conditions, such as intermittency. We introduce a statistic, γk, based on the spectral content of the inferred graph. We show that the γk statistic under high-dimensional dynamics converges to a normal distribution, and we employ the parameters of this distribution to construct a procedure to detect qualitative changes in the coupling structure of a dynamical system. Experimental investigations suggest that the γk statistic by itself is able to detect changes with modified area under curve (mAUC) of about 0.96 (for numerical simulation tests), and can, by itself, achieve a true positive rate of about 40% for detecting seizures from EEG signals. In addition, by incorporating this statistic with random forest, one of the best seizure detection methods, the seizure detection rate of the random forest method improves by 5% in 35% of the subjects. These studies of the network inferred from EEG signals suggest that γk can capture salient structural changes in the physiology of the process and can therefore serve as an effective feature for detecting seizures from EEG signals.

Connecting the Dots: Identifying Network Structure via Graph Signal Processing

Network topology inference is a prominent problem in Network Science. Most graph signal processing (GSP) efforts to date assume that the underlying network is known, and then analyze how the graph's algebraic and spectral characteristics impact the properties of the graph signals of interest. Such an assumption is often untenable beyond applications dealing with e.g., directly observable social and infrastructure networks; and typically adopted graph construction schemes are largely informal, distinctly lacking an element of validation. This tutorial offers an overview of graph learning methods developed to bridge the aforementioned gap, by using information available from graph signals to infer the underlying graph topology. Fairly mature statistical approaches are surveyed first, where correlation analysis takes center stage along with its connections to covariance selection and high-dimensional regression for learning Gaussian graphical models. Recent GSP-based network inference frameworks are also described, which postulate that the network exists as a latent underlying structure, and that observations are generated as a result of a network process defined in such a graph. A number of arguably more nascent topics are also briefly outlined, including inference of dynamic networks, nonlinear models of pairwise interaction, as well as extensions to directed graphs and their relation to causal inference. All in all, this paper introduces readers to challenges and opportunities for signal processing research in emerging topic areas at the crossroads of modeling, prediction, and control of complex behavior arising in networked systems that evolve over time.

Network Inference via the Time-Varying Graphical Lasso.

Many important problems can be modeled as a system of interconnected entities, where each entity is recording time-dependent observations or measurements. In order to spot trends, detect anomalies, and interpret the temporal dynamics of such data, it is essential to understand the relationships between the different entities and how these relationships evolve over time. In this paper, we introduce the time-varying graphical lasso (TVGL), a method of inferring time-varying networks from raw time series data. We cast the problem in terms of estimating a sparse time-varying inverse covariance matrix, which reveals a dynamic network of interdependencies between the entities. Since dynamic network inference is a computationally expensive task, we derive a scalable message-passing algorithm based on the Alternating Direction Method of Multipliers (ADMM) to solve this problem in an efficient way. We also discuss several extensions, including a streaming algorithm to update the model and incorporate new observations in real time. Finally, we evaluate our TVGL algorithm on both real and synthetic datasets, obtaining interpretable results and outperforming state-of-the-art baselines in terms of both accuracy and scalability.

Time-Dependent Gene Network Modelling by Sequential Monte Carlo.

Most existing methods used for gene regulatory network modeling are dedicated to inference of steady state networks, which are prevalent over all time instants. However, gene interactions evolve over time. Information about the gene interactions in different stages of the life cycle of a cell or an organism is of high importance for biology. In the statistical graphical models literature, one can find a number of methods for studying steady-state network structures while the study of time varying networks is rather recent. A sequential Monte Carlo method, namely particle filtering (PF), provides a powerful tool for dynamic time series analysis. In this work, the PF technique is proposed for dynamic network inference and its potentials in time varying gene expression data tracking are demonstrated. The data used for validation are synthetic time series data available from the DREAM4 challenge, generated from known network topologies and obtained from transcriptional regulatory networks of S. cerevisiae. We model the gene interactions over the course of time with multivariate linear regressions where the parameters of the regressive process are changing over time.

Assessing dynamics, spatial scale, and uncertainty in task-related brain network analyses

The brain is a complex network of interconnected elements, whose interactions evolve dynamically in time to cooperatively perform specific functions. A common technique to probe these interactions involves multi-sensor recordings of brain activity during a repeated task. Many techniques exist to characterize the resulting task-related activity, including establishing functional networks, which represent the statistical associations between brain areas. Although functional network inference is commonly employed to analyze neural time series data, techniques to assess the uncertainty—both in the functional network edges and the corresponding aggregate measures of network topology—are lacking. To address this, we describe a statistically principled approach for computing uncertainty in functional networks and aggregate network measures in task-related data. The approach is based on a resampling procedure that utilizes the trial structure common in experimental recordings. We show in simulations that this approach successfully identifies functional networks and associated measures of confidence emergent during a task in a variety of scenarios, including dynamically evolving networks. In addition, we describe a principled technique for establishing functional networks based on predetermined regions of interest using canonical correlation. Doing so provides additional robustness to the functional network inference. Finally, we illustrate the use of these methods on example invasive brain voltage recordings collected during an overt speech task. The general strategy described here—appropriate for static and dynamic network inference and different statistical measures of coupling—permits the evaluation of confidence in network measures in a variety of settings common to neuroscience.

Bayesian Inference for Sparse VAR(1) Models, with Application to Time Course Microarray Data

This paper considers the problem of undertaking fully Bayesian inference for both the parameters and structure of a vector autoregressive model on the basis of time course data in the ``p>> n scenario’’. The autoregressive matrix is assumed to be sparse, but of unknown structure. The resulting algorithm for dynamic Bayesian network inference is shown to be highly effective, and is applied to the problem of dynamic network inference from time course microarray data using a dataset concerned with the transient response of budding yeast to telomere damage.