Network Inference Research Articles

The impact of data resolution on dynamic causal inference in multiscale ecological networks.

While it is commonly accepted that ecosystem dynamics are nonlinear, what is often not acknowledged is that nonlinearity implies scale-dependence. With the increasing availability of high-resolution ecological time series, there is a growing need to understand how scale and resolution in the data affect the construction and interpretation of causal networks-specifically, networks mapping how changes in one variable drive changes in others as part of a shared dynamic system ("dynamic causation"). We use Convergent Cross Mapping (CCM), a method specifically designed to measure dynamic causation, to study the effects of varying temporal and taxonomic/functional resolution in data when constructing ecological causal networks. As the system is viewed at different scales relationships will appear and disappear. The relationship between data resolution and interaction presence is not random: the temporal scale at which a relationship is uncovered identifies a biologically relevant scale that drives changes in population abundance. Further, causal relationships between taxonomic aggregates (low-resolution) are shown to be influenced by the number of interactions between their component species (high-resolution). Because no single level of resolution captures all the causal links in a system, a more complete understanding requires multiple levels when constructing causal networks.

Efficient Batched Inference in Conditional Neural Networks

Efficient Batched Inference in Conditional Neural Networks

Scalable parallel photonic processing unit for various neural network accelerations

In recent years, integrated optical processing units (IOPUs) have demonstrated advantages in energy efficiency and computational speed for neural network inference applications. However, limited by optical integration technology, the practicality and versatility of IOPU face serious challenges. In this work, a scalable parallel photonic processing unit (SPPU) for various neural network accelerations based on high-speed phase modulation is proposed and implemented on a silicon-on-insulator platform, which supports parallel processing and can switch between multiple computational paradigms simply and without latency to infer different neural network structures, enabling to maximize the utility of on-chip components. The SPPU adopts a scalable and process-friendly architecture design, with a preeminent photonic-core energy efficiency of 0.83 TOPS/W, two to ten times higher than existing integrated solutions. In the proof-of-concept experiment, a convolutional neural network (CNN), a residual CNN, and a recurrent neural network (RNN) are all implemented on our photonic processor to handle multiple tasks of handwritten digit classification, signal modulation format recognition, and review emotion recognition. The SPPU achieves multi-task parallel processing capability, serving as a promising and attractive research route to maximize the utility of on-chip components under the constraints of integrated technology, which helps to make IOPU more practical and universal.

Gene Regulatory Network inference based on Modified Adaptive Lasso

Gene Regulatory Network inference based on Modified Adaptive Lasso

MicroNet-MIMRF: A Microbial Network Inference Approach Based on Mutual Information and Markov Random Fields

Abstract Motivation The human microbiome, comprises complex associations and communication networks among microbial communities, which are crucial for maintaining health. The construction of microbial networks is vital for elucidating these associations and co-occurrence patterns. However, existing microbial networks inference methods cannot solve the issues of zero-inflation and non-linear associations. Therefore, necessitating novel methods to improve the accuracy of microbial networks inference. Results In this study, we introduce the Microbial Network based on Mutual Information and Markov Random Fields (MicroNet-MIMRF) as a novel approach for inferring microbial networks. Abundance data of microbes are modeled through the zero-inflated Poisson distribution, and the discrete matrix is estimated for further calculation. Markov random fields based on mutual information are used to construct accurate microbial networks. MicroNet-MIMRF excels at estimating pairwise associations between microbes, effectively addressing zero-inflation and non-linear associations in microbial abundance data. It outperforms commonly used techniques in simulation experiments, achieving area under the curve values exceeding 0.75 for all parameters. A case study on inflammatory bowel disease data further demonstrates the method’s ability to identify insightful associations. Conclusively, MicroNet-MIMRF is a powerful tool for microbial network inference that handles the biases caused by zero-inflation and overestimation of associations. This novel tool has abundant applications in health research. Availability and implementation The MicroNet-MIMRF is provided at https://github.com/Fionabiostats/MicroNet-MIMRF.

Benchmarking In-Sensor Machine Learning Computing: An Extension to the MLCommons-Tiny Suite

This paper proposes a new benchmark specifically designed for in-sensor digital machine learning computing to meet an ultra-low embedded memory requirement. With the exponential growth of edge devices, efficient local processing is essential to mitigate economic costs, latency, and privacy concerns associated with the centralized cloud processing. Emerging intelligent sensors equipped with computing assets to run neural network inferences and embedded in the same package, which hosts the sensing elements, present new challenges due to their limited memory resources and computational skills. This benchmark evaluates models trained with Quantization Aware Training (QAT) and compares their performance with Post-Training Quantization (PTQ) across three use cases: Human Activity Recognition (HAR) by means of the SHL dataset, Physical Activity Monitoring (PAM) by means of the PAMAP2 dataset, and superficial electromyography (sEMG) regression with the NINAPRO DB8 dataset. The results demonstrate the effectiveness of QAT over PTQ in most scenarios, highlighting the potential for deploying advanced AI models on highly resource-constrained sensors. The INT8 versions of the models always outperformed their FP32, regarding memory and latency reductions, except for the activations for CNN. The CNN model exhibited reduced memory usage and latency with respect to its Dense counterpart, allowing it to meet the stringent 8KiB data RAM and 32 KiB program RAM limits of the ISPU. The TCN model proved to be too large to fit within the memory constraints of the ISPU, primarily due to its greater capacity in terms of number of parameters, designed for processing more complex signals like EMG. This benchmark aims to guide the development of efficient AI solutions for In-Sensor Machine Learning Computing, fostering innovation in the field of Edge AI benchmarking, such as the one conducted by the MLCommons-Tiny working group.

Efficient probabilistic inference in biochemical networks

Biochemical networks are usually modeled by ordinary differential equations that describe the time evolution of the concentrations of the interacting (biochemical) species for specific initial concentrations and certain values of the interaction rates. They consist, in general, of many variables and parameters. Parameter estimation of such systems using standard optimization algorithms is computationally expensive since a large number of numerical simulations must be performed for numerous values of initial conditions and parameters, making these approaches either inefficient or inaccurate. In this paper, we explain how biochemical networks can be approximated by dynamic Bayesian networks, a class of discrete probabilistic models. This type of approximation enables the use of Bayesian inference for performing tasks such as parameter estimation. We explain how to make this type of approximations and the subsequent parameter estimation task as accurate and computationally efficient as possible. Our approach is applied on concrete examples such as the EGF-NGF cellular signaling pathway.

Estimation of missing streamflow data using various artificial intelligence methods in peninsular Malaysia

ABSTRACT Missing streamflow data is a common issue in Peninsular Malaysia, as the technologies used in hydrological studies often fail to collect data accurately. Additionally, conventional methods are still widely used in the region, which are less accurate compared to artificial intelligence (AI) methods in estimating missing streamflow data. Therefore, this study aims to estimate the missing streamflow data from 11 stations in Peninsular Malaysia by using different AI methods and determine the most appropriate method. Four homogeneity tests were applied to check the quality of data, and the results of the tests indicated that the streamflow data in most stations were homogenous. Two AI methods were applied in this study, which were artificial neural network and artificial neuro-fuzzy inference systems (ANFIS). The proposed AI methods were compared with five different conventional methods. All streamflow missing data, constituting 30% of data from each year were estimated on a daily time scale, and evaluated using root mean square error, mean absolute error and correlation coefficient values. The results indicated that ANFIS was the best due to its learning abilities and the fuzzy inference systems, which enable it to handle complicated input–output patterns and provide highly accurate estimation results.

A benchmark of RNA-seq data normalization methods for transcriptome mapping on human genome-scale metabolic networks

Genome-scale metabolic models (GEMs) cover the entire list of metabolic genes in an organism and associated reactions, in a tissue/condition non-specific manner. RNA-seq provides crucial information to make the GEMs condition-specific. Integrative Metabolic Analysis Tool (iMAT) and Integrative Network Inference for Tissues (INIT) are the two most popular algorithms to create condition-specific GEMs from human transcriptome data. The normalization method of choice for raw RNA-seq count data affects the model content produced by these algorithms and their predictive accuracy. However, a benchmark of the RNA-seq normalization methods on the performance of iMAT and INIT algorithms is missing in the literature. Another important phenomenon is covariates such as age and gender in a dataset, and they can affect the predictivity of analysis. In this study, we aimed to compare five different RNA-seq data normalization methods (TPM, FPKM, TMM, GeTMM, and RLE) and covariate adjusted versions of the normalized data by mapping them on a human GEM using the iMAT and INIT algorithms to generate personalized metabolic models. We used RNA-seq data for Alzheimer’s disease (AD) and lung adenocarcinoma (LUAD) patients. The results demonstrated that RNA-seq data normalized by the RLE, TMM, or GeTMM methods enabled the production of condition-specific metabolic models with considerably low variability in terms of the number of active reactions compared to the within-sample normalization methods (FPKM, TPM). Using these models, we could more accurately capture the disease-associated genes (average accuracy of ~0.80 for AD and ~0.67 for LUAD) for the RLE, TMM, and GeTMM normalization methods. An increase in the accuracies was observed for all the methods when covariate adjustment was applied. We found a similar accuracy trend when we compared the metabolites of perturbed reactions to metabolome data for AD. Together, our benchmark study shows that the between-sample RNA-seq normalization methods reduce false positive predictions at the expense of missing some true positive genes when mapped on GEMs.

High expression of PLA2G2A in fibroblasts plays a crucial role in the early progression of carotid atherosclerosis

BackgroundIn mouse models of atherosclerosis, knockout of the PLA2G2A gene has been shown to reduce the volume of atherosclerotic plaques. Clinical trials have demonstrated the potential of using the sPLA2 inhibitor Varespladib in combination with statins to reduce lipid levels. However, this approach has not yielded the expected results in reducing the risk of cardiovascular events. Therefore, it is necessary to further investigate the mechanisms of PLA2G2A.MethodsSingle-cell transcriptome data from two sets of carotid plaques, combined with clinical patient information. were used to describe the expression characteristics of PLA2G2A in carotid plaques at different stages. In order to explore the mechanisms of PLA2G2A, we conducted enrichment analysis, cell–cell communication analysis and single-cell regulatory network inference and clustering analyses. We validated the above findings at the cellular level.ResultsOur findings indicate that PLA2G2A is primarily expressed in vascular fibroblasts and shows significant cell interactions with macrophages in the early-stage, especially in complement and inflammation-related pathways. We also found that serum sPLA2 levels have stronger diagnostic value in patients with mild carotid artery stenosis. Subsequent comparisons of single-cell transcriptomic data from early and late-stage carotid artery plaques corroborated these findings and predicted transcription factors that might regulate the progression of early carotid atherosclerosis (CA) and the expression of PLA2G2A.ConclusionsOur study discovered and validated that PLA2G2A is highly expressed by vascular fibroblasts and promotes plaque progression through the activation of macrophage complement and coagulation cascade pathways in the early-stage of CA.

Reservoir structure optimization of echo state networks: A detrended multiple cross-correlation pruning perspective

Reservoir structure optimization of echo state networks (ESN) is an important enabler for improving network performance. In this regard, pruning provides an effective means to optimize reservoir structure by removing redundant components in the network. Existing studies achieve reservoir pruning by removing insignificant neuronal connections. However, such processing causes the optimized neurons to still remain in the reservoir and thus hinder network inference by participating in computations, leading to suboptimal utilization of pruning benefits by the network. To solve this problem, this paper proposes an adaptive pruning algorithm for ESN within the detrended multiple cross-correlation (DMC2) framework, i.e., DMAP. On the whole, it contains two main functional parts: DMC2 measure of reservoir neurons and reservoir pruning. Specifically, the former is used to quantify the correlation among neurons. Based on this, the latter can remove neurons with high correlation from the reservoir completely, and finally obtain the optimal network structure by retraining the output weights. Experiment results show that DMAP-ESN outperforms its competitors in nonlinear approximation capability and reservoir stability.

SPLANG—a synthetic poisson-lognormal-based abundance and network generative model for microbial interaction inference algorithms

Microbes are pervasive and their interaction with each other and the environment can impact fields as diverse as health and agriculture. While network inference and related algorithms that use abundance data from pyrosequencing can infer microbial interaction networks, the ambiguity surrounding the actual underlying networks hampers the validation of these algorithms. This study introduces a generative model to synthesize both the underlying interactive network and observable abundance data, serving as a test bed for the existing and future network inference algorithms. We tested our generative model with four typical network inference algorithms; our results suggest that none of these algorithms demonstrate adequate accuracy for inferring ecologies of non-commensalistic species, either mutualistic or competitive. We further explored the potential for predictability by combining existing algorithms with an oracle algorithm built by fusing the results of several existing algorithms. The oracle algorithm reveals promising improvements in predictability, although it falls short when applied to networks characterized by dense interspecies taxa interactions. Our work underscores the need for the continued development and validation of algorithms to unravel the intricacies of microbial interaction networks.

Analysis and Optimization of Super Duplex Stainless Steel Deposition in Wire Arc Additive Manufacturing Using Machine Learning Techniques

<div>This article presents experimental investigations and machine learning-based analysis on depositions of super duplex stainless steel (SDSS ER2594) material in wire arc additive manufacturing (WAAM) considering the process parameters namely voltage, wire feed rate, torch travel speed, and gas flow rate. Deposition efficiency and surface height values of the accumulated material were measured to build machine learning models using artificial neural network (ANN) and adaptive neuro-fuzzy inference system (ANFIS). The developed ANN model could predict the deposition efficiency and surface height with mean absolute deviations (MADs) of 8.9% and 16.1%, respectively. The MAD for prediction of the two responses for ANFIS model was found to be 6.1% and 14.9% as compared to the experimental data. Multi-objective optimization was also performed to obtain optimal solutions to achieve desired deposition results. Mechanical properties and microstructures of the deposited materials with optimal processing parameters were found comparable to that of the base materials.</div>

Reconstructing Molecular Networks by Causal Diffusion Do-Calculus Analysis with Deep Learning.

Quantifying molecular regulations between genes/molecules causally from observed data is crucial for elucidating the molecular mechanisms underlying biological processes at the network level. Presently, most methods for inferring gene regulatory and biological networks rely on association studies or observational causal-analysis approaches. This study introduces a novel approach that combines intervention operations and diffusion models within a do-calculus framework by deep learning, i.e., Causal Diffusion Do-calculus (CDD) analysis, to infer causal networks between molecules. CDD can extract causal relations from observed data owing to its intervention operations, thereby significantly enhancing the accuracy and generalizability of causal network inference. Computationally, CDD has been applied to both simulated data and real omics data, which demonstrates that CDD outperforms existing methods in accurately inferring gene regulatory networks and identifying causal links from genes to disease phenotypes. Especially, compared with the Mendelian randomization algorithm and other existing methods, the CDD can reliably identify the disease genes or molecules for complex diseases with better performances. In addition, the causal analysis between various diseases and the potential factors in different populations from the UK Biobank database is also conducted, which further validated the effectiveness of CDD.

A dynamic authorizable ciphertext image retrieval algorithm based on security neural network inference.

In this paper, we propose a dynamic authorizable ciphertext image retrieval scheme based on secure neural network inference that effectively enhances the security of image retrieval while preserving privacy. To ensure the privacy of the original image and enable feature extraction without decryption operations, we employ a secure neural network for feature extraction during the index construction stage of encrypted images. Additionally, we introduce a dynamic authenticatable ciphertext retrieval algorithm to enhance system flexibility and security by enabling users to quickly and flexibly retrieve authorized images. Experimental results demonstrate that our scheme guarantees data image privacy throughout the entire process from upload to retrieval compared to similar literature schemes. Furthermore, our scheme ensures data availability while maintaining security, allowing users to conveniently perform image retrieval operations. Although overall efficiency may not be optimal according to experimental results, our solution satisfies practical application needs in cloud computing environments by providing an efficient and secure image retrieval solution.

STCO: Enhancing Training Efficiency via Structured Sparse Tensor Compilation Optimization

Network sparsification serves as an effective technique to accelerate Deep Neural Network (DNN) inference. However, existing sparsification techniques often rely on structured sparsity, which yields limited benefits. This is primarily due to the significant memory and computational overhead introduced by numerous sparse storage formats during address generation and gradient updates. Additionally, many of these solutions are tailored solely for the inference phase, neglecting the crucial training phase. In this paper, we introduce STCO, a novel Sparse Tensor Compilation Optimization technique that significantly enhances training efficiency through structured sparse tensor compilation. Central to STCO is the Tensorization-aware Index Entity (TIE) format, which effectively represents structured sparse tensors by eliminating redundant indices and minimizing storage overhead. The TIE format plays a pivotal role in the Address-carry flow (AC flow) pass, which optimizes the data layout at the computational graph level. This pass leverages the TIE format to enhance the efficiency of tensor representations, enabling more compact and efficient sparse tensor storage. Meanwhile, a shape inference pass utilizes the AC flow to derive optimized tensor shapes, further refining the performance of sparse tensor operations. Moreover, the Address-Carry TIE Flow dynamically tracks nonzero addresses, extending the benefits of sparse optimization to both forward and backward propagation. This seamless integration into the training pipeline enables a smooth transition to sparse tensor compilation without significant modifications to existing codebases. To further boost training performance, we implement an operator-level AC flow optimization pass tailored for structured sparse tensors. This pass generates efficient addresses, ensuring minimal computational overhead during sparse tensor operations. The flexibility of STCO allows it to be efficiently integrated into various frameworks or compilers, providing a robust solution for enhancing training efficiency with structured sparse tensors. Experiments demonstrated that STCO achieved impressive speedups of 3.64 ×, 5.43 ×, 4.89 ×, and 3.91 × when compared to state-of-the-art sparse formats on VGG16, ResNet-18, MobileNetV1, and MobileNetV2, respectively. These findings underscore the efficiency and superiority of our proposed approach in leveraging unstructured sparsity for Deep Neural Network inference acceleration.

Predicting chlorophyll-a dynamics in the Ashtamudi estuary using ANN and ANFIS techniques

Excessive nutrients and associated high chlorophyll-a concentration are of growing concern to the public health due to the occurrence of eutrophication in aquatic bodies. Continuous monitoring of chlorophyll-a has limitations, thereby necessitating the development of prediction models. Even though chlorophyll-a prediction models are numerous, they rarely fit into estuarine conditions, which is encountered with dynamic mixing of freshwater and sea water. Thus, the present study identifies the most appropriate driving factors that are relevant for estuarine conditions through chlorophyll-a prediction models developed using Artificial Neural Network and Adaptive Neuro Fuzzy Inference System (ANFIS) approaches in the Ashtamudi estuary, India. Salinity, which is an indirect measure of the estuarine mixing intensity, has been used to replicate the dynamic mixing in the estuary. The results indicate that the model incorporating salinity along with total nitrogen, total phosphorous and turbidity well-predicted chlorophyll-a concentration with a coefficient of determination ranging from 0.73 to 0.99. General Regression Neural Network (GRNN) and ANFIS models outperformed Back Propagation Neural Network (BPNN) and Recurrent Neural Network (RNN) models. Of the various ANFIS models, the ones that used Gaussian and Generalized Bell membership functions were most appropriate for the prediction of chlorophyll-a than the models with triangular and trapezoidal membership functions. The study identified that the models that considered salinity were less sensitive and can be used for modelling chlorophyll-a. The chlorophyll-a was observed to have a direct influence of salinity at salinity values greater than 5‰. The study highlighted that estuarine mixing further enhances the primary productivity through increased penetration of light leading to higher chlorophyll-a concentration. Further the study emplasized that the predictive models are important tools fitting well within estuarine environments due to its replicability.

Approximate inference on optimized quantum Bayesian networks

In recent years, there has been a significant upsurge in the interest surrounding Quantum machine learning, with researchers actively developing methods to leverage the power of quantum technology for solving highly complex problems across various domains. However, implementing gate-based quantum algorithms on noisy intermediate quantum devices (NISQ) presents notable challenges due to limited quantum resources and inherent noise. In this paper, we propose an innovative approach for representing Bayesian networks on quantum circuits, specifically designed to address these challenges and highlight the potential of combining optimized circuits with quantum hybrid algorithms for Bayesian network inference. Our aim is to minimize the required quantum resource needed to implement a Quantum Bayesian network (QBN) and implement quantum approximate inference algorithm on a quantum computer. Through simulations and experiments on IBM Quantum computers, we show that our circuit representation significantly reduces the resource requirements without decreasing the performance of the model. These findings underscore how our approach can better enable practical applications of QBN on currently available quantum hardware.

Sparse inference of the human haematopoietic system from heterogeneous and partially observed genomic data

Abstract Haematopoiesis is the process of blood cells’ formation, with progenitor stem cells differentiating into mature forms such as white and red blood cells or platelets. While progenitor cells share regulatory pathways involving common nuclear factors, specific networks shape their fate towards particular lineages. This paper analyses the complex regulatory network that drives the formation of mature red blood cells and platelets from their common precursors. Using the latest reverse transcription quantitative real-time PCR genomic data, we develop a dedicated graphical model that incorporates the effect of external genomic data and allows inference of regulatory networks from the high-dimensional and partially observed data.

Mechanically Stabilized Earth Wall Reliability Analysis Using Response Surface Methodology, ANN, ANFIS and Multi-objective Genetic Algorithm

Considering uncertainty in the analysis of geotechnical structures is a necessary condition for optimal and robust design. An alternative method for studying the reliability of a mechanically reinforced earth wall in granular soil is used to account for these uncertainties more rigorously. This allows for the inclusion of various uncertainties in a mathematical risk formulation based on random variables. The deterministic model is a benchmark taken from the literature used in a numerical simulation to determine the maximum horizontal displacement of the wall. In this case, the serviceability limit state is considered, allowing the wall's actual behavior to be described. ANOVA was used to identify the most influential parameters on the system's response. As uncorrelated random variables, only the parameters (E, φ and γ) were considered. The mathematical model serving as the limit state function was numerically predictedby three methods, response surface methodology (RSM), artificial neural network (ANN) and Adaptive Neuro-Fuzzy Inference System (ANFIS), and their predictive capacities were then compared. The results showed that the ANN model outperformed the RSM and ANFIS models regarding prediction. ANN models and multi-objective genetic algorithm (MOGA) are used to optimize the Hasofer-Lind reliability index. The analysis is then carried out by taking into account the various types of functions of parameter distributions, which allowed us to better appreciate the effects of the uncertainties and identify the set of parameters with a high incidence.