Representative Network Models Research Articles

Density Evolution for Noise Propagation Analysis in Biological Networks

Accurate prediction of noise propagation in biological networks is key to understanding faithful signal propagation in gene networks as well as for designing noise-tolerant synthetic gene circuits. Knowledge on how biological fluctuations propagate up the development ladder of biological systems is currently lacking. Similarly, little research effort has been devoted to the analysis of error propagation in biological networks. To capture and characterize error evolution, this paper considers a Boolean network (BN) model representation of a biological network such that nodes on the graph represent diverse biological entities, e.g., proteins, genes, messenger-RNAs, etc. In addition, the network edges capture the interactions between nodes. By conducting a density evolution analysis on the graphical model based on node functionalities, a recursive closed-form expression for error propagation is derived. Subsequently, the recursive equation allows us to obtain a necessary condition to guarantee noise-error elimination in dynamic discrete gene networks. Our analytical formulations provide a step toward achieving optimal network parameters for resilience against variability or noise in biology.

Anchored User Selection for Traffic Offloading Optimization in D2D-Aided Mobile-Edge Computing

Recently, integrated with the advanced communication technologies (e.g., 5G) and artificial intelligence (AI), mobile-edge intelligence (MEI) is regarded as the promising method to deal with the emerging challenges. Specifically, Device-to-Device (D2D) communications have been put forward to reduce the traffic pressure while extending cellular network capacity. However, the stability of the social network is important for the design of efficient and reliable traffic offloading strategy, which is often absent from the related work. Besides, most existing studies merely model the relation between a node pair as a binary or continuous value, neglecting the rich information between users. Moreover, many traditional models are conducted based on small-scale data sets or online Internet services, severely confining their applications in the D2D scenario. Thus, it is necessary to understand the network structure and select the key users to address the aforementioned challenges. In this article, we first propose a network representation model, named MPPT, to regard the multidimensional relations as a probability in a third-order (3-D) tensor space. Then, a mobile D2D social community is derived by integrating an edge base station (BS) and the nearby D2D users, and develop an anchored user selection algorithm to maintain the stability of multiple D2D social communities by choosing and retaining critical users adaptively under the limited network resources. Finally, we devise a probability-based onion layers anchored <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$(k,r)$ </tex-math></inline-formula> -core (P-OLAK) algorithm to identify the anchor users. The large-scale data sets-based experimental results show the superiorities of the proposed methods.

Determining Local Transport Properties in Gas Diffusion Layers Using Pore Network Modelling

Understanding the relationship between local liquid water distribution in gas diffusion layers (GDLs) and transport properties in polymer electrolyte fuel cells can lead to improved GDL and flow-field channel configurations for improved performance. Modelling studies have shown that transport properties can spatially vary in the through-plane direction and under different regions of the flow-field [1], [2]. Further modelling studies may unveil unique insights on spatially varying properties of the GDL at different operating conditions.In this study, x-ray synchrotron imaging was used in conjunction with pore network modelling to reveal transport properties local to the GDL in the land and channel regions. First, liquid water distribution within the GDL was obtained using in operando x-ray synchrotron imaging. Micro computed tomography images of the GDL were obtained and used to generate representative pore network models of the materials. Pore network models were partially invaded from stochastically selected inlet pores using invasion percolation to match the average experimental saturation of land, channel, and overall regions. Next, we performed transport simulations on the partially saturated pore networks to determine the diffusivity, permeability, and oxygen transport resistance of each GDL region. Oxygen transport resistance values determined from the simulations were compared to experimental values obtained via limiting current experiments to isolate the contribution of the GDL to transport resistance at limiting current.The methodology presented in this work can be used to evaluate transport properties of various GDL designs. The local transport properties obtained in this work will be incorporated into computational fluids dynamics modelling to improve simulation accuracy.[1] P. A. García-Salaberri, J. T. Gostick, G. Hwang, A. Z. Weber, and M. Vera, “Effective diffusivity in partially-saturated carbon-fiber gas diffusion layers: Effect of local saturation and application to macroscopic continuum models,” Journal of Power Sources, vol. 296, pp. 440–453, 2015, doi: 10.1016/j.jpowsour.2015.07.034.[2] N. Ge et al., “Resolving the gas diffusion layer substrate land and channel region contributions to the oxygen transport resistance of a partially-saturated substrate,” Electrochimica Acta, p. 135001, 2019, doi: 10.1016/j.electacta.2019.135001.

Fault-tolerant neural network training framework based on client-server

In order to realize low power consumption and real-time inference, AIoT devices have been applied in many fields of deep learning in recent years. However, some manufacturing processes cause some soft errors on AIOT devices in inference. For a neural network accelerator with a large amount of computation, it may lead to a large amount of computing error and a huge loss of prediction accuracy, which is intolerable for precision-sensitive applications such as autonomous drones. However, conventional fault tolerance techniques such as triple modular redundancy can incur considerable power consumption and performance penalty. In this paper, a client-server collaborative fault-tolerant neural network training framework is proposed. In the training, an AIoT processor with soft errors is used as the client, and the server learns the on-site computing errors with the application data of AIoT processor. Several representative neural network models were selected in the experiment. Compared with the off-line training model, the model trained by this method increases the top5 accuracy of the neural network by an average of 2.8%.

GEOGRAPHIC RATEMAKING WITH SPATIAL EMBEDDINGS

AbstractSpatial data are a rich source of information for actuarial applications: knowledge of a risk’s location could improve an insurance company’s ratemaking, reserving or risk management processes. Relying on historical geolocated loss data is problematic for areas where it is limited or unavailable. In this paper, we construct spatial embeddings within a complex convolutional neural network representation model using external census data and use them as inputs to a simple predictive model. Compared to spatial interpolation models, our approach leads to smaller predictive bias and reduced variance in most situations. This method also enables us to generate rates in territories with no historical experience.

Reinforced Neighbour Feature Fusion Object Detection with Deep Learning

Neural networks have enabled state-of-the-art approaches to achieve incredible results on computer vision tasks such as object detection. However, previous works have tried to improve the performance in various object detection necks but have failed to extract features efficiently. To solve the insufficient features of objects, this work introduces some of the most advanced and representative network models based on the Faster R-CNN architecture, such as Libra R-CNN, Grid R-CNN, guided anchoring, and GRoIE. We observed the performance of Neighbour Feature Pyramid Network (NFPN) fusion, ResNet Region of Interest Feature Extraction (ResRoIE) and the Recursive Feature Pyramid (RFP) architecture at different scales of precision when these components were used in place of the corresponding original members in various networks obtained on the MS COCO dataset. Compared to the experimental results after replacing the neck and RoIE parts of these models with our Reinforced Neighbour Feature Fusion (RNFF) model, the average precision (AP) is increased by 3.2 percentage points concerning the performance of the baseline network.

IMPACT OF SYSTEM STRENGTH AND HVDC CONTROL STRATEGIES ON DISTANCE PROTECTION PERFORMANCE

This paper presents comprehensive studies and tests for evaluating the impact of reduced system strength and different control strategies used by HVDC systems on the performance of distance protection. A Hardware-In-the-Loop (HIL) test setup is established to enable realistic testing of physical relays being used in the system, where simulated voltage and current waveforms are injected into the distance protection relay via an analogue amplifier, and the relay tripping signal is fed back to simulation and recorded for protection performance analysis. In the simulation, a reduced but representative transmission network model, which includes a Modular Multilevel Converter (MMC) based HVDC system, a synchronous condenser (SC), and a two-level converter representing non-synchronous generation (NSG), is developed in RSCAD for the RTDS simulator. The model can be flexibly configured to reflect different levels of system strength and synchronous compensation applied at the HVDC site. The HVDC system is implemented with a flexible controller, which can replicate typically used control strategies during faults (e.g. balanced current mode to eliminate negative sequence current, and constant active and reactive power modes to suppress the oscillations on the active and reactive power respectively), allowing the user to inject different levels of negative sequence current. From the studies, it was found that with decreased system strength, the impact of the HVDC system on the distance protection becomes apparent, i.e. protection performance could be compromised with delayed operation, and such impact, to some extent, is subject to the control strategies applied in the HVDC system. It was also observed that the installation of SC could facilitate the protection response, and such support is dependent on the SC capacity.

A physics-informed deep learning paradigm for car-following models

Car-following behavior has been extensively studied using physics-based models, such as Intelligent Driving Model (IDM). These models successfully interpret traffic phenomena observed in the real world but may not fully capture the complex cognitive process of driving. Deep learning models, on the other hand, have demonstrated their power in capturing observed traffic phenomena but require a large amount of driving data to train. This paper aims to develop a family of neural network based car-following models that are informed by physics-based models, which leverage the advantage of both physics-based (being data-efficient and interpretable) and deep learning based (being generalizable) models. We design physics-informed deep learning car-following model (PIDL-CF) architectures encoded with 4 popular physics-based models - the IDM, the Optimal Velocity Model, the Gazis-Herman-Rothery model, and the Full Velocity Difference Model. Acceleration is predicted for 4 traffic regimes: acceleration, deceleration, cruising, and emergency braking. The generalization of PIDL method is further validated using two representative neural network models: the artificial neural networks (ANN) and the long short-term memory (LSTM) model. Two types of PIDL-CF problems are studied, one to predict acceleration only and the other to jointly predict acceleration and discover model parameters. We also demonstrate the superior performance of PIDL with the Next Generation SIMulation (NGSIM) dataset over baselines, especially when the training data is sparse. The results demonstrate the superior performance of neural networks informed by physics over those without.

The Barron Space and the Flow-Induced Function Spaces for Neural Network Models

One of the key issues in the analysis of machine learning models is to identify the appropriate function space and norm for the model. This is the set of functions endowed with a quantity which can control the approximation and estimation errors by a particular machine learning model. In this paper, we address this issue for two representative neural network models: the two-layer networks and the residual neural networks. We define the Barron space and show that it is the right space for two-layer neural network models in the sense that optimal direct and inverse approximation theorems hold for functions in the Barron space. For residual neural network models, we construct the so-called flow-induced function space and prove direct and inverse approximation theorems for this space. In addition, we show that the Rademacher complexity for bounded sets under these norms has the optimal upper bounds.

A Scalable Redefined Stochastic Blockmodel

Stochastic blockmodel (SBM) is a widely used statistical network representation model, with good interpretability, expressiveness, generalization, and flexibility, which has become prevalent and important in the field of network science over the last years. However, learning an optimal SBM for a given network is an NP-hard problem. This results in significant limitations when it comes to applications of SBMs in large-scale networks, because of the significant computational overhead of existing SBM models, as well as their learning methods. Reducing the cost of SBM learning and making it scalable for handling large-scale networks, while maintaining the good theoretical properties of SBM, remains an unresolved problem. In this work, we address this challenging task from a novel perspective of model redefinition. We propose a novel redefined SBM with Poisson distribution and its block-wise learning algorithm that can efficiently analyse large-scale networks. Extensive validation conducted on both artificial and real-world data shows that our proposed method significantly outperforms the state-of-the-art methods in terms of a reasonable trade-off between accuracy and scalability. 1

Neuro-Symbolic Integration of Hopfield Neural Network for Optimal Maximum Random kSatisfiability (Maxrksat) Representation

Boolean satisfiability logical representation is a programming paradigm that has its foundations in mathematical logic. It has been classified as an NP-complete problem that difficult practical combinatorial optimization and search problems can be easily converted into it. Random Maximum kSatisfiability (MAX-RkSAT) composed of the most consistent mapping in a Boolean formula that generates a maximum number of random satisfied clauses. Many optimization and search problems can be easily expressed by mapping the problem into a Hopfield neural network (HNN) to minimize the optimal configuration of the corresponding Lyapunov energy function. In this paper, a hybrid computational model hs been proposed that incorporates the Random Maximum kSatisfiability (MAX-RkSAT) into the Hopfield neural network (HNN) for optimal Random Maximum kSatisfiability representation (HNN-MAX-RkSAT). Hopfield neural network learning will be integrated with the random maximum satisfiability to enhance the correct neural state of the network model representation. The computer simulation using C++ has been used to demonstrate the ability of MAX-RkSAT to be embedded optimally in Hopfield neural network to serve as Neuro-symbolic integration. The performance of the proposed hybrid HNN-MAXRkSAT model has been explored and compared with the existing model. The proposed HNN-MAXRkSAT demonstrates good agreement with the existing models measured in terms of Global minimum Ratio (Gm), Hamming Distance (HD), Mean Absolute Error (MAE) and network computation Time CPU time). The proposed framework explored that MAX-RkSAT can be optimally represented in HNN and subsequently provides an additional platform for neural-symbolic integration, representing the various types of satisfiability logic.

Classification method based on the deep structure and least squares support vector machine

Support vector machines (SVMs) are one of the most representative shallow network models and have good generalisation abilities in small data sets. In this Letter, a new classification method based on the deep structure and least squares SVM (LSSVM) is proposed. For large-scale data sets, the method builds the structures of a multi-layer SVM. Using edge detection and the K-means algorithm, the sample set is compressed into a smaller sample set, which is used to train the LSSVM model of each layer and the discriminant classification function is obtained. Finally, this method is applied to UCI data sets and compared with several density-dependent quantised LSSVM methods and other methods. The experimental results show that the method has good performance in solving the large-scale data set classification problem.

Fairness in Network Representation by Latent Structural Heterogeneity in Observational Data

While recent advances in machine learning put many focuses on fairness of algorithmic decision making, topics about fairness of representation, especially fairness of network representation, are still underexplored. Network representation learning learns a function mapping nodes to low-dimensional vectors. Structural properties, e.g. communities and roles, are preserved in the latent embedding space. In this paper, we argue that latent structural heterogeneity in the observational data could bias the classical network representation model. The unknown heterogeneous distribution across subgroups raises new challenges for fairness in machine learning. Pre-defined groups with sensitive attributes cannot properly tackle the potential unfairness of network representation. We propose a method which can automatically discover subgroups which are unfairly treated by the network representation model. The fairness measure we propose can evaluate complex targets with multi-degree interactions. We conduct randomly controlled experiments on synthetic datasets and verify our methods on real-world datasets. Both quantitative and quantitative results show that our method is effective to recover the fairness of network representations. Our research draws insight on how structural heterogeneity across subgroups restricted by attributes would affect the fairness of network representation learning.

Multi-View Low-Rank Coding-Based Network Data De-Anonymization

Social networks are extensively exploited by third-party consumers such as researchers and advertisers to understand user characteristics and behaviors. In general, before network data is published, sensitive relationships should be anonymized to prevent the compromise of individual privacy. To quantify the guarantee level of privacy-preserving mechanisms and mitigate users’ privacy concerns, numerous studies concerning network data de-anonymization have been carried out. However, most existing studies focus on single-view data, and privacy protection for multi-view data that is ubiquitous in the era of big data has not been yet extensively explored. In this study, we are interested in answering the following question: Are the traditional privacy protection methods still valid for the anonymization of multi-view data? In this study, we propose a Multi-View Low-Rank Coding (MVLRC) based network data de-anonymization framework to assess the vulnerability of privacy protection techniques by accurately reconstructing a large portion of the original data. Specifically, the framework assumes that in principle, the target and auxiliary networks have common structural patterns, and they can be modeled together to infer the hidden structure of the target network. The essential components of our work include the following: (1) a robust network representation model for structural pattern learning; (2) the network representation based multi-view modeling of target network and auxiliary network; (3) the inference of the anonymized links via target network reconstruction. Experimental results on synthetic networks and three real-world networks demonstrate that auxiliary networks can be utilized by malicious adversaries for privacy inference attacks. Thus, the privacy protection of multi-view network data needs more sophisticated anonymization techniques.

Differences in the intrinsic spatial dynamics of the chromatin contribute to cell differentiation.

Advances in chromosome conformation capture techniques as well as computational characterization of genomic loci structural dynamics open new opportunities for exploring the mechanistic aspects of genome-scale differences across different cell types. We examined here the dynamic basis of variabilities between different cell types by investigating their chromatin mobility profiles inferred from Hi-C data using an elastic network model representation of the chromatin. Our comparative analysis of sixteen cell lines reveals close similarities between chromosomal dynamics across different cell lines on a global scale, but notable cell-specific variations emerge in the detailed spatial mobilities of genomic loci. Closer examination reveals that the differences in spatial dynamics mainly originate from the difference in the frequencies of their intrinsically accessible modes of motion. Thus, even though the chromosomes of different types of cells have access to similar modes of collective movements, not all modes are deployed by all cells, such that the effective mobilities and cross-correlations of genomic loci are cell-type-specific. Comparison with RNA-seq expression data reveals a strong overlap between highly expressed genes and those distinguished by high mobilities in the present study, in support of the role of the intrinsic spatial dynamics of chromatin as a determinant of cell differentiation.

Prediction of Adverse Drug Reactions by Combining Biomedical Tripartite Network and Graph Representation Model.

As one of the primary contributors to high clinical attrition rates of drugs, toxicity evaluation is of critical significance to new drug discovery. Unsurprisingly, a vast number of computational methods have been developed at various stages of development pipeline to evaluate potential adverse drug reactions (ADRs). Despite previous success of these methods on individual ADR or certain drug family, there are great challenges to toxicity evaluation. In this study, a novel strategy was developed to predict the drug-ADR associations by combining deep learning and the biomedical tripartite network. This heterogeneous network contains biomedical linked data of three entities, for example, drugs, targets, and ADRs. For the first time, GraRep, a deep learning method for distributed representations, is introduced to learn graph representations and identify hidden features from the tripartite network which are further used for ADR prediction. Through this approach, drug-ADR associations could possibly be discovered from a systemic perspective. The accuracy of our method is 0.95 based on internal resource validation and 0.88 based on external resource validation. Moreover, our results show the prediction accuracy using the tripartite network is better than the one with bipartite network, suggesting the model performance can be improved with further enrichment on information. According to the result of 10-fold cross validation, the deep learning model outperforms two traditional methods (topology-based measures and chemical structure-based measures). Additionally, predictive models are also constructed using other deep learning methods, and comparable results are achieved. In summary, the biomedical tripartite network-based deep learning model proposed here proves to offer a promising solution for prediction of ADRs.

A network representation method based on edge information extraction

In recent years, network representation learning has attracted extensive attention in the academic field due to its significant application potential. However, most of the methods cannot explore edge information in the network deeply, resulting in poor performance at downstream tasks such as classification, clustering and link prediction. In order to solve this problem, we propose a novel way to extract network information. First, the original network is transformed into an edge network with structure and edge information. Then, edge representation vectors can be obtained directly by using an existing network representation model with edge network as its input. Node representation vectors can also be obtained by utilizing the relationships between edges and nodes. Compared with the structure of original network, the edge network is denser, which can help solving the problems caused by sparseness. Extensive experiments on several real-world networks demonstrate that edge network outperforms original network in various graph mining tasks, i.e., node classification and node clustering.

Prediction model for patients with acute respiratory distress syndrome: use of a genetic algorithm to develop a neural network model.

BackgroundAcute respiratory distress syndrome (ARDS) is associated with significantly increased risk of death, and early risk stratification may help to choose the appropriate treatment. The study aimed to develop a neural network model by using a genetic algorithm (GA) for the prediction of mortality in patients with ARDS.MethodsThis was a secondary analysis of two multicenter randomized controlled trials conducted in forty-four hospitals that are members of the National Heart, Lung, and Blood Institute, founded to create an acute respiratory distress syndrome Clinical Trials Network. Model training and validation were performed using the SAILS and OMEGA studies, respectively. A GA was employed to screen variables in order to predict 90-day mortality, and a neural network model was trained for the prediction. This machine learning model was compared to the logistic regression model and APACHE III score in the validation cohort.ResultsA total number of 1,071 ARDS patients were included for analysis. The GA search identified seven important variables, which were age, AIDS, leukemia, metastatic tumor, hepatic failure, lowest albumin, and FiO2. A representative neural network model was constructed using the forward selection procedure. The area under the curve (AUC) of the neural network model evaluated with the validation cohort was 0.821 (95% CI [0.753–0.888]), which was greater than the APACHE III score (0.665; 95% CI [0.590–0.739]; p = 0.002 by Delong’s test) and logistic regression model, albeit not statistically significant (0.743; 95% CI [0.669–0.817], p = 0.130 by Delong’s test).ConclusionsThe study developed a neural network model using a GA, which outperformed conventional scoring systems for the prediction of mortality in ARDS patients.

Impact of Non-Ideal Characteristics of Resistive Synaptic Devices on Implementing Convolutional Neural Networks

Emerging non-volatile memory (eNVM) based resistive synaptic devices have shown great potential for implementing deep neural networks (DNNs). However, the eNVM devices typically suffer from various non-ideal effects which may degrade the performance of the system. Based on a representative convolutional neural network (CNN) model for CIFAR-10 dataset, this paper comprehensively investigates the impact of those non-ideal characteristics, such as nonlinearity and asymmetry of conductance tuning, variations, endurance and retention, on the training/inference accuracy. The compact models of the device non-ideal effects are incorporated into the TensorFlow framework. Our simulation results suggest that 1) the training accuracy is more sensitive to the asymmetry of conductance tuning than the nonlinearity; 2) the conductance range variation does not degrade the training accuracy, instead, a small variation can even reduce the accuracy loss introduced by asymmetry; 3) device-to-device variation can also remedy the accuracy loss due to asymmetry while cycle-to-cycle variation leads to significant accuracy degradation; 4) the accuracy degradation will not be noticeable if the endurance cycles are more than 7,000 cycles; and 5) different drifting modes affect the inference accuracy differently, and the best case is where the conductance is drifting up/down randomly.

Pore network modelling of fluid flow in tight formations considering boundary layer effect and media deformation

Fluid flow in tight formations deviates significantly from that in conventional reservoirs due to the micro/nano pores and high confining pressure; non-linear flow occurs at low displacement pressure gradients. The effects of boundary layer and media deformation on fluid flow cannot be ignored. In this paper, a stochastic network model with anisotropic topology is firstly constructed to represent the complex structures of tight formations. Subsequently, a series of network models with different physical scales are generated to obtain the representative elementary volume (REV). Based on the representative network model, a pore network flow model coupling the boundary layer effect with media deformation is developed. Iterative methods are proposed to solve the strong nonlinear mathematical model. Microtube experiments with different radii and three corresponding representative networks with different distributions of throat radii are employed to verify the accuracy and reliability of the numerical results. After the validation, pore scale flow regularity considering the boundary layer effect and media deformation of tight formations is obtained. Furthermore, the factors influencing pore scale flow, such as the effective stress, aspect ratio, coordination number and fluid viscosity, are studied. The results show that pore scale flow velocity considering the boundary layer effect and media deformation is smaller than that without considering the two mechanisms. Nonlinear flow occurs at low displacement pressure gradients because of the boundary layer effect, and media deformation aggravates the nonlinear phenomenon. Similarly, the absolute permeability also decreases when the two mechanisms are considered. It is no longer constant when boundary layer effect is taken into consideration, which increases with increasing displacement pressure gradient and reaches a plateau when the displacement pressure gradient is large enough. The effects of different factors on pore scale flow considering boundary layer effect and media deformation differ from their roles in conventional flow network models. Although the absolute permeability increases with decreasing aspect ratio and increasing coordination number, the value is far smaller than that without considering the two mechanisms because they result in the compression of the effective flow space. Additionally, the absolute permeability no longer remains unchanged when the variation of fluid viscosity is included. This indicates that the absolute permeability considering the boundary layer effect is not only affected by the pore structures but also the fluid viscosity. The methodology developed in this work provides a novel way to study the effects of boundary layer and media deformation on pore scale flow. The results provide new insights into fluid flow in tight formations.