Large Network Models Research Articles

The research of multi-target tracking based on improved YOLOv3

Multi-target tracking based on detection is a hot topic in the field of computer vision. Aiming at solving the problem of inconvenient application of large network model, this paper proposes a multi-target tracking algorithm based on improved YOLOv3. We use MobileNetV3 to replace the deep network as the backbone of YOLOv3. Compared with Faster R-CNN and YOLOv3, the model of MobileNetV3-YOLOv3 is greatly reduced and the detection speed is improved. Compared with the MobileNetV1-YOLOv3, the accuracy is improved and FPS is higher. The improved YOLOv3 model is used in DeepSORT algorithm for multi-target tracking. The experimental results show that the algorithm used in this paper has the best detecting and tracking effect.

HetSeq: Distributed GPU Training on Heterogeneous Infrastructure

Modern deep learning systems like PyTorch and Tensorflow are able to train enormous models with billions (or trillions) of parameters on a distributed infrastructure. These systems require that the internal nodes have the same memory capacity and compute performance. Unfortunately, most organizations, especially universities, have a piecemeal approach to purchasing computer systems resulting in a heterogeneous infrastructure, which cannot be used to compute large models. The present work describes HetSeq, a software package adapted from the popular PyTorch package that provides the capability to train large neural network models on heterogeneous infrastructure. Experiments with language translation, text and image classification shows that HetSeq scales over heterogeneous systems. Additional information, support documents, source code are publicly available at https://github.com/yifding/hetseq.

T-GAN: A deep learning framework for prediction of temporal complex networks with adaptive graph convolution and attention mechanism

Complex network is graph network with non-trivial topological features often occurring in real systems, such as video monitoring networks, social networks and sensor networks. While there is growing research study on complex networks, the main focus has been on the analysis and modeling of large networks with static topology. Predicting and control of temporal complex networks with evolving patterns are urgently needed but have been rarely studied. In view of the research gaps we are motivated to propose a novel end-to-end deep learning based network model, which is called temporal graph convolution and attention (T-GAN) for prediction of temporal complex networks. To joint extract both spatial and temporal features of complex networks, we design new adaptive graph convolution and integrate it with Long Short-Term Memory (LSTM) cells. An encoder-decoder framework is applied to achieve the objectives of predicting properties and trends of complex networks. And we proposed a dual attention block to improve the sensitivity of the model to different time slices. Our proposed T-GAN architecture is general and scalable, which can be used for a wide range of real applications. We demonstrate the applications of T-GAN to three prediction tasks for evolving complex networks, namely, node classification, feature forecasting and topology prediction over 6 open datasets. Our T-GAN based approach significantly outperforms the existing models, achieving improvement of more than 4.7% in recall and 25.1% in precision. Additional experiments are also conducted to show the generalization of the proposed model on learning the characteristic of time-series images. Extensive experiments demonstrate the effectiveness of T-GAN in learning spatial and temporal feature and predicting properties for complex networks.

RF-Based Human Activity Recognition Using Signal Adapted Convolutional Neural Network

Human activity recognition (HAR) plays a critical role in a wide range of real-world applications, and it is traditionally achieved via wearable sensing. Recently, to avoid the burden and discomfort caused by wearable devices, device-free approaches exploiting radio-frequency (RF) signals arise as a promising alternative for HAR. Most of the latest device-free approaches require training a large deep neural network model in either time or frequency domain, entailing extensive storage to contain the model and intensive computations to infer human activities. Consequently, even with some major advances on device-free HAR, current device-free approaches are still far from practical in real-world scenarios where the computation and storage resources possessed by, for example, edge devices, are limited. To overcome these weaknesses, we introduce HAR-SAnet which is a novel RF-based HAR framework. It adopts an original signal adapted convolutional neural network architecture: instead of feeding the handcraft features of RF signals into a classifier, HAR-SAnet fuses them adaptively from both time and frequency domains to design an end-to-end neural network model. We apply point-wise grouped convolution and depth-wise separable convolutions to confine the model scale and to speed up the inference execution time. The experiment results show that the recognition accuracy of HAR-SAnet substantially outperforms the state-of-the-art algorithms and systems.

Connectivity and routes in large geometric network hypergraphs

Random geometric hypergraphs are considered as mathematical models of large wireless computer networks. The dependences of the mathematical expectation of the number of hyper-edges in random geometric hypergraphs on the radii of reliable reception / transmission of radio signals by network nodes, as well as on the number of vertices in the hyper- graph are studied. The concepts of the shortest route in a geometric hypergraph are discussed. Calculations of the probabil- ity of connectivity of large random geometric hypergraphs, the mathematical expectation of the diameter of hypergraphs and its change with a change in the radii of the nodes are carried out. The presentation of the results is accompanied by graphs.

Larger GPU-accelerated brain simulations with procedural connectivity.

Simulations are an important tool for investigating brain function but large models are needed to faithfully reproduce the statistics and dynamics of brain activity. Simulating large spiking neural network models has, until now, needed so much memory for storing synaptic connections that it required high performance computer systems. Here, we present an alternative simulation method we call 'procedural connectivity' where connectivity and synaptic weights are generated 'on the fly' instead of stored and retrieved from memory. This method is particularly well suited for use on graphical processing units (GPUs)-which are a common fixture in many workstations. Using procedural connectivity and an additional GPU code generation optimization, we can simulate a recent model of the macaque visual cortex with 4.13 × 106 neurons and 24.2 × 109 synapses on a single GPU-a significant step forward in making large-scale brain modeling accessible to more researchers.

RRAM-DNN: An RRAM and Model-Compression Empowered All-Weights-On-Chip DNN Accelerator

This article presents an energy-efficient deep neural network (DNN) accelerator with non-volatile embedded resistive random access memory (RRAM) for mobile machine learning (ML) applications. This DNN accelerator implements weight pruning, non-linear quantization, and Huffman encoding to store all weights on RRAM, enabling single-chip processing for large neural network models without external memory. A four-core parallel and programmable architecture adapts to various neural network configurations with high utilization. We introduce a customized RRAM macro with a dynamic clamping offset-canceling sense amplifier (DCOCSA) that achieves sub-microampere input offset. The on-chip decompression and memory error-resilient scheme enables 16 million (M) 8-bit (decompressed) weights on a single-chip using 24 Mb RRAM. The proposed RRAM-DNN is the first digital DNN accelerator featuring 24 Mb RRAM as all-on-chip weight storage to eliminate energy-consuming off-chip memory accesses. The fabricated design performs the complete inference process of the ResNet-18 model while consuming 127.9 mW power in TSMC-22 nm ULL CMOS. The RRAM-DNN accelerator achieves peak performance of 123 GOPs with 8-bit precision, exhibiting measured energy efficiency of 0.96 TOPs/W.

A Knowledge Distillation-Based Transportation System for Sensory Data Sharing Using LoRa

Internet of Things (IoT) provides an adequate and effective approach to make an Intelligent Transportation System (ITS). The ITS uses sensory data to identify driver behaviour, transportation mode, pollution level, road health, and so on. The processing of sensory data in ITS requires high computation and storage, which is possible only on high-end machines. Moreover, sharing of sensory data in ITS also consumes colossal energy. In this article, we propose a Knowledge Distillation-based Transportation (KDT) system for sharing the sensory data based on the requirement of users. We use knowledge distillation technique for compressing a large deep neural network model. The compressed model is suitable for processing the data on limited resource devices in ITS. Next, we use Long-Range (LoRa) for sharing the data to the end-users. We also use game model for selecting the suitable deep learning model and LoRa resources, which helps to transfer the data form LoRa nodes to the end-users effectively. We evaluate the performance of KDT system by using different game and deep neural network parameters.

Bayesian indirect inference for models with intractable normalizing functions

Inference for doubly intractable distributions is challenging because the intractable normalizing functions of these models include parameters of interest. Previous auxiliary variable MCMC algorithms are infeasible for multi-dimensional models with large data sets because they depend on expensive auxiliary variable simulation. We develop a fast Bayesian indirect algorithm by replacing an expensive auxiliary variable simulation from a probability model with a computationally cheap simulation from a surrogate model. We learn the relationship between the surrogate model parameters and the probability model parameters using Gaussian process approximations. We apply our methods to challenging examples, and illustrate that the algorithm addresses both computational and inferential challenges for doubly intractable distributions. Especially for a large social network model with 10 parameters, we show that our method can reduce computing time from about 2 weeks to 5 hours, compared to the previous method.

Steady air flow model for large sewer networks: a theoretical framework.

Modelling air movement in sewer networks is needed in order to address the issues related to sewer odour complaints and sewer corrosions due to hydrogen sulphide in sewers. Most of the existing air flow models can only be applied in small sewer networks or the trunk lines of sewer systems. The purpose of this paper is therefore to propose a theoretical approach to formulate a general governing equation set for modelling steady air movement in large sewer systems. This approach decomposes the sewer system of interest into its basic physical components as pipes and nodes, and builds local topology of each pipe and each node based on geographic information system data as the fundamentals of model formulation. It avoids manually identifying each branch of the sewer system, eliminates the effect of physically closed networks in sewer systems on the governing equations, and considers key sewer components and all known driving forces. The proposed approach was applied to a real sewer system with over 500 pipes. The results show that the proposed model is applicable in modelling air movement in a large sewer system and provides a general idea of sewer gases moving through the system and their emission.

Surrogate-assisted parallel tempering for Bayesian neural learning

Due to the need for robust uncertainty quantification, Bayesian neural learning has gained attention in the era of deep learning and big data. Markov Chain Monte-Carlo (MCMC) methods typically implement Bayesian inference which faces several challenges given a large number of parameters, complex and multimodal posterior distributions, and computational complexity of large neural network models. Parallel tempering MCMC addresses some of these limitations given that they can sample multimodal posterior distributions and utilize high-performance computing. However, certain challenges remain given large neural network models and big data. Surrogate-assisted optimization features the estimation of an objective function for models which are computationally expensive. In this paper, we address the inefficiency of parallel tempering MCMC for large-scale problems by combining parallel computing features with surrogate assisted likelihood estimation that describes the plausibility of a model parameter value, given specific observed data. Hence, we present surrogate-assisted parallel tempering for Bayesian neural learning for simple to computationally expensive models. Our results demonstrate that the methodology significantly lowers the computational cost while maintaining quality in decision making with Bayesian neural networks. The method has applications for a Bayesian inversion and uncertainty quantification for a broad range of numerical models.

Biophysically grounded mean-field models of neural populations under electrical stimulation.

Electrical stimulation of neural systems is a key tool for understanding neural dynamics and ultimately for developing clinical treatments. Many applications of electrical stimulation affect large populations of neurons. However, computational models of large networks of spiking neurons are inherently hard to simulate and analyze. We evaluate a reduced mean-field model of excitatory and inhibitory adaptive exponential integrate-and-fire (AdEx) neurons which can be used to efficiently study the effects of electrical stimulation on large neural populations. The rich dynamical properties of this basic cortical model are described in detail and validated using large network simulations. Bifurcation diagrams reflecting the network's state reveal asynchronous up- and down-states, bistable regimes, and oscillatory regions corresponding to fast excitation-inhibition and slow excitation-adaptation feedback loops. The biophysical parameters of the AdEx neuron can be coupled to an electric field with realistic field strengths which then can be propagated up to the population description. We show how on the edge of bifurcation, direct electrical inputs cause network state transitions, such as turning on and off oscillations of the population rate. Oscillatory input can frequency-entrain and phase-lock endogenous oscillations. Relatively weak electric field strengths on the order of 1 V/m are able to produce these effects, indicating that field effects are strongly amplified in the network. The effects of time-varying external stimulation are well-predicted by the mean-field model, further underpinning the utility of low-dimensional neural mass models.

Ultrafast Video Attention Prediction with Coupled Knowledge Distillation

Large convolutional neural network models have recently demonstrated impressive performance on video attention prediction. Conventionally, these models are with intensive computation and large memory. To address these issues, we design an extremely light-weight network with ultrafast speed, named UVA-Net. The network is constructed based on depth-wise convolutions and takes low-resolution images as input. However, this straight-forward acceleration method will decrease performance dramatically. To this end, we propose a coupled knowledge distillation strategy to augment and train the network effectively. With this strategy, the model can further automatically discover and emphasize implicit useful cues contained in the data. Both spatial and temporal knowledge learned by the high-resolution complex teacher networks also can be distilled and transferred into the proposed low-resolution light-weight spatiotemporal network. Experimental results show that the performance of our model is comparable to 11 state-of-the-art models in video attention prediction, while it costs only 0.68 MB memory footprint, runs about 10,106 FPS on GPU and 404 FPS on CPU, which is 206 times faster than previous models.

A Neural Network Model Compression Approach Based on Deep Feature Map Transfer

Neural network is widely used in computer vision. However, with the continuous expansion of the application field, high-precision large parameter neural network model is difficult to deploy on small equipment with limited resources. In order to obtain a small but efficient network, the soft output of the teacher network was used to train students through the teacher-student structure. A new method of neural network model compression based on deep feature map transfer (DFMT) is proposed in this paper, which uses visual system characteristics adequately. A small decoder is designed in the network to generate a deep feature map from the features extracted by the network, and the feature map is used to transfer knowledge. In addition, cosine similarity is used as the evaluation index of knowledge transfer. A smaller model with better precision can be obtained by the proposed method. Experiments on benchmark datasets prove the validity and advancement of the proposed approach.

Multivariate spatial autoregressive model for large scale social networks

The rapid growth of social network platforms generates a large amount of social network data, where multivariate responses are frequently collected from users. To statistically model such type of data, the multivariate spatial autoregressive (MSAR) model is studied. To estimate the model, the quasi maximum likelihood estimator (QMLE) is obtained under certain technical conditions. However, it is found that the computational cost of QMLE is expensive. To fix this problem, a least squares estimator (LSE) is developed. The corresponding identification conditions and asymptotic properties are investigated. To gauge the finite sample performance of various estimators, a number of simulation studies are conducted. Lastly, a Sina Weibo dataset is analyzed for illustration purpose.

Identifying mechanistically distinct pathways in kinetic transition networks.

We present an implementation of a scalable path deviation algorithm to find the k most kinetically relevant paths in a transition network, where each path is distinguished on the basis of having a distinct rate-limiting edge. The potential of the algorithm to identify distinct pathways that exist in separate regions of the configuration space is demonstrated for two benchmark systems with double-funnel energy landscapes, namely a model "three-hole" network embedded on a 2D potential energy surface and the cluster of 38 Lennard-Jones atoms (LJ38). The path cost profiles for the interbasin transitions of the two systems reflect the contrasting nature of the landscapes. There are multiple well-defined pathway ensembles for the three-hole system, whereas the transition in LJ38 effectively involves a single ensemble of pathways via disordered structures. A by-product of the algorithm is a set of edges that constitute a cut of the network, which is related to the discrete analog of a transition dividing surface. The algorithm ought to be useful for determining the existence, or otherwise, of competing mechanisms in large stochastic network models of dynamical processes and for assessing the kinetic relevance of distinguishable ensembles of pathways. This capability will provide insight into conformational transitions in biomolecules and other complex slow processes.

Hub-based truck platooning: Potentials and profitability

This paper presents a model for optimising truck platoons formed at a platooning hub. Different planning and dispatching strategies, from static to dynamic, are investigated with respect to profitability and fuel savings across a range of input variables. The problem is solved using a dynamic programming based local search heuristic. As a case study, a virtual platooning hub close to the German Elb Tunnel is examined using data from a large European transport network model. It is concluded that profitability crucially depends on; (i) dynamic outlook and (ii) if chauffeurs are allowed to rest while driving in platoons.

Normal Approximation in Large Network Models

We develop a methodology for proving central limit theorems in network models with strategic interactions and homophilous agents. We consider an asymptotic framework in which the size of the network tends to infinity, which is useful for inference in the typical setting in which the sample consists of a single large network. In the presence of strategic interactions, network moments are generally complex functions of network components, where a node's component consists of all alters to which it is directly or indirectly connected. We find that a modification of exponential conditions from the stochastic geometry literature provides a useful formulation of weak dependence for moments of this type. Our first contribution is to prove a CLT for a large class of network moments satisfying stabilization and a moment condition. Our second contribution is a methodology for deriving primitive sufficient conditions for stabilization using results in branching process theory. We apply the methodology to static and dynamic models of network formation and discuss how it can be used more broadly.

Computation of Robust Minimal Intervention Sets in Multi-Valued Biological Regulatory Networks.

Enabled by rapid advances in computational sciences, in silico logical modeling of complex and large biological networks is more and more feasible making it an increasingly popular approach among biologists. Automated high-throughput, drug target identification is one of the primary goals of this in silico network biology. Targets identified in this way are then used to mine a library of drug chemical compounds in order to identify appropriate therapies. While identification of drug targets is exhaustively feasible on small networks, it remains computationally difficult on moderate and larger models. Moreover, there are several important constraints such as off-target effects, efficacy and safety that should be integrated into the identification of targets if the intention is translation to the clinical space. Here we introduce numerical constraints whereby efficacy is represented by efficiency in response and robustness of outcome. This paper introduces an algorithm that relies on a Constraint Satisfaction (CS) technique to efficiently compute the Minimal Intervention Sets (MIS) within a set of often complex clinical safety constraints with the aim of identifying the smallest least invasive set of targets pharmacologically accessible for therapy that most efficiently and reliably achieve the desired outcome.

Two-dimensional macroscopic model for large scale traffic networks

In this article we introduce a new traffic flow model for a dense urban area. We consider a two-dimensional conservation law in which the velocity magnitude is given by the fundamental diagram and the velocity direction is constructed following the network geometry and assuming we do not have precise information of drivers trajectories. We validate the model using synthetic data from Aimsun and propose a reconstruction technique to recover the 2D density from the data of individual vehicles. A comparison between the model and the data is shown.