Finite-size Networks Research Articles

An Alternative Learning-Based Approach for Economic Dispatch in Smart Grid

This article tries to provide a new alternative approach to solve the economic dispatch (ED) problem in a smart grid system. Such a problem has been widely studied recently with several advanced numerical optimization algorithms being proposed. However, most of these numerical algorithms may suffer from high computational cost for on-line optimization. In this article, we aim to address this problem by proposing a learning-based optimization strategy. The key idea is to regard the optimization strategy of the ED problem as an unknown mapping relationship. With the help of traditional ED optimization algorithms to obtain the ground truth, we employ a deep neural network (DNN) to learn the ED optimization strategy and use it for online ED. In particular, our main contribution in this article is to theoretically show that one popular ED algorithm, i.e., <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\lambda $ </tex-math></inline-formula> -iteration algorithm, can be accurately approximated by a well-constructed DNN with finite network size. Moreover, dynamic units status of dispatchable generators is also considered and can be well solved by our proposed approach. Furthermore, several simulation case studies implemented on a 3-unit power system and an IEEE-30 bus power system validate the effectiveness of our proposed method.

Power law decay of stored pattern stability in sparse Hopfield neural networks

Hopfield neural networks on scale-free networks display the power law relation between the stability of patterns and the number of patterns. The stability is measured by the overlap between the output state and the stored pattern which is presented to a neural network. In simulations the overlap declines to a constant by a power law decay. Here we provide the explanation for the power law behavior through the signal-to-noise ratio analysis. We show that on sparse networks storing a plenty of patterns the stability of stored patterns can be approached by a power law function with the exponent −0.5. There is a difference between analytic and simulation results that the analytic results of overlap decay to 0. The difference exists because the signal and noise term of nodes diverge from the mean-field approach in the sparse finite size networks.

Power-law distribution of degree–degree distance: A better representation of the scale-free property of complex networks

Whether real-world complex networks are scale free or not has long been controversial. Recently, in Broido and Clauset [A. D. Broido, A. Clauset, Nat. Commun. 10, 1017 (2019)], it was claimed that the degree distributions of real-world networks are rarely power law under statistical tests. Here, we attempt to address this issue by defining a fundamental property possessed by each link, the degree-degree distance, the distribution of which also shows signs of being power law by our empirical study. Surprisingly, although full-range statistical tests show that degree distributions are not often power law in real-world networks, we find that in more than half of the cases the degree-degree distance distributions can still be described by power laws. To explain these findings, we introduce a bidirectional preferential selection model where the link configuration is a randomly weighted, two-way selection process. The model does not always produce solid power-law distributions but predicts that the degree-degree distance distribution exhibits stronger power-law behavior than the degree distribution of a finite-size network, especially when the network is dense. We test the strength of our model and its predictive power by examining how real-world networks evolve into an overly dense stage and how the corresponding distributions change. We propose that being scale free is a property of a complex network that should be determined by its underlying mechanism (e.g., preferential attachment) rather than by apparent distribution statistics of finite size. We thus conclude that the degree-degree distance distribution better represents the scale-free property of a complex network.

Modeling and Estimating User Influence in Social Networks

It is of importance to model and estimate the user influence in social networks, especially for advertisers who conduct viral marketing. In this paper, we are interested in the number of received messages incurred by a node generating a message, and introduce the concepts of individual influence and type influence, while type influence is got by averaging out individual influence over nodes of the same type. We propose a user behavior model and use generating function to analyze type influence (including the mean and variance) and diffusion threshold, and find these results are not accurate in finite-size networks. We then classify nodes into subtypes and redefine the network model, which achieves much more accurate results. We also propose a scalable approach to estimate individual influence, and find it can get good approximates for individual influence, subtype influence and type influence by only considering local neighbors and out-of-date information, which is useful in large-scale networks. All analysis results are verified by simulations in real-world networks. Models in this paper can be extended to consider more realistic situations, and we believe these results are of use in understanding the diffusion dynamics in social networks.

Noise-induced macroscopic oscillations in a network of synaptically coupled quadratic integrate-and-fire neurons.

We consider the effect of small independent local noise on a network of quadratic integrate-and-fire neurons, globally coupled via synaptic pulses of finite width. The Fokker-Planck equation for a network of infinite size is reduced to a low-dimensional system of ordinary differential equations using the recently proposed perturbation theory based on circular cumulants. A bifurcation analysis of the reduced equations is performed, and areas in the parameter space, where the noise causes macroscopic oscillations of the network, are determined. The validity of the reduced equations is verified by comparing their solutions with "exact" solutions of the Fokker-Planck equation, as well as with the results of direct simulation of stochastic microscopic dynamics of a finite-size network.

Distributed Uplink Power Control in an Ultra-Dense Millimeter Wave Network: A Mean-Field Game Approach

In this letter, a novel mean-field game (MFG) framework is proposed for uplink power control in an ultra-dense millimeter wave network. The proposed MFG considers the time evolution of the mobile users’ (MUs) orientations as well as the energy available in their batteries, under adaptive user association. The objective of each MU is then to find the optimal transmission power that maximizes its energy efficiency. The expression of the energy efficiency is analytically derived for the realistic case of a finite size network. Simulation results show that the proposed approach yields gains of up to 24%, in terms of energy efficiency, compared to a baseline in which the nodes transmit according to the path loss compensating power control policy.

Localization and non-ergodicity in clustered random networks

Abstract We consider clustering in rewired Erdős–Rényi networks with conserved vertex degree and in random regular graphs from the localization perspective. It has been found in Avetisov et al. (2016, Phys. Rev. E, 94, 062313) that at some critical value of chemical potential $\mu_{\rm cr}$ of closed triad of bonds, the evolving networks decay into the maximally possible number of dense subgraphs. The adjacency matrix acquires above $\mu_{\rm cr}$ the two-zonal support with the triangle-shaped main (perturbative) zone separated by a wide gap from the side (non-perturbative) zone. Studying the distribution of gaps between neighbouring eigenvalues (the level spacing), we demonstrate that in the main zone the level spacing matches the Wigner–Dyson law and is delocalized, however it shares the Poisson statistics in the side zone, which is the signature of localization. In parallel with the evolutionary designed networks, we consider ‘instantly’ ad hoc prepared networks with in- and cross-cluster probabilities exactly as at the final stage of the evolutionary designed network. For such ‘instant’ networks the eigenvalues are delocalized in both zones. We speculate about the difference in eigenvalue statistics between ‘evolutionary’ and ‘instant’ networks from the perspective of a possible phase transition between ergodic and non-ergodic network patterns with a strong ‘memory dependence’, thus advocating possible existence of non-ergodic delocalized states in the clustered random networks at least at finite network sizes.

Stability of stochastic finite-size spiking-neuron networks: Comparing mean-field, 1-loop correction and quasi-renewal approximations.

We examine the stability and qualitative dynamics of stochastic neuronal networks specified as multivariate non-linear Hawkes processes and related point-process generalized linear models that incorporate both auto- and cross-history effects. In particular, we adapt previous theoretical approximations based on mean field and mean field plus 1-loop correction to incorporate absolute refractory periods and other auto-history effects. Furthermore, we extend previous quasi-renewal approximations to the multivariate case, i.e. neuronal networks. The best sensitivity and specificity performance, in terms of predicting stability and divergence to nonphysiologically high firing rates in the examined simulations, was obtained by a variant of the quasi-renewal approximation.

A Geometrical Analysis of Global Stability in Trained Feedback Networks.

Recurrent neural networks have been extensively studied in the context of neuroscience and machine learning due to their ability to implement complex computations. While substantial progress in designing effective learning algorithms has been achieved, a full understanding of trained recurrent networks is still lacking. Specifically, the mechanisms that allow computations to emerge from the underlying recurrent dynamics are largely unknown. Here we focus on a simple yet underexplored computational setup: a feedback architecture trained to associate a stationary output to a stationary input. As a starting point, we derive an approximate analytical description of global dynamics in trained networks, which assumes uncorrelated connectivity weights in the feedback and in the random bulk. The resulting mean-field theory suggests that the task admits several classes of solutions, which imply different stability properties. Different classes are characterized in terms of the geometrical arrangement of the readout with respect to the input vectors, defined in the high-dimensional space spanned by the network population. We find that such an approximate theoretical approach can be used to understand how standard training techniques implement the input-output task in finite-size feedback networks. In particular, our simplified description captures the local and the global stability properties of the target solution, and thus predicts training performance.

Optimized brute-force algorithms for the bifurcation analysis of a binary neural network model.

Bifurcation theory is a powerful tool for studying how the dynamics of a neural network model depends on its underlying neurophysiological parameters. However, bifurcation theory of neural networks has been developed mostly for mean-field limits of infinite-size spin-glass models, for finite-size dynamical systems whose units have a graded, continuous output, and for models with discrete-output neurons that evolve in continuous time. To allow progress on understanding the dynamics of some widely used classes of neural network models with discrete units and finite size, which could not be studied thoroughly with the previous methodology, here we introduced algorithms that perform a semianalytical bifurcation analysis of a finite-size firing-rate neural network model with binary firing rates and discrete-time evolution. In particular, we focus on the case of small networks composed of tens of neurons, to which existing statistical methods are not applicable. Our approach is based on a numerical brute-force search of the stationary and oscillatory solutions of the model, from which we derive analytical expressions of its bifurcation structure by means of the state-to-state transition probability matrix. Our algorithms determine how the network parameters affect the degree of multistability, the emergence and the period of the neural oscillations, and the formation of spontaneous symmetry breaking in the neural populations. While this technique can be applied to networks with arbitrary (generally asymmetric) connectivity matrices, in particular we introduce a highly efficient algorithm for the bifurcation analysis of sparse networks. We also provide some examples of the obtained bifurcation diagrams and a python implementation of the algorithms.

Mathematical studies of the dynamics of finite-size binary neural networks: A review of recent progress.

Several mathematical approaches to studying analytically the dynamics of neural networks rely on mean-field approximations, which are rigorously applicable only to networks of infinite size. However, all existing real biological networks have finite size, and many of them, such as microscopic circuits in invertebrates, are composed only of a few tens of neurons. Thus, it is important to be able to extend to small-size networks our ability to study analytically neural dynamics. Analytical solutions of the dynamics of small-size neural networks have remained elusive for many decades, because the powerful methods of statistical analysis, such as the central limit theorem and the law of large numbers, do not apply to small networks. In this article, we critically review recent progress on the study of the dynamics of small networks composed of binary neurons. In particular, we review the mathematical techniques we developed for studying the bifurcations of the network dynamics, the dualism between neural activity and membrane potentials, cross-neuron correlations, and pattern storage in stochastic networks. Then, we compare our results with existing mathematical techniques for studying networks composed of a finite number of neurons. Finally, we highlight key challenges that remain open, future directions for further progress, and possible implications of our results for neuroscience.

Consistent Tomography Under Partial Observations Over Adaptive Networks

This paper studies the problem of inferring whether an agent is directly influenced by another agent over a network. Agent $i$ influences agent $j$ if they are connected (according to the network topology), and if agent $j$ uses the data from agent $i$ to update its online learning algorithm. The solution of this inference task is challenging for two main reasons. First, only the output of the learning algorithm is available to the external observer that must perform the inference based on these indirect measurements. Second, only output measurements from a fraction of the network agents is available, with the total number of agents itself being also unknown. The main focus of this paper is ascertaining under these demanding conditions whether consistent tomography is possible , namely, whether it is possible to reconstruct the interaction profile of the observable portion of the network, with negligible error as the network size increases. We establish a critical achievability result, namely, that for symmetric combination policies and for any given fraction of observable agents, the interacting and non-interacting agent pairs split into two separate clusters as the network size increases. This remarkable property then enables the application of clustering algorithms to identify the interacting agents influencing the observations. We provide a set of numerical experiments that verify the results for finite network sizes and time horizons. The numerical experiments show that the results hold for asymmetric combination policies as well, which is particularly relevant in the context of causation .

Model reduction for Kuramoto models with complex topologies.

Synchronization of coupled oscillators is a ubiquitous phenomenon, occurring in topics ranging from biology and physics to social networks and technology. A fundamental and long-time goal in the study of synchronization has been to find low-order descriptions of complex oscillator networks and their collective dynamics. However, for the Kuramoto model, the most widely used model of coupled oscillators, this goal has remained surprisingly challenging, in particular for finite-size networks. Here, we propose a model reduction framework that effectively captures synchronization behavior in complex network topologies. This framework generalizes a collective coordinates approach for all-to-all networks [G. A. Gottwald, Chaos 25, 053111 (2015)CHAOEH1054-150010.1063/1.4921295] by incorporating the graph Laplacian matrix in the collective coordinates. We first derive low dimensional evolution equations for both clustered and nonclustered oscillator networks. We then demonstrate in numerical simulations for Erdős-Rényi networks that the collective coordinates capture the synchronization behavior in both finite-size networks as well as in the thermodynamic limit, even in the presence of interacting clusters.

Random Matrix Analysis for Gene Interaction Networks in Cancer Cells

Investigations of topological uniqueness of gene interaction networks in cancer cells are essential for understanding the disease. Although cancer is considered to originate from the topological alteration of a huge molecular interaction network in cellular systems, the theoretical study to investigate such complex networks is still insufficient. It is necessary to predict the behavior of a huge complex interaction network from the behavior of a finite size network. Based on the random matrix theory, we study the distribution of the nearest neighbor level spacings P(s) of interaction matrices of gene networks in human cancer cells. The interaction matrices are computed using the Cancer Network Galaxy (TCNG) database which is a repository of gene interactions inferred by a Bayesian network model. 256 NCBI GEO entries regarding gene expressions in human cancer cells have been used for the inference. We observe the Wigner distribution of P(s) when the gene networks are dense networks that have more than ~38,000 edges. In the opposite case, when the networks have smaller numbers of edges, the distribution P(s) becomes the Poisson distribution. We investigate relevance of P(s) both to the sparseness of the networks and to edge frequency factor which is the reliance (likelihood) of the inferred gene interactions.

General formulation of long-range degree correlations in complex networks.

We provide a general framework for analyzing degree correlations between nodes separated by more than one step (i.e., beyond nearest neighbors) in complex networks. One joint and four conditional probability distributions are introduced to fully describe long-range degree correlations with respect to degrees k and k^{'} of two nodes and shortest path length l between them. We present general relations among these probability distributions and clarify the relevance to nearest-neighbor degree correlations. Unlike nearest-neighbor correlations, some of these probability distributions are meaningful only in finite-size networks. Furthermore, as a baseline to determine the existence of intrinsic long-range degree correlations in a network other than inevitable correlations caused by the finite-size effect, the functional forms of these probability distributions for random networks are analytically evaluated within a mean-field approximation. The utility of our argument is demonstrated by applying it to real-world networks.

Symmetry breaking in two interacting populations of quadratic integrate-and-fire neurons.

We analyze the dynamics of two coupled identical populations of quadratic integrate-and-fire neurons, which represent the canonical model for class I neurons near the spiking threshold. The populations are heterogeneous; they include both inherently spiking and excitable neurons. The coupling within and between the populations is global via synapses that take into account the finite width of synaptic pulses. Using a recently developed reduction method based on the Lorentzian ansatz, we derive a closed system of equations for the neuron's firing rates and the mean membrane potentials in both populations. The reduced equations are exact in the infinite-size limit. The bifurcation analysis of the equations reveals a rich variety of nonsymmetric patterns, including a splay state, antiphase periodic oscillations, chimera-like states, and chaotic oscillations as well as bistabilities between various states. The validity of the reduced equations is confirmed by direct numerical simulations of the finite-size networks.

Delay Analysis of Epidemic Schemes in Sparse and Dense Heterogeneous Contact Networks

Epidemic algorithms have found their way into many areas of computer science, such as databases and distributed systems, and recently for communication in Opportunistic or Delay Tolerant Networks (DTNs). To ensure analytical tractability, existing analyses of epidemic spreading predominantly consider homogeneous contact rates between nodes. However, this assumption is generally not true in real scenarios. In this paper, we consider classes of contact/mobility models with heterogeneous contact rates. Through an asymptotic analysis, we prove that a first-order, mean value approximation for the basic epidemic spreading step becomes exact in the limiting case (large network size). We further derive simple closed form approximations, based on higher order statistics of the mobility heterogeneity, for the case of finite-size networks. To demonstrate the utility of our results, we use them to predict the delay of epidemic-based routing schemes and analyze scenarios with node selfishness. We validate the analytic results through extensive simulations on synthetic scenarios, as well as on real traces to demonstrate that our expressions can be useful also in scenarios with significantly more complex structure. We believe these results are an important step forward towards analyzing the effects of heterogeneity (of mobility and/or other characteristics) on the performance of epidemic-based algorithms.

A stochastic-field description of finite-size spiking neural networks.

Neural network dynamics are governed by the interaction of spiking neurons. Stochastic aspects of single-neuron dynamics propagate up to the network level and shape the dynamical and informational properties of the population. Mean-field models of population activity disregard the finite-size stochastic fluctuations of network dynamics and thus offer a deterministic description of the system. Here, we derive a stochastic partial differential equation (SPDE) describing the temporal evolution of the finite-size refractory density, which represents the proportion of neurons in a given refractory state at any given time. The population activity—the density of active neurons per unit time—is easily extracted from this refractory density. The SPDE includes finite-size effects through a two-dimensional Gaussian white noise that acts both in time and along the refractory dimension. For an infinite number of neurons the standard mean-field theory is recovered. A discretization of the SPDE along its characteristic curves allows direct simulations of the activity of large but finite spiking networks; this constitutes the main advantage of our approach. Linearizing the SPDE with respect to the deterministic asynchronous state allows the theoretical investigation of finite-size activity fluctuations. In particular, analytical expressions for the power spectrum and autocorrelation of activity fluctuations are obtained. Moreover, our approach can be adapted to incorporate multiple interacting populations and quasi-renewal single-neuron dynamics.

Contact process with exogenous infection and the scaled SIS process

Abstract Propagation of contagion in networks depends on the graph topology. This article is concerned with studying the time-asymptotic behaviour of the extended contact processes on static, undirected, finite-size networks. This is a contact process with nonzero exogenous infection rate (also known as the $\epsilon$-susceptible-infected-susceptible model). The only known analytical characterization of the equilibrium distribution of this process is for complete networks. For large networks with arbitrary topology, it is infeasible to numerically solve for the equilibrium distribution since it requires solving the eigenvalue-eigenvector problem of a matrix that is exponential in $N$, the size of the network. We derive a condition on the infection rates under which, depending on the degree distribution of the network, the equilibrium distribution of extended contact processes on arbitrary, finite-size networks is well approximated by a closed-form formulation. We confirm the goodness of the approximation with small networks answering inference questions like the distribution of the percentage of infected individuals and the most-probable equilibrium configuration. We then use the approximation to analyse the equilibrium distribution of the extended contact process on the 4941-node US Western power grid.

Understanding the anomalous frequency responses of composite materials using very large random resistor-capacitor networks

In this paper large resistor-capacitor (RC) networks that consist of randomly distributed conductive and capacitive elements which are much larger than those previously explored are studied using an efficient algorithm. We investigate the emergent power-law scaling of the conductance and the percolation and saturation limits of the networks at the high and low frequency bounds in order to compare with a modification of the classical Effective Medium Approximation (EMA) that enables its extension to finite network sizes. It is shown that the new formula provides a simple analytical description of the network response that accurately predicts the effects of finite network size and composition and it agrees well with the new numerical calculations on large networks and is a significant improvement on earlier EMA formulae. Avenues for future improvement and explanation of the formula are highlighted. Finally, the statistical variation of network conductivity with network size is observed and explained. This work provides a deeper insight into the response of large resistor-capacitor networks to understand the AC electrical properties, size effects, composition effects and statistical variation of properties of a range of heterogeneous materials and composite systems.