Hierarchical Modular Networks Research Articles

Effects of network heterogeneity on phases of the quenched contact process in directed complex networks

We investigate the quenched contact process within directed complex networks, including directed random networks and directed scale-free networks. Additionally, we explore the contact process within directed general small-world networks and directed hierarchical modular networks. We observed the presence of a Griffiths phase characterized by continuously varying critical exponents of the order parameter. Notably, the directionality of the networks plays a significant role in influencing the Griffiths phase within the quenched contact process. We suggest that the presence of strongly directed connecting clusters is responsible for the changes in the continuously varying critical exponents of the order parameter within directed networks.

Learning Hierarchical Modular Networks for Video Captioning.

Video captioning aims to generate natural language descriptions for a given video clip. Existing methods mainly focus on end-to-end representation learning via word-by-word comparison between predicted captions and ground-truth texts. Although significant progress has been made, such supervised approaches neglect semantic alignment between visual and linguistic entities, which may negatively affect the generated captions. In this work, we propose a hierarchical modular network to bridge video representations and linguistic semantics at four granularities before generating captions: entity, verb, predicate, and sentence. Each level is implemented by one module to embed corresponding semantics into video representations. Additionally, we present a reinforcement learning module based on the scene graph of captions to better measure sentence similarity. Extensive experimental results show that the proposed method performs favorably against the state-of-the-art models on three widely-used benchmark datasets, including microsoft research video description corpus (MSVD), MSR-video to text (MSR-VTT), and video-and-TEXt (VATEX).

Nonuniversal power-law dynamics of susceptible infected recovered models on hierarchical modular networks.

Power-law (PL) time-dependent infection growth has been reported in many COVID-19 statistics. In simple susceptible infected recovered (SIR) models, the number of infections grows at the outbreak as I(t)∝t^{d-1} on d-dimensional Euclidean lattices in the endemic phase, or it follows a slower universal PL at the critical point, until finite sizes cause immunity and a crossover to an exponential decay. Heterogeneity may alter the dynamics of spreading models, and spatially inhomogeneous infection rates can cause slower decays, posing a threat of a long recovery from a pandemic. COVID-19 statistics have also provided epidemic size distributions with PL tails in several countries. Here I investigate SIR-like models on hierarchical modular networks, embedded in 2d lattices with the addition of long-range links. I show that if the topological dimension of the network is finite, average degree-dependent PL growth of prevalence emerges. Supercritically, the same exponents as those of regular graphs occur, but the topological disorder alters the critical behavior. This is also true for the epidemic size distributions. Mobility of individuals does not affect the form of the scaling behavior, except for the d=2 lattice, but it increases the magnitude of the epidemic. The addition of a superspreader hot spot also does not change the growth exponent and the exponential decay in the herd immunity regime.

A New Approach to Constructing a Decentralized Hierarchical Modular Network for Solving Complex Problems in the Paradigm of Training Artificial Neural Networks with a Teacher

A New Approach to Constructing a Decentralized Hierarchical Modular Network for Solving Complex Problems in the Paradigm of Training Artificial Neural Networks with a Teacher

Heterogeneous excitable systems exhibit Griffiths phases below hybrid phase transitions

In $d > 2$ dimensional, homogeneous threshold models discontinuous transition occur, but the mean-field solution provides $1/t$ power-law activity decay and other power-laws, thus it is called mixed-order or hybrid type. It has recently been shown that the introduction of quenched disorder rounds the discontinuity and second order phase transition and Griffiths phases appear. Here we provide numerical evidence, that even in case of high graph dimensional hierarchical modular networks the Griffiths phase of the $K=2$ threshold model is present below the hybrid phase transition. This is due to the fragmentation of the activity propagation by modules, which are connected via single links. This provides a widespread mechanism in case of threshold type of heterogeneous systems, modeling the brain or epidemics for the occurrence of dynamical criticality in extended Griffiths phase parameter spaces. We investigate this in synthetic modular networks with and without inhibitory links as well as in the presence of refractory states.

Optimal Interplay between Synaptic Strengths and Network Structure Enhances Activity Fluctuations and Information Propagation in Hierarchical Modular Networks.

In network models of spiking neurons, the joint impact of network structure and synaptic parameters on activity propagation is still an open problem. Here, we use an information-theoretical approach to investigate activity propagation in spiking networks with a hierarchical modular topology. We observe that optimized pairwise information propagation emerges due to the increase of either (i) the global synaptic strength parameter or (ii) the number of modules in the network, while the network size remains constant. At the population level, information propagation of activity among adjacent modules is enhanced as the number of modules increases until a maximum value is reached and then decreases, showing that there is an optimal interplay between synaptic strength and modularity for population information flow. This is in contrast to information propagation evaluated among pairs of neurons, which attains maximum value at the maximum values of these two parameter ranges. By examining the network behavior under the increase of synaptic strength and the number of modules, we find that these increases are associated with two different effects: (i) the increase of autocorrelations among individual neurons and (ii) the increase of cross-correlations among pairs of neurons. The second effect is associated with better information propagation in the network. Our results suggest roles that link topological features and synaptic strength levels to the transmission of information in cortical networks.

Explosive, continuous and frustrated synchronization transition in spiking Hodgkin–Huxley neural networks: The role of topology and synaptic interaction

Synchronization is an important collective phenomenon in interacting oscillatory agents. Many functional features of the brain are related to synchronization of neurons. The type of synchronization transition that may occur (explosive vs. continuous) has been the focus of intense attention in recent years, mostly in the context of phase oscillator models for which collective behavior is independent of the mean-value of natural frequency. However, synchronization properties of biologically-motivated neural models depend on the firing frequencies. In this study we report a systematic study of gamma-band synchronization in spiking Hodgkin–Huxley neurons which interact via electrical or chemical synapses. We use various network models in order to define the connectivity matrix. We find that the underlying mechanisms and types of synchronization transitions in gamma-band differs from beta-band. In gamma-band, network regularity suppresses transition while randomness promotes a continuous transition. Heterogeneity in the underlying topology does not lead to any change in the order of transition, however, correlation between number of synapses and frequency of a neuron will lead to explosive synchronization in heterogeneous networks with electrical synapses. Furthermore, small-world networks modeling a fine balance between clustering and randomness (as in the cortex), lead to explosive synchronization with electrical synapses, but a smooth transition in the case of chemical synapses. We also find that hierarchical modular networks, such as the connectome, lead to frustrated transitions. We explain our results based on various properties of the network, paying particular attention to the competition between clustering and long-range synapses.

Disturbance-Observer-Based Tracking Controller for Neural Network Driving Policy Transfer

The neural network policies are widely explored in the autonomous driving field, thanks to their capability of handling complicated driving tasks. However, the practical deployment of such policies is slowed down due to their lack of robustness against modeling gap and external disturbances. In our prior work, we proposed a planner-controller architecture and applied a disturbance-observer-based (DOB) robust tracking controller to reject the disturbances and achieved zero-shot policy transfer. In this paper, we present our latest progress on improving the policy transfer performance under this framework. Concretely, we applied adaptive DOB, so as to more accurately model the inverse system dynamics and increase the cut-off frequency of the Q-filter in the DOB. A closed-loop reference path smoothing algorithm is introduced to alleviate the step disturbance input imposed by the reference trajectory re-planning. On the neural network control policy side, we applied the parallel attribute networks, a hierarchical modular policy network to dynamically handle various driving tasks. We have carried out various simulations and experiments to validate the capability of our proposed method to achieve sim-to-sim and sim-to-real policy transfer. The proposed method achieves the most outstanding performance among a series of baseline control schemes.

Analysis of disease comorbidity patterns in a large-scale China population

BackgroundDisease comorbidity is popular and has significant indications for disease progress and management. We aim to detect the general disease comorbidity patterns in Chinese populations using a large-scale clinical data set.MethodsWe extracted the diseases from a large-scale anonymized data set derived from 8,572,137 inpatients in 453 hospitals across China. We built a Disease Comorbidity Network (DCN) using correlation analysis and detected the topological patterns of disease comorbidity using both complex network and data mining methods. The comorbidity patterns were further validated by shared molecular mechanisms using disease-gene associations and pathways. To predict the disease occurrence during the whole disease progressions, we applied four machine learning methods to model the disease trajectories of patients.ResultsWe obtained the DCN with 5702 nodes and 258,535 edges, which shows a power law distribution of the degree and weight. It further indicated that there exists high heterogeneity of comorbidities for different diseases and we found that the DCN is a hierarchical modular network with community structures, which have both homogeneous and heterogeneous disease categories. Furthermore, adhering to the previous work from US and Europe populations, we found that the disease comorbidities have their shared underlying molecular mechanisms. Furthermore, take hypertension and psychiatric disease as instance, we used four classification methods to predicte the disease occurrence using the comorbid disease trajectories and obtained acceptable performance, in which in particular, random forest obtained an overall best performance (with F1-score 0.6689 for hypertension and 0.6802 for psychiatric disease).ConclusionsOur study indicates that disease comorbidity is significant and valuable to understand the disease incidences and their interactions in real-world populations, which will provide important insights for detection of the patterns of disease classification, diagnosis and prognosis.

Heterogeneity effects in power grid network models.

We have compared the phase synchronization transition of the second-order Kuramoto model on two-dimensional (2D) lattices and on large, synthetic power grid networks, generated from real data. The latter are weighted, hierarchical modular networks. Due to the inertia the synchronization transitions are of first-order type, characterized by fast relaxation and hysteresis by varying the global coupling parameter K. Finite-size scaling analysis shows that there is no real phase transition in the thermodynamic limit, unlike in the mean-field model. The order parameter and its fluctuations depend on the network size without any real singular behavior. In case of power grids the phase synchronization breaks down at lower global couplings, than in case of 2D lattices of the same sizes, but the hysteresis is much narrower or negligible due to the low connectivity of the graphs. The temporal behavior of desynchronization avalanches after a sudden quench to low K values has been followed and duration distributions with power-law tails have been detected. This suggests rare region effects, caused by frozen disorder, resulting in heavy-tailed distributions, even without a self-organization mechanism as a consequence of a catastrophic drop event in the couplings.

Topological dimension tunes activity patterns in hierarchical modular networks

Connectivity patterns of relevance in neuroscience and systems biology can be encoded in hierarchical modular networks (HMNs). Recent studies highlight the role of hierarchical modular organization in shaping brain activity patterns, providing an excellent substrate to promote both segregation and integration of neural information. Here, we propose an extensive analysis of the critical spreading rate (or ‘epidemic’ threshold)—separating a phase with endemic persistent activity from one in which activity ceases—on diverse HMNs. By employing analytical and computational techniques we determine the nature of such a threshold and scrutinize how it depends on general structural features of the underlying HMN. We critically discuss the extent to which current graph-spectral methods can be applied to predict the onset of spreading in HMNs and, most importantly, we elucidate the role played by the network topological dimension as a relevant and unifying structural parameter, controlling the epidemic threshold.

Artificial neural network based screening of cervical cancer using a hierarchical modular neural network architecture (HMNNA) and novel benchmark uterine cervix cancer database

The work reported in this paper presents a novel hierarchical modular neural network architecture (HMNNA) for automated screening of cervical cancer. HMNNA consists of three neural networks trained specifically on different areas of problem space under consideration, and the trained networks are then arranged in a tree structure forming hierarchical modular neural network architecture. The three specialized neural networks are trained by Levenberg–Maarquardt neural network algorithm. As compared to the standard back propagation algorithm, Levenberg–Maarquardt is fast and stable for convergence with only one drawback, i.e., storage requirement for estimated Hessian Matrix. For training and testing of HMNNA, a huge primary database is created which contains 8091 cervical cell images pertaining to 200 clinical cases collected from two health care institutions of northern India. The raw cases of cervical cancer in the form of Pap smear slides were photographed under a multi-headed digital microscope. Individual cells were manually cropped off from these slide images which were then passed through a feature extraction module for morphological profiling. Each cell was calibrated on the basis of 40 features from both cytoplasm and nucleus. After profiling, these cells were vigilantly assigned cell classes as per the latest 2001-Bethesda system of cervical cancer cell classification, by trained cytotechnicians and histopathologists. HMNNA is also trained and tested on the Herlev Benchmark dataset created by the Denmark University, which consists of 1417 cervical cancer cells. Both the datasets have seven classes of diagnosis, i.e., superficial squamous, intermediate squamous, columnar, mild dysplasia, moderate dysplasia, severe dysplasia, and carcinoma in situ, corresponding to the level of abnormality in cervical cells. These datasets are available in public domain at http://digitalpapsmeardb.in/ and http://mde-lab.aegean.gr/index.php/downloads. The screening potential of the HMNNA is compared with 25 well-known machine learning algorithms available in MatlabR2016 (Machine learning and statistics toolbox 10.2) and monolithic neural network algorithms available in Matlab neural network pattern recognition toolbox. The HMNNA outperformed in all the 25 algorithms for both the datasets. For the Novel Benchmark database, it produced a classification accuracy of 95.32% with an F-value of 0.949310 and classification accuracy of 88.41% with an F-value of 0.89145 for the Herlev dataset. The screening potential of HMNNA was also evaluated and compared with the other diagnostic systems available in the recently published literature and was found to be performing much better than the counterparts on multiple parameters of performance evaluation.

Hierarchical modular Bayesian networks for low-power context-aware smartphone

Abstract Nowadays, smartphone has a tremendous number of applications using sensors and devices for several applications such as healthcare and game. However, serious consideration to increase the duration of battery use of phone is required because of the limited battery capacity. In this paper, we propose a hybrid system to increase the longevity of phone with hierarchical modular Bayesian networks that recognize the user's contexts, and device management rules that infer the unnecessary devices in smartphone. Inferring the user's contexts with sensor data and considering the device status, the context inferred and user's tendency, we determine the superfluous devices that are consuming the battery as dispensable. The experiments with the real log data collected from 28 people for 6 months verify that the proposed system performs the accuracy of 85.68% and the reduction of battery consumption of about 6%.

Griffiths phase on hierarchical modular networks with small-world edges.

The Griffiths phase has been proposed to induce a stretched critical regime that facilitates self-organizing of brain networks for optimal function. This phase stems from the intrinsic structural heterogeneity of brain networks, i.e., the hierarchical modular structure. In this work, the Griffiths phase is studied in modified hierarchical networks with small-world connections based on the 3-regular Hanoi network. Through extensive simulations, the hierarchical level-dependent inter-module wiring probabilities are identified to determine the emergence of the Griffiths phase. Numerical results and the complementary spectral analysis of the relevant networks can be helpful for a deeper understanding of the essential structural characteristics of finite-dimensional networks to support the Griffiths phase.

Mechanisms of Self-Sustained Oscillatory States in Hierarchical Modular Networks with Mixtures of Electrophysiological Cell Types.

In a network with a mixture of different electrophysiological types of neurons linked by excitatory and inhibitory connections, temporal evolution leads through repeated epochs of intensive global activity separated by intervals with low activity level. This behavior mimics “up” and “down” states, experimentally observed in cortical tissues in absence of external stimuli. We interpret global dynamical features in terms of individual dynamics of the neurons. In particular, we observe that the crucial role both in interruption and in resumption of global activity is played by distributions of the membrane recovery variable within the network. We also demonstrate that the behavior of neurons is more influenced by their presynaptic environment in the network than by their formal types, assigned in accordance with their response to constant current.

Is it right to estimate inter-modular connectivity from local field potentials?

Human brains with hundreds of billions of neurons are organized in a hierarchical modular network. There have been many attempts to estimate inter-modular connectivity utilizing coherent neuronal activities of a huge number of neurons, such as the electro-encephalogram, the magneto-encephalogram, and the functional magnetic resonance imaging. Here we ask a question: Is the inter-modular connectivity estimated from the modular activities consistent with the inter-modular connectivity that could be extracted from the network connectivity of individual nodes? To answer this question, we introduce a method of estimating the inter-modular connectivity based on the analysis of inverse phase synchronization [1,2]. For coupled phase oscillators on a hierarchical modular network shown in the Figure 1(A), the local field potential corresponding to a module is defined as the mean phase of oscillators belonging to a subset of the module. Figure 1 (A) A hierarchical modular network model 256-p-q-r, where p denotes the number of links of one node with nodes of its lower-level module, q links with nodes of the rest modules in its upper-level module, and r links with nodes of any modules from the ... For strong coupling strength, it is shown in Figure 1(B) that the inverse phase synchronization index grows linearly with the number of links connecting two modules. This result enables us to estimate the inter-modular connectivity in various complex systems from the inverse phase synchronization index of the mesoscopic modular activities.

Griffiths phases and localization in hierarchical modular networks.

We study variants of hierarchical modular network models suggested by Kaiser and Hilgetag [ Front. in Neuroinform., 4 (2010) 8] to model functional brain connectivity, using extensive simulations and quenched mean-field theory (QMF), focusing on structures with a connection probability that decays exponentially with the level index. Such networks can be embedded in two-dimensional Euclidean space. We explore the dynamic behavior of the contact process (CP) and threshold models on networks of this kind, including hierarchical trees. While in the small-world networks originally proposed to model brain connectivity, the topological heterogeneities are not strong enough to induce deviations from mean-field behavior, we show that a Griffiths phase can emerge under reduced connection probabilities, approaching the percolation threshold. In this case the topological dimension of the networks is finite, and extended regions of bursty, power-law dynamics are observed. Localization in the steady state is also shown via QMF. We investigate the effects of link asymmetry and coupling disorder, and show that localization can occur even in small-world networks with high connectivity in case of link disorder.

Estimation of inter-modular connectivity from the local field potentials in a hierarchical modular network

We propose a method of estimating inter-modular connectivity in a hierarchical modular network. The method is based on an analysis of inverse phase synchronization applied to the local field potentials on a hierarchical modular network of phase oscillators. For a strong-coupling strength, the inverse phase synchronization index of the local field potentials for two modules depends linearly on the corresponding inter-modular connectivity defined as the number of links connecting the modules. The method might enable us to estimate the inter-modular connectivity in various complex systems from the inverse phase synchronization index of the mesoscopic modular activities.

Sustained oscillations, irregular firing, and chaotic dynamics in hierarchical modular networks with mixtures of electrophysiological cell types.

The cerebral cortex exhibits neural activity even in the absence of external stimuli. This self-sustained activity is characterized by irregular firing of individual neurons and population oscillations with a broad frequency range. Questions that arise in this context, are: What are the mechanisms responsible for the existence of neuronal spiking activity in the cortex without external input? Do these mechanisms depend on the structural organization of the cortical connections? Do they depend on intrinsic characteristics of the cortical neurons? To approach the answers to these questions, we have used computer simulations of cortical network models. Our networks have hierarchical modular architecture and are composed of combinations of neuron models that reproduce the firing behavior of the five main cortical electrophysiological cell classes: regular spiking (RS), chattering (CH), intrinsically bursting (IB), low threshold spiking (LTS), and fast spiking (FS). The population of excitatory neurons is built of RS cells (always present) and either CH or IB cells. Inhibitory neurons belong to the same class, either LTS or FS. Long-lived self-sustained activity states in our network simulations display irregular single neuron firing and oscillatory activity similar to experimentally measured ones. The duration of self-sustained activity strongly depends on the initial conditions, suggesting a transient chaotic regime. Extensive analysis of the self-sustained activity states showed that their lifetime expectancy increases with the number of network modules and is favored when the network is composed of excitatory neurons of the RS and CH classes combined with inhibitory neurons of the LTS class. These results indicate that the existence and properties of the self-sustained cortical activity states depend on both the topology of the network and the neuronal mixture that comprises the network.

Structural and Functional Brain Networks: From Connections to Cognition

How rich functionality emerges from the invariant structural architecture of the brain remains a major mystery in neuroscience. Recent applications of network theory and theoretical neuroscience to large-scale brain networks have started to dissolve this mystery. Network analyses suggest that hierarchical modular brain networks are particularly suited to facilitate local (segregated) neuronal operations and the global integration of segregated functions. Although functional networks are constrained by structural connections, context-sensitive integration during cognition tasks necessarily entails a divergence between structural and functional networks. This degenerate (many-to-one) function-structure mapping is crucial for understanding the nature of brain networks. The emergence of dynamic functional networks from static structural connections calls for a formal (computational) approach to neuronal information processing that may resolve this dialectic between structure and function.