Hierarchical Network Model Research Articles

Face detection in untrained deep neural networks

Face-selective neurons are observed in the primate visual pathway and are considered as the basis of face detection in the brain. However, it has been debated as to whether this neuronal selectivity can arise innately or whether it requires training from visual experience. Here, using a hierarchical deep neural network model of the ventral visual stream, we suggest a mechanism in which face-selectivity arises in the complete absence of training. We found that units selective to faces emerge robustly in randomly initialized networks and that these units reproduce many characteristics observed in monkeys. This innate selectivity also enables the untrained network to perform face-detection tasks. Intriguingly, we observed that units selective to various non-face objects can also arise innately in untrained networks. Our results imply that the random feedforward connections in early, untrained deep neural networks may be sufficient for initializing primitive visual selectivity.

Reconstruction of a Fully Paralleled Auditory Spiking Neural Network and FPGA Implementation.

This paper presents a field-programmable gate array (FPGA) implementation of an auditory system, which is biologically inspired and has the advantages of robustness and anti-noise ability. We propose an FPGA implementation of an eleven-channel hierarchical spiking neuron network (SNN) model, which has a sparsely connected architecture with low power consumption. According to the mechanism of the auditory pathway in human brain, spiking trains generated by the cochlea are analyzed in the hierarchical SNN, and the specific word can be identified by a Bayesian classifier. Modified leaky integrate-and-fire (LIF) model is used to realize the hierarchical SNN, which achieves both high efficiency and low hardware consumption. The hierarchical SNN implemented on FPGA enables the auditory system to be operated at high speed and can be interfaced and applied with external machines and sensors. A set of speech from different speakers mixed with noise are used as input to test the performance our system, and the experimental results show that the system can classify words in a biologically plausible way with the presence of noise. The method of our system is flexible and the system can be modified into desirable scale. These confirm that the proposed biologically plausible auditory system provides a better method for on-chip speech recognition. Compare to the state-of-the-art, our auditory system achieves a higher speed with a maximum frequency of 65.03 MHz and a lower energy consumption of 276.83 μJ for a single operation. It can be applied in the field of brain-computer interface and intelligent robots.

Generation of γ-Alumina Digital Twins Using a Nitrogen Porosimetry Simulation

A γ-alumina digital twin is created by means of a hierarchical pore network model. For this digital twin, nitrogen porosimetry is simulated, and the parameters of the pore network model are modifie...

Internet-like brain hierarchical network model: Alzheimer's disease study as an example

Background and ObjectiveThe modular structure and hierarchy are important topological characteristics in real complex networks such as brain networks on temporal scale. However, there are few studies investigating the hierarchical structure at the spatial scale of brain networks, the application of which still remains to be further studied. MethodsIn this study, a novel model of brain hierarchical network based on the hierarchical characteristic of Internet topology is proposed for the first time, which is called Internet-like brain hierarchical network (IBHN). In this model, the whole brain network is partitioned into multiple levels: brain wide area network (Brain-WAN), brain metropolitan network (Brain-MAN), and brain local area network (Brain-LAN). A Brain-MAN is formed by the interconnection of multiple Brain-LANs, and the interconnection of multiple Brain-MANs forms a Brain-WAN. A multivariate analysis method is employed to measure overall functional connectivity between two brain networks at the same network level rather than detecting the change of each node pair's functional connection. Furthermore, we demonstrate the utility of IBHN model with application to a practical case-control study involving 64 patients with Alzheimer's disease and 75 healthy controls. ResultsThe proposed model identified enhanced functional connectivity (P-value<0.05) at Brain-WAN level and reduced functional connectivity (P-value=0.004) at Brain-LAN level of Alzheimer's disease patients, which can be used as a multi-dimension functional reference for AD's diagnosis. ConclusionsThis study not only provides insight into AD pathophysiology, but also further proves the effectiveness of the proposed IBHN model. In addition, the IBHN model makes it possible to explore the brain's functional organization from multiple dimensions and offers a multi-level perspective for the research of complex brain network.

Role of Asymptomatic COVID-19 Cases in Viral Transmission: Findings From a Hierarchical Community Contact Network Model.

As part of ongoing efforts to contain the coronavirus disease (COVID-19) pandemic, understanding the role of asymptomatic patients in the transmission system is essential for infection control. However, the optimal approach to risk assessment and management of asymptomatic cases remains unclear. This study proposed a Susceptible, Exposed, Infectious, No symptoms, Hospitalized and reported, Recovered, Death (SEINRHD) epidemic propagation model. The model was constructed based on epidemiological characteristics of COVID-19 in China and accounting for the heterogeneity of social contact networks. The early community outbreaks in Wuhan were reconstructed and fitted with the actual data. We used this model to assess epidemic control measures for asymptomatic cases in three dimensions. The impact of asymptomatic cases on epidemic propagation was examined based on the effective reproduction number, abnormally high transmission events, and type and structure of transmission. Management of asymptomatic cases can help flatten the infection curve. Tracing 75% of the asymptomatic cases corresponds to a 32.5% overall reduction in new cases (compared with tracing no asymptomatic cases). Regardless of population-wide measures, household transmission is higher than other types of transmission, accounting for an estimated 50% of all cases. The magnitude of tracing of asymptomatic cases is more important than the timing; when all symptomatic patients were traced, tested, and isolated in a timely manner, the overall epidemic was not sensitive to the time of implementing the measures to trace asymptomatic patients. Disease control and prevention within families should be emphasized during an epidemic. Note to Practitioners—This article addresses the urgent need to assess the risk of another COVID-19 outbreak caused by asymptomatic cases and to find the optimal, most practical approach to asymptomatic case management. Previous studies mostly focused on the clinical and statistical characteristics of asymptomatic cases; few have evaluated the impact of asymptomatic case measures using mathematical modeling at the community scale. This study proposed a Susceptible, Exposed, Infectious, No symptoms, Hospitalized and reported, Recovered, Death (SEINRHD) propagation model based on local community structures and social contact networks, according to the development characteristics and trend of COVID-19 in a Chinese community. The conclusion provides theoretical support for emergency work of relevant departments in different periods of an epidemic. In the early stages of the epidemic, timely detection and isolation of symptomatic patients should be a priority. Where there are surplus resources for epidemic prevention, the authorities should consider increasing the proportion of asymptomatic patients being traced. Epidemic prevention measures among family members should be a primary focus of attention. This combination of strategies can help reduce the rate of viral transmission and result in extinguishing the epidemic.

A Bayesian Hierarchical Network Model for Daily Streamflow Ensemble Forecasting

AbstractA novel Bayesian Hierarchical Network Model (BHNM) for ensemble forecasts of daily streamflow is presented that uses the spatial dependence induced by the river network topology and hydrometeorological variables from the upstream contributing area between station gauges. Model parameters are allowed to vary with time as functions of selected covariates for each day. Using the network structure to incorporate flow information from upstream gauges and precipitation from the immediate contributing area as covariates allows one to model the spatial correlation of flows simultaneously and parsimoniously. An application to daily monsoon period (July–August) streamflow at three gauges in the Narmada basin in central India for the period 1978–2014 is presented. The best set of covariates include daily streamflow from upstream gauges or from the gauge above the upstream gauges depending on travel times and daily precipitation from the area between two stations. The model validation indicates that the model is highly skillful relative to a null‐model of generalized linear regression, which represents the analogous non‐Bayesian forecast. The ensemble spread of BHNM accounts for the forecast uncertainty leading to reliable and skillful streamflow predictions.

Modeling Autobiographical Memory for Believable Agents

We present a multi-layer hierarchical connectionist network model for simulating human autobiographical memory in believable agents. Grounded in psychological theory, this model improves on previous attempts to model agents’ event knowledge by providing a more dynamic and non-deterministic representation of autobiographical memories. From this model, a Java-based proof-of-concept prototype system was created for use as an enabling technology in video games. This prototype was leveraged in the creation of a Minecraft modification (mod) implementation of the model that is able to demonstrate context-dependent recall and the effects of recency on memory recall. Wider implications of the model in agent and game design are discussed.

Learning exact enumeration and approximate estimation in deep neural network models

A system for approximate number discrimination has been shown to arise in at least two types of hierarchical neural network models—a generative Deep Belief Network (DBN) and a Hierarchical Convolutional Neural Network (HCNN) trained to classify natural objects. Here, we investigate whether the same two network architectures can learn to recognise exact numerosity. A clear difference in performance could be traced to the specificity of the unit responses that emerged in the last hidden layer of each network. In the DBN, the emergence of a layer of monotonic ‘summation units’ was sufficient to produce classification behaviour consistent with the behavioural signature of the approximate number system. In the HCNN, a layer of units uniquely tuned to the transition between particular numerosities effectively encoded a thermometer-like ‘numerosity code’ that ensured near-perfect classification accuracy. The results support the notion that parallel pattern-recognition mechanisms may give rise to exact and approximate number concepts, both of which may contribute to the learning of symbolic numbers and arithmetic.

Breaking the curse of dimensionality: hierarchical Bayesian network model for multi-view clustering

Clustering high-dimensional data under the curse of dimensionality is an arduous task in many applications domains. The wide dimension yields the complexity-related challenges and the limited number of records leads to the overfitting trap. We propose to tackle this problematic using the graphical and probabilistic power of the Bayesian network. Our contribution is a new loose hierarchical Bayesian network model that encloses latent variables. These hidden variables are introduced for ensuring a multi-view clustering of the records. We propose a new framework for learning our proposed Bayesian network model. It starts by extracting the cliques of highly dependent features and it proceeds to learn representative latent variable for each features’ clique. The experimental results of our comparative analysis prove the efficiency of our model in tackling the distance concentration challenge. They also show the effectiveness of our model learning framework in skipping the overfitting trap, on benchmark high-dimensional datasets.

Hierarchical Network Models for Exchangeable Structured Interaction Processes

Network data often arises via a series of structured interactions among a population of constituent elements. E-mail exchanges, for example, have a single sender followed by potentially multiple receivers. Scientific articles, on the other hand, may have multiple subject areas and multiple authors. We introduce a statistical model, termed the Pitman-Yor hierarchical vertex components model (PY-HVCM), that is well suited for structured interaction data. The proposed PY-HVCM effectively models complex relational data by partial pooling of local information via a latent, shared population-level distribution. The PY-HCVM is a canonical example of hierarchical vertex components models—a subfamily of models for exchangeable structured interaction-labeled networks, that is, networks invariant to interaction relabeling. Theoretical analysis and supporting simulations provide clear model interpretation, and establish global sparsity and power law degree distribution. A computationally tractable Gibbs sampling algorithm is derived for inferring sparsity and power law properties of complex networks. We demonstrate the model on both the Enron e-mail dataset and an ArXiv dataset, showing goodness of fit of the model via posterior predictive validation.

Integrated Biophysical Modeling of the SARS-CoV-2 Spike Protein Binding and Allosteric Interactions with Antibodies.

Structural and biochemical studies of the severe acute respiratory syndrome (SARS)-CoV-2 spike glycoproteins and complexes with highly potent antibodies have revealed multiple conformation-dependent epitopes highlighting conformational plasticity of spike proteins and capacity for eliciting specific binding and broad neutralization responses. In this study, we used coevolutionary analysis, molecular simulations, and perturbation-based hierarchical network modeling of the SARS-CoV-2 spike protein complexes with a panel of antibodies targeting distinct epitopes to explore molecular mechanisms underlying binding-induced modulation of dynamics and allosteric signaling in the spike proteins. Through coevolutionary analysis of the SARS-CoV-2 spike proteins, we identified highly coevolving hotspots and functional clusters that enable a functional cross-talk between distant allosteric regions in the SARS-CoV-2 spike complexes with antibodies. Coarse-grained and all-atom molecular dynamics simulations combined with mutational sensitivity mapping and perturbation-based profiling of the SARS-CoV-2 receptor-binding domain (RBD) complexes with CR3022 and CB6 antibodies enabled a detailed validation of the proposed approach and an extensive quantitative comparison with the experimental structural and deep mutagenesis scanning data. By combining in silico mutational scanning, perturbation-based modeling, and network analysis of the SARS-CoV-2 spike trimer complexes with H014, S309, S2M11, and S2E12 antibodies, we demonstrated that antibodies can incur specific and functionally relevant changes by modulating allosteric propensities and collective dynamics of the SARS-CoV-2 spike proteins. The results provide a novel insight into regulatory mechanisms of SARS-CoV-2 S proteins showing that antibody-escaping mutations can preferentially target structurally adaptable energy hotspots and allosteric effector centers that control functional movements and allosteric communication in the complexes.

A deep attention model to forecast the Length Of Stay and the in-hospital mortality right on admission from ICD codes and demographic data

Leveraging the Electronic Health Records (EHR) longitudinal data to produce actionable clinical insights has always been a critical issue for recent studies. Non-forecasted extended hospitalizations account for a disproportionate amount of resource use, the mediocre quality of inpatient care, and avoidable fatalities. The capability to predict the Length of Stay (LoS) and mortality in the early stages of the admission provides opportunities to improve care and prevent many preventable losses. Forecasting the in-hospital mortality is important in providing clinicians with enough insights to make decisions and hospitals to allocate resources, hence predicting the LoS and mortality within the first day of admission is a difficult but a paramount endeavor. The biggest challenge is that few data are available by this time, thus the prediction has to bring in the previous admissions history and free text diagnosis that are recorded immediately on admission. We propose a model that uses the multi-modal EHR structured medical codes and key demographic information to classify the LoS in 3 classes; Short Los (LoS⩽10 days), Medium LoS (10<LoS⩽30 days) and Long LoS (LoS>30 days) as well as mortality as a binary classification of a patient’s death during current admission. The prediction has to use data available only within 24 h of admission. The key predictors include previous ICD9 diagnosis codes, ICD9 procedures, key demographic data, and free text diagnosis of the current admission recorded right on admission. We propose a Hierarchical Attention Network (HAN-LoS and HAN-Mor) model and train it to a dataset of over 45321 admissions recorded in the de-identified MIMIC-III dataset. For improved prediction, our attention mechanisms can focus on the most influential past admissions and most influential codes in these admissions. For fair performance evaluation, we implemented and compared the HAN model with previous approaches. With dataset balancing techniques HAN-LoS achieved an AUROC of over 0.82 and a Micro-F1 score of 0.24 and HAN-Mor achieved AUC-ROC of 0.87 hence outperforming the existing baselines that use structured medical codes as well as clinical time series for LoS and Mortality forecasting. By predicting mortality and LoS using the same model, we show that with little tuning the proposed model can be used for other clinical predictive tasks like phenotyping, decompensation,re-admission prediction, and survival analysis.

An interpretable deep learning model to predict symptomatic knee osteoarthritis

Purpose: Purpose- Early prediction of knee osteoarthritis may help to initiate potential interventions or treatments sooner, which can result in preventing or delaying knee osteoarthritis development. In this study, we aim to develop a deep learning (DL) prediction model to predict the incident symptomatic radiographic knee osteoarthritis over the next 78 months using radiographs at baseline. Methods: 427 knee radiography images from the Osteoarthritis Initiative (OAI) dataset were used to train and validate our model, with a mean age of 55.7 (SD: 3.2, range: 50 - 62) years and a mean BMI of 31.7 (SD: 3.7, range: 26.1 - 47.1) kg/m2. All subjects had Kellgren-Lawrence (KL) grades of 0 or 1 without symptoms at baseline. Incident symptomatic knee osteoarthritis was defined as knee pain and a definite tibiofemoral osteophyte in the same knee developed at 78-month follow up. Attention scores and probability of symptomatic knee osteoarthritis from radiography images were first extracted using an interpretable hierarchical attention-based convolutional neural network (CNN) model. The proposed model leveraged a transfer learning approach based on VGG-16 architecture. A 5-fold stratified cross-validation was used to evaluate and tune the neural network with 20% of training data as validation set in each fold. Twelve clinical variables including: age, body mass index (BMI), gender, KL grade, varus malalignment, history of knee injury, mild symptoms, presence of Heberden nodes, joint line tenderness, crepitus, morning stiffness and postmenopausal status, along with the attention and probability scores from our trained CNN, were then fed into a logistic regression model to predict the 78-month probability of knee osteoarthritis incident for each individual. The developed model was evaluated using an independent test set of 108 baseline knee radiographs from the OAI dataset (age, BMI, KL distribution, no symptoms at baseline). Results: The developed multi-modal model using radiographs and 12 clinical variables demonstrated an area under the receiver operating characteristic curve (AUC) of 0.91 (Figure 1), which outperforms the results reported in the literature using deep learning based methods. Moreover, the proposed model is interpretable, which can detect the specific location of potential abnormalities within knee images using the learned attention scores (Figure 2). Conclusions: A novel deep learning-based predictive model that takes knee radiographs and clinical variables as inputs was developed. It can predict the risk of symptomatic knee osteoarthritis incident over the next 6.5 years with a high AUC value. The attention maps on knee images confirm that the model predictions at baseline match the knee progression pattern in the course of the disease.View Large Image Figure ViewerDownload Hi-res image Download (PPT)

A Hierarchical Attractor Network Model of perceptual versus intentional decision updates

Changes of Mind are a striking example of our ability to flexibly reverse decisions and change our own actions. Previous studies largely focused on Changes of Mind in decisions about perceptual information. Here we report reversals of decisions that require integrating multiple classes of information: 1) Perceptual evidence, 2) higher-order, voluntary intentions, and 3) motor costs. In an adapted version of the random-dot motion task, participants moved to a target that matched both the external (exogenous) evidence about dot-motion direction and a preceding internally-generated (endogenous) intention about which colour to paint the dots. Movement trajectories revealed whether and when participants changed their mind about the dot-motion direction, or additionally changed their mind about which colour to choose. Our results show that decision reversals about colour intentions are less frequent in participants with stronger intentions (Exp. 1) and when motor costs of intention pursuit are lower (Exp. 2). We further show that these findings can be explained by a hierarchical, multimodal Attractor Network Model that continuously integrates higher-order voluntary intentions with perceptual evidence and motor costs. Our model thus provides a unifying framework in which voluntary actions emerge from a dynamic combination of internal action tendencies and external environmental factors, each of which can be subject to Change of Mind.

Study of Cascading Failure in Multisubnet Composite Complex Networks

Current research on the cascading failure of coupling networks is mostly based on hierarchical network models and is limited to a single relationship. In reality, many relationships exist in a network system, and these relationships collectively affect the process and scale of the network cascading failure. In this paper, a composite network is constructed based on the multisubnet composite complex network model, and its cascading failure is proposed combined with multiple relationships. The effect of intranetwork relationships and coupling relationships on network robustness under different influencing factors is studied. It is shown that cascading failure in composite networks is different from coupling networks, and increasing the strength of the coupling relationship can significantly improve the robustness of the network.

Detection of sitting posture using hierarchical image composition and deep learning.

Human posture detection allows the capture of the kinematic parameters of the human body, which is important for many applications, such as assisted living, healthcare, physical exercising and rehabilitation. This task can greatly benefit from recent development in deep learning and computer vision. In this paper, we propose a novel deep recurrent hierarchical network (DRHN) model based on MobileNetV2 that allows for greater flexibility by reducing or eliminating posture detection problems related to a limited visibility human torso in the frame, i.e., the occlusion problem. The DRHN network accepts the RGB-Depth frame sequences and produces a representation of semantically related posture states. We achieved 91.47% accuracy at 10 fps rate for sitting posture recognition.

Excitatory-inhibitory tone shapes decision strategies in a hierarchical neural network model of multi-attribute choice.

We are constantly faced with decisions between alternatives defined by multiple attributes, necessitating an evaluation and integration of different information sources. Time-varying signals in multiple brain areas are implicated in decision-making; but we lack a rigorous biophysical description of how basic circuit properties, such as excitatory-inhibitory (E/I) tone and cascading nonlinearities, shape attribute processing and choice behavior. Furthermore, how such properties govern choice performance under varying levels of environmental uncertainty is unknown. We investigated two-attribute, two-alternative decision-making in a dynamical, cascading nonlinear neural network with three layers: an input layer encoding choice alternative attribute values; an intermediate layer of modules processing separate attributes; and a final layer producing the decision. Depending on intermediate layer E/I tone, the network displays distinct regimes characterized by linear (I), convex (II) or concave (III) choice indifference curves. In regimes I and II, each option’s attribute information is additively integrated. In regime III, time-varying nonlinear operations amplify the separation between offer distributions by selectively attending to the attribute with the larger differences in input values. At low environmental uncertainty, a linear combination most consistently selects higher valued alternatives. However, at high environmental uncertainty, regime III is more likely than a linear operation to select alternatives with higher value. Furthermore, there are conditions where readout from the intermediate layer could be experimentally indistinguishable from the final layer. Finally, these principles are used to examine multi-attribute decisions in systems with reduced inhibitory tone, leading to predictions of different choice patterns and overall performance between those with restrictions on inhibitory tone and neurotypicals.

Efficacy-specific herbal group detection from traditional Chinese medicine prescriptions via hierarchical attentive neural network model

BackgroundMining massive prescriptions in Traditional Chinese Medicine (TCM) accumulated in the lengthy period of several thousand years to discover essential herbal groups for distinct efficacies is of significance for TCM modernization, thus starting to draw attentions recently. However, most existing methods for the task treat herbs with different surface forms orthogonally and determine efficacy-specific herbal groups based on the raw frequencies an herbal group occur in a collection of prescriptions. Such methods entirely overlook the fact that prescriptions in TCM are formed empirically by different people at different historical stages, and thus full of herbs with different surface forms expressing the same material, or even noisy and redundant herbs.MethodsWe propose a two-stage approach for efficacy-specific herbal group detection from prescriptions in TCM. For the first stage we devise a hierarchical attentive neural network model to capture essential herbs in a prescription for its efficacy, where herbs are encoded with dense real-valued vectors learned automatically to identify their differences on the semantical level. For the second stage, frequent patterns are mined to discover essential herbal groups for an efficacy from distilled prescriptions obtained in the first stage.ResultsWe verify the effectiveness of our proposed approach from two aspects, the first one is the ability of the hierarchical attentive neural network model to distill a prescription, and the second one is the accuracy in discovering efficacy-specific herbal groups.ConclusionThe experimental results demonstrate that the hierarchical attentive neural network model is capable to capture herbs in a prescription essential to its efficacy, and the distilled prescriptions significantly could improve the performance of efficacy-specific herbal group detection.

Paradoxical sensory reactivity induced by functional disconnection in a robot model of neurodevelopmental disorder

Neurodevelopmental disorders are characterized by heterogeneous and non-specific nature of their clinical symptoms. In particular, hyper- and hypo-reactivity to sensory stimuli are diagnostic features of autism spectrum disorder and are reported across many neurodevelopmental disorders. However, computational mechanisms underlying the unusual paradoxical behaviors remain unclear. In this study, using a robot controlled by a hierarchical recurrent neural network model with predictive processing and learning mechanism, we simulated how functional disconnection altered the learning process and subsequent behavioral reactivity to environmental change. The results show that, through the learning process, long-range functional disconnection between distinct network levels could simultaneously lower the precision of sensory information and higher-level prediction. The alteration caused a robot to exhibit sensory-dominated and sensory-ignoring behaviors ascribed to sensory hyper- and hypo-reactivity, respectively. As long-range functional disconnection became more severe, a frequency shift from hyporeactivity to hyperreactivity was observed, paralleling an early sign of autism spectrum disorder. Furthermore, local functional disconnection at the level of sensory processing similarly induced hyporeactivity due to low sensory precision. These findings suggest a computational explanation for paradoxical sensory behaviors in neurodevelopmental disorders, such as coexisting hyper- and hypo-reactivity to sensory stimulus. A neurorobotics approach may be useful for bridging various levels of understanding in neurodevelopmental disorders and providing insights into mechanisms underlying complex clinical symptoms.

Efficiency measurements of Diponegoro University faculty using the Data Envelopment Analysis hierarchical network model

Diponegoro University (UNDIP) is a state university located in Semarang city. UNDIP is aiming to become a world-class university. To become a world-class university, UNDIP has to maintain its quality in academic, research, and community service. But UNDIP was facing some challenges, like has to compete with other universities that growing rapidly. To achieve the target, the efficiency of every faculty in UNDIP should be continuously monitored and improved. Measuring the overall efficiency of every faculty is the aim of this research. The samples used in this research were all faculty of undergraduate programs in UNDIP. The hierarchical network data envelopment analysis model measuring the efficiency level of every faculty at UNDIP for systems with a hierarchical structure. In this research, the input variables used were the number of lecturers, the number of students, and amount of operational expenses. While the output variables were the number of graduates, the number of researches, the number of publications, the number of researches, the number of guidance & consultation, and the number of activities. The results of this study would help the management of university especially the dean of faculty to identify the inefficiency at every faculty in UNDIP and then plan for improvement.