Heterogeneous Neural Networks Research Articles

Distributed Chiller Loading via Collaborative Neurodynamic Optimization With Heterogeneous Neural Networks

Distributed Chiller Loading via Collaborative Neurodynamic Optimization With Heterogeneous Neural Networks

Quasi-synchronization of heterogeneous neural networks with hybrid time delays via sampled-data saturating impulsive control

This paper focuses on the quasi-synchronization matter of directed coupled heterogeneous neural networks with actuator saturation and hybrid time-varying delays under the sampled-data impulsive control. By utilizing the extended impulsive comparison principle and a dead-zone function representation of saturation nonlinearity function, multiple sufficient criteria are established for attaining the quasi-synchronization of heterogeneous dynamic neural networks. Meanwhile, the maximum estimating domain of attraction is acquired by solving an optimization problem which is confined by the linear matrix inequalities converted from the bilinear matrix inequalities. In addition, through utilizing the extended formula for the variation of parameters with hybrid delays, precise exponential convergence rate and theoretical error bound are calculated. Ultimately, the theoretical results are illustrated by means of a numerical simulation.

Fractional-order heterogeneous memristive Rulkov neuronal network and its medical image watermarking application

This article proposes a novel fractional heterogeneous neural network by coupling a Rulkov neuron with a Hopfield neural network (FRHNN), utilizing memristors for emulating neural synapses. The study firstly demonstrates the coexistence of multiple firing patterns through phase diagrams, Lyapunov exponents (LEs), and bifurcation diagrams. Secondly, the parameter related firing behaviors are described through two-parameter bifurcation diagrams. Subsequently, local attraction basins reveal multi-stability phenomena related to initial values. Moreover, the proposed model is implemented on a microcomputer-based ARM platform, and the experimental results correspond to the numerical simulations. Finally, the article explores the application of digital watermarking for medical images, illustrating its features of excellent imperceptibility, extensive key space, and robustness against attacks including noise and cropping.

DeepWind: a heterogeneous spatio-temporal model for wind forecasting

Deep learning (DL) has shown great potential in enhancing the performance of traditional numerical weather prediction (NWP) methods in weather forecasting. Certain applications such as wind power generation desire more accurate wind predictions which is challenging due to limited observations and complex dynamics. To this end, this paper introduces a DL-based heterogeneous network named DeepWind for NWP correction, which can simultaneously correct the NWP of diverse wind variables across multiple weather stations. In particular, it first exerts the meteorological domain knowledge to achieve an effective transformation of target variables and then develops a heterogeneous neural network to learn spatio-temporal representations. A novel difference loss function is further designed for stable temporal learning. Moreover, this study might be the first to expose an underlying evaluation problem in deep forecasting, which we call evaluation inconsistency, thereby necessitating the assessment of model performance across diverse evaluation metrics. Experimental results demonstrate the superiority of the proposed approach over strong DL baselines, which makes it positioned for deployment in the real-world production environment. Source code is released at https://github.com/Rittersss/DeepWind.

Neural heterogeneity controls computations in spiking neural networks.

The brain is composed of complex networks of interacting neurons that express considerable heterogeneity in their physiology and spiking characteristics. How does this neural heterogeneity influence macroscopic neural dynamics, and how might it contribute to neural computation? In this work, we use a mean-field model to investigate computation in heterogeneous neural networks, by studying how the heterogeneity of cell spiking thresholds affects three key computational functions of a neural population: the gating, encoding, and decoding of neural signals. Our results suggest that heterogeneity serves different computational functions in different cell types. In inhibitory interneurons, varying the degree of spike threshold heterogeneity allows them to gate the propagation of neural signals in a reciprocally coupled excitatory population. Whereas homogeneous interneurons impose synchronized dynamics that narrow the dynamic repertoire of the excitatory neurons, heterogeneous interneurons act as an inhibitory offset while preserving excitatory neuron function. Spike threshold heterogeneity also controls the entrainment properties of neural networks to periodic input, thus affecting the temporal gating of synaptic inputs. Among excitatory neurons, heterogeneity increases the dimensionality of neural dynamics, improving the network's capacity to perform decoding tasks. Conversely, homogeneous networks suffer in their capacity for function generation, but excel at encoding signals via multistable dynamic regimes. Drawing from these findings, we propose intra-cell-type heterogeneity as a mechanism for sculpting the computational properties of local circuits of excitatory and inhibitory spiking neurons, permitting the same canonical microcircuit to be tuned for diverse computational tasks.

Event-triggered impulsive synchronization of heterogeneous neural networks

Event-triggered impulsive synchronization of heterogeneous neural networks

H-Net: Heterogeneous Neural Network for Multi-Classification of Neuropsychiatric Disorders.

Clinical studies have proved that both structural magnetic resonance imaging (sMRI) and functional magnetic resonance imaging (fMRI) are implicitly associated with neuropsychiatric disorders (NDs), and integrating multi-modal to the binary classification of NDs has been thoroughly explored. However, accurately classifying multiple classes of NDs remains a challenge due to the complexity of disease subclass. In our study, we develop a heterogeneous neural network (H-Net) that integrates sMRI and fMRI modes for classifying multi-class NDs. To account for the differences between the two modes, H-Net adopts a heterogeneous neural network strategy to extract information from each mode. Specifically, H-Net includes an multi-layer perceptron based (MLP-based) encoder, a graph attention network based (GAT-based) encoder, and a cross-modality transformer block. The MLP-based and GAT-based encoders extract semantic features from sMRI and features from fMRI, respectively, while the cross-modality transformer block models the attention of two types of features. In H-Net, the proposed MLP-mixer block and cross-modality alignment are powerful tools for improving the multi-classification performance of NDs. H-Net is validate on the public dataset (CNP), where H-Net achieves 90% classification accuracy in diagnosing multi-class NDs. Furthermore, we demonstrate the complementarity of the two MRI modalities in improving the identification of multi-class NDs. Both visual and statistical analyses show the differences between ND subclasses.

Complex Motion Behavior and Synchronization Analysis of Heterogeneous Neural Network

Complex Motion Behavior and Synchronization Analysis of Heterogeneous Neural Network

Deep learning based retinal vessel segmentation and hypertensive retinopathy quantification using heterogeneous features cross-attention neural network.

Retinal vessels play a pivotal role as biomarkers in the detection of retinal diseases, including hypertensive retinopathy. The manual identification of these retinal vessels is both resource-intensive and time-consuming. The fidelity of vessel segmentation in automated methods directly depends on the fundus images' quality. In instances of sub-optimal image quality, applying deep learning-based methodologies emerges as a more effective approach for precise segmentation. We propose a heterogeneous neural network combining the benefit of local semantic information extraction of convolutional neural network and long-range spatial features mining of transformer network structures. Such cross-attention network structure boosts the model's ability to tackle vessel structures in the retinal images. Experiments on four publicly available datasets demonstrate our model's superior performance on vessel segmentation and the big potential of hypertensive retinopathy quantification.

Heterogeneous multi-agent task allocation based on graph neural network ant colony optimization algorithms

Heterogeneous multi-agent task allocation is a key optimization problem widely used in fields such as drone swarms and multi-robot coordination. This paper proposes a new paradigm that innovatively combines graph neural networks and ant colony optimization algorithms to solve the assignment problem of heterogeneous multi-agents. The paper introduces an innovative Graph-based Heterogeneous Neural Network Ant Colony Optimization (GHNN-ACO) algorithm for heterogeneous multi-agent scenarios. The multi-agent system is composed of unmanned aerial vehicles, unmanned ships, and unmanned vehicles that work together to effectively respond to emergencies. This method uses graph neural networks to learn the relationship between tasks and agents, forming a graph representation, which is then integrated into ant colony optimization algorithms to guide the search process of ants. Firstly, the algorithm in this paper constructs heterogeneous graph data containing different types of agents and their relationships and uses the algorithm to classify and predict linkages for agent nodes. Secondly, the GHNN-ACO algorithm performs effectively in heterogeneous multi-agent scenarios, providing an effective solution for node classification and link prediction tasks in intelligent agent systems. Thirdly, the algorithm achieves an accuracy rate of 95.31% in assigning multiple tasks to multiple agents. It holds potential application prospects in emergency response and provides a new idea for multi-agent system cooperation.

Event-triggered synchronization of a two-layer heterogeneous neural network via hybrid control

Multi-layer network contains multiple subsystems and different types of interaction, which is more practical than traditional single-layer network. On this observation, this paper explores point-to-point quasi-synchronization in a two-layer heterogeneous neural network system. The sufficient conditions for global exponential synchronization in the driving layer are given a coupling term. Based on the global exponential synchronization in the driver layer, this paper further studies the global point-to-point quasi-synchronization between the driver layer and the response layer of the two-layer neural network under the hybrid control and event-triggered control. Under this hybrid control strategies, the event can be activated if and only if the trigger condition is violated and the dynamical adaptive control is also involved. Moreover, the Zeno behavior under this strategy is eliminated by Lyapunov method and matrix theory. Finally, a numerical example is given to verify the theoretical results.

BCU-Net: Bridging ConvNeXt and U-Net for medical image segmentation

Medical image segmentation enables doctors to observe lesion regions better and make accurate diagnostic decisions. Single-branch models such as U-Net have achieved great progress in this field. However, the complementary local and global pathological semantics of heterogeneous neural networks have not yet been fully explored. The class-imbalance problem remains a serious issue. To alleviate these two problems, we propose a novel model called BCU-Net, which leverages the advantages of ConvNeXt in global interaction and U-Net in local processing. We propose a new multilabel recall loss (MRL) module to relieve the class imbalance problem and facilitate deep-level fusion of local and global pathological semantics between the two heterogeneous branches. Extensive experiments were conducted on six medical image datasets including retinal vessel and polyp images. The qualitative and quantitative results demonstrate the superiority and generalizability of BCU-Net. In particular, BCU-Net can handle diverse medical images with diverse resolutions. It has a flexible structure owing to its plug-and-play characteristics, which promotes its practicality.

Waveform-to-Waveform End-to-End Learning Framework in a Seamless Fiber-Terahertz Integrated Communication System

Seamless fiber-terahertz integrated communication has emerged as a promising technology in the special field of 6G, including mobile fronthaul and wireless bridges. Electronic terahertz communication systems have the advantages of high-level integration, small form factors, and potentially low costs, but their drawbacks are their small bandwidths and high harmonic interference levels. In this paper, an end-to-end learning-based waveform-to-waveform automatic equalization framework (W2WAEF) is proposed to overcome the above shortcomings in a seamless fiber-terahertz integrated communication system. An attention-based three-tributary heterogeneous neural network (ATTH) is designed to simulate the channel model of a seamless fiber-wireless communication system, taking fiber optics, optical-to-electrical conversion, and electrogenerated terahertz waves into account. With the proposed method, a data rate of 80.78 Gbps is experimentally demonstrated for discrete multitone modulation (DMT) signals over 5 km of fiber and 1 m of 209-GHz terahertz signals under a 20% soft decision-forward error correction (SD-FEC) threshold of 2.4 × 10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">−2</sup> . Compared with an approach without preprocessing, a receiver sensitivity gain exceeding 1.3 dB is successfully achieved at a data rate of 60 Gbps. The proposed method is a promising scheme for meeting the high speed and low-cost demands of future seamless fiber-terahertz integrated communication systems.

Intruder Detection in VANET Data Streams Using Federated Learning for Smart City Environments

Vehicular networks improve quality of life, security, and safety, making them crucial to smart city development. With the rapid advancement of intelligent vehicles, the confidentiality and security concerns surrounding vehicular ad hoc networks (VANETs) have garnered considerable attention. VANETs are intrinsically more vulnerable to attacks than wired networks due to high mobility, common network medium, and lack of centrally managed security services. Intrusion detection (ID) servers are the first protection layer against cyberattacks in this digital age. The most frequently used mechanism in a VANET is intrusion detection systems (IDSs), which rely on vehicle collaboration to identify attackers. Regrettably, existing cooperative IDSs get corrupted and cause the IDSs to operate abnormally. This article presents an approach to intrusion detection based on the distributed federated learning (FL) of heterogeneous neural networks for smart cities. It saves time and resources by using the most efficient intruder detection approach. First, vehicles use a federated learning technique to develop local, deep learning-based IDS classifiers for VANET data streams. They then share their locally learned classifiers upon request, significantly reducing communication overhead with neighboring vehicles. Then, an ensemble of federated heterogeneous neural networks is constructed for each vehicle, including locally and remotely trained classifiers. Finally, the global ensemble model is again shared with local devices for their updating. The effectiveness of the suggested method for intrusion detection in VANETs is evaluated using performance indicators such as attack detection rates, classification accuracy, precision, recall, and F1 scores over a ToN-IoT data stream. The ID model shows 0.994 training and 0.981 testing accuracy.

AINNS: All-Inclusive Neural Network Scheduling Via Accelerator Formalization

Driven by the rapid development of accelerators and diverse efficiency requirements of the naturally heterogeneous neural network computation, recent years have seen increased heterogeneity in neural network accelerator systems in terms of network structures, accelerator dataflows and implementations. However, existing research fails to schedule and map the heterogeneous neural networks on heterogeneous accelerators efficiently. They rely on clumpy exhaustive search or complicated ad hoc mapping approaches due to the semantic gap between the networks and accelerators. This paper proposes a systematic method to transform various accelerators into standard parameterized containers of the neural network loops, which builds a direct connection between the computation and the underlying hardware resources. This enables us to match the neural networks with accelerators based on their essential characteristics (e.g., reuse opportunities and bandwidth requirements) without diving into the detailed architectures. To this end, we propose AINNS, an all-inclusive neural network scheduler, that automatically schedules and maps the NN computation on heterogeneous accelerators with just one universal algorithm. Our experimental results show the proposed AINNS not only performs well in the traditional neural network acceleration but also improves the system throughput and energy efficiency by 1.8x and 1.7x respectively in the most challenging heterogeneous acceleration system.

Heterogeneous separation consistency training for adaptation of unsupervised speech separation

Recently, supervised speech separation has made great progress. However, limited by the nature of supervised training, most existing separation methods require ground-truth sources and are trained on synthetic datasets. This ground-truth reliance is problematic, because the ground-truth signals are usually unavailable in real conditions. Moreover, in many industry scenarios, the real acoustic characteristics deviate far from the ones in simulated datasets. Therefore, the performance usually degrades significantly when applying the supervised speech separation models to real applications. To address these problems, in this study, we propose a novel separation consistency training, termed SCT, to exploit the real-world unlabeled mixtures for improving cross-domain unsupervised speech separation in an iterative manner, by leveraging upon the complementary information obtained from heterogeneous (structurally distinct but behaviorally complementary) models. SCT follows a framework using two heterogeneous neural networks (HNNs) to produce high confidence pseudo labels of unlabeled real speech mixtures. These labels are then updated and used to refine the HNNs to produce more reliable consistent separation results for real mixture pseudo-labeling. To maximally utilize the large complementary information between different separation networks, a cross-knowledge adaptation is further proposed. Together with simulated dataset, those real mixtures with high confidence pseudo labels are then used to update the HNN separation models iteratively. In addition, we find that combing the heterogeneous separation outputs by a simple linear fusion can further slightly improve the final system performance. In this paper, we use cross-dataset to simulate the cross-domain situation in real-life. The term of “source domain” and “target domain” refer to the simulation set for model pre-training and the real unlabeled mixture for model adaptation. The proposed SCT is evaluated on both public reverberant English and anechoic Mandarin cross-domain separation tasks. Results show that, without any available ground-truth of target domain mixtures, the SCT can still significantly outperform our two strong baselines with up to 1.61 dB and 3.44 dB scale-invariant signal-to-noise ratio (SI-SNR) improvements, on the English and Mandarin cross-domain conditions, respectively.

Inferring disease-associated circRNAs by multi-source aggregation based on heterogeneous graph neural network

Emerging evidence has proved that circular RNAs (circRNAs) are implicated in pathogenic processes. They are regarded as promising biomarkers for diagnosis due to covalently closed loop structures. As opposed to traditional experiments, computational approaches can identify circRNA-disease associations at a lower cost. Aggregating multi-source pathogenesis data helps to alleviate data sparsity and infer potential associations at the system level. The majority of computational approaches construct a homologous network using multi-source data, but they lose the heterogeneity of the data. Effective methods that use the features of multi-source data are considered as a matter of urgency. In this paper, we propose a model (CDHGNN) based on edge-weighted graph attention and heterogeneous graph neural networks for potential circRNA-disease association prediction. The circRNA network, micro RNA network, disease network and heterogeneous network are constructed based on multi-source data. To reflect association probabilities between nodes, an edge-weighted graph attention network model is designed for node features. To assign attention weights to different types of edges and learn contextual meta-path, CDHGNN infers potential circRNA-disease association based on heterogeneous neural networks. CDHGNN outperforms state-of-the-art algorithms in terms of accuracy. Edge-weighted graph attention networks and heterogeneous graph networks have both improved performance significantly. Furthermore, case studies suggest that CDHGNN is capable of identifying specific molecular associations and investigating biomolecular regulatory relationships in pathogenesis. The code of CDHGNN is freely available at https://github.com/BioinformaticsCSU/CDHGNN.

Event-Based Impulsive Control for Heterogeneous Neural Networks with Communication Delays

The quasi-synchronization for a class of general heterogeneous neural networks is explored by event-based impulsive control strategy. Compared with the traditional average impulsive interval (AII) method, instead, an event-triggered mechanism (ETM) is employed to determine the impulsive instants, in which case the subjectivity of selecting the controlling sequence can be eliminated. In addition, considering the fact that communication delay is inevitable between the allocation and execution of instructions in practice, we further nominate an ETM centered on communication delays and aperiodic sampling, which is more accessible and affordable, yet can straightforwardly avoid Zeno behavior. Hence, on the basis of the novel event-triggered impulsive control strategy, quasi-synchronization of heterogeneous neural network model is investigated and some general conditions are also achieved. Finally, two numerical simulations are afforded to validate the efficacy of theoretical results.

Asynchronously intermittent decentralized control for synchronization of stochastic delayed networks

This paper studies the asynchronously intermittent decentralized control for the synchronization of stochastic complex delayed networks, where the “asynchronously intermittent” means that the intermittent control for each subsystem is activated asynchronously, i.e., each subsystem possesses its own work or rest time. By designing an auxiliary timer and using a graph-theoretic technique as well as the Halanay’s inequality for piecewise continuous function, we obtain a novel synchronization criterion that is independent of the bound of time-varying delay. Finally, a class of coupled heterogeneous neural networks is considered to show the effectiveness and the superiority of the main result.

Multi-Vehicle Tracking Using Heterogeneous Neural Networks for Appearance And Motion Features

Multi-Vehicle Tracking Using Heterogeneous Neural Networks for Appearance And Motion Features