Efficient long-distance latent relation-aware graph neural network for multi-modal federated emotion recognition

Glycans play an indispensable role in various bio-logical processes, such as cancer and autoimmune diseases. The function of glycan is closely determined by its structure. Due to the branch and nonlinear properties of glycans, previous research treats the glycans graph structure as a topological graph to represent glycans data effectively. Graph neural networks (GNNs) are an efficient graph mining method and have many applications in bioinformatics. Therefore, researchers have successfully used handcrafted GNNs to predict glycan immunogenicity. However, a GNN architecture contains many different components, and designing GNN architectures for specific graphs in the bioinformatics field is time-consuming and expert-dependent. To address this challenge, we propose an efficient automatic graph neural network method called EAGNN that can efficiently and automatically construct GNN architecture for glycan immunogenicity prediction. We design an effective graph attention pooling (GAP) search space. We use differential architecture search to efficiently create the optimal GNN architecture in the search space to build the GNN model for glycan immunogenicity prediction. We test EAGNN on the data set SugarBase based on the glycan immunogenicity prediction task. The experiment results show that EAGNN can work more superiorly than the baseline model and achieve the best performance.

Emotion Recognition plays a key role in human social activities. Most current methods choose facial information for emotion recognition. But facial information is easily limited by distance in large scenes. Gait information can break this limitation, but the amount of publicly available data is small. This paper proposes two feature fusion network based on efficient deep residual shrinkage spatial-temporal graph convolution to achieve automatic emotion recognition from gaits in large scenes. Firstly chebyshev polynomial approximation graph convolution is combined with efficient deep residual shrinkage network to obtain efficient deep residual shrinkage spatial-temporal graph convolution network to extract deep features. Then sixteen temporal affective features extracted from gaits are used as input of efficient deep residual shrinkage network to extract new affective features. Finally the deep features is spliced with new affective features to divided emotion into happy,angry,sad and neutral. We performed a ten-flod cross-validation experiment on the E-gait real datasets. The highest recognition accuracy is 0.87, with an average of 0.85. Experimental results show that the proposed model is effective in dealing with emotion recognition promblem from gait in fewer sample situation.

With the increasing availability of large-scale datasets and powerful computing paradigms, convolutional neural networks (CNNs) have empowered a wide range of intelligent embedded vision tasks, which span from image classification to downstream vision tasks, such as on-device object recognition, detection, and tracking. In the past few years, convolutional networks have been evolving deeper and wider in order to maintain superior accuracy on target task. This rule of thumb, despite its efficacy, leads to an exponential growth in the number of floating-point operations (FLOPs) and parameters. For example, ResNet50, as one of the most representative convolutional networks, consists of over 4 billion FLOPs and 25 million parameters. The prohibitive network complexity, as a result, further enlarges the computational gap between computation-intensive CNNs and resource-constrained embedded platforms, making it challenging to develop hardware-friendly network solutions to accommodate the limited available computational resources in real-world embedded scenarios towards embedded intelligence. This thesis focuses on alleviating the above computational gap from the perspective of hardware-aware neural architecture search (NAS) and compression. First of all, we introduce SurgeNAS for efficient architecture search. Specifically, SurgeNAS turns back to one-level optimization for accurate and consistent gradient estimation, which also features an effective identity mapping scheme in order to avoid the search collapse. In addition, we introduce an efficient ordered differentiable sampling approach to reduce the memory consumption to the single-path level, while at the same time maintaining strict search fairness. An efficient graph neural networks (GNNs) based latency predictor is further proposed and integrated into the search engine to avoid tedious on-device latency measurements during the search process. Finally, we introduce the paradigm of Comfort Zone, which allows us to scale up the searched architecture candidates to achieve better accuracy on target task without degrading the inference efficiency on target hardware. Furthermore, we introduce LightNAS for flexible architecture search. The motivation behind LightNAS is that previous relevant NAS methods, including SurgeNAS, simply focus on reducing the explicit search cost -- the time for one single search, while ignoring the huge implicit search cost -- the time for manual hyper-parameter tuning to derive the required architecture candidate. In practice, previous relevant NAS methods have to perform manual hyper-parameter tuning in order to navigate the required architecture candidate that satisfies the specified latency constraint, which empirically involves 10 trial-and-errors and thus significantly increases the total search cost by 10 times. In contrast, LightNAS only requires one single search for any specified latency constraint (i.e., you only search once). In addition, we introduce an efficient yet reliable proxy, namely batchwise training estimation (BTE), which can be seamlessly integrated into LightNAS to enable channel-level explorations at low computational cost. This further boosts the attainable accuracy on target task without degrading the efficiency on target hardware. Finally, we introduce Domino for efficient network compression, in which we pioneer to revisit the trade-off dilemma between accuracy and efficiency from a fresh perspective of linearity and non-linearity. Specifically, Domino focuses on trading the less important network non-linearity for better network efficiency. To this end, Domino leverages two efficient performance predictors, including one vanilla latency predictor and one meta-accuracy predictor, to explore less important non-linear building blocks, which are then grafted with their linear counterparts. The resulting grafted network is further trained on target task to achieve decent accuracy. Finally, we reparameterize each grafted linear building block that consists of multiple consecutive linear layers, including multiple convolutional, batch normalization (BN), and grafted linear activation layers, into one single convolutional layer to aggressively boost the efficiency on target hardware, and more importantly, without sacrificing the accuracy on target task since the network maintains the same output regardless of linear reparameterization. In summary, this thesis focuses on hardware-aware neural architecture search and compression to deliver efficient network solutions for resource-constrained embedded platforms to empower embedded intelligence. Future research will continue to explore more general search spaces and more advanced search/compression techniques to develop more efficient networks for intelligent embedded applications.

Heterogeneous graph neural networks (HGNNs) have the powerful capability to embed rich structural and semantic information of a heterogeneous graph into node representations. Existing HGNNs inherit many mechanisms from graph neural networks (GNNs) designed for homogeneous graphs, especially the attention mechanism and the multi-layer structure. These mechanisms bring excessive complexity, but seldom work studies whether they are really effective on heterogeneous graphs. In this paper, we conduct an in-depth and detailed study of these mechanisms and propose the Simple and Efficient Heterogeneous Graph Neural Network (SeHGNN). To easily capture structural information, SeHGNN pre-computes the neighbor aggregation using a light-weight mean aggregator, which reduces complexity by removing overused neighbor attention and avoiding repeated neighbor aggregation in every training epoch. To better utilize semantic information, SeHGNN adopts the single-layer structure with long metapaths to extend the receptive field, as well as a transformer-based semantic fusion module to fuse features from different metapaths. As a result, SeHGNN exhibits the characteristics of a simple network structure, high prediction accuracy, and fast training speed. Extensive experiments on five real-world heterogeneous graphs demonstrate the superiority of SeHGNN over the state-of-the-arts on both accuracy and training speed.

Efficient complementary graph convolutional network without negative sampling for item recommendation

Graph neural networks have been widely used to learn node representations for many real-world static graphs. In general, they learn node representations by recursively aggregating information from neighbors. However, graphs in many applications are dynamic, evolving with continuous graph events, such as node feature and graph structure updates. These events require the node representations to be updated accordingly. Currently, due to the real-time requirement, how to efficiently and reliably update node representations under continuous graph events is still an open problem. Recent studies propose two solutions to partially address this problem, but their performance is still limited. First, local-based GNNs only update the nodes directly involved in events, suffering from the quality-deficit issue, since they neglect the other nodes affected by these events. Second, neighbor-sampling GNNs propose to sample neighbors to accelerate neighbor aggregation computations, encountering the neighbor-redundant issue. These sampled neighbors may be similar and cannot reflect the distribution of all neighbors, leading that node representations aggregated on these redundant neighbors may differ from those aggregated on all neighbors. In this paper, we propose an efficient and reliable graph neural network, namely EARLY, to update node representations for dynamic graphs. We first identify the top-k influential nodes that are most affected by graph events. Then, to sample neighbors diversely, we propose a diversity-aware layer-wise sampling technique. We theoretically demonstrate that this technique can decrease the sampling expectation error and learn more reliable node representations. Therefore, the top-k nodes selection and diversity-aware sampling enable EARLY to efficiently update node representations in a reliable way. Extensive experiments on the five real-world graphs demonstrate the effectiveness and efficiency of our proposed EARLY.

In the era of Internet of Things (IoT), intelligent recommendation is playing an important role in our daily life. How to provide personalized information to users is the core concern of Internet content service providers. To improve the recommendation quality, it is a hot topic to go beyond merely user–item interaction records and take social relations into account in IoT. Recently, emerged graph neural networks (GNNs) shine a light on simulating the recursive social diffusion process, to refine user embedding learning. Nevertheless, two key issues have not been well studied in previous studies: 1) they usually model user preference and social influence within a same semantic space and fail to simultaneously inject high-order connectivity information reflected in both user–item interaction graph and user–user social graph and 2) they typically rely on negative sampling to optimize a recommendation model, which makes them highly sensitive to the design of the sampler and hardly makes full use of GPU’s computing ability. In light of this, we propose a novel framework for item recommendation, namely, an efficient adaptive graph convolutional network (EAGCN). Specifically, we introduce a space-adaptive graph convolutional module, which could jointly explore the propagation process of user interest and social influence. Furthermore, a user-specific gating mechanism is designed to aggregate user representations from both spaces. To make EAGCN practical in social IoT, we devise a fast nonsampling leaner to optimize EAGCN’s parameters with better leveraging matrix computing of GPU. Extensive experiments under four scenarios show that our solution consistently and significantly outperforms strong baseline methods in both model effectiveness and training efficiency.

As a frontier field of individualized therapy, microRNA (miRNA) pharmacogenomics facilitates the understanding of different individual responses to certain drugs and provides a reasonable reference for clinical treatment. However, the known drug resistance-associated miRNAs are not yet sufficient to support precision medicine. Although existing methods are effective, they all focus on modelling miRNA-drug resistance interaction graphs, making their performance bounded by the interaction density. In this study, we propose a framework for miRNA-drug resistance prediction through efficient neural architecture search and graph isomorphism networks (NASMDR). NASMDR uses attribute information instead of the commonly used interactive graph information. In the cross-validation experiment, the proposed framework can achieve an AUC of 0.9468 on the ncDR dataset, which is 2.29% higher than the state-of-the-art method. In addition, we propose a novel sequence characterization approach, k-mer Sparse Nonnegative Matrix Factorization (KSNMF). The results show that NASMDR provides novel insights for integrating efficient neural architecture search and graph isomorphic networks into a unified framework to predict drug resistance-related miRNAs. The codes for NASMDR are available at https://github.com/kaizheng-academic/NASMDR.

A graph is a widely used and effective data structure in many applications; it describes the relationships among nodes or entities. Currently, most semi-supervised or unsupervised graph neural network models are trained based on a very basic operation called negative sampling. Usually, the purpose of the learning objective is to maximize the similarity between neighboring nodes while minimizing the similarity between nodes that are not close to each other. Negative sampling can reduce the time complexity by sampling a small fraction of the negative nodes instead of using all of the negative nodes when optimizing the objective. However, sampling of the negative nodes may fail to deliver stable model performance due to the uncertainty in the sampling procedure. To avoid such disadvantages, we provide an efficient Non-Sampling Graph Neural Network (NS-GNN) framework. The main idea is to use all the negative samples when optimizing the learning objective to avoid the sampling process. Of course, directly using all of the negative samples may cause a large increase in the model training time. To mitigate this problem, we rearrange the origin loss function into a linear form and take advantage of meticulous mathematical derivation to reduce the complexity of the loss function. Experiments on benchmark datasets show that our framework can provide better efficiency at the same level of prediction accuracy compared with existing negative sampling-based models.

Graph Neural Networks (GNNs) have attracted increasing research interest for their remarkable capability to model graph-structured knowledge. However, GNNs suffer from intensive data exchange and poor data locality, which will cause critical performance and energy bottlenecks under conventional complementary metal oxide semiconductor (CMOS)-based von-Neumann computing architectures (graphics processing unit (GPU), central processing unit (CPU)) for the “Memory Wall” issue. Fortunately, memristive crossbar-based computation has emerged as one of the most promising neuromorphic computing architectures, which has been widely studied as the computing platform for convolutional neural network (CNNs), recurrent neural network (RNNs), spiking neural network (SNNs), etc. This paper proposes the deployment of spectral graph convolutional networks (GCNs) on memristive crossbars. Further, based on the structure of GCNs (extremely high sparsity and unbalanced non-zero data distribution) and the neuromorphic characteristics of memristive crossbar circuit, we propose the acceleration method that consists of Sparse Laplace Matrix Reordering and Diagonal Block Matrix Multiplication. The simulated experiment on memristor crossbars achieves 90.3% overall accuracy on the supervised learning graph dataset (QM7), and compared with the original computation, the proposed acceleration computing framework (with half-size diagonal blocks) achieves a 27.3% reduction of memristor number. Additionally, on the unsupervised learning dataset (Karate club), our method shows no loss of accuracy with half-size diagonal block mapping and reaches a 32.2% reduction of memristor number.

Drowsy driving is a significant contributor to road traffic accidents, necessitating the development of robust and efficient detection systems. Existing approaches for detecting drowsy drivers often rely on single modalities such as image-based facial analysis or vehiclebased metrics, which may lack precision and be susceptible to environmental variations. Furthermore, the existing models might be computationally expensive and result in delayed responses. To address these limitations, this paper proposes a novel Multimodal Iterative Deep Graph Network Wearable Interface that leverages a rich set of data including heart rate, respiration, accelerometer movement, location (speed), traffic, and weather data. These data samples are processed through a fusion of different Recurrent Neural Network processes. The Bidirectional Long Short-Term Memory (BiLSTM) which is used to generate augmented features, is fused with Bidirectional Gated Recurrent Unit (BiGRU) which assists in estimating multidomain feature sets. These features are then used to train an efficient Deep Graph Recurrent Network (DGRN), which facilitates the real-time identification of drowsy drivers. The proposed model significantly enhances the precision of drowsy driver detection by 8.5%, accuracy by 9.5%, recall by 8.3%, and Area Under the Curve (AUC) by 4.9%, while also improving specificity by 5.5% and reducing delay by 10.4%. These improvements collectively contribute to a substantial enhancement in road safety, potentially saving lives by preventing accidents due to drowsy driving. The proposed system, with its real-time processing capability and high accuracy, stands as a significant improvement for identifying drowsy drivers, paving the way for safer and smarter transportation systems.

In general-purpose particle detectors, the particle-flow algorithm may be used to reconstruct a comprehensive particle-level view of the event by combining information from the calorimeters and the trackers, significantly improving the detector resolution for jets and the missing transverse momentum. In view of the planned high-luminosity upgrade of the CERN Large Hadron Collider (LHC), it is necessary to revisit existing reconstruction algorithms and ensure that both the physics and computational performance are sufficient in an environment with many simultaneous proton–proton interactions (pileup). Machine learning may offer a prospect for computationally efficient event reconstruction that is well-suited to heterogeneous computing platforms, while significantly improving the reconstruction quality over rule-based algorithms for granular detectors. We introduce MLPF, a novel, end-to-end trainable, machine-learned particle-flow algorithm based on parallelizable, computationally efficient, and scalable graph neural network optimized using a multi-task objective on simulated events. We report the physics and computational performance of the MLPF algorithm on a Monte Carlo dataset of top quark–antiquark pairs produced in proton–proton collisions in conditions similar to those expected for the high-luminosity LHC. The MLPF algorithm improves the physics response with respect to a rule-based benchmark algorithm and demonstrates computationally scalable particle-flow reconstruction in a high-pileup environment.

Several brain disorders can be detected by observing alterations in the brain’s structural and functional connectivities. Neurological findings suggest that early diagnosis of brain disorders, such as mild cognitive impairment (MCI), can prevent and even reverse its development into Alzheimer’s disease (AD). In this context, recent studies aimed to predict the evolution of brain connectivities over time by proposing machine learning models that work on brain images. However, such an approach is costly and time-consuming. Here, we propose to use brain connectivities as a more efficient alternative for time-dependent brain disorder diagnosis by regarding the brain as instead a large interconnected graph characterizing the interconnectivity scheme between several brain regions. We term our proposed method Recurrent Brain Graph Mapper (RBGM), a novel efficient edge-based recurrent graph neural network that predicts the time-dependent evaluation trajectory of a brain graph from a single baseline. Our RBGM contains a set of recurrent neural network-inspired mappers for each time point, where each mapper aims to project the ground-truth brain graph onto its next time point. We leverage the teacher forcing method to boost training and improve the evolved brain graph quality. To maintain the topological consistency between the predicted brain graphs and their corresponding ground-truth brain graphs at each time point, we further integrate a topological loss. We also use l1 loss to capture time-dependency and minimize the distance between the brain graph at consecutive time points for regularization. Benchmarks against several variants of RBGM and state-of-the-art methods prove that we can achieve the same accuracy in predicting brain graph evolution more efficiently, paving the way for novel graph neural network architecture and a highly efficient training scheme. Our RBGM code is available at https://github.com/basiralab/RBGM.

On large-scale graphs, many graph neural networks are problematic in capturing long-range dependencies due to the oversmoothing problem. Recently, Graph Equilibrium Models (GEQs) arise as a promising solution to this issue. Their output is the equilibrium of a fixed-point equation, which can be seen as the result of iterating a GNN layer for infinite times, so that they inherently have global receptive fields. However, to find the equilibrium, GEQs require running costly full-batch root-finding algorithms from scratch during each model update, which leads to severe efficiency and scalability issues that prevent them from scaling to large graphs. To address these limitations, we propose VEQ, an efficient learning method to scale GEQs to large graphs. Instead of initializing the equilibrium from scratch in full-batch training, VEQ uses the latest equilibrium of in-batch nodes and their 1-hop neighbors (dubbed Virtual Equilibrium) to accelerate and calibrate the root-finding process in mini-batch training. With virtual equilibrium as an informative prior, VEQ is able to reach the equilibrium in fewer steps while still capturing global dependencies. Theoretically, we provide convergence analysis for the forward and backward pass of VEQ. Empirically, VEQ significantly outperforms existing GEQs by a large margin (more than 1.5%) on all benchmark datasets, with much less training time and memory. Also, VEQ achieves competitive and even superior performance to many highly engineered explicit GNNs on large-scale benchmark datasets like ogbn-arxiv and ogbn-products. VEQ shows that after we resolve the efficiency and scalability issues, GEQs are indeed favorable on large graphs due to their advantage of capturing long-range dependencies.

Graph neural networks are attractive for learning properties of atomic structures thanks to their intuitive graph encoding of atoms and bonds. However, conventional encoding does not include angular information, which is critical for describing atomic arrangements in disordered systems. In this work, we extend the recently proposed ALIGNN (Atomistic Line Graph Neural Network) encoding, which incorporates bond angles, to also include dihedral angles (ALIGNN-d). This simple extension leads to a memory-efficient graph representation that captures the complete geometry of atomic structures. ALIGNN-d is applied to predict the infrared optical response of dynamically disordered Cu(II) aqua complexes, leveraging the intrinsic interpretability to elucidate the relative contributions of individual structural components. Bond and dihedral angles are found to be critical contributors to the fine structure of the absorption response, with distortions that represent transitions between more common geometries exhibiting the strongest absorption intensity. Future directions for further development of ALIGNN-d are discussed.