Graph Transformation Research Articles

Micro-expression recognition based on a novel GCN-transformer cooperation model for IoT-eHealth

If a person is truly healthy, his/her well-being encompasses both physical and psychological health. However, the existing IoT-eHealth system typically focus only on monitoring the user’s physical data through various sensors, neglecting their mental state. To enhance the intelligence level of IoT-eHealth system and enable it to have the psychological monitoring ability, a novel collaborative model based on Graph Convolutional Network (GCN) and Transformer is designed for Micro-Expression (ME) recognition in this paper. Firstly, facial information within each frame is transformed into a Spatial Topological Relationship Graph (STRG) by using facial landmarks detection and psychological relationship of local patches. Then, in order to automatically aggregate the key information on facial patches that contribute to ME recognition from the structured graph data, a Hierarchical Adaptive Graph Pooling (HAGP) module is designed for obtaining discriminative frame-level feature based on GCN utilizing graph structure and vertex global dependencies. Finally, in order to model the long-term dependencies among frames and capture the key frame-level features that are beneficial for ME recognition, a Temporal Sensitive Self-Attention (TSSA) mechanism is designed, and a novel Temporal Sensitive Transformer (TST) encoder is constructed based on TSSA to explore the evolution law of facial patterns and obtain discriminative video-level features that are helpful for ME recognition. In the comparative experiments of standard dataset verification and practical dataset testing, designed collaborative model is superior to other methods and can achieve the highest recognition accuracy, which almost can meet the application requirements of IoT-eHealth system.

A Health Monitoring Model for Circulation Water Pumps in a Nuclear Power Plant Based on Graph Neural Network Observer.

The health monitoring of CRF (circulation water) pumps is essential for prognostics and management in nuclear power plants. However, the operational status of CRF pumps can vary due to environmental factors and human intervention, and the interrelationships between monitoring parameters are often complex. Consequently, the existing methods face challenges in effectively assessing the health status of CRF pumps. In this study, we propose a health monitoring model for CRF pumps utilizing a meta graph transformer (MGT) observer. Initially, the meta graph transformer, a temporal-spatial graph learning model, is employed to predict trends across the various monitoring parameters of the CRF pump. Subsequently, a fault observer is constructed to generate early warnings of potential faults. The proposed model was validated using real data from CRF pumps in a nuclear power plant. The results demonstrate that the average Mean Absolute Percentage Error (MAPE), Mean Absolute Error (MAE), and Root Mean Square Error (RMSE) of normal predictions were reduced to 1.2385, 0.5614, and 2.6554, respectively. These findings indicate that our model achieves higher prediction accuracy compared to the existing methods and can provide fault warnings at least one week in advance.

Multidimensional graph transformer networks for trajectory prediction in urban road intersections

Multidimensional graph transformer networks for trajectory prediction in urban road intersections

Transformer technology in molecular science

AbstractA transformer is the foundational architecture behind large language models designed to handle sequential data by using mechanisms of self‐attention to weigh the importance of different elements, enabling efficient processing and understanding of complex patterns. Recently, transformer‐based models have become some of the most popular and powerful deep learning (DL) algorithms in molecular science, owing to their distinctive architectural characteristics and proficiency in handling intricate data. These models leverage the capacity of transformer architectures to capture complex hierarchical dependencies within sequential data. As the applications of transformers in molecular science are very widespread, in this review, we only focus on the technical aspects of transformer technology in molecule domain. Specifically, we will provide an in‐depth investigation into the algorithms of transformer‐based machine learning techniques in molecular science. The models under consideration include generative pre‐trained transformer (GPT), bidirectional and auto‐regressive transformers (BART), bidirectional encoder representations from transformers (BERT), graph transformer, transformer‐XL, text‐to‐text transfer transformer, vision transformers (ViT), detection transformer (DETR), conformer, contrastive language‐image pre‐training (CLIP), sparse transformers, and mobile and efficient transformers. By examining the inner workings of these models, we aim to elucidate how their architectural innovations contribute to their effectiveness in processing complex molecular data. We will also discuss promising trends in transformer models within the context of molecular science, emphasizing their technical capabilities and potential for interdisciplinary research. This review seeks to provide a comprehensive understanding of the transformer‐based machine learning techniques that are driving advancements in molecular science.This article is categorized under: Data Science > Chemoinformatics Data Science > Artificial Intelligence/Machine Learning

Fault Line Selection Method for Power Distribution Network Based on Graph Transformation and ResNet50 Model

Fault Line Selection Method for Power Distribution Network Based on Graph Transformation and ResNet50 Model

Self-supervised BGP-graph reasoning enhanced complex KBQA via SPARQL generation

Knowledge base question answering aims to answer complex questions from large-scale knowledge bases. Although existing generative language models that translate questions into SPARQL queries have achieved promising results, there are still generation errors due to redundancies or errors in the knowledge fed to the generative models and difficulties in representing the implicit logic of knowledge as the specific syntax of SPARQL. To address above issues, we propose TrackerQA, a novel self-supervised reasoning framework based on basic graph patterns (BGP) to determine precise paths and enhance SPARQL generation. First, we develop a contrastive learning semantic matching model to reduce the large knowledge searching space. Then, we built a BGP parser that parses the recalled knowledge and constraints into BGP graphs, which can deconstruct complex knowledge into BGP triples and naturally obtain supervision from gold SPARQL. Next, we design a self-supervised BGP graph neural network that encodes knowledge through graph transformation layers with directed message-passing control and employs a question-aware attention mechanism to predict the exact BGP paths. Finally, a SPARQL generator integrates the paths into a pre-trained language model to improve the performance of SPARQL generation. Experiments on the KQA Pro dataset show that our model achieves state-of-the-art answering accuracy scores of 95.32%, being the closest to the human level at 97.5%, and reasons out KB paths with F1 scores of 0.98 for nodes and 0.99 for edges.

AG-SpTRSV: An Automatic Framework to Optimize Sparse Triangular Solve on GPUs

Sparse Triangular Solve (SpTRSV) has long been an essential kernel in the field of scientific computing. Due to its low computational intensity and internal data dependencies, SpTRSV is hard to implement and optimize on GPUs. Based on our experimental observations, existing implementations on GPUs fail to achieve the optimal performance due to their sub-optimal parallelism setups and code implementations, and lack of consideration of the irregular data distribution. Moreover, their algorithm design lacks the adaptability to different input matrices, which may involve substantial manual efforts of algorithm redesigning and parameter tuning for performance consistency. In this work, we propose AG-SpTRSV, an automatic framework to optimize SpTRSV on GPUs, which provides high performance on various matrices while eliminating the costs of manual design. AG-SpTRSV abstracts the procedures of optimizing an SpTRSV kernel as a scheme and constructs a comprehensive optimization space based on it. By defining a unified code template and preparing code variants, AG-SpTRSV enables fine-grained dynamic parallelism and adaptive code optimizations to handle various tasks. Through computation graph transformation and multi-hierarchy heuristic scheduling, AG-SpTRSV generates schemes for task partitioning and mapping, which effectively address the issues of irregular data distribution and internal data dependencies. AG-SpTRSV searches for the best scheme to optimize the target kernel for the specific matrix. A learned lightweight performance model is also introduced to reduce search costs and provide an efficient end-to-end solution. Experimental results with SuiteSparse Matrix Collection on NVIDIA Tesla A100 and RTX 3080 Ti show that AG-SpTRSV outperforms state-of-the-art implementations with geometric average speedups of 2.12x ∼ 3.99x. With the performance model enabled, AG-SpTRSV can provide an efficient end-to-end solution, with preprocessing times ranging from 3.4 to 245 times of the execution time.

HGTDR: Advancing drug repurposing with heterogeneous graph transformers.

Drug repurposing is a viable solution for reducing the time and cost associated with drug development. However, thus far, the proposed drug repurposing approaches still need to meet expectations. Therefore, it is crucial to offer a systematic approach for drug repurposing to achieve cost savings and enhance human lives. In recent years, using biological network-based methods for drug repurposing has generated promising results. Nevertheless, these methods have limitations. Primarily, the scope of these methods is generally limited concerning the size and variety of data they can effectively handle. Another issue arises from the treatment of heterogeneous data, which needs to be addressed or converted into homogeneous data, leading to a loss of information. A significant drawback is that most of these approaches lack end-to-end functionality, necessitating manual implementation and expert knowledge in certain stages. We propose a new solution, Heterogeneous Graph Transformer for Drug Repurposing (HGTDR), to address the challenges associated with drug repurposing. HGTDR is a three-step approach for knowledge graph-based drug repurposing: (1) constructing a heterogeneous knowledge graph, (2) utilizing a heterogeneous graph transformer network, and (3) computing relationship scores using a fully connected network. By leveraging HGTDR, users gain the ability to manipulate input graphs, extract information from diverse entities, and obtain their desired output. In the evaluation step, we demonstrate that HGTDR performs comparably to previous methods. Furthermore, we review medical studies to validate our method's top 10 drug repurposing suggestions, which have exhibited promising results. We also demonstrated HGTDR's capability to predict other types of relations through numerical and experimental validation, such as drug-protein and disease-protein inter-relations. The source code and data are available at https://github.com/bcb-sut/HGTDR and http://git.dml.ir/BCB/HGTDR.

Graph transformer embedded deep learning for short-term passenger flow prediction in urban rail transit systems: A multi-gate mixture-of-experts model

Urban rail transit (URT) plays a crucial role in mitigating urban traffic congestion by offering faster and higher-quality travel services. Short-term passenger flow predictions have practical significance for metro management and operation. However, the complex spatiotemporal characteristics and the relationship between entry and exit passenger flows make it challenging to detect the dynamic evolution patterns. This study proposes a Spatio-Temporal Graph Transformer (STGT) under the multi-task learning framework, utilizing Graph Transformer network and gated residual units to select and aggregate features. To account for the correlation between entry and exit passenger flow prediction tasks, the STGT model integrates a Multi-gate Mixture-of-Experts (MMoE) approach, which combines different expert networks for diverse input and explicitly learns to model passenger flow relationships in various scenarios. Metro-related characteristics such as weather conditions, train operation characteristics, and accessibility of nearby bus stops are incorporated to enhance prediction accuracy. Experimental evaluations are conducted using real-world historical passenger travel records from the Beijing subway. The results demonstrate the superior robustness and advantages of the STGT-MMoE model over basic and advanced benchmarks for passenger flow prediction tasks. These findings provide compelling evidence to address the challenges of short-term inflow and outflow prediction in urban rail transit systems.

A spatiotemporal graph transformer approach for Alzheimer’s disease diagnosis with rs-fMRI

Alzheimer’s disease (AD) is a neurodegenerative disease accompanied by cognitive impairment. Early diagnosis is crucial for the timely treatment and intervention of AD. Resting-state functional magnetic resonance imaging (rs-fMRI) records the temporal dynamics and spatial dependency in the brain, which have been utilized for automatically diagnosis of AD in the community. Existing approaches of AD diagnosis using rs-fMRI only assess functional connectivity, ignoring the spatiotemporal dependency mining of rs-fMRI. In addition, it is difficult to increase diagnosis accuracy due to the shortage of rs-fMRI sample and the poor anti-noise ability of model. To deal with these problems, this paper proposes a novel approach for the automatic diagnosis of AD, namely spatiotemporal graph transformer network (STGTN). The proposed STGTN can effectively extract spatiotemporal features of rs-fMRI. Furthermore, to solve the sample-limited problem and to improve the anti-noise ability of the proposed model, an adversarial training strategy is adopted for the proposed STGTN to generate adversarial examples (AEs) and augment training samples with AEs. Experimental results indicate that the proposed model achieves the classification accuracy of 92.58%, and 85.27% with the adversarial training strategy for AD vs. normal control (NC), early mild cognitive impairment (eMCI) vs. late mild cognitive impairment (lMCI) respectively, outperforming the state-of-the-art methods. Besides, the spatial attention coefficients reflected from the designed model reveal the importance of brain connections under different classification tasks.

A Spatio-temporal Graph Transformer driven model for recognizing fine-grained data human activity

Human activity recognition(HAR) is an important research focus in ubiquitous computing. It has been widely applied in various domains, such as smart homes, healthcare assistance, and sports training. Accurate human activity recognition directly impacts the performance of downstream tasks. Existing methods for human activity recognition primarily rely on extracting temporal or spatial features from the data. These features are used for the task of human activity recognition. The existing methods mainly use CNN or RNN models to model the Euclidean correlations among spatially adjacent sensors or channels. However, non-Euclidean pairwise correlations among all sensors or channels are even critical for accurate classification, which has been ignored by the existing methods. In this paper, we incorporate fine-grained spatial structural data into the model to overcome these limitations. A novel deep learning model for human activity recognition is proposed, which is called the fine-grained data-oriented Spatial-Temporal Graph Transformer network (STGT). The introduction of the STGT model can eliminate the limitations of existing spatial feature extraction methods by leveraging a novel data organization approach proposed in this study. This model enhances the spatial features within the time feature extraction module for effective spatio-temporal feature extraction. We conducted experiments on four large-scale real-world HAR datasets to evaluate its performance. The experimental results demonstrate the superiority of our method over state-of-the-art approaches.

Formulate an Adaptive Technique to Validate Near Field Communication Technology using Attributed Graph Grammar-AGG tool

Recently, developments in such Near field Communication Technology (NFC) and associated integrated devices have pace out an acceleration the demand of transactions in electronic devices system such wearable readers which have short-power consumption and proficiency in high transferring data. NFC has showed the alternative active key solution to existing smart electronic devices that depend on huge batteries to supply the energy in paying bills though such devices. NFC widely utilized via wireless power transfer, where devices share and connect with each other, to increase the efficiency of transferring data, with high superb properties such as permeability. NFC has emerged as a powerful technology and its apparatus devices. The ongoing of NFC growth could be related to its extensibility and more sustainable. NFC can deliver different shapes at providing software application capabilities to connect with each other. NFCs demonstrates an adaptable method to enhance and offer current elements accessible to applications across the Internet. NFCs are a new fad and modern technology that shares embedded and ubiquitous devices with others, various corporations are finding the advantage by the NFC. This research will propose a formulation technique to explore and adapt an approach of developing and validating the NFC which permits to exchange data via technology wireless (NFC) between the parties. This technique will facilitate the feasibility of having e.g., a system to reduce of losing duration time and cost for users, as well as, an agile technique for validating the suggested approach via Attributed Graph Grammar (AGG) tools.

CGT: A Clause Graph Transformer Structure for aspect-based sentiment analysis

In the realm of natural language processing (NLP), aspect-based sentiment analysis plays a pivotal role. Recently, there has been a growing emphasis on techniques leveraging Graph Convolutional Neural Network (GCN). However, there are several challenges associated with current approaches: (1) Due to the inherent transitivity of CGN, training inevitably entails the acquisition of irrelevant semantic information. (2) Existing methodologies heavily depend on the dependency tree, neglecting to consider the contextual structure of the sentence. (3) Another limitation of the majority of methods is their failure to account for the interactions occurring between different aspects. In this study, we propose a Clause Graph Transformer Structure (CGT) to alleviate these limitations. Specifically, CGT comprises three modules. The preprocessing module extracts aspect clauses from each sentence by bi-directionally traversing the constituent tree, reducing reliance on syntax trees and extracting semantic information from the perspective of clauses. Additionally, we assert that a word’s vector direction signifies its underlying attitude in the semantic space, a feature often overlooked in recent research. Without the necessity for additional parameters, we introduce the Clause Attention encoder (CA-encoder) to the clause module to effectively capture the directed cross-correlation coefficient between the clause and the target aspect. To enhance the representation of the target component, we propose capturing the connections between various aspects. In the inter-aspect module, we intricately design a Balanced Attention encoder (BA-encoder) that forms an aspect sequence by navigating the extracted phrase tree. To effectively capture the emotion of implicit components, we introduce a Top-K Attention Graph Convolutional Network (KA-GCN). Our proposed method has showcased state-of-the-art (SOTA) performance through experiments conducted on four widely used datasets. Furthermore, our model demonstrates a significant improvement in the robustness of datasets subjected to disturbances.

DFSTrack: Dual-stream fusion Siamese network for human pose tracking in videos

Human pose tracking is a challenging task that involves estimating the human pose and tracking it across multiple frames in a video sequence. In recent years, deep learning-based methods have made significant progress in this field, achieving state-of-the-art performance. However, due to complex background and occlusion among people missed detection and incorrect association matching are still the challenging problems. To address these issues, we adopt a top-down framework to perform human pose tracking in the paper. We propose a human detection prediction recovery module (HDP module) to recover missed detection, and propose a dual-stream fusion Siamese network for human matching (DFSTrack). Specifically, we design a residual graph convolutional block (RGCN block) for spatial position encoding of human keypoints, and use spatial self-attention and temporal cross-attention to design a dual-stream spatial–temporal fusion transformer (DST Transformer). The graph convolutional block and transformer are cascaded to simultaneously obtain information on the spatial and temporal positions of human keypoints, allowing the Siamese network to solve the erroneous human matching. Experimental results on the PoseTrack17 dataset, PoseTrack18 dataset and PoseTrack21 dataset demonstrate that our proposed method outperforms state-of-the-art methods on human pose tracking tasks. Our code and pretrained models are available at https://github.com/yhtian2023/DFSTrack.

Spatial-temporal graph Transformer for object tracking against noise spoofing interference

Achieving accurate object tracking against noise spoofing interference caused by similar backgrounds, similar objects, occlusion, or illumination variations is a challenge, especially since Siamese trackers largely ignore the spatial-temporal and part-level information of object structure. To handle this problem, we present a novel object tracking network with spatial-temporal awareness, which learns the correlation of local feature changes and extracts global spatial-temporal fusion features of the object to prevent noise spoofing interference. Specifically, our tracker integrates an object graph reconstruction representation module and a spatial-temporal graph Transformer module. The graph reconstruction representation module models the object structure and search region as part-to-part graph node correspondences, propagating object information to achieve part-level feature aggregation in the search region. The proposed spatial-temporal graph Transformer module fuses the temporal and spatial features of the object to enhance the contextual features and correlation between temporal and spatial feature variations, recognizing the noise spoofing interference to enhance the tracking performance. Meanwhile, our tracker learns object motion information to improve object state awareness, enhancing tracking accuracy under similar objects interference. Extensive experiments on six public datasets validate that our tracker outperforms related state-of-the-art trackers and achieves accurate tracking results against noise spoofing interference.

Efficiently localizing system anomalies for cloud infrastructures: a novel Dynamic Graph Transformer based Parallel Framework

Cloud environment is a virtual, online, and distributed computing environment that provides users with large-scale services. And cloud monitoring plays an integral role in protecting infrastructures in the cloud environment. Cloud monitoring systems need to closely monitor various KPIs of cloud resources, to accurately detect anomalies. However, due to the complexity and highly dynamic nature of the cloud environment, anomaly detection for these KPIs with various patterns and data quality is a huge challenge, especially those massive unlabeled data. Besides, it’s also difficult to improve the accuracy of the existing anomaly detection methods. To solve these problems, we propose a novel Dynamic Graph Transformer based Parallel Framework (DGT-PF) for efficiently detect system anomalies in cloud infrastructures, which utilizes Transformer with anomaly attention mechanism and Graph Neural Network (GNN) to learn the spatio-temporal features of KPIs to improve the accuracy and timeliness of model anomaly detection. Specifically, we propose an effective dynamic relationship embedding strategy to dynamically learn spatio-temporal features and adaptively generate adjacency matrices, and soft cluster each GNN layer through Diffpooling module. In addition, we also use nonlinear neural network model and AR-MLP model in parallel to obtain better detection accuracy and improve detection performance. The experiment shows that the DGT-PF framework have achieved the highest F1-Score on 5 public datasets, with an average improvement of 21.6% compared to 11 anomaly detection models.

Progressive prediction: Video anomaly detection via multi‐grained prediction

AbstractVideo Anomaly Detection (VAD) has been an active research field for several decades. However, most existing approaches merely extract a single type of feature from videos and define a single paradigm to indicate the extent of abnormalities. A coarse‐to‐fine three‐level prediction is built by integrating different levels of spatio‐temporal representations, better highlighting the difference between normal and abnormal behaviors. First, an object‐level trajectory prediction is proposed to model human historical position using a graph transformer network. Subsequently, skeleton‐level prediction is achieved by incorporating the positional information from the trajectory prediction. More importantly, based on the predicted skeleton, a skeleton‐guided pixel‐level region prediction is performed. A novel Skeleton Conditioned Generative Adversarial Network (SCGAN) is designed to explore the correlation between skeleton‐level and pixel‐level motion prediction. Benefiting from SCGAN, the prediction of human regions is contributed by both coarse‐grained and fine‐grained motion features. This three‐level prediction, namely Progressive Prediction Video Anomaly Detection (P3VAD), enlarges the prediction error on irregular motion patterns. Besides, a pixel‐level analysis method is proposed to achieve Background‐bias Elimination (BE) and denoise the predicted region. Experimental results validate the effectiveness of P3VAD on the four benchmark datasets (ShanghaiTech, CUHK Avenue, IITB‐Corridor, and ADOC).

Research on spatial-temporal synergistic sensor fault diagnosis method for top-blowing furnace

Top-blowing furnace systems, characterized by a large number of sensors and harsh working environments, are prone to sensor failures due to factors like component aging and external interference. These failures can significantly impact the system's safe and reliable operation. However, traditional sensor fault diagnosis methods often neglect the exploration of spatial-temporal characteristics and focus solely on learning temporal relationships between sensors, failing to effectively consider their spatial relationships. In this study, we propose a spatial correlation model based on the maximal information-based graph convolutional network (MI-GCN) by constructing a sensor network knowledge graph using maximal mutual information. The MI-GCN leverages the graph convolution mechanism to extract multi-scale spatial features and capture the spatial relationships between sensors. Additionally, we develop a spatial-temporal graph-level prediction model, known as the spatial-temporal graph transformer (STGT), to extract temporal features. By combining the spatial features extracted by the MI-GCN with the temporal features captured by the STGT, accurate predictions can be achieved. Sensor fault diagnosis is conducted by analysing the normalized residuals between the predicted values and the ground truth. Finally, the feasibility and effectiveness of the proposed method are validated using test data from a top-blowing furnace system in the nickel smelting process.

Fluid identification with Graph Transformer using well logging data

The prediction of fluid through well logging is a cornerstone in guiding exploratory efforts in the energy sector. Comprehending the fluid composition beneath the surface empowers exploration teams to effectively gauge the extent, reserves, and caliber of oil and gas resources. This leads to enhanced strategies in exploration and the judicious use of resources. We introduce an innovative machine learning framework named “Graph Transformer” for predicting fluid. This model melds graph convolutional layers with a Transformer module. It excels in decoding spatial and temporal patterns within well logging data, thus unraveling complex geological dependencies by factoring in the interconnectedness of various data points. Additionally, it features a Positional Encoding module to enhance understanding of sequential data points in terms of depth, thereby overcoming the limitations of sequence independence. The Transformer's Multi-Head Self-Attention mechanism is pivotal in discerning and integrating spatial and temporal interconnections across various depths, elevating its capability to represent geological structures. Initially, the model harnesses key well log data like Density, Acoustic, Gamma-ray, and Compensated Neutron Logs for extracting geological features. These insights are then processed through the Graph Transformer to establish relationship between fluid characteristics and logging parameters. Furthermore, we compare this model with other leading models using precision, recall, and accuracy metrics. Experimental findings affirm the model's high accuracy in predicting fluid within intricate geological settings. Its exceptional adaptability makes it apt for various geological conditions and logging tools. Thus, our Graph Transformer model stands out as a sophisticated, efficient machine learning solution in the realm of well logging fluid prediction, offering geologists and engineers precise tools for exploration and development.

MTSecurity: Privacy-Preserving Malicious Traffic Classification Using Graph Neural Network and Transformer

MTSecurity: Privacy-Preserving Malicious Traffic Classification Using Graph Neural Network and Transformer