Local Embedding Research Articles

Deciphering abbreviations in Malaysian clinical notes using machine learning.

This is the first Malaysian machine learning model to detect and disambiguate abbreviations in clinical notes. The model has been designed to be incorporated into MyHarmony, a Natural Language Processing system, that extracts clinical information for healthcare management. The model utilizes word embedding to ensure feasibility of use, not in real-time but for secondary analysis, within the constraints of low-resource settings. A Malaysian clinical embedding, based on Word2Vec model, was developed using 29,895 electronic discharge summaries. The embedding was compared against conventional rule-based and FastText embedding on two tasks: abbreviation detection and abbreviation disambiguation. Machine Learning classifiers were applied to assess performance. The Malaysian clinical word embedding contained 7 million word-tokens, 24,352 unique vocabularies, and 100 dimensions. For abbreviation detection, the Decision Tree classifier augmented with the Malaysian clinical embedding showed the best performance (F-score of 0.9519). For abbreviation disambiguation, the classifier with the Malaysian clinical embedding had the best performance for most of the abbreviations (F-score of 0.9903). Despite having a smaller vocabulary and dimension, our local clinical word embedding performed better than the larger non-clinical FastText embedding. Word embedding with simple machine learning algorithms can decipher abbreviations well. It also requires lower computational resources and is suitable for implementation in low-resource settings such as Malaysia. The integration of this model into MyHarmony will improve recognition of clinical terms, thus improving the information generated for monitoring Malaysian healthcare services and policymaking.

GLMAFuse: A Dual-Stream Infrared and Visible Image Fusion Framework Integrating Local and Global Features with Multi-Scale Attention

Integrating infrared and visible-light images facilitates a more comprehensive understanding of scenes by amalgamating dual-sensor data derived from identical environments. Traditional CNN-based fusion techniques are predominantly confined to local feature emphasis due to their inherently limited receptive fields. Conversely, Transformer-based models tend to prioritize global information, which can lead to a deficiency in feature diversity and detail retention. Furthermore, methods reliant on single-scale feature extraction are inadequate for capturing extensive scene information. To address these limitations, this study presents GLMAFuse, an innovative dual-stream encoder–decoder network, which utilizes a multi-scale attention mechanism to harmoniously integrate global and local features. This framework is designed to maximize the extraction of multi-scale features from source images while effectively synthesizing local and global information across all layers. We introduce the global-aware and local embedding (GALE) module to adeptly capture and merge global structural attributes and localized details from infrared and visible imagery via a parallel dual-branch architecture. Additionally, the multi-scale attention fusion (MSAF) module is engineered to optimize attention weights at the channel level, facilitating an enhanced synergy between high-frequency edge details and global backgrounds. This promotes effective interaction and fusion of dual-modal features. Extensive evaluations using standard datasets demonstrate that GLMAFuse surpasses the existing leading methods in both qualitative and quantitative assessments, highlighting its superior capability in infrared and visible image fusion. On the TNO and MSRS datasets, our method achieves outstanding performance across multiple metrics, including EN (7.15, 6.75), SD (46.72, 47.55), SF (12.79, 12.56), MI (2.21, 3.22), SCD (1.75, 1.80), VIF (0.79, 1.08), Qbaf (0.58, 0.71), and SSIM (0.99, 1.00). These results underscore its exceptional proficiency in infrared and visible image fusion.

DenseGNN: universal and scalable deeper graph neural networks for high-performance property prediction in crystals and molecules

Modern generative models based on deep learning have made it possible to design millions of hypothetical materials. To screen these candidate materials and identify promising new materials, we need fast and accurate models to predict material properties. Graphical neural networks (GNNs) have become a current research focus due to their ability to directly act on the graphical representation of molecules and materials, enabling comprehensive capture of important information and showing excellent performance in predicting material properties. Nevertheless, GNNs still face several key problems in practical applications: First, although existing nested graph network strategies increase critical structural information such as bond angles, they significantly increase the number of trainable parameters in the model, resulting in a increase in training costs; Second, extending GNN models to broader domains such as molecules, crystalline materials, and catalysis, as well as adapting to small data sets, remains a challenge. Finally, the scalability of GNN models is limited by the over-smoothing problem. To address these issues, we propose the DenseGNN model, which combines Dense Connectivity Network (DCN), hierarchical node-edge-graph residual networks (HRN), and Local Structure Order Parameters Embedding (LOPE) strategies to create a universal, scalable, and efficient GNN model. We have achieved state-of-the-art performance (SOAT) on several datasets, including JARVIS-DFT, Materials Project, QM9, Lipop, FreeSolv, ESOL, and OC22, demonstrating the generality and scalability of our approach. By merging DCN and LOPE strategies into GNN models in computing, crystal materials, and molecules, we have improved the performance of models such as GIN, Schnet, and Hamnet on materials datasets such as Matbench. The LOPE strategy optimizes the embedding representation of atoms and allows our model to train efficiently with a minimal level of edge connections. This substantially reduces computational costs and shortens the time required to train large GNNs while maintaining accuracy. Our technique not only supports building deeper GNNs and avoids performance penalties experienced by other models, but is also applicable to a variety of applications that require large deep learning models. Furthermore, our study demonstrates that by using structural embeddings from pre-trained models, our model not only outperforms other GNNs in distinguishing crystal structures but also approaches the standard X-ray diffraction (XRD) method.

Semisupervised Subspace Learning With Adaptive Pairwise Graph Embedding.

Graph-based semisupervised learning can explore the graph topology information behind the samples, becoming one of the most attractive research areas in machine learning in recent years. Nevertheless, existing graph-based methods also suffer from two shortcomings. On the one hand, the existing methods generate graphs in the original high-dimensional space, which are easily disturbed by noisy and redundancy features, resulting in low-quality constructed graphs that cannot accurately portray the relationships between data. On the other hand, most of the existing models are based on the Gaussian assumption, which cannot capture the local submanifold structure information of the data, thus reducing the discriminativeness of the learned low-dimensional representations. This article proposes a semisupervised subspace learning with adaptive pairwise graph embedding (APGE), which first builds a k1 -nearest neighbor graph on the labeled data to learn local discriminant embeddings for exploring the intrinsic structure of the non-Gaussian labeled data, i.e., the submanifold structure. Then, a k2 -nearest neighbor graph is constructed on all samples and mapped to GE learning to adaptively explore the global structure of all samples. Clustering unlabeled data and its corresponding labeled neighbors into the same submanifold, sharing the same label information, improves embedded data's discriminative ability. And the adaptive neighborhood learning method is used to learn the graph structure in the continuously optimized subspace to ensure that the optimal graph matrix and projection matrix are finally learned, which has strong robustness. Meanwhile, the rank constraint is added to the Laplacian matrix of the similarity matrix of all samples so that the connected components in the obtained similarity matrix are precisely equal to the number of classes in the sample, which makes the structure of the graph clearer and the relationship between the near-neighbor sample points more explicit. Finally, multiple experiments on several synthetic and real-world datasets show that the method performs well in exploring local structure and classification tasks.

Discriminative Domain Adaption Network for Simultaneously Removing Batch Effects and Annotating Cell Types in Single-Cell RNA-Seq.

Machine learning techniques have become increasingly important in analyzing single-cell RNA and identifying cell types, providing valuable insights into cellular development and disease mechanisms. However, the presence of batch effects poses major challenges in scRNA-seq analysis due to data distribution variation across batches. Although several batch effect mitigation algorithms have been proposed, most of them focus only on the correlation of local structure embeddings, ignoring global distribution matching and discriminative feature representation in batch correction. In this paper, we proposed the discriminative domain adaption network (D2AN) for joint batch effects correction and type annotation with single-cell RNA-seq. Specifically, we first captured the global low-dimensional embeddings of samples from the source and target domains by adversarial domain adaption strategy. Second, a contrastive loss is developed to preliminarily align the source domain samples. Moreover, the semantic alignment of class centroids in the source and target domains is achieved for further local alignment. Finally, a self-paced learning mechanism based on inter-domain loss is adopted to gradually select samples with high similarity to the target domain for training, which is used to improve the robustness of the model. Experimental results demonstrated that the proposed method on multiple real datasets outperforms several state-of-the-art methods.

Diagnosing Necrotizing Enterocolitis via Fine-Grained Visual Classification.

Necrotizing Enterocolitis (NEC) is a devastating condition affecting prematurely born neonates. Reviewing Abdominal X-rays (AXRs) is a key step in NEC diagnosis, staging and treatment decision-making, but poses significant challenges due to the subtle, difficult-to-identify radiological signs of the disease. In this paper, we propose AIDNEC - AI Diagnosis of NECrotizing enterocolitis, a deep learning method to automatically detect and stratify the severity (surgical or medical) of NEC from no pathology in AXRs. The model is trainable end-to-end and integrates a Detection Transformer and Graph Convolution modules for localizing discriminative areas in AXRs, used to formulate subtle local embeddings. These are then combined with global image features to perform Fine-Grained Visual Classification (FGVC). We evaluate AIDNEC on our GOSH NEC dataset of 1153 images from 334 patients, achieving 79.7% accuracy in classifying NEC against No Pathology. AIDNEC outperforms the backbone by 2.6%, FGVC models by 2.5% and CheXNet by 4.2%, with statistically significant (two-tailed p 0.05) improvements, while providing meaningful discriminative regions to support the classification decision. Additional validation in the publicly available Chest X-ray14 dataset yields comparable performance to state-of-the-art methods, illustrating AIDNEC's robustness in a different X-ray classification task.

Maximizing Influence in Social Networks Using Combined Local Features and Deep Learning-Based Node Embedding.

The influence maximization problem has several issues, including low infection rates and high time complexity. Many proposed methods are not suitable for large-scale networks due to their time complexity or free parameter usage. To address these challenges, this article proposes a local heuristic called Embedding Technique for Influence Maximization (ETIM) that uses shell decomposition, graph embedding, and reduction, as well as combined local structural features. The algorithm selects candidate nodes based on their connections among network shells and topological features, reducing the search space and computational overhead. It uses a deep learning-based node embedding technique to create a multidimensional vector of candidate nodes and calculates the dependency on spreading for each node based on local topological features. Finally, influential nodes are identified using the results of the previous phases and newly defined local features. The proposed algorithm is evaluated using the independent cascade model, showing its competitiveness and ability to achieve the best performance in terms of solution quality. Compared with the collective influence global algorithm, ETIM is significantly faster and improves the infection rate by an average of 12%.

Federated Node Classification over Distributed Ego-Networks with Secure Contrastive Embedding Sharing.

Federated learning on graphs (a.k.a., federated graph learning- FGL) has recently received increasing attention due to its capacity to enable collaborative learning over distributed graph datasets without compromising local clients' data privacy. In previous works, clients of FGL typically represent institutes or organizations that possess sets of entire graphs (e.g., molecule graphs in biochemical research) or parts of a larger graph (e.g., sub-user networks of e-commerce platforms). However, another natural paradigm exists where clients act as remote devices retaining the graph structures of local neighborhoods centered around the device owners (i.e., ego-networks), which can be modeled for specific graph applications such as user profiling on social ego-networks and infection prediction on contact ego-networks. FGL in such novel yet realistic ego-network settings faces the unique challenge of incomplete neighborhood information for non-ego local nodes since they likely appear and have different sets of neighbors in multiple ego-networks. To address this challenge, we propose an FGL method for distributed ego-networks in which clients obtain complete neighborhood information of local nodes through sharing node embeddings with other clients. A contrastive learning mechanism is proposed to bridge the gap between local and global node embeddings and stabilize the local training of graph neural network models, while a secure embedding sharing protocol is employed to protect individual node identity and embedding privacy against the server and other clients. Comprehensive experiments on various distributed ego-network datasets successfully demonstrate the effectiveness of our proposed embedding sharing method on top of different federated model sharing frameworks, and we also provide discussions on the potential efficiency and privacy drawbacks of the method as well as their future mitigation.

Optimizing Virtual Network Embedding for Avionics with Time-Triggered Traffic Scheduling

Network virtualization technology abstracts the nodes and links resources in the physical network, and enables multiple virtual networks (VNs) to share the substrate network (SN) resources through the Virtual Netw ork Embedding (VNE) method. For time-triggered Ethernet (TTE) used in avionics, a time-triggered traffic scheduling oriented virtual network embedding (TT-VNE) method is proposed. While meeting the total resource constraints of traditional VNE problem, it ensures the strict periodicity of time-triggered (TT) traffic. During the solution process, the method sorts the virtual nodes according to the TT traffic bandwidth requirements and the total bandwidth requirements of the links connected to the virtual nodes, embeds the virtual nodes using the breadth first search algorithm, and plans the virtual links in candidate shortest path aggregation. If the TT traffic in the current virtual link is not schedulable, the iterative design of local virtual node re embedding and path planning is carried out. The simulation shows that the request acceptance rate of TT-VNE is not lower than that of existing methods, and when the number of virtual network requests in the star topology exceeds 30 , its request acceptance rate is about 14.3%higher than the VNE-NTANRC-D method which only considers the network topology and resource attributes.

Cross-domain zero-shot learning for enhanced fault diagnosis in high-voltage circuit breakers

Ensuring the stability of high-voltage circuit breakers (HVCBs) is crucial for maintaining an uninterrupted supply of electricity. Existing fault diagnosis methods typically rely on extensive labeled datasets, which are challenging to obtain due to the unique operational contexts and complex mechanical structures of HVCBs. Additionally, these methods often cater to specific HVCB models and lack generalizability across different types, limiting their practical applicability. To address these challenges, we propose a novel cross-domain zero-shot learning (CDZSL) approach specifically designed for HVCB fault diagnosis. This approach incorporates an adaptive weighted fusion strategy that combines vibration and current signals. To bypass the constraints of manual fault semantics, we develop an automatic semantic construction method. Furthermore, a multi-channel residual convolutional neural network is engineered to distill deep, low-level features, ensuring robust cross-domain diagnostic capabilities. Our model is further enhanced with a local subspace embedding technique that effectively aligns semantic features within the embedding space. Comprehensive experimental evaluations demonstrate the superior performance of our CDZSL approach in diagnosing faults across various HVCB types.

A novel multi-layer discriminative dictionary learning approach for image classification

Discriminative dictionary learning (DDL) has been confirmed to be effective for image classification. However, existing DDL approaches often fail to extract deep hierarchical information due to the single-layer dictionary learning framework. Moreover, they overlook the atoms-label information in the dictionary, leading to reduced feature distinctiveness and lower classification accuracy. To overcome the above problem, a novel DDL method, called the Multi-layer local constraint and label embedding dictionary learning (M-LCLEDL), is proposed. Specifically, the novel multi-layer DDL framework, which is formed by stacking the DL process one by one, is designed to learn the deep hierarchical and nonlinear features. The layer-by-layer stacking of the DL process in the multi-layer DDL framework allows for the elimination of redundant and interfering features. This step-by-step elimination process enhances the stability and robustness of the framework. Additionally, to leverage the label information carried by the labeled training samples, atoms with label embedding and locality structure are introduced. The proposed approach includes a fast iteration strategy for efficient optimization. Experimental results demonstrate that the approach is relatively insensitive to dictionary size, achieving promising performance and greater stability compared to most DDL-based image classification approaches.

Personalized identification of autism-related bacteria in the gut microbiome using explainable artificial intelligence

Autism spectrum disorder (ASD) affects social interaction and communication. Emerging evidence links ASD to gut microbiome alterations, suggesting that microbial composition may play a role in the disorder. This study employs explainable artificial intelligence (XAI) to examine the contributions of individual microbial species to ASD. By using local explanation embeddings and unsupervised clustering, the research identifies distinct ASD subgroups, underscoring the disorder's heterogeneity. Specific microbial biomarkers associated with ASD are revealed, and the best classifiers achieved an AU-ROC of 0.965± 0.005 and an AU-PRC of 0.967± 0.008. The findings support the notion that gut microbiome composition varies significantly among individuals with ASD. This work's broader significance lies in its potential to inform personalized interventions, enhancing precision in ASD management and classification. These insights highlight the importance of individualized microbiome profiles for developing tailored therapeutic strategies for ASD.

Local core expanding-based label diffusion and local deep embedding for fast community detection algorithm in social networks

Community detection is a key task in social network analysis, as it reveals the underlying structure and function of the network. Various global and local techniques exist for uncovering community structures in social networks wherein diffusion-based algorithms are proposed as novel methods for local community detection, particularly suited for large-scale networks. The efficacy of diffusion processes and initial detection is paramount in the successful identification of community structures within social networks. This effectiveness hinges significantly on the meticulous selection of the label diffuser core, which serves as the foundation for propagating labels through the network, and the precise labeling of boundary nodes. Addressing the constraints of current community detection algorithms, notably their time complexity and efficiency, this paper proposes a novel local community detection algorithm that combines core expansion with label diffusion, and deep embedding techniques. In the proposed method, a new centrality measure is introduced for appropriate core selection to facilitate precise label diffusion in the initial phase. Subsequently, a deep embedding technique is employed for updating labels of boundary and core nodes using the GraphSage embedding method. Finally, a rapid merging step is executed to amalgamate initially proximate communities into finalized community structures in large-scale social networks. We evaluate our algorithm on 14 real-world and 4 synthetic networks and show that it outperforms existing methods in terms of NMI, F-measure, ARI, and modularity. According to numerical results, the proposed method shows approximately 1.04 %, 1.03 %, and 1.12 % improvement in F-measure, NMI, and ARI measures respectively, compared to the second-best method, LBLD, in the networks with ground-truth. In addition, our method is able to accurately identify communities in large-scale networks such as Orkut, YouTube, and LiveJournal, where it ranks among the top-performing methods. Our approach exhibits the best performance in terms of ARI compared to other algorithms under comparison.

Enhancing local representation learning through global–local integration with functional connectivity for EEG-based emotion recognition

Emotion recognition based on electroencephalogram (EEG) signals is crucial in understanding human affective states. Current research has limitations in extracting local features. The representation capabilities of local features are limited, making it difficult to comprehensively capture emotional information. In this study, a novel approach is proposed to enhance local representation learning through global–local integration with functional connectivity for EEG-based emotion recognition. By leveraging the functional connectivity of brain regions, EEG signals are divided into global embeddings that represent comprehensive brain connectivity patterns throughout the entire process and local embeddings that reflect dynamic interactions within specific brain functional networks at particular moments. Firstly, a convolutional feature extraction branch based on the residual network is designed to extract local features from the global embedding. To further improve the representation ability and accuracy of local features, a multidimensional collaborative attention (MCA) module is introduced. Secondly, the local features and patch embedded local embeddings are integrated into the feature coupling module (FCM), which utilizes hierarchical connections and enhanced cross-attention to couple region-level features, thereby enhancing local representation learning. Experimental results on three public datasets show that compared with other methods, this method improves accuracy by 4.92% on the DEAP, by 1.11% on the SEED, and by 7.76% on the SEED-IV, demonstrating its superior performance in emotion recognition tasks.

On CR maps between hyperquadrics and Winkelmann hypersurfaces

In this paper, we study CR maps between hyperquadrics and Winkelmann hypersurfaces. Based on a previous study on the CR Ahlfors derivative of Lamel–Son and a recent result of Huang–Lu–Tang–Xiao on CR maps between hyperquadrics, we prove that a transversal CR map from a hyperquadric into a hyperquadric or a Winkelmann hypersurface extends to a local holomorphic isometric embedding with respect to certain Kähler metrics if and only if the Hermitian part of its CR Ahlfors derivative vanishes on an open set of the source. Our proof is based on relating the geometric rank of a CR map into a hyperquadric and its CR Ahlfors derivative.

Compound-SNE: Comparative alignment of t-SNEs for multiple single-cell omics data visualisation.

One of the first steps in single-cell omics data analysis is visualization, which allows researchers to see how well-separated cell-types are from each other. When visualizing multiple datasets at once, data integration/batch correction methods are used to merge the datasets. While needed for downstream analyses, these methods modify features space (e.g. gene expression)/PCA space in order to mix cell-types between batches as well as possible. This obscures sample-specific features and breaks down local embedding structures that can be seen when a sample is embedded alone. Therefore, in order to improve in visual comparisons between large numbers of samples (e.g., multiple patients, omic modalities, different time points), we introduce Compound-SNE, which performs what we term a soft alignment of samples in embedding space. We show that Compound-SNE is able to align cell-types in embedding space across samples, while preserving local embedding structures from when samples are embedded independently. Python code for Compound-SNE is available for download at https://github.com/HaghverdiLab/Compound-SNE. Available online. Provides algorithmic details and additional tests.

Learning hierarchical embedding space for image-text matching

There are two mainstream strategies for image-text matching at present. The one, termed as joint embedding learning, aims to model the semantic information of both image and sentence in a shared feature subspace, which facilitates the measurement of semantic similarity but only focuses on global alignment relationship. To explore the local semantic relationship more fully, the other one, termed as metric learning, aims to learn a complex similarity function to directly output score of each image-text pair. However, it significantly suffers from more computation burden at retrieval stage. In this paper, we propose a hierarchically joint embedding model to incorporate the local semantic relationship into a joint embedding learning framework. The proposed method learns the shared local and global embedding spaces simultaneously, and models the joint local embedding space with respect to specific local similarity labels which are easy to access from the lexical information of corpus. Unlike the methods based on metric learning, we can prepare the fixed representations of both images and sentences by concatenating the normalized local and global representations, which makes it feasible to perform the efficient retrieval. And experiments show that the proposed model can achieve competitive performance when compared to the existing joint embedding learning models on two publicly available datasets Flickr30k and MS-COCO.

EGPDI: identifying protein-DNA binding sites based on multi-view graph embedding fusion.

Mechanisms of protein-DNA interactions are involved in a wide range of biological activities and processes. Accurately identifying binding sites between proteins and DNA is crucial for analyzing genetic material, exploring protein functions, and designing novel drugs. In recent years, several computational methods have been proposed as alternatives to time-consuming and expensive traditional experiments. However, accurately predicting protein-DNA binding sites still remains a challenge. Existing computational methods often rely on handcrafted features and a single-model architecture, leaving room for improvement. We propose a novel computational method, called EGPDI, based on multi-view graph embedding fusion. This approach involves the integration of Equivariant Graph Neural Networks (EGNN) and Graph Convolutional Networks II (GCNII), independently configured to profoundly mine the global and local node embedding representations. An advanced gated multi-head attention mechanism is subsequently employed to capture the attention weights of the dual embedding representations, thereby facilitating the integration of node features. Besides, extra node features from protein language models are introduced to provide more structural information. To our knowledge, this is the first time that multi-view graph embedding fusion has been applied to the task of protein-DNA binding site prediction. The results of five-fold cross-validation and independent testing demonstrate that EGPDI outperforms state-of-the-art methods. Further comparative experiments and case studies also verify the superiority and generalization ability of EGPDI.

Using Genshin Impact as an Example, Examining How Local Cultural Embedding Can Enhance the Player's Gaming Experience

This research examines how the Genshin game developers organically incorporate culture into their games and improve player experience. Using quantitative methods, researchers analyzed 3 interviews with Genshin users to explore their attitudes and feelings towards the integration of Chinese cultural characteristics into the games. Through this game, many Chinese players have developed a sense of cultural pride, and the developers are very good at the details of the game. This game also spreads Chinese culture to the world. In many foreign websites, such as YouTube, Twitter can see a lot of foreign players comments and suggestions to these games. Also, the age coverage of this game is very wide too. From middle-aged people to high school students, all show a strong interest in this game. Researchers intend to study what kind of indigenous Chinese culture is embedded in this game and to understand how indigenous Chinese culture is organically embedded in the game by comparing game screenshots with real-life photos. Next, it is also necessary to check what kind of attitude players hold towards the embedding of Chinese indigenous culture appearing in Genshin Impact on social media platforms and select a few players to interview. Based on the former, it is important to find out how the embedding of Chinese indigenous culture in the game attracts players and improves their gaming experience. Finally, it will be generalized to the general game development, how a game can attract more players to play and improve the player experience through the organic integration of indigenous culture.

Dynamic spatial aware graph transformer for spatiotemporal traffic flow forecasting

Accurately predicting traffic flow is a crucial upstream technique in intelligent transportation systems for future travel plans, the efficiency of urban transport, and the regulation of transport departments, etc. The mainstream spatiotemporal graph convolutional neural networks are usually based on prior knowledge to predefine adjacency matrix graphs for spatial dependencies of the road network. However, modeling spatial correlation statically limits these models to accurately predict traffic flow, since the spatial correlations of road segments change over time. To address these issues, we propose a spatiotemporal gated transformer network with a graph latent information learning structure, termed GL-STGTN, for spatiotemporal traffic flow forecasting. First, we propose a graph latent information learning structure to dynamically learn the spatial dependencies for road network conditions from global and local learning perspectives. Second, we design a spatiotemporal gated transformer network (STGTN) block, which consists of a localized geographically aware block to extract the local embedding of spatial correlations and a temporal-aware enlarged block to extract local temporal dependencies. The learned spatial and temporal feature embeddings are further aggregated in a spatial multi-head attention module and a temporal multi-head attention module, respectively. In the end, a spatiotemporal fusion layer fuses the spatial and temporal information from the stacked STGTN blocks. Experiments on two public real-world benchmark datasets show that our model outperforms six state-of-the-art models for multi-step traffic flow forecasting.