Attentive Recurrent Network Research Articles

Traffic flow forecasting based on augmented multi-component recurrent graph attention network

ABSTRACT Accurate real-time traffic flow forecasting has been a challenge due to the complex spatial–temporal dependencies and uncertainties associated with the dynamic changes in traffic flow. To overcome this problem, a traffic flow forecasting model based on an Augmented Multi-Component Recurrent Graph Attention Network (AMR-GAT) is proposed in this paper to model the spatial–temporal correlations and periodic offset of traffic flows. This paper introduces an augmented multi-component module to address periodic temporal offset in traffic flow forecasting. It proposes an encoder-decoder architecture combining 1D convolution and LSTM via a Temporal Correlation Learner (TCL) to capture temporal characteristics, while a Graph Attention Network (GAT) handles spatial features. TCL and GAT are integrated to manage spatial-temporal correlations, and the decoder uses TCL and convolutional neural networks to generate high-dimensional representations based on spatial-temporal sequences. Experiments on two datasets demonstrate superior prediction performance of the proposed AMR-GAT model.

Intercity round-trip multi-region demand prediction based on multi-task fusion recurrent graph attention network

Intercity round-trip multi-region demand prediction based on multi-task fusion recurrent graph attention network

Wavelet Pyramid Recurrent Structure-Preserving Attention Network for Single Image Super-Resolution.

Many single image super-resolution (SISR) methods that use convolutional neural networks (CNNs) learn the relationship between low-and high-resolution images directly, without considering the context structure and detail fidelity. This can limit the potential of CNNs and result in unrealistic, distorted edges and textures in the reconstructed images. A more effective approach is to incorporate prior knowledge about the image into the model to aid in image reconstruction. In this study, we propose a novel recurrent structure-preserving mechanism that innovatively uses the multiscale wavelet transform (WT) as an image prior, namely, wavelet pyramid recurrent structure-preserving attention network (WRSANet), to process both low-and high-frequency subnetworks at each level separately and recursively. We propose a novel structure scale preservation (SSP) architecture that differs from traditional WTs. This architecture allows us to incorporate and learn structure preservation subnetworks at each level. By using our proposed structure scale fusion (SSF) combined with inverse WT, we can recursively restore and preserve rich low-frequency image structure through the combination of SSP at various levels. Furthermore, we also propose novel low-to-high-frequency information transmission (L2HIT) and detail enhancement (DE) mechanisms to address the issue of detail distortion in high-frequency images by transferring information from structure preservation subnetworks. This allows us to preserve the low-frequency structure while reconstructing high-frequency details, improving detail fidelity and avoiding structural distortion. Finally, a joint loss function is also used to balance the fusion of low-and high-frequency information at different degrees, with hyperparameters being adjusted during training. The experimental results demonstrate that the proposed WRSANet achieves better performance and visual presentation than the state-of-the-art (SOTA) on synthetic and real datasets, especially in terms of context structure and texture details.

TG-PGAT: An AIS Data-Driven Dynamic Spatiotemporal Prediction Model for Ship Traffic Flow in the Port

Accurate prediction of ship traffic flow is essential for developing intelligent maritime transportation systems. To address the complexity of ship traffic flow data in the port and the challenges of capturing its dynamic spatiotemporal dependencies, a dynamic spatiotemporal model called Temporal convolutional network-bidirectional Gated recurrent unit-Pearson correlation coefficient-Graph Attention Network (TG-PGAT) is proposed for predicting traffic flow in port waters. This model extracts spatial features of traffic flow by combining the adjacency matrix and spatial dynamic coefficient correlation matrix within the Graph Attention Network (GAT) and captures temporal features through the concatenation of the Temporal Convolutional Network (TCN) and Bidirectional Gated Recurrent Unit (BiGRU). The proposed TG-PGAT model demonstrates higher prediction accuracy and stability than other classic traffic flow prediction methods. The experimental results from multiple angles, such as ablation experiments and robustness tests, further validate the critical role and strong noise resistance of different modules in the TG-PGAT model. The experimental results of visualization demonstrate that this model not only exhibits significant predictive advantages in densely trafficked areas of the port but also outperforms other models in surrounding areas with sparse traffic flow data.

Managing remaining useful life of cyber-aeroengine systems using a graph spatio-temporal attention recurrent network with phase-lag index

The decision-making process concerning the maintenance of equipment significantly impacts the safety and reliability of the aeroengine system. A cyber-aeroengine system (CAS) integrates intricate aeroengine components with a cyber decision-making space based on the communication network, thereby facilitating computational support for forecasting the health status of the aeroengine system. Within the cyber realm, we introduce a novel approach termed the graph spatio-temporal attention recurrent network with phase-lag index (PLI-GSTARN) to assess the degradation states of aeroengine systems. This method constructs a spatio-temporal graph based on sensor signal correlations and information reachability across distributed sensors within the physical equipment. Subsequently, the proposed framework employs a graph attention network to capture spatial features and a long short-term memory (LSTM) layer equipped with an attention mechanism to capture temporal features inherent in the constructed graph. Ultimately, this methodology is applied to cyber–physical systems within aircraft engines to manage remaining health life and diagnose faults, yielding commendable outcomes.

An efficient channel recurrent Criss-cross attention network for epileptic seizure prediction

Epilepsy is a chronic disease caused by repeated abnormal discharge of neurons in the brain. Accurately predicting the onset of epilepsy can effectively improve the quality of life for patients with the condition. While there are many methods for detecting epilepsy, EEG is currently considered one of the most effective analytical tools due to the abundant information it provides about brain activity. The aim of this study is to explore potential time-frequency and channel features from multi-channel epileptic EEG signals and to develop a patient-specific seizure prediction network. In this paper, an epilepsy EEG signal classification algorithm called Channel Recurrent Criss-cross Attention Network (CRCANet) is proposed. Firstly, the spectrograms processed by the short-time fourier transform is input into a Convolutional Neural Network (CNN). Then, the spectrogram feature map obtained in the previous step is input into the channel attention module to establish correlations between channels. Subsequently, the feature diagram containing channel attention characteristics is input into the recurrent criss-cross attention module to enhance the information content of each pixel. Finally, two fully connected layers are used for classification. We validated the method on 13 patients in the public CHB-MIT scalp EEG dataset, achieving an average accuracy of 93.8 %, sensitivity of 94.3 %, and specificity of 93.5 %. The experimental results indicate that CRCANet can effectively capture the time-frequency and channel characteristics of EEG signals while improving training efficiency.

Recognizing text lines in handwritten archival document images using octave convolutional and attention recurrent neural networks

Recognizing text lines in handwritten archival document images using octave convolutional and attention recurrent neural networks

A systematic study of DNN based speech enhancement in reverberant and reverberant-noisy environments

Deep learning has led to dramatic performance improvements for the task of speech enhancement, where deep neural networks (DNNs) are trained to recover clean speech from noisy and reverberant mixtures. Most of the existing DNN-based algorithms operate in the frequency domain, as time-domain approaches are believed to be less effective for speech dereverberation. In this study, we employ two DNNs: ARN (attentive recurrent network) and DC-CRN (densely-connected convolutional recurrent network), and systematically investigate the effects of different components on enhancement performance, such as window sizes, loss functions, and feature representations. We conduct evaluation experiments in two main conditions: reverberant-only and reverberant-noisy. Our findings suggest that incorporating larger window sizes is helpful for dereverberation, and adding transform operations (either convolutional or linear) to encode and decode waveform features improves the sparsity of the learned representations, and boosts the performance of time-domain models. Experimental results demonstrate that ARN and DC-CRN with proposed techniques achieve superior performance compared with other strong enhancement baselines.

Squeeze-and-excitation 3D convolutional attention recurrent network for end-to-end speech emotion recognition

Speech emotion recognition (SER) is difficult since emotions are complex and dynamic processes involving multiple dimensions and sub-dimensions. Feature extraction is a challenging step in SER, where relevant features are extracted from the speech to identify emotional states accurately. Overcoming these challenges is essential to ensuring the effectiveness and robustness of the SER system. 3D-convolutional neural networks (3D-CNNs) are successfully used for feature extraction in SER. Speech signals can be converted into spectrogram-like representations where one axis represents time, another frequency, and the third can represent additional context or features. For modeling the dynamic nature of speech, a squeeze-and-excitation-based 3D-CNN model is employed for capturing temporal and spatial features of speech. Attention Gated Recurrent Units (AGRU) are applied to the extracted features for learning long-range temporal dependencies and selecting more informative features. The extraction and selection of the spatio-temporal feature representation lead to a hierarchical representation of the input speech. The fusion of squeeze-and-excitation 3D-CNN and AGRU (named SE3D-CARN) is evaluated on two datasets, EMO-BD and IEMOCAP, to identify various emotional states in speech. The proposed SER model reached an accuracy of 94.2% and 81.1% on the EMO-BD and IEMOCAP datasets, respectively.

Object detection in drone video based on recurrent motion attention

In recent years, object detection in drone videos has garnered widespread attention. Conventional image-based detectors have encountered a performance bottleneck for drone images due to the severe appearance deterioration of targets and numerous background clutters. Video object detection (VOD) is a promising paradigm for such problems as it can effectively enhance target appearance by aggregating features from nearby frames in drone videos. However, struggling with the feature alignment problem which is particularly important for drone images where the background varies frequently. Following the VOD philosophy, we propose the Recurrent Motion Attention Network (RMA-Net) to extract spatial-temporal image features for object detection in drone vision. A recurrent fashion is designed to effectively and efficiently learn motion features in long sequences, only requiring a few frames of optical flows for initialization. Experiments demonstrate that the proposed method could easily promote existing object detectors to achieve high performance on drone videos. Specifically, our RMA adapted YOLOv7 outperforms the state-of-the-arts by a large margin on the challenging VisDrone2019-VID dataset, not only boosts the mean average precision by 6.99 % but also reaches a batch-1 inference speed of 31 frames per second.

Fusion of satellite and street view data for urban traffic accident hotspot identification

As the number of vehicles and the volume of traffic swell in urban centers, cities have experienced a concomitant increase in traffic accidents. Proactively identifying accident-prone hotspots in urban environments holds the promise of preventing traffic mishaps, thereby curtailing the incidence of accidents and reducing property damage. This research introduces the Two-Branch Contextual Feature-Guided Converged Network (TCFGC-Net) utilizing multimodal satellite and street view data. Designed to extract global structural features from satellite imagery and dynamic continuous features from street view imagery, the model aims to improve the accuracy of detecting urban accident hotspots. For the satellite imagery branch, we propose the Contextual Feature Coupled Convolutional Neural Network (Trans-CFCCNN) designed to extract global spatial features and discern feature correlations across adjacent regions. For the street view imagery branch, we develop the Sequential Feature Recurrent Attention Network (SFRAN) to assimilate and integrate dynamic scene features captured from successive street view images. We designed the Multi-Branch Feature Adaptive Fusion Structure (MBFAF) to aggregate different branch features for accurate identification of accident hotspots. Experimental results show that the model performs well, with an overall accuracy of 93.7 %. Ablation studies confirm that relative to standalone street view and satellite branch analyses, implementing multimodal fusion enhances the model's accuracy by 12.05 % and 17.86 %, respectively. The innovative fusion structure proposed herein garners a 4.22 % increase in model accuracy, outpacing conventional feature concatenation techniques. Furthermore, the model outperforms existing deep learning models in terms of overall efficacy. Additionally, to showcase the efficacy of the proposed model structure, we utilize Class Activation Maps (CAM) to provide visual interpretability for the model. These results suggest that the dual-branch fusion model effectively decreases false alarm occurrences and directs the model's focus toward regions more pertinent to accident hotspots. Finally, the code and model used for identifying hotspots of urban traffic accidents in this study are available for access: https://github.com/gwt-ZJU/TCFGC-Net.

Learning spatial–temporal pairwise and high-order relationships for short-term passenger flow prediction in urban rail transit

Short-term passenger flow prediction (STPFP) helps ease traffic congestion and optimize urban rail transit (URT) system resource allocation. Although graph-based models have been widely used in STPFP because of their ability to model complex spatial–temporal dependencies, accurate STPFP still faces challenges. The graph-based methods exploit pairwise stations to model spatial dependencies without considering the non-pairwise high-order relationships between multiple stations. Besides, the event’s occurrence will make the passenger flow at a particular moment have a more profound impact on the passenger flow at a future moment. To solve these issues, a spatial–temporal hypergraph attention recurrent network (STHGARN) method is proposed for STPFP. Specifically, STHGARN first models passenger flow data’s recent, daily, and weekly characteristics through three parallel components. Then, each parallel component uses the hypergraph attention recurrent network (HGARN) cell to jointly learn the pairwise and high-order relationships of multiple stations. Finally, a temporal Hawkes attention mechanism is constructed and concatenated after each parallel HGARN cell to simulate the influence of events and capture the historical key moments that affect the future passenger flow. Numerous experiments on three metro ridership datasets show that STHGARN is superior to the most advanced methods, and good prediction results can lay the foundation for the operation and management of URT.

No-Reference Video Quality Assessment Using Transformers and Attention Recurrent Networks

No-Reference Video Quality Assessment Using Transformers and Attention Recurrent Networks

Hypergraph Attention Recurrent Network for Cellular Traffic Prediction

Hypergraph Attention Recurrent Network for Cellular Traffic Prediction

Spatio-temporal Fourier enhanced heterogeneous graph learning for traffic forecasting

Traffic flow prediction is of paramount importance in the field of spatio-temporal forecasting. In recent years, research efforts have primarily been directed towards developing intricate graph convolutional networks (GCNs) to capture spatial complexities. However, this has inadvertently led to the neglect of the inherent temporal correlations in traffic prediction, as well as the heterogeneity of graph structures. As a result, existing models show limited efficacy when dealing with the complex nature of traffic data. To address this issue, this paper introduces a novel traffic prediction model: the Fourier-enhanced heterogeneous graph convolution attention recurrent network (FEHGCARN). This model integrates historical information and incorporates a graph convolution attention recurrent unit (GCARU), meticulously engineered to effectively capture spatio-temporal dependencies. Additionally, it features a Fourier-enhanced heterogeneous graph learning module, which facilitates the acquisition of complex relationships among nodes in the frequency domain. Notably, this memory network excels at recognizing abrupt traffic conditions. To validate our approach, we conducted comprehensive comparisons using three authentic datasets and benchmarked our model against six state-of-the-art baseline methods. The experimental results unequivocally demonstrate the superior performance of our model across all evaluation metrics.

Enhancing Dimensional Emotion Recognition from Speech through Modulation-Filtered Cochleagram and Parallel Attention Recurrent Network

Dimensional emotion can better describe rich and fine-grained emotional states than categorical emotion. In the realm of human–robot interaction, the ability to continuously recognize dimensional emotions from speech empowers robots to capture the temporal dynamics of a speaker’s emotional state and adjust their interaction strategies in real-time. In this study, we present an approach to enhance dimensional emotion recognition through modulation-filtered cochleagram and parallel attention recurrent neural network (PA-net). Firstly, the multi-resolution modulation-filtered cochleagram is derived from speech signals through auditory signal processing. Subsequently, the PA-net is employed to establish multi-temporal dependencies from diverse scales of features, enabling the tracking of the dynamic variations in dimensional emotion within auditory modulation sequences. The results obtained from experiments conducted on the RECOLA dataset demonstrate that, at the feature level, the modulation-filtered cochleagram surpasses other assessed features in its efficacy to forecast valence and arousal. Particularly noteworthy is its pronounced superiority in scenarios characterized by a high signal-to-noise ratio. At the model level, the PA-net attains the highest predictive performance for both valence and arousal, clearly outperforming alternative regression models. Furthermore, the experiments carried out on the SEWA dataset demonstrate the substantial enhancements brought about by the proposed method in valence and arousal prediction. These results collectively highlight the potency and effectiveness of our approach in advancing the field of dimensional speech emotion recognition.

Capsule Endoscopy Image Deblurring Method Based on Multi-scale Recurrent Network

During the process of acquiring capsule endoscope images, image motion blur may result from errors made by the operating physician. A multi-scale recurrent attention network is proposed to address the issue of motion blur in collected images. Considering the issue of over-exposure during image acquisition in capsule endoscopy, which can adversely affect network generation results, an image preprocessing module has been incorporated into the network to eliminate highlights caused by over-exposure in input images. To enhance network sampling efficiency, the encoder is equipped with a Convolutional Block Attention Module (CBAM) to extract fuzzy features more effectively. Additionally, depthwise separable convolution is employed to reduce parameter count. Since the structure of images obtained through capsule endoscopy is relatively simple and stable, we have opted for the mean square error as our loss function due to its heightened sensitivity to data errors. The experimental results demonstrate that the proposed network achieves a Peak Signal-to-Noise Ratio (PSNR) of 31.096 and a Structural Similarity Index Measure (SSIM) of 0.925, indicating a significant improvement over DeblurGAN-v2 and Pix2pix.

Adaboosting graph attention recurrent network: A deep learning framework for traffic speed forecasting in dynamic transportation networks with spatial-temporal dependencies

In construction engineering, transportation is a key factor affecting the construction schedule, and Transportation Speed Prediction (TSP) provides essential information for the precise scheduling of construction transportation. TSP is a challenging task due to the complex spatial-temporal traffic correlations and the dynamic variation of transportation routes. Given the superiority in topology representation of traffic networks, graph-based networks are becoming the prevalent traffic prediction solutions. However, existing methods have difficulty in dealing with dynamic transportation network structures and insufficiency in extracting representative spatial-temporal features. To address such issues, in this article, an Adaboosting Graph Attention Recurrent Network (Ada-GARN) is proposed. In the network, a graph attention recurrent unit is developed that integrates graph attention convolution with gated recurrent structures to extract spatial-temporal features of the transportation network with changing structures. On this basis, considering the effect of the time lag in traffic propagation, the attention space of graph convolution is extended to the past state of neighboring nodes based on traffic feature graphs, which enhances the representation of spatial evolution characteristics. Furthermore, to effectively integrate the multi-scale spatial-temporal information, the model uses an Adaboost framework to ensemble graph attention recurrent units instead of directly stacking spatial-temporal layers, which measures the feature differences among layer-wise neighbors and adaptively adjusts node weights in training. Experiments conducted on a large infrastructure project transportation dataset and a highway dataset show the model's adaptability to different scenarios. The proposed model outperforms state-of-the-art methods and reduces 11.2%–28% and 23%–23.1% in terms of RMSE and MAE metrics, respectively.

A Lightweight Recurrent Grouping Attention Network for Video Super-Resolution.

Effective aggregation of temporal information of consecutive frames is the core of achieving video super-resolution. Many scholars have utilized structures such as sliding windows and recurrences to gather the spatio-temporal information of frames. However, although the performances of constructed video super-resolution models are improving, the sizes of the models are also increasing, exacerbating the demand on the equipment. Thus, to reduce the stress on the device, we propose a novel lightweight recurrent grouping attention network. The parameters of this model are only 0.878 M, which is much lower than the current mainstream model for studying video super-resolution. We have designed a forward feature extraction module and a backward feature extraction module to collect temporal information between consecutive frames from two directions. Moreover, a new grouping mechanism is proposed to efficiently collect spatio-temporal information of the reference frame and its neighboring frames. The attention supplementation module is presented to further enhance the information gathering range of the model. The feature reconstruction module aims to aggregate information from different directions to reconstruct high-resolution features. Experiments demonstrate that our model achieves state-of-the-art performance on multiple datasets.

CARDIAN: a novel computational approach for real-time end-diastolic frame detection in intravascular ultrasound using bidirectional attention networks.

Changes in coronary artery luminal dimensions during the cardiac cycle can impact the accurate quantification of volumetric analyses in intravascular ultrasound (IVUS) image studies. Accurate ED-frame detection is pivotal for guiding interventional decisions, optimizing therapeutic interventions, and ensuring standardized volumetric analysis in research studies. Images acquired at different phases of the cardiac cycle may also lead to inaccurate quantification of atheroma volume due to the longitudinal motion of the catheter in relation to the vessel. As IVUS images are acquired throughout the cardiac cycle, end-diastolic frames are typically identified retrospectively by human analysts to minimize motion artefacts and enable more accurate and reproducible volumetric analysis. In this paper, a novel neural network-based approach for accurate end-diastolic frame detection in IVUS sequences is proposed, trained using electrocardiogram (ECG) signals acquired synchronously during IVUS acquisition. The framework integrates dedicated motion encoders and a bidirectional attention recurrent network (BARNet) with a temporal difference encoder to extract frame-by-frame motion features corresponding to the phases of the cardiac cycle. In addition, a spatiotemporal rotation encoder is included to capture the IVUS catheter's rotational movement with respect to the coronary artery. With a prediction tolerance range of 66.7 ms, the proposed approach was able to find 71.9%, 67.8%, and 69.9% of end-diastolic frames in the left anterior descending, left circumflex and right coronary arteries, respectively, when tested against ECG estimations. When the result was compared with two expert analysts' estimation, the approach achieved a superior performance. These findings indicate that the developed methodology is accurate and fully reproducible and therefore it should be preferred over experts for end-diastolic frame detection in IVUS sequences.