Recurrent Network Research Articles

VTNet: A multi-domain information fusion model for long-term multi-variate time series forecasting with application in irrigation water level

Time series forecasting is intricately tied to production and life, garnering widespread attention over an extended period. Enhancing the performance of long-term multivariate time series forecasting (MTSF) poses a highly challenging task, as it requires mining complicated and obscure temporal patterns in many aspects. For this reason, this paper proposes a long-term forecasting model based on multi-domain fusion (VTNet) to adaptively capture and refine multi-scale intra- and inter-variate dependencies. In contrast to previous techniques, we devise a dual-stream learning architecture. Firstly, the fast Fourier Transform (FFT) is adopted to extract frequency domain information. The original sequences are then transformed into 2D visual features in the temporal-frequency domain, and a 2D-TBlock is designed for multi-scale dynamic learning. Secondly, a combination of convolution and recurrent networks continues to explore the local temporal features and preserve the global trend. Finally, multi-modal circulant fusion is applied to achieve a more comprehensive and enriched feature fusion representation, further promoting overall performance. Extensive experiments are conducted on 9 public benchmark datasets and the real-world irrigation water level to showcase VTNet’s promoted performance and generalization. Moreover, VTNet yields 46.93% and 25.36% relative improvements for water level forecasting, revealing its potential application value in water-saving planning and extreme event early warning.

HybridCBAMNet: Enhancing time series binary classification with convolutional recurrent networks and attention mechanisms

The rapid advancement of Internet of Things technology and the increasing availability of big data have resulted in an exponential growth of time series data, highlighting a pressing need for effective classification methods. This study introduces HybridCBAMNet, a novel convolutional recurrent neural network model enhanced with recurrent networks and attention mechanisms for binary time series classification. The architecture integrates Conv1D-based feature extraction modules to extract relevant features, alongside attention enhancement modules and convolutional block attention modules. Additionally, bidirectional recurrent units are utilized to capture temporal dependencies and contextual information in both forward and backward directions. The model achieves top F1-scores in seven out of thirty-six binary classification tasks, significantly surpassing the performance of fourteen existing state-of-the-art models from the UCR archive. These results demonstrate that HybridCBAMNet not only enhances classification accuracy but also improves the model generalization capabilities, contributing valuable insights to the field of time series analysis.

State space model-based Runge–Kutta gated recurrent unit networks for structural response prediction

State space model-based Runge–Kutta gated recurrent unit networks for structural response prediction

Tide level prediction during typhoons based on variable topology in graph convolution recurrent neural networks

Accurate tide level prediction is crucial for storm surge prevention and mitigation. In this study, we propose a regional multi-station tide-level prediction model (RMSTL) based on a graph convolution recurrent network that utilizes spatial and temporal information from historical tide levels and meteorological data to predict tide levels at multiple stations. Six stations in Changjiang Estuary, China, were selected for this case study. We explored the change in the accuracy of forecasting the tide-level during typhoons under a distinct topology of the tidal station network and the weights of the stations. The results showed that: (1) the prediction accuracy of RMSTL model behaved better than that of the other three widely used models; (2) the RMSTL model can be made more accurate by adding the weights of the edges or the number of edges of the topology; the value of RMSE decreased from 0.369 to 0.278, and the value of MAE decreased from 0.260 to 0.189; (3) when the dataset was limited, increasing the weights of the edges had a slightly better effect than increasing the weights of the edges in terms of improving the accuracy of the prediction results. The proposed method opens new avenues for forecasting storm surges.

Deep Complex Gated Recurrent Networks-Based IoT Network Intrusion Detection Systems.

The explosive growth of the Internet of Things (IoT) has highlighted the urgent need for strong network security measures. The distinctive difficulties presented by Internet of Things (IoT) environments, such as the wide variety of devices, the intricacy of network traffic, and the requirement for real-time detection capabilities, are difficult for conventional intrusion detection systems (IDS) to adjust to. To address these issues, we propose DCGR_IoT, an innovative intrusion detection system (IDS) based on deep neural learning that is intended to protect bidirectional communication networks in the IoT environment. DCGR_IoT employs advanced techniques to enhance anomaly detection capabilities. Convolutional neural networks (CNN) are used for spatial feature extraction and superfluous data are filtered to improve computing efficiency. Furthermore, complex gated recurrent networks (CGRNs) are used for the temporal feature extraction module, which is utilized by DCGR_IoT. Furthermore, DCGR_IoT harnesses complex gated recurrent networks (CGRNs) to construct multidimensional feature subsets, enabling a more detailed spatial representation of network traffic and facilitating the extraction of critical features that are essential for intrusion detection. The effectiveness of the DCGR_IoT was proven through extensive evaluations of the UNSW-NB15, KDDCup99, and IoT-23 datasets, which resulted in a high detection accuracy of 99.2%. These results demonstrate the DCG potential of DCGR-IoT as an effective solution for defending IoT networks against sophisticated cyber-attacks.

Based on Gated Recurrent network analysis of advanced manufacturing cluster and unified large market to promote regional economic development

This study evaluates the catalytic effects of advanced manufacturing industry clusters and unified large markets on regional economic development from a computer science perspective, revealing their underlying mechanisms. It employs a Gated Recurrent Network (GRN) model optimized with Gradient Boosting Decision Tree (GBDT) technology to conduct empirical analysis through comprehensive data collection and analysis. The primary objectives are to assess these catalytic effects, highlight the importance of innovation and environmental indicators, determine the contribution levels of various factors, and test the computational fit and predictive accuracy of the model. Key findings indicate that the GBDT-GRN model demonstrates a significant improvement in data computation accuracy, ranging from 20% to 52%, and an increase in response time by 23% to 52%. The model achieves a computational fit of 92% to 99% when analyzing regional economic development. The proposed GBDT-GRN model is highly accurate and reliable in evaluating catalytic effects, providing strong support for policy-making and business decision-making. Innovation and environmental indicators play a crucial role, with varying contributions from different factors. This study offers an effective solution for sequence data prediction problems, supports policy-making and business decisions, and points to promising directions for future research.

Deep beamforming for speech enhancement and speaker localization with an array response-aware loss function

Recent research advances in deep neural network (DNN)-based beamformers have shown great promise for speech enhancement under adverse acoustic conditions. Different network architectures and input features have been explored in estimating beamforming weights. In this paper, we propose a deep beamformer based on an efficient convolutional recurrent network (CRN) trained with a novel ARray RespOnse-aWare (ARROW) loss function. The ARROW loss exploits the array responses of the target and interferer by using the ground truth relative transfer functions (RTFs). The DNN-based beamforming system, trained with ARROW loss through supervised learning, is able to perform speech enhancement and speaker localization jointly. Experimental results have shown that the proposed deep beamformer, trained with the linearly weighted scale-invariant source-to-noise ratio (SI-SNR) and ARROW loss functions, achieves superior performance in speech enhancement and speaker localization compared to two baselines.

Long-term trend prediction of pandemic combining the compartmental and deep learning models

Predicting the spread trends of a pandemic is crucial, but long-term prediction remains challenging due to complex relationships among disease spread stages and preventive policies. To address this issue, we propose a novel approach that utilizes data augmentation techniques, compartmental model features, and disease preventive policies. We also use a breakpoint detection method to divide the disease spread into distinct stages and weight these stages using a self-attention mechanism to account for variations in virus transmission capabilities. Finally, we introduce a long-term spread trend prediction model for infectious diseases based on a bi-directional gated recurrent unit network. To evaluate the effectiveness of our model, we conducted experiments using public datasets, focusing on the prediction of COVID-19 cases in four countries over a period of 210 days. Experiments shown that the Adjust-R2 index of our model exceeds 0.9914, outperforming existing models. Furthermore, our model reduces the mean absolute error by 0.85–4.52% compared to other models. Our combined approach of using both the compartmental and deep learning models provides valuable insights into the dynamics of disease spread.

SSGCRTN: a space-specific graph convolutional recurrent transformer network for traffic prediction

SSGCRTN: a space-specific graph convolutional recurrent transformer network for traffic prediction

Effective Public Health Assessment: Research on Alzheimer Disease Diagnosis Model With Communication Technology

Rapid identification and appropriate management of Alzheimer's pose important challenges, particularly within public health contexts. This research introduces a unique framework for diagnosing Alzheimer's disease using communication technologies (such as wearable sensors). Novel chimp search-driven ReLu recurrent network (CS-RRN) strategy is introduced in this study for effectively detecting Alzheimer's disease. The ReLu function is employed in long/short-term memory (LSTM) to learn nonlinear dependencies, and then the chimp optimization strategy is applied to enhance the detection performance of the LSTM network. The gait signal data is collected from various wearable sensor devices to analyze the effectiveness of the suggested CS-RRN m technique. The performance of the proposed method is analyzed through a Python-driven implementation platform in terms of various metrics. The CS-RRN approach effectively achieved the maximum performance in detecting Alzheimer's disease when compared with other existing approaches. Based on the findings of this research, communication technology may substantially boost public health by providing a low-cost and high-performing method for Alzheimer disease identification.

Sensing with shallow recurrent decoder networks

Sensing is a universal task in science and engineering. Downstream tasks from sensing include inferring full-state estimates of a system (system identification), control decisions and forecasting. We propose a shallow recurrent decoder (SHRED) neural network structure for sensing, which incorporates (i) a recurrent neural network to learn a latent representation of the temporal dynamics of the sensors and (ii) a shallow decoder that learns a mapping between this latent representation and the high-dimensional state space. SHRED enables accurate reconstructions with far fewer sensors, outperforms existing techniques when more measurements are available and is more robust to random sensor placements. In the example cases explored, complex spatio-temporal dynamics are characterized by exceedingly limited sensors that can be randomly placed with minimal loss of performance.

Pyramid Convolutional Recurrent Network for Serial Medical Image Registration With Adaptive Motion Regularizations

Pyramid Convolutional Recurrent Network for Serial Medical Image Registration With Adaptive Motion Regularizations

CGFTrans: Cross-Modal Global Feature Fusion Transformer for Medical Report Generation.

Medical report generation, as a cross-modal automatic text generation task, can be highly significant both in research and clinical fields. The core is to generate diagnosis reports in clinical language from medical images. However, several limitations persist, including a lack of global information, inadequate cross-modal fusion capabilities, and high computational demands. To address these issues, we propose cross-modal global feature fusion Transformer (CGFTrans) to extract global information meanwhile reduce computational strain. Firstly, we introduce mesh recurrent network to capture inter-layer information at different levels to address the absence of global features. Then, we design feature fusion decoder and define 'mid-fusion' strategy to separately fuse visual and global features with medical report embeddings, which enhances the ability of the cross-modal joint learning. Finally, we integrate shifted window attention into Transformer encoder to alleviate computational pressure and capture pathological information at multiple scales. Extensive experiments conducted on three datasets demonstrate that the proposed method achieves average increments of 2.9%, 1.5%, and 0.7% in terms of the BLEU-1, METEOR and ROUGE-L metrics, respectively. Besides, it achieves average increments -22.4% and 17.3% training time and images throughput, respectively.

Mechanical properties and flow stress constitutive relationship of Ti–6Al–4V alloy with equiaxed microstructure at cryogenic temperatures

This paper investigates the uniaxial tensile mechanical properties and flow behavior of Ti-6Al-4V alloys with equiaxed microstructure at cryogenic temperatures ranging from 77 K to 298 K and strain rates from 10−4/s to 10−2/s. Scanning electron microscopy is utilized to analyze the fracture morphology, aiming to reveal the fracture behavior at various temperatures. The applicability of the Zener-Hollomon parameter and the Johnson-Cook model in describing the flow stress of Ti-6Al-4V at cryogenic temperatures is analyzed. Moreover, a constitutive relationship modeling method based on the variational recurrent networks is proposed. Mechanical test results show a significant increase in the strength of equiaxed Ti-6Al-4V alloy under cryogenic conditions while the plastic deformation process is shortened. However, the fracture analysis indicates that even at 77 K, the fracture process is still dominated by ductile fracture, and brittle fracture does not occur within the range of 77 K to 298 K. The fitting results validate the performance of the Zener-Hollomon parameter and the Johnson-Cook model in describing the deformation flow stress of Ti-6Al-4V alloy at cryogenic temperatures. The results also indicate that the proposed constitutive relationship modeling method based on the variational recurrent network performs better, making it a potential method for widespread applications.

Latent Relations at Steady-state with Associative Nets.

Models of word meaning that exploit patterns of word usage across large text corpora to capture semantic relations, like the topic model and word2vec, condense word-by-context co-occurrence statistics to induce representations that organize words along semantically relevant dimensions (e.g., synonymy, antonymy, hyponymy, etc.). However, their reliance on latent representations leaves them vulnerable to interference, makes them slow learners, and commits to a dual-systems account of episodic and semantic memory. We show how it is possible to construct the meaning of words online during retrieval to avoid these limitations. We implement a spreading activation account of word meaning in an associative net, a one-layer highly recurrent network of associations, called a Dynamic-Eigen-Net, that we developed to address the limitations of earlier variants of associative nets when scaling up to deal with unstructured input domains like natural language text. We show that spreading activation using a one-hot coded Dynamic-Eigen-Net outperforms the topic model and reaches similar levels of performance as word2vec when predicting human free associations and word similarity ratings. Latent Semantic Analysis vectors reached similar levels of performance when constructed by applying dimensionality reduction to the Shifted Positive Pointwise Mutual Information but showed poorer predictability for free associations when using an entropy-based normalization. An analysis of the rate at which the Dynamic-Eigen-Net reaches asymptotic performance shows that it learns faster than word2vec. We argue in favor of the Dynamic-Eigen-Net as a fast learner, with a single-store, that is not subject to catastrophic interference. We present it as an alternative to instance models when delegating the induction of latent relationships to process assumptions instead of assumptions about representation.

SFormer: An end-to-end spatio-temporal transformer architecture for deepfake detection

Growing AI advancements are continuously pacing GAN enhancement that eventually facilitates the generation of deepfake media. Manipulated media poses serious risks pertaining court proceedings, journalism, politics, and many more where digital media have a substantial impact on society. State-of-the-art techniques for deepfake detection rely on convolutional networks for spatial analysis, and recurrent networks for temporal analysis. Since transformers are capable of recognizing wide-range dependencies with a global spatial view and along temporal sequence too, a novel approach called “SFormer” is proposed in this paper, utilizing a transformer architecture for both spatial and temporal analysis to detect deepfakes. Further, state-of-the-art techniques suffer from high computational complexity and overfitting which causes loss in generalizability. The proposed model utilized a Swin Transformer for spatial analysis that resulted in low complexity, thereby enhancing its generalization ability and robustness against the different manipulation types. Proposed end-to-end transformer based model, SFormer, is proven to be effective for numerous deepfake datasets, including FF++, DFD, Celeb-DF, DFDC and Deeper-Forensics, and achieved an accuracy of 100%, 97.81%, 99.1%, 93.67% and 100% respectively. Moreover, SFormer has demonstrated superior performance compared to existing spatio-temporal and transformer-based approaches for deepfake detection.

A Spatiotemporal Locomotive Axle Temperature Prediction Approach Based on Ensemble Graph Convolutional Recurrent Unit Networks

Spatiotemporal axle temperature forecasting is crucial for real-time failure detection in locomotive control systems, significantly enhancing reliability and facilitating early maintenance. Motivated by the need for more accurate and reliable prediction models, this paper proposes a novel ensemble graph convolutional recurrent unit network. This innovative approach aims to develop a highly reliable and accurate spatiotemporal axle temperature forecasting model, thereby increasing locomotive safety and operational efficiency. The modeling structure involves three key steps: (1) the GCN module extracts and aggregates spatiotemporal temperature data and deep feature information from the raw data of different axles; (2) these features are fed into GRU and BiLSTM networks for modeling and forecasting; (3) the ICA algorithm optimizes the fusion weight coefficients to combine the forecasting results from GRU and BiLSTM, achieving superior outcomes. Comparative experiments demonstrate that the proposed model achieves RMSE values of 0.2517 °C, 0.2011 °C, and 0.2079 °C across three temperature series, respectively, indicating superior prediction accuracy and reduced errors compared to benchmark models in all experimental scenarios. The Wilcoxon signed-rank test further confirms the statistical significance of the result improvements with high confidence.

Managing remaining useful life of cyber-aeroengine systems using a graph spatio-temporal attention recurrent network with phase-lag index

The decision-making process concerning the maintenance of equipment significantly impacts the safety and reliability of the aeroengine system. A cyber-aeroengine system (CAS) integrates intricate aeroengine components with a cyber decision-making space based on the communication network, thereby facilitating computational support for forecasting the health status of the aeroengine system. Within the cyber realm, we introduce a novel approach termed the graph spatio-temporal attention recurrent network with phase-lag index (PLI-GSTARN) to assess the degradation states of aeroengine systems. This method constructs a spatio-temporal graph based on sensor signal correlations and information reachability across distributed sensors within the physical equipment. Subsequently, the proposed framework employs a graph attention network to capture spatial features and a long short-term memory (LSTM) layer equipped with an attention mechanism to capture temporal features inherent in the constructed graph. Ultimately, this methodology is applied to cyber–physical systems within aircraft engines to manage remaining health life and diagnose faults, yielding commendable outcomes.

Human movement monitoring system for classification of strength exercises and verification of their execution technique

Human movement analysis is critical to optimizing sports training and influencing exercise intensity and effectiveness. In the age of modern technology, more and more advanced systems are emerging to support coaches and expand the range of analysis performed. This article aims to verify that artificial intelligence, together with machine learning algorithms, can accurately classify exercises in a dynamic gym environment and effectively assess the correctness of their performance. For the initial analysis of movement, the Google MediaPipe Pose model was used, which was responsible for detecting the human silhouette and determining the coordinates of the position of critical joints. Based on these coordinates, the angles between each joint were calculated, and then their sequences were further analyzed. The sequences were analyzed using the following three algorithms: support vector machine (SVM), dense neural network, and LSTM recurrent network. As a result, the system based on recurrent LSTM networks achieved the best prediction efficiency of approximately 98%, enabling accurate exercise classification. Subsequently, verification of the activities' correctness was also carried out, and the system, based on recursive LSTM networks, again achieved the best efficiency, this time equal to 96% on average for all exercises. On this basis, it was concluded that the discussed approach enables practical analysis of human movement, which can significantly improve training methods and facilitate coaching work.

High-percentage new energy distribution network line loss frequency division prediction based on wavelet transform and BIGRU-LSTM.

The access of new energy improves the flexibility of distribution network operation, but also leads to more complex mechanism of line loss. Therefore, starting from the nonlinear, fluctuating and multi-scale characteristics of line loss data, and based on the idea of decomposition prediction, this paper proposes a new method of line loss frequency division prediction based on wavelet transform and BIGRU-LSTM (Bidirectional Gated Recurrent Unit-Long Short Term Memory Network).Firstly, the grey relation analysis and the improved NARMA (Nonlinear Autoregressive Moving Average) correlation analysis method are used to extract the non-temporal and temporal influencing factors of line loss, and the corresponding feature data set is constructed. Then, the historical line loss data is decomposed into physical signals of different frequency bands by using wavelet transform, and the multi-dimensional input data of the prediction network is formed with the above characteristic data set. Finally, the BIGRU-LSTM prediction network is built to realize the probabilistic prediction of high-frequency and low-frequency components of line loss. The effectiveness and applicability of the method proposed in this paper were verified through numerical simulation. By dividing the line loss data into different frequency bands for frequency prediction, the mapping relationship between different line loss components and influencing factors was accurately matched, thereby improving the prediction accuracy.