Short-term Memory Network Research Articles

A Predictive Model for Sintering Ignition Temperature Based on a CNN-LSTM Neural Network with an Attention Mechanism

The sintering ignition process parameters fluctuate frequently and significantly, resulting in large variations in ignition temperature, which in severe cases can exceed 200 °C. This not only increases gas consumption but also affects the quality of the sinter. Because the intelligent control model based on feedback mechanisms struggles to deal with high-frequency fluctuation conditions over time, the prediction of sintering ignition temperature using feedforward regulation is attracting increasing attention. Given the multi-variable, time-sequential and strongly coupled characteristics of the sintering ignition process, a convolutional neural network (CNN) and a long short-term memory (LSTM) network are deeply integrated, with an attention mechanism incorporated to develop the sintering ignition temperature prediction model, enabling the accurate prediction of the ignition temperature. The research demonstrates that the combination of a CNN and the attention mechanism effectively addresses the challenges posed by the multi-variable and strongly coupled nature of sintering ignition data to predictive modeling. The LSTM network resolves the sequential data issues through its gating mechanism. As a result, the coefficient of determination (R2 ) of the CNN_LSTM-Attention model in predicting the sintering ignition temperature can reach 0.97, with a mean absolute error (MAE) as low as 10.23 °C. The predicted values closely match the actual values, achieving a hit rate of 93% within the acceptable error range. These performance metrics are significantly superior to those of the CNN-Attention and LSTM-Attention models, greatly enhancing the control accuracy of the ignition temperature.

Leveraging advanced AI algorithms with transformer-infused recurrent neural networks to optimize solar irradiance forecasting

Solar energy (SE) is vital for renewable energy generation, but its natural fluctuations present difficulties in maintaining grid stability and planning. Accurate forecasting of solar irradiance (SI) is essential to address these challenges. The current research presents an innovative forecasting approach named as Transformer-Infused Recurrent Neural Network (TIR) model. This model integrates a Bi-Directional Long Short-Term Memory (BiLSTM) network for encoding and a Gated Recurrent Unit (GRU) network for decoding, incorporating attention mechanisms and positional encoding. This model is proposed to enhance SI forecasting accuracy by effectively utilizing meteorological weather data, handling overfitting, and managing data outliers and data complexity. To evaluate the model’s performance, a comprehensive comparative analysis is conducted, involving five algorithms: Artificial Neural Network (ANN), BiLSTM, GRU, hybrid BiLSTM-GRU, and Transformer models. The findings indicate that employing the TIR model leads to superior accuracy in the analyzed area, achieving R2 value of 0.9983, RMSE of 0.0140, and MAE of 0.0092. This performance surpasses those of the alternative models studied. The integration of BiLSTM and GRU algorithms with the attention mechanism and positional encoding has been optimized to enhance the forecasting of SI. This approach mitigates computational dependencies and minimizes the error terms within the model.

Development of a Digital Twin Driven by a Deep Learning Model for Fault Diagnosis of Electro-Hydrostatic Actuators

The first quarter of the 21st century has witnessed many technological innovations in various sectors. Likewise, the COVID-19 pandemic triggered the acceleration of digital transformation in organizations driven by artificial intelligence and communication technologies in Industry 4.0 and Industry 5.0. Aiming at the construction of digital twins, virtual representations of a physical system allow real-time bidirectional communication. This will allow the monitoring of operations, identification of possible failures, and decision making based on technical evidence. In this study, a fault diagnosis solution is proposed, based on the construction of a digital twin, for a cloud-based Industrial Internet of Things (IIoT) system contemplating the control of electro-hydrostatic actuators (EHAs). The system was supported by a deep learning model using Long Short-Term Memory (LSTM) networks for an effective diagnostic approach. The implemented study considers data preparation and integration and system development and application to evaluate the performance against the fault diagnosis problem. According to the results obtained, positive results are shown in the construction of the digital twin using a deep learning model for the fault diagnosis problem of an active EHA-IIoT configuration.

Ground subsidence prediction with high precision: a novel spatiotemporal prediction model with Interferometric Synthetic Aperture Radar technology

ABSTRACT As the extraction of mineral resources intensifies, ground subsidence in mining areas has escalated, posing substantial challenges to sustainable development and operational safety. This subsidence, resulting directly from mining activities, significantly compromises the safety of nearby residents by damaging residential structures and infrastructure. Thus, developing precise and dependable methods for predicting ground subsidence is crucial. This study introduces an innovative Cabs-Unet model, which enhances the U-Net architecture by integrating a Convolutional Block Attention Module (CBAM) and Depthwise Separable Convolutions (DSC). This model aims to predict the spatiotemporal dynamics of the Interferometric Synthetic Aperture Radar (InSAR) time series. Employing Small Baseline Subset Interferometric Synthetic Aperture Radar (SBAS InSAR) technology, we gathered and validated data on ground subsidence at the Pengzhuang coal mine from May 2017 to November 2021, covering 130 scenes, with its accuracy corroborated by levelling survey results. An empirical evaluation of the Cabs-Unet model in two distinct subsidence zones demonstrated superior performance over conventional methods like Convolutional Long Short-Term Memory (ConvLSTM) and Predictive Recurrent Neural Network (PredRNN), with Root Mean Square Error (RMSE) values of 1.44 and 1.70, respectively. These findings highlight the model’s efficacy in accurately predicting spatiotemporal InSAR ground subsidence. Further predictive analysis using InSAR data indicated an expected increase in subsidence, projecting cumulative declines of −457 mm in Area A and −1278 mm in Area B by 17 July 2022. Our model proves effective in assessing subsidence, promptly detecting potential risks and facilitating the rapid implementation of risk mitigation strategies.

Analysis and prediction of mesoscale eddy kinetic energy variations in the Kuroshio extension

The Kuroshio Extension (KE) region, a crucial area in the Northwest Pacific Ocean, exhibits eddy kinetic energy with various scales of periodicity. Understanding how to extract its characteristic features and analyze and predict their periodic correlations has become vital for studying the regulatory mechanisms of eddy kinetic energy in the KE region. This paper first introduces a mesoscale eddy hybrid identification algorithm based on the flow field vector and the closed flow field. Using this algorithm, we gather mesoscale eddy identification data from the KE region to extract the monthly average series of five typical features of the KE region. Subsequently, wavelet theory is applied to analyze the cycles of these main features, identifying the common cycles of the KE region as the primary focus for analyzing the vorticity kinetic energy. This analysis includes cycle correlations with globally recognized indices, and it predicts these correlations. Further analysis of the main characteristic cycles through wavelet theory reveals that the KE region's eddy kinetic energy is significantly influenced by solar activity over long periods and by the North Pacific ocean-atmosphere interaction over shorter, interannual periods. Finally, this paper introduces a W-LSTM (Wavelet Decomposition based Long Short-term Memory Networks) prediction model based on wavelet decomposition for the KE region, covering January 2023–December 2023. The model demonstrates its effectiveness, achieving a Root Mean Square Error (RMSE) of 0.2530 and a correlation coefficient of 0.8259 between the predicted data and the actual observations.

High Stability Control of a Magnetic Suspension Flywheel Based on SA-BPNN and CNN+LSTM+ATTENTION

Compared to traditional, static-based flywheel systems, vehicle-mounted magnetic suspension flywheels face more complex operating conditions, and existing control strategies usually regard disturbances in vehicles under different operating conditions to be the same problem. Therefore, it is necessary to determine the interference from complex operating conditions and reasonably distinguish among them under different operating conditions to provide flywheel systems with strong stability (the rotor offset was less than 0.025 mm). Thus, this paper proposes a high-stability control strategy for flywheels based on the classification of vehicle-driving conditions and designs its control strategy by taking the vehicle-mounted magnetic suspension flywheel with a virtual inertia spindle as an example. First, according to the different vehicle working conditions and the varying interference intensities affecting the flywheel system, the working mode is divided into four modes. Considering the obvious differences in each working mode, it is proposed to use BP neural network optimization based on the simulated annealing algorithm (SA-BPNN) to determine the flywheel’s working condition. A relatively simple neural network can improve the response speed of the whole system. It also has a good effect. Secondly, it is proposed to use deep learning models based on convolutional neural networks, long short-term memory networks and attention mechanisms (CNN+LSTM+ATTENTION) to train the corresponding control parameters under each working condition to judge and predict the control parameters under different working conditions. Three evaluation parameters are used to evaluate the training results, and all achieved good results. Finally, the classification of working conditions and performance tests are carried out. The experimental results show the effectiveness and superiority of the proposed control strategy.

DAFE-MSGAT: Dual-Attention Feature Extraction and Multi-Scale Graph Attention Network for Polyphonic Piano Transcription

Automatic music transcription (AMT) aims to convert raw audio signals into symbolic music. This is a highly challenging task in the fields of signal processing and artificial intelligence, and it holds significant application value in music information retrieval (MIR). Existing methods based on convolutional neural networks (CNNs) often fall short in capturing the time-frequency characteristics of audio signals and tend to overlook the interdependencies between notes when processing polyphonic piano with multiple simultaneous notes. To address these issues, we propose a dual attention feature extraction and multi-scale graph attention network (DAFE-MSGAT). Specifically, we design a dual attention feature extraction module (DAFE) to enhance the frequency and time-domain features of the audio signal, and we utilize a long short-term memory network (LSTM) to capture the temporal features within the audio signal. We introduce a multi-scale graph attention network (MSGAT), which leverages the various implicit relationships between notes to enhance the interaction between different notes. Experimental results demonstrate that our model achieves high accuracy in detecting the onset and offset of notes on public datasets. In both frame-level and note-level metrics, DAFE-MSGAT achieves performance comparable to the state-of-the-art methods, showcasing exceptional transcription capabilities.

Modulation Pattern Identification Algorithm for Model Transfer Based on Machine Learning

To overcome the shortcomings of traditional modulation identification methods in small sample signal processing, this paper proposes a new model transfer modulation identification algorithm that integrates machine learning. The algorithm first uses a combination of convolutional neural network (CNN) and long short-term memory network (LSTM) to fully tap and utilize its feature extraction capabilities in the spatial and temporal dimensions. It significantly improves the performance of the model in these two dimensions through a parallel structure, which can ensure high efficiency and accuracy in signal recognition tasks. Then, through model migration technology, the common features in the pre-trained model are retained and fine-tuned to adapt to new signal recognition needs, effectively solving the problem of small sample signal recognition. Experimental results show that the algorithm proposed in this paper significantly improves signal recognition accuracy, with an average accuracy rate of 96%. In the 16APSK and 16QAM signal recognition tasks, the accuracy is as high as 100% and 99%, respectively, which demonstrates excellent performance.

Enhancing International Logistics and Supply Chain Management: Deep Learning Strategies for Enhanced Route Planning and Warehouse Optimization

Logistics and Supply Chain Management (SCM) are both key areas in modern industry and commerce which need better route planning and warehouse optimization. Traditional methods that are in practice have often fall short in of addressing the dynamic complexities of modern logistics which results in inefficient travel times, fuel consumption, and space utilization. To counter these limitations this study introduces an integrated model that combines Long Short-Term Memory (LSTM) networks, Radial Basis Function Neural Networks (RBNN), and the Non-dominated Sorting Genetic Algorithm II (NSGA-II) for route planning and warehouse optimization. The proposed model employ LSTM to predict traffic patterns and RBNN to optimize space utilization in warehouse. The NSGA-II model is then utilized for multi-objective optimization of minimizing travel times and maximizing warehouse space utilization. In experiment analysis the proposed model achieved the highest accuracy and least variability in predictions, with mean MAE, RMSE, and MAPE values of 0.57, 1.12, and 5.9%, respectively.

Discrete Time Series Forecasting of Hive Weight, In-Hive Temperature, and Hive Entrance Traffic in Non-Invasive Monitoring of Managed Honey Bee Colonies: Part I.

From June to October, 2022, we recorded the weight, the internal temperature, and the hive entrance video traffic of ten managed honey bee (Apis mellifera) colonies at a research apiary of the Carl Hayden Bee Research Center in Tucson, AZ, USA. The weight and temperature were recorded every five minutes around the clock. The 30 s videos were recorded every five minutes daily from 7:00 to 20:55. We curated the collected data into a dataset of 758,703 records (280,760-weight; 322,570-temperature; 155,373-video). A principal objective of Part I of our investigation was to use the curated dataset to investigate the discrete univariate time series forecasting of hive weight, in-hive temperature, and hive entrance traffic with shallow artificial, convolutional, and long short-term memory networks and to compare their predictive performance with traditional autoregressive integrated moving average models. We trained and tested all models with a 70/30 train/test split. We varied the intake and the predicted horizon of each model from 6 to 24 hourly means. Each artificial, convolutional, and long short-term memory network was trained for 500 epochs. We evaluated 24,840 trained models on the test data with the mean squared error. The autoregressive integrated moving average models performed on par with their machine learning counterparts, and all model types were able to predict falling, rising, and unchanging trends over all predicted horizons. We made the curated dataset public for replication.

An innovative attention infused- BiRecurrenTwin network assisted hybrid segmentation technique for accurate heart disease prediction

Heart disease is a noteworthy global health concern, profoundly affecting public health and individual well-being. Early prediction and diagnosis are crucial for reducing its negative impact. This work focuses on developing an effectual heart disease detection employing advanced optimization approaches and data processing. The proposed system incorporates a hybrid segmentation algorithm, Hybrid K-means-Fuzzy-C-means (HKMFCM) clustering, which assigns membership degrees to data points, enabling soft clustering and accommodating data uncertainty. Additionally, the Artificial Bee Colony (ABC) Optimization method is applied for feature selection, mimicking the foraging behavior of honeybees to identify the most discriminative attributes from the dataset. This optimization algorithm iteratively explores the feature space to select features that enhance the model's predictive accuracy. Furthermore, a novel classification architecture, termed the Attention-infused BiRecurrenTwin Network, is introduced to accurately predict heart disease based on segmented and extracted patient data profiles. This classifier leverages both Bidirectional Gated Recurrent Unit (BiGRU) and Bidirectional Long Short-Term Memory (BiLSTM) networks, with their bidirectional nature capturing both past and future contexts, thus increasing the classifier's capability to detect subtle temporal patterns in patient data. In addition, the proposed system addresses traditional approaches limitations through its advanced components like HKMFCM (effectively manages data uncertainty by enabling soft clustering), ABC Optimization technique (enabling an iterative, global search for the most discriminative attributes) Attention-infused BiRecurrenTwin Network (surpasses conventional classifiers by capturing both past and future temporal patterns in patient data). The simulation outcomes demonstrate that the developed system attains improved performance, with accuracy, ROC, F-measure, precision, recall, sensitivity, and specificity values of 96.09%, 97%, 95%, 96.6%, 94.3%, 95.14%, and 97.05%, respectively which is higher than the baseline models with an average of 7.75 % increase in accuracy. These results indicate that the developed predictive models show promise in accurately classifying individuals based on their extracted features, thereby facilitating the early recognition of heart disease.

Development of analytical “aroma wheels” for Oolong tea infusions (Shuixian and Rougui) and prediction of dynamic aroma release and colour changes during “Chinese tea ceremony” with machine learning

The flavour of tea as a worldwide popular beverage has been studied extensively. This study aimed to apply established flavour analysis techniques (GC–MS, GC-O-MS and APCI-MS/MS) in innovative ways to characterise the flavour profile of oolong tea infusions for two types of oolong tea (type A- Shuixian, type B- Rougui). GC–MS identified 48 aroma compounds, with type B having a higher abundance of most compounds. GC-O-MS analysis determined the noticeable aroma difference based on 20 key aroma compounds, facilitating the creation of an analytical “Aroma Wheel” with 8 key odour descriptors. APCI-MS/MS assessed real-time aroma release during successive brews linked with the “Chinese tea ceremony” (Gongfu Cha). Multivariate Polynomial Regression (MPR) and Long Short-Term Memory (LSTM) network approaches were applied to aroma and colour data from seven successive brews. The results revealed a progressive decline in both colour and aroma with seven repeated brews, particularly notable after the fourth brew.

In-depth research on fault diagnosis of turbine rotor utilizing NGSABO-optimized VMD and CNN-BiLSTM

Abstract To solve the problem of difficulty in extracting and identifying fault types during turbine rotor operation, a fault diagnosis method based on improved subtraction mean optimizer (NGSABO) algorithm to optimize variational mode decomposition (VMD) and CNN-BiLSTM neural network is proposed. Firstly, three improvements are made to the subtraction average optimizer algorithm. Secondly, the optimal VMD parameter combination of NGSABO adaptive selection mode decomposition number K and penalty factor is used to decompose the rotor fault signal, and the minimum sample entropy is used as the fitness function for feature extraction. Combining convolutional neural network and bidirectional long short-term memory network to identify and classify features. Compared with other methods, this method has outstanding performance in the diagnosis of single and coupled rotor faults. The accuracy of fault diagnosis reaches 98.5714%, which has good practical application value.

Enhancing Bearing Fault Diagnosis With Deep Learning Model Fusion and Semantic Web Technologies

Given the limited accuracy of a singular deep learning model in bearing fault diagnosis, this study seeks to investigate and validate the efficacy of a deep learning model fusion strategy. It also aims to enhance the performance of deep learning models in bearing fault diagnosis using semantic web technology. Utilizing a publicly available bearing dataset, we employ semantic web to represent data in a structured format that is easily interpretable by machines. We then train separate convolutional neural network (CNN) and long short-term memory network (LSTM) models, and implement three distinct fusion strategies: voting fusion, weighted fusion, and stacking fusion. The experimental results indicate that the model fusion strategy significantly improves both the accuracy and robustness of fault diagnosis. Notably, the stacking fusion strategy outperforms the others in recognizing complex fault patterns. This study offers novel ideas and methodologies for the application of deep learning in the realm of bearing fault diagnosis.

Advancing Mental Health and Ethics Education for College Students

This study explores the implementation of the DeepPsy framework at a university in Hebei, China, assessing its impact on student mental health and ethical behavior. By integrating a 2D convolutional neural network (2D-CNN) with long short-term memory (LSTM) networks, DeepPsy analyzes complex online behavior patterns and academic performance data. The methodology involves diverse data collection techniques, providing a comprehensive understanding of students' mental health. Results indicate improved identification of at-risk students, leading to timely interventions and enhanced mental health outcomes. This study highlights the potential of advanced data analytics in fostering supportive educational environments that promote student mental health and ethical decision making.

Flow prediction via adaptive dynamic graph with spatio-temporal correlations

Flow prediction has shown its significance in optimizing and smoothing resource allocation. However, it still faces challenges due to inherent temporal variability and dynamic spatial correlations of flow data. Although many methods have been introduced to solve the problem, they often rely on predefined graphs to capture shared patterns, which is likely to overlook the critical spatiotemporal dependencies that are necessary for mid-term and long-term prediction tasks. To address the issue, a novel flow prediction model, termed Dynamic Graph Multi-Step Prediction via Long Short-Term Memory Network and Adaptive Learning (DMLSA), is proposed. DMLSA employs Node Adaptive Learning (NAL) to learn node-specific patterns without relying on predefined static adjacency matrices. Moreover, DMLSA utilizes Graph Adaptive Generation (GAG) to extract dynamic features from node attributes and automatically infer interdependencies among flow sequences, thus enhancing the ability to predict complex flow dynamics. Additionally, DMLSA integrates multi-head self-attention mechanisms with gated recurrent units, effectively capturing both long-term and short-term node dependencies, therefore allowing for more precise identification of spatiotemporal patterns at the individual node level. Extensive experiments on five datasets demonstrate that DMLSA outperformed existing baseline models in terms of accuracy and stability for multi-step prediction tasks.

Improved decision-making through life event prediction: A case study in the financial services industry

Life event prediction is an important tool for customer relationship management (CRM), because life events shift customers’ preferences towards different products and services. Existing life event research mainly uses cross-sectional data, whereas in the CRM field, incorporating longitudinal data is increasingly common. Because longitudinal data can capture the dynamics of customer behavior, opportunities arise to benchmark the power of longitudinal customer data for predictions of cross-sectional versus longitudinal life events. Therefore, this study compares statistical and machine learning (SaML) classifiers, such as logistic regression, random forest, and XGBoost, with long- and short-term memory networks (LSTM), using data represented in both cross-sectional and longitudinal setups for life event prediction. Through a real-life longitudinal customer data set from a European bank, the authors represent the longitudinal data in a cross-sectional data format, using featurization in the form of aggregation. The available data cover 42 end-of-month snapshots for 760,438 unique customers. For marketing decision-making literature, this article (1) introduces three novel life events (i.e., primary, secondary, and rental residence purchases) to life event predictions; (2) offers guidance for how to leverage longitudinal customer data, according to the comparison of various featurization approaches and benchmarking SaML classifiers against LSTM; and (3) clarifies the importance of features and timing for improving marketing decision-making dynamically. The results show that aggregating features over time is preferable as a featurization approach for cross-sectional modeling using SaML classifiers. Furthermore, LSTM can capture behavioral changes over time, unlike SaML classifiers. It also performs significantly better than SaML classifiers on the area under curve and F1 metrics. Insights into the uses of integrated gradients reveal that feature importance changes over time. An integrated gradients method can assist decision-makers in their efforts to plan effective communication with customers in advance, such as by allocating more resources to customers who exhibit high probabilities of a particular life event occurrence.

A Hybrid Deep Learning Approach with Generative Adversarial Network for Credit Card Fraud Detection

Credit card fraud detection is a critical challenge in the financial industry, with substantial economic implications. Conventional machine learning (ML) techniques often fail to adapt to evolving fraud patterns and underperform with imbalanced datasets. This study proposes a hybrid deep learning framework that integrates Generative Adversarial Networks (GANs) with Recurrent Neural Networks (RNNs) to enhance fraud detection capabilities. The GAN component generates realistic synthetic fraudulent transactions, addressing data imbalance and enhancing the training set. The discriminator, implemented using various DL architectures, including Simple RNN, Long Short-Term Memory (LSTM) networks, and Gated Recurrent Units (GRUs), is trained to distinguish between real and synthetic transactions and further fine-tuned to classify transactions as fraudulent or legitimate. Experimental results demonstrate significant improvements over traditional methods, with the GAN-GRU model achieving a sensitivity of 0.992 and specificity of 1.000 on the European credit card dataset. This work highlights the potential of GANs combined with deep learning architectures to provide a more effective and adaptable solution for credit card fraud detection.

The Use of eXplainable Artificial Intelligence and Machine Learning Operation Principles to Support the Continuous Development of Machine Learning-Based Solutions in Fault Detection and Identification

Machine learning (ML) revolutionized traditional machine fault detection and identification (FDI), as complex-structured models with well-designed unsupervised learning strategies can detect abnormal patterns from abundant data, which significantly reduces the total cost of ownership. However, their opaqueness raised human concern and intrigued the eXplainable artificial intelligence (XAI) concept. Furthermore, the development of ML-based FDI models can be improved fundamentally with machine learning operations (MLOps) guidelines, enhancing reproducibility and operational quality. This study proposes a framework for the continuous development of ML-based FDI solutions, which contains a general structure to simultaneously visualize and check the performance of the ML model while directing the resource-efficient development process. A use case is conducted on sensor data of a hydraulic system with a simple long short-term memory (LSTM) network. Proposed XAI principles and tools supported the model engineering and monitoring, while additional system optimization can be made regarding input data preparation, feature selection, and model usage. Suggested MLOps principles help developers create a minimum viable solution and involve it in a continuous improvement loop. The promising result motivates further adoption of XAI and MLOps while endorsing the generalization of modern ML-based FDI applications with the HITL concept.

Forecasting Lake Nokoué Water Levels Using Long Short-Term Memory Network

The forecasting of hydrological flows (rainfall depth or rainfall discharge) is becoming increasingly important in the management of hydrological risks such as floods. In this study, the Long Short-Term Memory (LSTM) network, a state-of-the-art algorithm dedicated to time series, is applied to predict the daily water level of Lake Nokoué in Benin. This paper aims to provide an effective and reliable method to enable the reproduction of the future daily water level of Lake Nokoué, which is influenced by a combination of two phenomena: rainfall and river flow (runoff from the Ouémé River, the Sô River, the Porto-Novo lagoon, and the Atlantic Ocean). Performance analysis based on the forecasting horizon indicates that LSTM can predict the water level of Lake Nokoué up to a forecast horizon of t + 10 days. Performance metrics such as Root Mean Square Error (RMSE), coefficient of correlation (R2), Nash–Sutcliffe Efficiency (NSE), and Mean Absolute Error (MAE) agree on a forecast horizon of up to t + 3 days. The values of these metrics remain stable for forecast horizons of t + 1 day, t + 2 days, and t + 3 days. The values of R2 and NSE are greater than 0.97 during the training and testing phases in the Lake Nokoué basin. Based on the evaluation indices used to assess the model’s performance for the appropriate forecast horizon of water level in the Lake Nokoué basin, the forecast horizon of t + 3 days is chosen for predicting future daily water levels.