CNN-LSTM Models Research Articles

Forecasting Maritime and Financial Market Trends: Leveraging CNN-LSTM Models for Sustainable Shipping and China’s Financial Market Integration

With the acceleration of economic globalization, China’s financial market has emerged as a vital force in the global financial system. The Baltic Dry Index (BDI) and China Container Freight Index (CCFI) serve as key indicators of the shipping sector’s health, reflecting their sensitivity to shifts in China’s financial landscape. This study utilizes an innovative CNN-LSTM deep learning model to forecast the BDI and CCFI, using 25,974 daily data points from the Chinese financial market between 5 May 2015 and 30 November 2022. The model achieves high predictive accuracy across diverse samples, frequencies, and structural variations, with an R2 of 97.2%, showcasing its robustness. Beyond its predictive strength, this research underscores the critical role of China’s financial market in advancing sustainable practices within the global shipping industry. By merging advanced analytics with sustainable shipping strategies, the findings offer stakeholders valuable tools for optimizing operations and investments, reducing emissions, and promoting long-term environmental sustainability in both sectors. Additionally, this study enhances the resilience and stability of financial and shipping ecosystems, laying the groundwork for an eco-friendly, efficient, and sustainable global logistics network in the digital era.

Neural Networks for Audio Classification: Multi-scale CNN-LSTM Approach to Animal Sound Recognition

Abstract. This research explores the application of neural networks, specifically CNN-LSTM models, for classifying sound signals from dogs, frogs, and cats, selected from the ESC-50 dataset. The sound data was preprocessed using Mel-frequency cepstral coefficients (MFCCs) and augmented through time stretching, pitch shifting, and noise addition to enhance model generalization in varied acoustic environments. We compared two deep learning models: a traditional CNN-LSTM and an improved version with multi-scale feature extraction, allowing for capturing both short-term and long-term sound patterns. Our findings show that the multi-scale CNN-LSTM architecture outperforms the traditional CNN-LSTM, achieving a test accuracy of 86.11% compared to 80.56%. These results highlight the effectiveness of multi-scale feature extraction for handling complex audio signals. This research offers valuable insights into bioacoustics and has broader applications in areas such as environmental sound monitoring, wildlife preservation, and animal behavior analysis.

Lower Limb Motion Recognition Based on sEMG and CNN-TL Fusion Model.

To enhance the classification accuracy of lower limb movements, a fusion recognition model integrating a surface electromyography (sEMG)-based convolutional neural network, transformer encoder, and long short-term memory network (CNN-Transformer-LSTM, CNN-TL) was proposed in this study. By combining these advanced techniques, significant improvements in movement classification were achieved. Firstly, sEMG data were collected from 20 subjects as they performed four distinct gait movements: walking upstairs, walking downstairs, walking on a level surface, and squatting. Subsequently, the gathered sEMG data underwent preprocessing, with features extracted from both the time domain and frequency domain. These features were then used as inputs for the machine learning recognition model. Finally, based on the preprocessed sEMG data, the CNN-TL lower limb action recognition model was constructed. The performance of CNN-TL was then compared with that of the CNN, LSTM, and SVM models. The results demonstrated that the accuracy of the CNN-TL model in lower limb action recognition was 3.76%, 5.92%, and 14.92% higher than that of the CNN-LSTM, CNN, and SVM models, respectively, thereby proving its superior classification performance. An effective scheme for improving lower limb motor function in rehabilitation and assistance devices was thus provided.

An Investigation into the Utilisation of CNN with LSTM for Video Deepfake Detection

Video deepfake detection has emerged as a critical field within the broader domain of digital technologies driven by the rapid proliferation of AI-generated media and the increasing threat of its misuse for deception and misinformation. The integration of Convolutional Neural Network (CNN) with Long Short-Term Memory (LSTM) has proven to be a promising approach for improving video deepfake detection, achieving near-perfect accuracy. CNNs enable the effective extraction of spatial features from video frames, such as facial textures and lighting, while LSTM analyses temporal patterns, detecting inconsistencies over time. This hybrid model enhances the ability to detect deepfakes by combining spatial and temporal analysis. However, the existing research lacks systematic evaluations that comprehensively assess their effectiveness and optimal configurations. Therefore, this paper provides a comprehensive review of video deepfake detection techniques utilising hybrid CNN-LSTM models. It systematically investigates state-of-the-art techniques, highlighting common feature extraction approaches and widely used datasets for training and testing. This paper also evaluates model performance across different datasets, identifies key factors influencing detection accuracy, and explores how CNN-LSTM models can be optimised. It also compares CNN-LSTM models with non-LSTM approaches, addresses implementation challenges, and proposes solutions for them. Lastly, open issues and future research directions of video deepfake detection using CNN-LSTM will be discussed. This paper provides valuable insights for researchers and cyber security professionals by reviewing CNN-LSTM models for video deepfake detection contributing to the advancement of robust and effective deepfake detection systems.

Machine learning for base transceiver stations power failure prediction: A multivariate approach

The widespread deployment of cellular networks has improved communication access, driving economic growth and enhancing social connections across diverse regions. Base Transceiver Stations (BTSs), are foundational to mobile networks but are vulnerable to power failures, disrupting service delivery and causing user inconvenience. This paper proposes a machine-learning-based framework for preemptive BTS power failure prediction using multivariate time-series data from power and environmental monitoring systems. We employ a combination of deep learning architectures, including Convolutional Neural Networks (CNNs), Long Short-Term Memory (LSTM) networks, and hybrid CNN-LSTM models, to achieve accurate and timely predictions of BTS power failures. CNNs were selected for extracting dependencies among features of a multivariate time-series data, while LSTMs effectively capture temporal dependencies, making them suitable for predicting power failures.The proposed models exhibit noteworthy predictive performance, with the LSTM network emerging as the most accurate model (MSE: 0.001, MAPE: 2.528), followed by the hybrid CNN-LSTM (MSE: 0.001, MAPE: 2.843) and the CNN (MSE: 0.223, MAPE: 2.843). This work demonstrates deep learning’s effectiveness in preemptive BTS failure prediction, enabling proactive maintenance and improved network resilience.

Methodology and Experimental Verification for Predicting the Remaining Useful Life of Milling Cutters Based on Hybrid CNN-LSTM-Attention-PSA

In modern manufacturing, the prediction of the remaining useful life (RUL) of computer numerical control (CNC) milling cutters is crucial for improving production efficiency and product quality. This study proposes a hybrid CNN-LSTM-Attention-PSA model that combines convolutional neural networks (CNN), long short-term memory (LSTM) networks, and attention mechanisms to predict the RUL of CNC milling cutters. The model integrates cutting force, vibration, and current signals for multi-channel feature extraction during cutter wear. The model’s hyperparameters are optimized using a PID-based search algorithm (PSA), and comparative experiments were conducted with different predictive models. The experimental results demonstrate the proposed model’s superior performance compared to CNN, LSTM, and hybrid CNN-LSTM models, achieving an R2 score of 99.42% and reducing MAE, RMSE, and MAPE by significant margins. The results validate that the proposed method has significant reference and practical value for RUL prediction research of CNC milling cutters.

A Monitoring Device and Grade Prediction System for Grain Mildew.

Mildew infestation is a significant cause of loss during grain storage. The growth and metabolism of mildew leads to changes in gas composition and temperature within granaries. Recent advances in sensor technology and machine learning enable the prediction of grain mildew during storage. Current research primarily focuses on predicting mildew occurrence or grading using simple machine learning methods, without in-depth exploration of the time series characteristics of mildew process data. A monitoring device was designed and developed to capture high-quality microenvironment parameters and image data during a simulated mildew process experiment. Using the "Yongyou 15" rice varieties from Zhejiang Province, five simulation experiments were conducted under varying temperature and humidity conditions between January and May 2023. Mildew grades were defined through manual analysis to construct a multimodal dataset for the rice mildew process. This study proposes a combined model (CNN-LSTM-A) that integrates convolutional neural networks (CNN), long short-term memory (LSTM) networks, and attention mechanisms to predict the mildew grade of stored rice. The proposed model was compared with LSTM, CNN-LSTM, and LSTM-Attention models. The results indicate that the proposed model outperforms the others, achieving a prediction accuracy of 98%. The model demonstrates superior accuracy and more stable performance. The generalization performance of the prediction model was evaluated using four experimental datasets with varying storage temperature and humidity conditions. The results show that the model achieves optimal prediction stability when the training set contains similar storage temperatures, with prediction accuracy exceeding 99.8%. This indicates that the model can effectively predict the mildew grades in rice under varying environmental conditions, demonstrating significant potential for grain mildew prediction and early warning systems.

Improving Pre-trained CNN-LSTM Models for Image Captioning with Hyper-Parameter Optimization

Improving Pre-trained CNN-LSTM Models for Image Captioning with Hyper-Parameter Optimization

Empowering NGOs with Remote Sensing and CNN-LSTM Models for Social and Environmental Transformation

Empowering NGOs with Remote Sensing and CNN-LSTM Models for Social and Environmental Transformation

Spatial-temporal simulation and prediction of root zone soil moisture based on Hydrus-1D and CNN-LSTM-Attention in the Yutian Oasis, Southern Xinjiang, China

Root zone soil moisture (RZSM) plays a critical role in land-atmosphere hydrological cycles and serves as the primary water source for vegetation growth. However, the correlation between RZSM and its associated variables, including surface soil moisture (SSM), often exhibit nonlinearities that are challenging to identify and quantify using conventional statistical techniques. Therefore, this study presents a CNN-LSTM-Attention (CLA) model for predicting RZSM. Owing to the scarcity of soil moisture observation data, the physical model Hydrus-1D was employed to simulate a comprehensive dataset of spatiotemporal vertical soil moisture. Meteorological data and MODIS vegetation characterization parameters were used as predictor variables for the training and validation of the CLA model. The results of the CLA model for soil moisture prediction within the root zone were significantly enhanced compared with those of the traditional LSTM and CNN-LSTM models. This was particularly notable at depth of 80–100 cm, where the coefficient of determination (R²) reached nearly 0.9298. Moreover, the RMSE was reduced by 49% and 57% compared with those of the LSTM and CNN-LSTM models, respectively. This study demonstrates that the integration of physical modeling and deep learning techniques provides a more comprehensive and accurate understanding of spatial and temporal soil moisture variations in the root zone.

An Arrhythmia Classification Model Based on a CNN-LSTM-SE Algorithm

Arrhythmia is the main cause of sudden cardiac death, and ECG signal analysis is a common method for the noninvasive diagnosis of arrhythmia. In this paper, we propose an arrhythmia classification model based on the combination of a channel attention mechanism (SE module), convolutional neural network (CNN), and long short-term memory neural network (LSTM). The data of this model use the MIT-BIH arrhythmia database, and after noise reduction of raw ECG data by the EEMD denoising algorithm, a CNN-LSTM is used to learn features from the data, and the fusion channel attention mechanism is used to adjust the weight of the feature map. The CNN-LSTM-SE model is compared with the LSTM, CNN-LSTM, and LSTM-attention models, and the models are evaluated using Precision, Recall, and F1-Score. The classification performance of the tested CNN-LSTM-SE classification prediction model is better, with a classification accuracy of 98.5%, a classification precision rate of more than 97% for each label, a recall rate of more than 98%, and an F1-score of more than 0.98. It meets the requirements of arrhythmia classification prediction and has a certain practical value.

Prediction of Remaining Useful Life of Aero-engines Based on CNN-LSTM-Attention

Accurately predicting the remaining useful life (RUL) of aircraft engines is crucial for maintaining financial stability and aviation safety. To further enhance the prediction accuracy of aircraft engine RUL, a deep learning-based RUL prediction method is proposed. This method possesses the potential to strengthen the recognition of data features, thereby improving the prediction accuracy of the model. First, the input features are normalized and the CMAPSS (Commercial Modular Aero-Propulsion System Simulation) dataset is utilized to calculate the RUL for aircraft engines. After extracting attributes from the input data using a convolutional neural network (CNN), the extracted data are input into a long short-term memory (LSTM) network model, with the addition of attention mechanisms to predict the RUL of aircraft engines. Finally, the proposed aircraft engine model is evaluated and compared through ablation studies and comparative model experiments. The results indicate that the CNN-LSTM-Attention model exhibits superior prediction performance for datasets FD001, FD002, FD003, and FD004, with RMSEs of 15.977, 14.452, 13.907, and 16.637, respectively. Compared with CNN, LSTM, and CNN-LSTM models, the CNN-LSTM model demonstrates better prediction performance across datasets. In comparison with other models, this model achieves the highest prediction accuracy on the CMAPSS dataset, showcasing strong reliability and accuracy.

Advancing Heart Disease Forecasting: Integrating CNN-LSTM Models with Feature Enhancement for Improved Predictive Accuracy

Background: Heart failure (HF) is the leading cause of death worldwide, with accurate early diagnosis being challenging due to the disease's subtle symptoms and the requirement for extensive medical expertise. The development of predictive models is crucial for early intervention. Objective: This study proposes a method for Dynamic Forecasting (DF) of Heart Diseases (HD) using a Deep Learning (DL) model that integrates Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM) networks. The aim is to create an automatic, accurate, and cost-effective system for predicting cardiac valve failure. Methods: The study introduces a Hybrid DL model combining CNN and LSTM with Feature Enhancement (FE) and Comprehensible Artificial Intelligence (CAI) techniques. The CNN extracts features from clinical data, which are then processed by the LSTM model to handle temporal dependencies. The model's effectiveness was evaluated using a freely available cardiovascular disease dataset. Accuracy was assessed with and without FE. Results: The CNN-LSTM model achieved an accuracy of 83.5% with FE and 88.2% without FE. The model demonstrated superior performance in terms of sensitivity (80.04% with FE), specificity (87.11% with FE), and overall accuracy compared to other methods including Support Vector Machines (SVM), Decision Trees (DT), and Random Forests (RF). Conclusion: The proposed CNN-LSTM model offers a significant improvement in forecasting heart disease dynamics by effectively integrating CNN for feature extraction and LSTM for temporal analysis. This hybrid approach, combined with advanced feature enhancement and explainable AI techniques, provides a more accurate and comprehensible prediction tool for early heart disease detection.

Sentiment Analysis of Sirekap Tweets Using CNN Algorithm

Background: The research investigates the application of deep learning models for sentiment analysis on Twitter data related to Indonesia's Sirekap system. Sentiment analysis is crucial for understanding public opinion and enhancing the transparency and reliability of election result recapitulation processes. Objective: The objective of this study is to compare the performance of Convolutional Neural Networks (CNN) and CNN-LSTM models in analyzing sentiments from tweets about the Sirekap system. The study aims to identify the most effective model and preprocessing techniques to improve sentiment classification accuracy. Methods: A comprehensive data preprocessing pipeline was implemented, including cleansing, case folding, tokenizing, normalization, stopword removal, and stemming. To address class imbalance, the SMOTE technique was applied. The models were trained and evaluated using accuracy, precision, recall, and F1-score metrics. Pre-trained word embeddings were used to enhance model performance. Results: The CNN model achieved an accuracy of 85.90%, outperforming the CNN-LSTM model, which achieved 79.91% accuracy. Additionally, the CNN model demonstrated superior precision, recall, and F1-score metrics compared to the CNN-LSTM model. The thorough preprocessing and handling of class imbalance significantly contributed to the enhanced performance of the CNN model. Conclusion: The research emphasizes the effectiveness of deep learning approaches, particularly CNNs, in sentiment analysis tasks. The findings highlight the importance of comprehensive preprocessing and class imbalance handling. The use of pre-trained word embeddings and various evaluation metrics ensures robust model performance. These insights contribute to improving the accuracy and efficiency of sentiment classification, thereby enhancing the reliability and transparency of election result recapitulation processes.

Enhancing pressure transient analysis in reservoir characterization through deep learning neural networks

Deep learning neural networks exhibit remarkable efficiency in numerous tasks within machine learning applications across various disciplines, including reservoir engineering. Traditional pressure transient analysis (PTA) methodologies, however, are challenged by limitations such as human error in accurately estimating critical reservoir parameters like permeability and wellbore damage (skin). In this study, we evaluate the efficiency and accuracy of various machine learning models in pressure transient analysis (PTA) tasks. Our investigation initially focused on integrating Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM) networks to enhance the precision in estimating reservoir parameters and identifying reservoir models. However, through comprehensive testing and validation using simulated well-test data and out-of-distribution samples, we discovered that tree-based models, specifically XGBoost and Random Forest, exhibit superior performance, particularly in generalization tasks. These models outperform CNN-LSTM and 1D-CNN models, demonstrating enhanced robustness and accuracy in handling diverse and unseen data, thereby addressing traditional challenges in pressure transient analysis more effectively. Moreover, Savitzky–Golay Filters were used to handle the noise problems of data, and this method proved effective for data with noise levels up to a standard deviation of 2. Compared with other models, the XGBoost model has a better performance in reservoir model identification with an out-of-distribution accuracy of 92%. On the other hand, Random Forest showed a remarkable result in the task of reservoir parameter estimation, especially for the permeability and skin with out-of-distribution R2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$R^2$$\\end{document} of 98% and 97%, respectively. The findings of this study can help for a better understanding of the different deep learning models and their applications in pressure transient analysis. Specifically, the superior performance of the tree-based models in overcoming traditional pressure transient analysis challenges demonstrates their potential to enhance reservoir characterization predictions. Employing these advanced machine learning techniques can significantly reduce human error, increase the accuracy of parameter estimation, and improve the analysis process, thereby aiding strategic decision-making in reservoir management

SQL injection attack: Detection, prioritization & prevention

Web applications have become central in the digital landscape, providing users instant access to information and allowing businesses to expand their reach. Injection attacks, such as SQL injection (SQLi), are prominent attacks on web applications, given that most web applications integrate a database system. While there have been solutions proposed in the literature for SQLi attack detection using learning-based frameworks, the problem is often formulated as a binary, single-attack vector problem without considering the prioritization and prevention component of the attack. In this work, we propose a holistic solution, SQLR34P3R, that formulates the SQLi attack as a multi-class, multi-attack vector, prioritization, and prevention problem. For attack detection and classification, we gathered 457,233 samples of benign and malicious network traffic, as well as 70,023 samples that had SQLi and benign payloads. After evaluating several machine-learning-based algorithms, the hybrid CNN-LSTM models achieve an average F1-Score of 97% in web and network traffic filtering. Furthermore, by using CVEs of SQLi vulnerabilities, SQLR34P3R incorporates a novel risk analysis approach which reduces additional effort while maintaining reasonable coverage to assist businesses in allocating resources effectively by focusing on patching vulnerabilities with high exploitability. We also present an in-the-wild evaluation of the proposed solution by integrating SQLR34P3R into the pipeline of known vulnerable web applications such as Damn Vulnerable Web Application (DVWA) and Vulnado and via network traffic captured using Wireshark from SQLi DNS exfiltration conducted with SQLMap for real-time detection. Finally, we provide a comparative analysis with state-of-the-art SQLi attack detection and risk ratings solutions.

Enhancing anomaly detection: A comprehensive approach with MTBO feature selection and TVETBO[sbnd]Optimized Quad-LSTM classification

The rapid growth of computer networks has heightened system vulnerabilities, challenging traditional Machine Learning (ML) techniques in network anomaly detection due to the abundance of data, imbalanced attack classes, and large-scale datasets. This paper introduces a novel approach leveraging the AWID3 dataset, utilizing advanced preprocessing techniques such as data splitting and normalization. We propose the Mountaineering Team-Based Optimization (MTBO) algorithm for robust feature selection, inspired by social behavior and human collaboration. Additionally, we deploy a Quad-LSTM framework, integrating ConvLSTM, CNN-LSTM, E3D-LSTM, and SAConvLSTM models, fine-tuned by the Technical and Vocational Education and Training-Based Optimizer (TVETBO). Our approach achieves a remarkable 99.91 % accuracy, significantly surpassing existing models, thus enhancing network security by effectively addressing the complexities of anomaly detection.

The Use of Attention-Enhanced CNN-LSTM Models for Multi-Indicator and Time-Series Predictions of Surface Water Quality

The Use of Attention-Enhanced CNN-LSTM Models for Multi-Indicator and Time-Series Predictions of Surface Water Quality

Time series prediction model using LSTM-Transformer neural network for mine water inflow

Mine flooding accidents have occurred frequently in recent years, and the predicting of mine water inflow is one of the most crucial flood warning indicators. Further, the mine water inflow is characterized by non-linearity and instability, making it difficult to predict. Accordingly, we propose a time series prediction model based on the fusion of the Transformer algorithm, which relies on self-attention, and the LSTM algorithm, which captures long-term dependencies. In this paper, Baotailong mine water inflow in Heilongjiang Province is used as sample data, and the sample data is divided into different ratios of the training set and test set in order to obtain optimal prediction results. In this study, we demonstrate that the LSTM-Transformer model exhibits the highest training accuracy when the ratio is 7:3. To improve the efficiency of search, the combination of random search and Bayesian optimization is used to determine the network model parameters and regularization parameters. Finally, in order to verify the accuracy of the LSTM-Transformer model, the LSTM-Transformer model is compared with LSTM, CNN, Transformer and CNN–LSTM models. The results prove that LSTM-Transformer has the highest prediction accuracy, and all the indicators of its model are well improved.

Research on precise lithium battery state of charge estimation method based on CALSE-LSTM model and pelican algorithm

This paper presents an innovative fusion model called “CALSE-LSTM,” which integrates Convolutional Neural Networks (CNNs), Long Short-Term Memory Networks (LSTMs), self-attention mechanisms, and squeeze-and-excitation attention mechanisms to optimize the estimation accuracy of the State of Charge (SoC). The model incorporates battery historical data as input and employs a dual-attention mechanism based on CNN-LSTM to extract diverse features from the input data, thereby enhancing the model's ability to learn hidden information. To further improve model performance, we fine-tune the model parameters using the Pelican algorithm. Experiments conducted under Urban Dynamometer Driving Schedule (UDDS) conditions show that the CALSE-LSTM model achieves a Root Mean Squared Error (RMSE) of only 1.73 % in lithium battery SoC estimation, significantly better than GRU, LSTM, and CNN-LSTM models, reducing errors by 31.9 %, 31.3 %, and 15 %, respectively. Ablation experiments further confirm the effectiveness of the dual-attention mechanism and its potential to improve SoC estimation performance. Additionally, we validate the learning efficiency of CALSE-LSTM by comparing model training time with the number of iterations. Finally, in the comparative experiment with the Kalman filtering method, the model in this paper significantly improved its performance by incorporating power consumption as an additional feature input. This further verifies the accuracy of CALSE-LSTM in estimating the State of Charge (SoC) of lithium batteries.