CNN-LSTM Model Research Articles

A CNN-LSTM Model for Short-Term Passenger Flow Forecast Considering the Built Environment in Urban Rail Transit Stations

A CNN-LSTM Model for Short-Term Passenger Flow Forecast Considering the Built Environment in Urban Rail Transit Stations

A Novel Method of Bridge Deflection Prediction Using Probabilistic Deep Learning and Measured Data

The deflection control of the main girder in suspension bridges, as flexible structures, is critically important during their operation. To predict the vertical deflection of existing suspension bridge girders under the combined effects of stochastic traffic loads and environmental temperature, this paper proposes an integrated deflection interval prediction method based on a Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), a probability density estimation layer, and bridge monitoring data. A time-series training dataset consisting of environmental temperature, vehicle load, and deflection monitoring data was built based on bridge health monitoring data. The CNN-LSTM combined layer is used to capture both local features and long-term dependencies in the time series. A Gaussian distribution (GD) is adopted as the probability density function, and its parameters are estimated using the maximum likelihood method, which outputs the optimal deflection prediction and probability intervals. Furthermore, this paper proposes a method for identifying abnormal deflections of the main girder in existing suspension bridges and establishes warning thresholds. This study indicates that, for short time scales, the CNN-LSTM-GD model achieves a 47.22% improvement in Root Mean Squared Error (RMSE) and a 12.37% increase in the coefficient of determination (R2) compared to the LSTM model. When compared to the CNN-LSTM model, it shows an improvement of 28.30% in RMSE and 6.55% in R2. For long time scales, the CNN-LSTM-GD model shows a 54.40% improvement in RMSE and a 10.22% increase in R2 compared to the LSTM model. Compared to the CNN-LSTM model, it improves RMSE by 38.43% and R2 by 5.31%. This model is instrumental in more accurately identifying abnormal deflections and determining deflection thresholds, making it applicable to bridge deflection early-warning systems.

Novel state of charge estimation method of containerized Lithium–Ion battery energy storage system based on deep learning

State of charge (SOC) is a critical indicator for lithium–ion battery energy storage system. However, model-driven SOC estimation is challenging due to the coupling of internal charging and discharging processes, ion diffusion, and chemical reactions in the electrode materials. These factors result in frequent changes in their internal characteristics. Therefore, we devise a novel deep learning SOC estimation method based on the attention mechanism (A) and convolutional neural networks-long short term memory (CNN–LSTM) network model. The CNN layers are employed to extract the critical features of the battery dataset. The LSTM layers are utilized to capture the long-term dependencies in the time series data. The attention mechanism can assign attention weights to different features within the output of the convolution layer. The A–CNN–LSTM method not only amplifies the positive influence of critical knowledge but also provides a global reference. The comparative experiments of three datasets and ablation study are carried out. The root mean square error, mean absolute error, and mean absolute percentage error obtained from the A–CNN–LSTM model decreased by 32.75 %, 45.88 %, and 53.36 %, respectively, compared with the best results obtained from the existing CNN, LSTM, gate recurrent unit (GRU), CNN–LSTM, and CNN–GRU models. The proposed model demonstrates greater stability and consistency than other models, underscoring its efficacy in SOC estimation.

Research on Momentum prediction algorithm for athletes based on the CNN-LSTM Model

Momentum has been widely recognized as a psychological factor affecting athletes' performance during sports events. Quantifying the momentum of both athletes in a match and predicting its future trends and reversals can provide effective information for coaches to make tactical decisions during a match. In this paper, taking the Wimbledon dataset as an example, the quantitative model of momentum is firstly constructed by the rank-sum-ratio evaluation method based on EWMA. Subsequently, this paper introduces the CNN network to process the data input based on the traditional LSTM network, takes the output data of the CNN network as the input layer of the LSTM network, and constructs a CNN-LSTM model to predict the change of momentum of athletes in the match, and the reversal time of momentum of athletes of both sides points. Through the model comparison analysis, this paper found that the prediction accuracy of the CNN-LSTM in different time periods is up to 85%, surpassing the LSTM network by 10%. This paper helps coaches and athletes to identify the future trend of momentum in the course of the match so that they can make reasonable tactical arrangements to achieve better performance in sports events.

A hybrid CNN-LSTM approach for intelligent cyber intrusion detection system

As the technology is advancing more and more in the era of increasing digitalization, safeguarding networks from cyber threats is crucial. As cyber-attacks on critical infrastructure are becoming more and more sophisticated, enhancing cyber intrusion detection systems (IDS) is imperative. This paper proposes and evaluates a deep learning-based IDS using the NSL-KDD dataset, a benchmark for intrusion detection. The system pre-processes data with Recursive Feature Elimination (RFE) and a Decision Tree classifier to identify the most significant features, optimizing model performance. Various deep learning models, including ANN, LSTM, BiLSTM, CNN-LSTM, GRU, and BiGRU, have been evaluated. The CNN-LSTM model outperformed the others, with 95 % accuracy, 0.89 recall, and 0.94 f1-score. These results prove the effectiveness of the proposed IDS in accurately distinguishing between malicious and benign network traffic. Future research can explore ensemble techniques like boosting or bagging to further enhance IDS performance.

Sarcasm Detection Based on Sentiment Analysis of Audio Corpus Using Deep Learning

With the rapid progress in digitalization, hundreds of services and applications have begun to depend on human-machine communication, primarily voice-based. As technology advances, it has become increasingly imperative to develop systems that can discern sophisticated emotions from humans, like sarcasm. Despite research that has been conducted on the detection and analysis of speech signals to try and understand emotion, the detection of sarcasm, which is often not overtly expressed, is quite far from something that has received the attention it deserves in recent years. It plays a very important role in any utterance, searching to identify the psychology, mood, or even health behind it. Hence, it does give interactions with humans a more human-like flavor while raising the feeling of understanding human emotions. This article tries to identify sarcasm in speech using audio data derived from real-world, spontaneous, monolingual corpora. In this classification process, a deep learning classifier known as CNN-LSTM model is used. Results of the study showed that a robust combination between feature extraction through convolutional layers and sequential learning through LSTM layers can identify intricate speech patterns in sarcasm. Experimented and validated in terms of performance, this model shows the model to be very good at discriminating between sarcastic and no sarcastic speech, thus promoting it for more extensive use in various applications related to sentiment analysis and in human-computer interaction applications. The results show that this system provides far much closer approximations to human interaction styles, but its impact would extend into a wide area from customer services, monitoring mental health, and even AI-driven communication systems. It is a monolingual and spontaneous corpus that will open the gates for further work and development down the line and possibly extend to multilingual and more diversified speech contexts, thus fine-tuning accuracy in practical scenarios of sarcasm detection.

The role of news sentiment in salmon price prediction using deep learning

This paper employs deep learning models and sentiment analysis to predict salmon spot prices. Our data includes historical price data and sentiment scores from 2018 to 2022. We extract sentiment scores from salmon-related news headlines by using FinBERT and TextBlob. We begin with price prediction using only historical price data and then introduce sentiment scores to improve the prediction accuracy of deep learning models. We find that the prediction performance of deep learning models outperforms traditional prediction methods in the salmon market. Our primary hybrid CNN-LSTM model outperforms other deep learning models and traditional models. Additionally, deep learning models incorporating sentiment scores exhibit reduced prediction errors. Our findings confirm the value of sentiment information in improving forecasting performance. These findings highlight the effectiveness and robustness of our CNN-LSTM model combined with sentiment analysis for price prediction in the salmon market.

On-line recognition of mixture control chart patterns using hybrid CNN and LSTM for auto-correlated processes

Statistical process control (SPC), designed to detect and identify process disturbances, is an effective quality control method for ensuring process stability. Owing to the growing automation of the industry, the manufacturing process can be autocorrelated. Engineer process control (EPC) is typically used to address autocorrelation. However, process disturbances can be offset by feedback compensation, making control chart patterns (CCPs) difficult to identify. Most studies on control chart pattern recognition (CCPR) techniques are based on a single abnormal control chart pattern (CCP). However, a mixture of CCPs can occur during real-world manufacturing processes. With the development of intelligent manufacturing systems, early detection of abnormal concurrent CCPs has become an important issue. In this study, a hybrid model combining a convolutional neural network (CNN) and long short-term memory (LSTM) in an online detection system was used to recognize the concurrent CCPR problem in SPC-EPC processes. The results showed that the average accuracy of the deep learning CNN-LSTM method was 99.83%, which was significantly better than that of the machine learning method. In addition, the running time of the CNN-LSTM model was shortened. In a comparison of online monitoring between the deep learning method and the machine learning method, the CNN-LSTM model for an online monitoring system identified abnormal concurrent patterns faster. Therefore, the proposed CNN-LSTM online monitoring scheme can be applied confidently and successfully to identify mixture CCPs in an SPC-EPC process.

HybridSecNet: In-Vehicle Security on Controller Area Networks Through a Hybrid Two-Step LSTM-CNN Model

HybridSecNet: In-Vehicle Security on Controller Area Networks Through a Hybrid Two-Step LSTM-CNN Model

Lateral spread prediction based on hybrid CNN-LSTM model for hot strip finishing mill

Lateral spread prediction based on hybrid CNN-LSTM model for hot strip finishing mill

Fault Diagnosis of Pumped Storage Units—A Novel Data-Model Hybrid-Driven Strategy

Pumped storage units serve as a crucial support for power systems to adapt to large-scale and high-proportion renewable energy sources by providing a stable and flexible energy supply. However, due to the coupling effects of electric power load demands and the complex multi-source factors within the water–mechanical–electrical system, the interrelationship between unit parameters becomes more intricate, posing significant threats to the operational reliability and health status of the units. The complexity of fault diagnosis is further aggravated by the intricate and varied nature of fault characteristics, as well as the challenges in signal extraction under conditions of strong electromagnetic interference and high noise levels. To address these issues, this paper proposes a novel data-model hybrid-driven strategy that analyzes vibration signals to achieve rapid and accurate fault diagnosis of the units. Firstly, the spectral kurtosis theory is employed to enhance the traditional empirical mode decomposition, achieving optimal decomposition and noise reduction effects for vibration signals. Secondly, the intrinsic mode functions (IMFs) obtained from the decomposition are reconstructed, and the entropy values of effective IMFs are calculated as fault feature vectors. Subsequently, the CNN-LSTM model is utilized for fault diagnosis. The effectiveness and feasibility of the proposed method are verified through actual operational data from pumped storage units in a specific region. Through analysis, the fault diagnosis accuracy of the method proposed in this paper can be maintained above 95%, demonstrating robustness in complex engineering environments and effectively ensuring the safe and stable operation of pumped storage units.

Aspect based sentiment analysis of Twitter mobile phone reviews using LSTM and Convolutional Neural Network

The proliferation of online shopping has led to a surge in product reviews, providing valuable information to consumers. However, the overwhelming volume and subjective nature of these reviews make it difficult to assess product performance accurately. We propose a machine learning-based system that extracts sentiment from online reviews to address this challenge. Our system effectively identifies positive, negative, and neutral sentiments and classifies sentiment for specific product aspects. By offering concise and clear information, our system empowers consumers to make informed purchasing decisions and assists manufacturers in improving their products. Our proposed LSTMCNN model, trained on a dataset of 62,563 reviews, achieved impressive results with an accuracy of 95.84%, precision of 95.6% and recall of 95.8%. This significantly outperforms existing models, demonstrating the effectiveness of our approach. Moving forward, we aim to further enhance the accuracy of our system, track sentiment changes over time, and develop personalized product recommendations. These advancements will continue to increase the value and utility of online reviews in e-commerce.

Experimental Study of Long Short-Term Memory and Transformer Models for Fall Detection on Smartwatches.

Falls are the second leading cause of unintentional injury deaths worldwide. While numerous wearable fall detection devices incorporating AI models have been developed, none of them are used successfully in a fall detection application running on commodity-based smartwatches in real time. The system misses some falls, and generates an annoying amount of False Positives for practical use. We have investigated and experimented with an LSTM model for fall detection on a smartwatch. Even though the LSTM model has high accuracy during offline testing, the good performance of offline LSTM models cannot be translated to the equivalence of real-time performance. Transformers, on the other hand, can learn long-sequence data and patterns intrinsic to the data due to their self-attention mechanism. This paper compares three variants of LSTM and two variants of Transformer models for learning fall patterns. We trained all models using fall and activity data from three datasets, and the real-time testing of the model was performed using the SmartFall App. Our findings showed that in the offline training, the CNN-LSTM model was better than the Transformer model for all the datasets. However, the Transformer is a preferable choice for deployment in real-time fall detection applications.

E-WebGuard: Enhanced neural architectures for precision web attack detection

Web applications have become a favored tool for organizations to disseminate vast amounts of information to the public. With the increasing adoption and inherent openness of these applications, there is an observed surge in web-based attacks exploited by adversaries. However, most of the web attack detection works are based on public datasets that are outdated or do not cover a sufficient quantity of web application attacks. Furthermore, most of them are binary detection (i.e., normal or attack) and there is little work on multi-class web attack detection. This highlights the crucial need for automated web attack detection models to bolster web security. In this study, a suite of integrated machine learning and deep learning models is designed to detect web attacks. Specifically, this study employs the Character-level Support Vector Machine (Char-SVM), Character-level Long Short-Term Memory (Char-LSTM), Convolutional Neural Network - SVM (CNN-SVM), and CNN-Bi-LSTM models to differentiate between standard HTTP requests and HTTP-based attacks in both the CSIC 2010 and SR-BH 2020 datasets. Note that the CSIC 2010 dataset involves binary classification, while the SR-BH 2020 dataset involves multi-class classification, specifically with 13 classes. Notably, the input data is first converted to the character level before being fed into any of the proposed model architectures. In the binary classification task, the Char-SVM model with a linear kernel outperforms other models, achieving an accuracy rate of 99.60%. The CNN-Bi-LSTM model closely follows with a 99.41% accuracy, surpassing the performance of the CNN-LSTM model presented in previous research. In the context of multi-class classification, the CNN-Bi-LSTM model demonstrates outstanding performance with a 99.63% accuracy rate. Furthermore, the multi-class classification models, namely Char-LSTM and CNN-Bi-LSTM, achieve validation accuracies above 98%, outperforming the two machine learning-based methods mentioned in the original research.

Integrating deep learning techniques for effective river water quality monitoring and management

Effective river water quality monitoring is essential for sustainable water resource management. In this study, we established a comprehensive monitoring system along the Kaveri River, capturing real-time data on multiple critical water quality parameters. The parameters collected encompassed water contamination levels, turbidity, pH measurements, temperature, and total dissolved solids (TDS), providing a holistic view of river water quality. The monitoring system was meticulously set up with strategically positioned sensors at various river locations, ensuring data collection at regular 5-min intervals. This data was then transmitted to a cloud-based web portal, facilitating storage and analysis. To assess water quality, we introduced a novel hybrid approach, combining Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM) networks. The proposed CNN-LSTM model achieved a validation accuracy of 98.40%, surpassing the performance of other state-of-the-art methods. Notably, the practical application of this system includes real-time alerts, promptly notifying stakeholders when water quality parameters exceed predefined thresholds. This feature aids in making informed decisions in water resource management. The study's contributions lie in its effective river water quality monitoring system, which encompassing various parameters, and its potential to positively impact environmental conservation efforts by providing a valuable tool for informed decision-making and timely interventions.

Navigating the emotional maze: Understanding Adolescent suicidal ideation using CNN-LSTM model

Teenage suicidal ideation is on the rise, which emphasizes how crucial it is to recognize and comprehend the variables that contribute to this problem. Convolutional neural networks (CNNs), which are complex machine learning models capable of analysing intricate relationships within a network, are one possible strategy for addressing this issue. In our study, we employed a CNN-LSTM hybrid model to explore the complex relationships between teen suicide ideation and various risk variables, including depression, anxiety, and social support by analysing a substantial dataset of mental health surveys, seeking patterns and risk factors associated with suicidal thoughts. Our objective was clear: identify adolescents prone to suicidal ideation. With 24 parameters and a sample size of 3075 subjects, our model achieved an impressive F1-score of 97.8%. These findings provide valuable insights which helps in developing effective preventive interventions to address adolescent suicidal ideation, finding out the important patterns and risk variables related to suicidal thoughts. The study results offer important direction for developing preventive interventions that successfully address adolescent suicidal ideation.

A Deep Learning-Based Hybrid CNN-LSTM Model for Location-Aware Web Service Recommendation

Advertising is the most crucial part of all social networking sites. The phenomenal rise of social media has resulted in a general increase in the availability of customer tastes and preferences, which is a positive development. This information may be used to improve the service that is offered to users as well as target advertisements for customers who already utilize the service. It is essential while delivering relevant advertisements to consumers, to take into account the geographic location of the consumers. Customers will be ecstatic if the offerings displayed to them are merely available in their immediate vicinity. As the user’s requirements will vary from place to place, location-based services are necessary for gathering this essential data. To get users to stop thinking about where they are and instead focus on an ad, location-based advertising (LBA) uses their mobile device’s GPS to pinpoint nearby businesses and provide useful information. Due to the increased two-way communication between the marketer and the user, mobile consumers’ privacy concerns and personalization issues are becoming more of a barrier. In this research, we developed a collaborative filtering-based hybrid CNN-LSTM model for recommending geographically relevant online services using deep neural networks. The proposed hybrid model is made using two neural networks, i.e., CNN and LSTM. Geographical information systems (GIS) are used to acquire initial location data to collect precise locational details. The proposed LBA for GIS is built in a Python simulation environment for evaluation. Hybrid CNN-LSTM recommendation performance beats existing location-aware service recommender systems in large simulations based on the WS dream dataset.

A Research of Fine-Tuning CNN-LSTM Model for Gold Price Prediction

Gold price and foreign exchange price prediction are usually regarded as serialized data with strong time correlation. It is precisely because of this feature that the performance of traditional convolutional neural networks is often not as good as that of recurrent neural networks. This paper fine-tunes the CNN-LSTM model in two ways. The first is to simplify the input data, convert the time series into binary data, and change the model task from predicting prices to predicting price fluctuations. The second is to add a dropout layer and use the dropout layer to improve the model's overfitting problem. The results demonstrate that the improved model outperforms the first model, with prediction accuracy rising from 54% to 57%. Furthermore, the refined model is not limited to processing the gold price series; it can also be applied to other time series, and when paired with other models, it could yield even better outcomes.

Deep Learning-Based Obesity Identification System for Young Adults Using Smartphone Inertial Measurements.

Obesity recognition in adolescents is a growing concern. This study presents a deep learning-based obesity identification framework that integrates smartphone inertial measurements with deep learning models to address this issue. Utilizing data from accelerometers, gyroscopes, and rotation vectors collected via a mobile health application, we analyzed gait patterns for obesity indicators. Our framework employs three deep learning models: convolutional neural networks (CNNs), long-short-term memory network (LSTM), and a hybrid CNN-LSTM model. Trained on data from 138 subjects, including both normal and obese individuals, and tested on an additional 35 subjects, the hybrid model achieved the highest accuracy of 97%, followed by the LSTM model at 96.31% and the CNN model at 95.81%. Despite the promising outcomes, the study has limitations, such as a small sample and the exclusion of individuals with distorted gait. In future work, we aim to develop more generalized models that accommodate a broader range of gait patterns, including those with medical conditions.

Prediction of Remaining Useful Life of Aero-engines Based on CNN-LSTM-Attention

Accurately predicting the remaining useful life (RUL) of aircraft engines is crucial for maintaining financial stability and aviation safety. To further enhance the prediction accuracy of aircraft engine RUL, a deep learning-based RUL prediction method is proposed. This method possesses the potential to strengthen the recognition of data features, thereby improving the prediction accuracy of the model. First, the input features are normalized and the CMAPSS (Commercial Modular Aero-Propulsion System Simulation) dataset is utilized to calculate the RUL for aircraft engines. After extracting attributes from the input data using a convolutional neural network (CNN), the extracted data are input into a long short-term memory (LSTM) network model, with the addition of attention mechanisms to predict the RUL of aircraft engines. Finally, the proposed aircraft engine model is evaluated and compared through ablation studies and comparative model experiments. The results indicate that the CNN-LSTM-Attention model exhibits superior prediction performance for datasets FD001, FD002, FD003, and FD004, with RMSEs of 15.977, 14.452, 13.907, and 16.637, respectively. Compared with CNN, LSTM, and CNN-LSTM models, the CNN-LSTM model demonstrates better prediction performance across datasets. In comparison with other models, this model achieves the highest prediction accuracy on the CMAPSS dataset, showcasing strong reliability and accuracy.