Convolutional Long Short-term Memory Research Articles

Using machine learning and satellite data to improve flood forecasting: the case of the Ouémé basin at the Bétérou outlet

ABSTRACT Flooding is a major concern for the scientific community, and it has been exacerbated by climate change. Accurate prediction of these extreme events is crucial for adequate preparedness. This study investigates the potential of advanced artificial intelligence (AI) techniques to enhance the accuracy of flood prediction and provide actionable insights for flood management. This study focuses on the African context, where data are scarce and the weak capacity of governments to react to floods makes populations vulnerable. It adopted advanced recurrent neural network architectures such as the long short-term memory (LSTM) and the convolutional long short-term memory (ConvLSTM) models, focusing on hydrological modeling innovation. The results indicated a high performance of these models in simulated runoff. The coefficient of determination (R2) and Nash–Sutcliff efficiency between observed and simulated runoff are approximately 0.96 and 0.95, respectively, for the ConvLSTM model and 0.95 and 0.95 for the LSTM model. This study also generated detailed maps of areas at risk of flooding. These maps represent a significant decision-making tool for flood management. This research confirms the effectiveness of deep learning in hydrology and proposes an innovative methodological framework for sustainable water resource management in the African context.

CMADRL: Cross-Modal Attention Based Deep Reinforcement Learning for Mobile Robot’s Obstacle Avoidance

Abstract The multimodal perception is very crucial for mobile robots to achieve safe autonomous
steering in complex and changing environments without colliding with obstacles. In spite of the fact that a rich environmental information can be provided by the multimodal data, how to integrate the complementary features from different sources effectively remains an open issue. In this paper, a simple yet effective multimodal deep reinforcement learning (DRL) method, CMADRL, by fusing two input modalities such as depth images and pseudo-LiDAR data generated by RGB-D cameras, is proposed to improve the performance of single sensor in environmental perception tasks. And the Convolutional LSTM Network (ConvLSTM) layers are also adopted to extract spatiotemporal features and capture temporal relationships in consecutive time steps, which allow the robot to predict dynamic changes of the environment. In addition, A crossmodal fusion module is designed to optimize the multimodal data fusion scheme, with dynamic weighting and merging of features from different modalities, allowing the agent to focus more reasonably on the current state and improving the accuracy and efficiency of the obstacle avoidance strategy. The experimental results show that the proposed approach achieved significant performance in cumulative rewards, convergence speed, and success rate. Additionally, tests in both simulated and real-world environments further verify that collisions are effectively avoided using only a low-cost RGB-D camera, demonstrating the method’s strong generalization capabilities.

A Hybrid CNN-BiLSTM-Based Approach for Roman Urdu Sentiment Analysis Using Enhanced Roman Urdu Corpus

The fast growth of the technology allowed for the easy and extensive dissemination of information on various topics, including business, marketing, news, and viewpoints on geopolitical situation. Although the number of studies on opinion mining is fast increasing, most of these studies focus on issues related to resource-rich languages. Resource-poor languages like Roman Urdu have been neglected for a long time, although having vast potential for research and the representation of almost 500 million people. This research in Roman Urdu is done using machine learning methods because of the nonavailability of the large and standard corpus. Due to research gaps in Roman Urdu sentiment analysis, there are few publicly available corpus for research purposes that are not large in size and not good in quality to get the promising results using Deep Learning (DL) methods. The key contributions of this research work are the enhancement of the existing Roman Urdu corpus and the use of hybrid Convolutional Neural Network-Bidirectional LSTM (CNN-BiLSTM) model on enhanced Roman Urdu corpus with fine tuning. For enhancement of the Roman Urdu corpus, the reviews are collected from multiple online sources such as politics, sports, entertainment, and food. Moreover, the enhanced corpus is annotated manually by two annotators A and B, following the guidelines given by researchers and statically validated by computing the Cohen’s Kappa score, which is moderate. The conflicted reviews are validated through another annotator C. Finally, the experiments are performed in binary class and multi-class using hybrid DL methods as well as the ML-based models. The experiments show the outperformance of hybrid CNN-BiLSTM model as compared to existing models of Recurrent Convolutional Neural Network (RCNN), RNN, LSTM, SVM, CRF, and Rule-based model as per the evaluation parameters such as with accuracy of 0.774 binary class and 0.721 multi-class on the enhanced Roman Urdu corpus.

A hybrid deep learning air pollution prediction approach based on neighborhood selection and spatio-temporal attention

Air pollution is a critical global environmental issue, further exacerbated by rapid industrialization and urbanization. Accurate prediction of air pollutant concentrations is essential for effective pollution prevention and control measures. The complex nature of pollutant data is influenced by fluctuating meteorological conditions, diverse pollution sources, and propagation processes, underscores the crucial importance of the spatial and temporal feature extraction for accurately predicting air pollutant concentrations. To address the challenges of data redundancy and diminished long-term prediction accuracy observed in previous studies, this paper presents an innovative approach to predict air pollutant concentrations leveraging advanced data analysis and deep learning methods. The proposed approach, termed KSC-ConvLSTM, integrates the k-nearest neighbors (KNN) algorithm, spatio-temporal attention (STA) mechanism, the residual block, and convolutional long short-term memory (ConvLSTM) neural network. The KNN algorithm adaptively selects highly correlated neighboring domains, while the residual block, enhanced with the STA mechanism, extracts spatial features from the input data. ConvLSTM further processes the output from STA-ConvNet to capture high-dimensional temporal and spatial features. The effectiveness of the KSC-ConvLSTM approach was validated through predictions of PM2.5 concentrations in Beijing and its surrounding urban agglomeration. The experimental results indicate that the KSC-ConvLSTM approach outperforms benchmark approaches in single-step, multi-step, and trend prediction. It demonstrates superior fitting accuracy and predictive performance. Quantitatively, the proposed KSC-ConvLSTM approach reduces the root mean square error (RMSE) by 4.216–8.458 for prediction averages of 1–12 h of PM2.5 in Beijing, compared with the benchmark approach. The findings show that the KSC-ConvLSTM approach shows considerable potential for predicting, preventing, and controlling air pollution.

Enhancing Environmental Monitoring of Harmful algal blooms with ConvLSTM Image Prediction

Abstract This study employs advanced predictive modeling to forecast Harmful algal blooms (HABs) concentrations in South Korea, adopt Convolutional Long Short-Term Memory (ConvLSTM) models. The research integrates 3D universal kriging for spatial interpolation, leveraging latitude, longitude, and elevation data to create a comprehensive dataset. The ConvLSTM model, which integrates convolutional neural networks (CNNs) for spatial feature extraction and long short-term memory (LSTM) units for capturing temporal dependencies, was utilized on spatiotemporal data. The results demonstrate effectiveness in predicting HABs concentrations, offering valuable insights for environmental monitoring and management. This approach supports the development of sustainable strategies in line with environmental, social, and governance (ESG) principles, enhancing our ability to mitigate the adverse impacts of HABs on ecosystems and public health.

A novel hybrid architecture for video frame prediction: combining convolutional LSTM and 3D CNN

A novel hybrid architecture for video frame prediction: combining convolutional LSTM and 3D CNN

Short-term urban traffic forecasting in smart cities: a dynamic diffusion spatial-temporal graph convolutional network

Short-term traffic forecasting is an important part of intelligent transportation systems. Accurately predicting short-term traffic trends can avoid traffic congestion and plan travel routes, which is of great significance to urban management and traffic scheduling. The difficulty of short-term urban traffic forecasting is that the traffic flow is random and will be dynamically changed by the traffic conditions of nearby nodes. In order to solve this problem, this paper proposes a model based on Dynamic Diffusion Spatial-Temporal Graph Convolutional Network. It first combines the dynamic generation matrix and the static distance matrix to grasp real-time traffic conditions, and then introduces the diffusion random walk strategy to capture the correlation of spatial nodes. Finally, the convolutional LSTM module is used to mine the spatiotemporal dependence of traffic data to improve the accuracy of traffic prediction. Compared to several baseline models, the experimental results show that the model is 7% better than other models on several metrics and demonstrates the necessity of the module through ablation experiments.

CONVOLUTIONAL LSTM FOR EGG PRICES FORECASTING IN INDONESIA’S HIGH STUNTING PREVALENCE PROVINCE

The Sustainable Development Goals (SDGs) are a series of 17 goals fixed by the United Nations and adopted by 193 countries in 2015, including Indonesia. By 2030, to end all forms of malnutrition, targeting on stunting and wasting in children under 5 years of age is one of the targets from Goals Number 2. One affordable source of protein and nutrition used as a solution to overcome malnutrition problems such as stunting is eggs. Egg price modeling was carried out to see the affordability of prices for the community. Weekly dataset of egg price in NTT Province from 2018 to 2023 used to modeling with Convolutional LSTM. The Convolutional LSTM components are Adam optimizer, ReLU activation function, Huber loss function, with batch size and neurons of 32. The MAPE value obtained from the model is relatively small, with MAPE for training, validation, and testing of 1.97%, 1%, and 1.19% respectively. The results of egg price forecasting for December 11, 2023, to January 8, 2024, show that egg prices tend to continue to decline per week. Thus, a decrease in egg prices can be a good thing in providing more affordable nutrition for the community.

AI-Assisted Forecasting of a Mitigated Multiple Steam Generator Tube Rupture Scenario in a Typical Nuclear Power Plant

This study is focused on developing a machine learning (ML) meta-model to predict the progression of a multiple steam generator tube rupture (MSGTR) accident in the APR1400 reactor. The accident was simulated using the thermal–hydraulic code RELAP5/SCDAPSIM/MOD3.4. The model incorporates a mitigation strategy executed through operator interventions. Following this, uncertainty quantification employing the Best Estimate Plus Uncertainty (BEPU) methodology was undertaken by coupling RELAP5/SCDAPSIM/MOD3.4 with the statistical software, DAKOTA 6.14.0. The analysis concentrated on critical safety parameters, including Reactor Coolant System (RCS) pressure and temperature, as well as reactor vessel upper head (RVUH) void fraction. These simulations generated a comprehensive dataset, which served as the foundation for training three ML architectures: Gated Recurrent Unit (GRU), Long Short-Term Memory (LSTM), and Convolutional LSTM (CNN+LSTM). Among these models, the CNN+LSTM hybrid configuration demonstrated superior performance, excelling in both predictive accuracy and computational efficiency. To bolster the model’s transparency and interpretability, Integrated Gradients (IGs)—an advanced Explainable AI (XAI) technique—was applied, elucidating the contribution of input features to the model’s predictions and enhancing its trustworthiness.

Predicting mental workload of using exoskeletons for construction work: a deep learning approach

Exoskeletons are gaining attention as a potential solution for addressing back injury in the construction industry. However, using active back-support exoskeletons in construction can trigger unintended consequences which could increase the mental workload of workers. Prolonged increase in mental workload could impact workers’ wellbeing and productivity. Predicting mental workload during exoskeleton use could inform strategies to mitigate the triggers. This study investigates two machine-learning frameworks for predicting mental workload using an active back-support exoskeleton for construction work. Laboratory experiments were conducted wherein electroencephalography (EEG) data was collected from participants wearing an active back-support exoskeleton to perform flooring tasks. The EEG data underwent preprocessing, including band filtering, notch filtering, and independent component analysis, to remove artifacts and ensure data quality. A regression-based Long Short-Term Memory (LSTM) network and a hybrid model of convolutional neural network and LSTM were trained to forecast future time steps of the processed EEG data. The performance of the networks was evaluated using root mean square error and r-squared. An average root mean square error of 0.162 and r-squared of 0.939 indicate that the LSTM network has a better predictive power across all the EEG channels. Results of the comparison between the actual and predicted mental workload also show that about 75% of the variance in the actual mental workload is captured in the predicted mental workload. This study enhances understanding of the unintended consequences of using exoskeletons in construction work. The results highlight the effectiveness of various convolutional neural network methods in identifying key EEG data features, offering guidance for algorithm selection in future applications. Additionally, the study identifies the most suitable brain channels for assessing mental workload during exoskeleton use, aiding the development of EEG devices that optimize cost-effectiveness, explanatory power, and minimal channels. This study provides valuable insights for stakeholders to understand the impact of mental workload while using exoskeletons and discovering opportunities for mitigation.

Brain Glial Cell Tumor Classification through Ensemble Deep Learning with APCGAN Augmentation

Classification of brain tumor plays a vital role in medical imaging for accurate diagnosis, treatment, and monitoring. Deep learning approaches have gained significant traction in this industry because of their ability to extract relevant features from medical images. The research suggests employing an ensemble classifier with a weighted voting mechanism to categorize glial cell brain malignancies such as Astrocytoma, Glioblastoma multiforme, Oligodendroglioma, and Ependymoma. The proposed ensemble technique employs three main classifiers: Convolutional Neural Network (CNN), Deep Convolutional Long Short Term Memory (C-LSTM), and Deep Convolutional Neural Network + Conditional Random Fields (DCNN+CRF). Deep learning algorithms require a huge amount of input data to avoid overfitting. The Adaptive Progressive Convolutional Generative Adversarial Networks (APCGANs) are used to produce realistic artificial images to efficiently train the proposed methodology. Overall, the proposed ensemble method with weighted voting strategy consistently outperforms the other tested algorithms (CNN, C-LSTM, and DCNN+CRF). Ensemble method attained an accuracy of 99.4 %, recall - 99.1%, precision- 98.0%, and F1-score of 99.2%. Ensemble method consistently demonstrates superior performance in accurately classifying brain tumors, making it a promising algorithm for brain tumor analysis tasks.

Cardiac arrhythmia classification of imbalanced data using convolutional autoencoder and LSTM techniques

Cardiac arrhythmia classification of imbalanced data using convolutional autoencoder and LSTM techniques

Short-term load forecasting for distribution stations based on convolutional LSTM network considering attention mechanism

Short-term load forecasting for distribution stations based on convolutional LSTM network considering attention mechanism

Blending Enterprise Resource Planning on Supply Chain Management in Aerospace Sector in India and Analysis using Multi-scale Adaptive Dilated Convolutional LSTM

Blending Enterprise Resource Planning on Supply Chain Management in Aerospace Sector in India and Analysis using Multi-scale Adaptive Dilated Convolutional LSTM

Mapping of high-resolution daily particulate matter (PM2.5) concentration at the city level through a machine learning-based downscaling approach.

PM2.5 pollution is a major global concern, especially in Vietnam, due to its harmful effects on health and the environment. Monitoring local PM2.5 levels is crucial for assessing air quality. However, Vietnam's state-of-the-art (SOTA) dataset with a 3km resolution needs to be revised to depict spatial variation in smaller regions accurately. In this research, we investigated machine learning-based downscaling methods to improve the spatial resolution and quality of Vietnam's existing 3km PM2.5 products using different approaches: traditional machine learning models (random forest, XGBoost, Catboost, support vector regression (SVR), mixed effect model (MEM)) and deep learning models (long short-term memory (LSTM), convolutional neural network (CNN), convolutional LSTM (ConvLSTM)). Overall, the CatBoost 2-day lag model exhibited superior performance. In terms of modeling, integrating temporal factors into tree-based models can enhance predictive accuracy. Furthermore, when faced with small datasets, traditional machine learning models demonstrate superior performance over complex deep learning approaches. The validation of machine and deep learning models based on their PM2.5 generated maps is requested because these models can obtain very high results for model evaluation but are unrealistic for application. In this study, compared to the state-of-the-art (SOTA) PM2.5 maps in Vietnam and the SOTA global maps, the proposed CatBoost 2-day lag model's maps showed a 57% increase in the correlation coefficient (Pearson R), as well as 42-73%, 28-75%, and 39-75% reductions in root mean squared error (RMSE), mean relative error (MRE), and mean absolute error (MAE), respectively. Additionally, the daily, monthly, and year-average maps generated by the Catboost 2-day lag model effectively capture the spatial distribution and seasonal variations of PM2.5 in Ho Chi Minh City. These findings indicate a substantial enhancement in the accuracy and reliability of downscaled PM2.5 maps.

Application of deep learning in cloud cover prediction using geostationary satellite images

ABSTRACT Predicting cloud cover is essential in fields, such as agriculture, climatology, and meteorology, where accurate weather forecasting can significantly impact decision-making. Traditional methods for cloud cover prediction encounter significant limitations in capturing complete spatial and temporal cloud dynamics. To address the problem, this study employs high-resolution data from the new-generation geostationary satellite Geostationary Korea Multi-Purpose Satellite 2A (GEO-KOMPSAT-2A; GK2A) and enables more accurate and timely predictions when combined with advanced deep learning techniques. We explore the effectiveness of advanced deep learning techniques – specifically 3D Convolutional Neural Networks, Long Short-Term Memory networks, and Convolutional Long Short-Term Memory (ConvLSTM) – using GK2A cloud detection data, which provides updates every 10 minutes at 2 km spatial resolution. Our model utilizes training sequences of four past hourly images to predict cloud cover up to 4 hours ahead. For improved computational efficiency, each image is divided into four patches during training. Notably, this research incorporates a dynamic learning model, continuously updating with the most recent data, in contrast with static models which do not update their parameters with new data once trained. Results show that ConvLSTM tends to exhibit stable and relatively higher performance across prediction times compared to the other models in August 2021. While transformer models, such as Video Swin Transformer and TimeSformer, showed strong training performance, they struggled with overfitting, particularly on smaller datasets. In contrast, ConvLSTM demonstrated better generalization to test data, highlighting its suitability for tasks with limited training data and simpler structures. Year-long validation demonstrates the robustness of the ConvLSTM model, which consistently outperforms the other models in all major metrics, including a precision of 0.79, recall of 0.80, F1-score of 0.80 (which balances precision and recall), and accuracy of 0.78 throughout 2021. However, results show that the model’s performance in terms of F1-score, recall, and precision is positively correlated with cloud fraction, with a slight tendency for higher accuracy during the summer compared to winter, indicating sensitivity to seasonal cloud cover variations.

Enhancing unmanned aerial vehicle and smart grid communication security using a ConvLSTM model for intrusion detection

The emergence of small-drone technology has revolutionized the way we use drones. Small drones leverage the Internet of Things (IoT) to deliver location-based navigation services, making them versatile tools for various applications. Unmanned aerial vehicle (UAV) communication networks and smart grid communication protocols share several similarities, particularly in terms of their architecture, the nature of the data they handle, and the security challenges they face. To ensure the safe, secure, and reliable operation of both, it is imperative to establish a secure and dependable network infrastructure and to develop and implement robust security and privacy mechanisms tailored to the specific needs of this domain. The research evaluates the performance of deep learning models, including convolutional neural networks (CNN), long short-term memory (LSTM), CNN-LSTM, and convolutional long short-term memory (ConvLSTM), in detecting intrusions within UAV communication networks. The study utilizes five diverse and realistic datasets, namely, KDD Cup-99, NSL-KDD, WSN-DS, CICIDS 2017, and Drone, to simulate real-world intrusion scenarios. Notably, the ConvLSTM model consistently achieves an accuracy of 99.99%, showcasing its potential in securing UAVs from cyber threats. By demonstrating its superior performance, this work highlights the importance of tailored security mechanisms in safeguarding UAV technology against evolving cyber threats. Ultimately, this research contributes to the growing body of knowledge on UAV security, emphasizing the necessity of high-quality datasets and advanced models in ensuring the safe, secure, and reliable operation of UAV systems across various industries.

Self-Attention (SA)-ConvLSTM Encoder–Decoder Structure-Based Video Prediction for Dynamic Motion Estimation

Video prediction, which is the task of predicting future video frames based on past observations, remains a challenging problem because of the complexity and high dimensionality of spatiotemporal dynamics. To address the problems associated with spatiotemporal prediction, which is an important decision-making tool in various fields, several deep learning models have been proposed. Convolutional long short-term memory (ConvLSTM) can capture space and time simultaneously and has shown excellent performance in various applications, such as image and video prediction, object detection, and semantic segmentation. However, ConvLSTM has limitations in capturing long-term temporal dependencies. To solve this problem, this study proposes an encoder–decoder structure using self-attention ConvLSTM (SA-ConvLSTM), which retains the advantages of ConvLSTM and effectively captures the long-range dependencies through the self-attention mechanism. The effectiveness of the encoder–decoder structure using SA-ConvLSTM was validated through experiments on the MovingMNIST, KTH dataset.

Deep learning-based recognition of construction activities in real construction site environment

PurposeToday’s technological advancements have had a significant impact on the construction industry. Managing and controlling complex construction projects has been made significantly easier using technological tools. One such advancement is the automatic identification of workers’ activities. This study aims to classify construction worker activities by analyzing real-time motion data collected from sensors.Design/methodology/approachIn accordance with our specific goals, we utilized advanced deep-learning methodologies such as deep neural networks, convolutional neural network, long short-term memory and convolutional long short-term memory to analyze the data thoroughly. This involved experimenting with various window sizes and overlap ratios to determine the optimal combination that would result in the most accurate predictions.FindingsBased on the analysis results, the convolutional long short-term memory (ConvLSTM) deep learning model with a window size of 4.8 s and an overlap rate of 75% was found to be the most accurate prediction model. This model correctly predicted 98.64% of the basic construction worker activities in a real construction site environment.Originality/valuePrevious studies have mainly been conducted in laboratory environments and have focused on basic construction activities such as lifting, moving, sawing and hammering. However, this study collected data from real workers in a real construction site environment. Various deep learning models were employed to determine the most accurate one. Additionally, several options were tested to determine the optimal window size and overlap ratio during the data segmentation phase, aiming to select the most suitable ones for preparing the data for the model.

Vision-Based Prediction of Flashover Using Transformers and Convolutional Long Short-Term Memory Model

The prediction of fire growth is crucial for effective firefighting and rescue operations. Recent advancements in vision-based techniques using RGB vision and infrared (IR) thermal imaging data, coupled with artificial intelligence and deep learning techniques, have shown promising solutions to be applied in the detection of fire and the prediction of its behavior. This study introduces the use of Convolutional Long Short-term Memory (ConvLSTM) network models for predicting room fire growth by analyzing spatiotemporal IR thermal imaging data acquired from full-scale room fire tests. Our findings revealed that SwinLSTM, an enhanced version of ConvLSTM combined with transformers (a deep learning architecture based on a new mechanism called multi-head attention) for computer vision purposes, can be used for the prediction of room fire flashover occurrence. Notably, transformer-based ConvLSTM deep learning models, such as SwinLSTM, demonstrate superior prediction capability, which suggests a new vision-based smart solution for future fire growth prediction tasks. The main focus of this work is to perform a feasibility study on the use of a pure vision-based deep learning model for analysis of future video data to anticipate behavior of fire growth in room fire incidents.