Convolutional Long Short-term Memory Research Articles

Portfolio management based on a reinforcement learning framework

AbstractPortfolio management is crucial for investors. We propose a dynamic portfolio management framework based on reinforcement learning using the proximal policy optimization algorithm. The two‐part framework includes a feature extraction network and a full connected network. First, the majority of the previous research on portfolio management based on reinforcement learning has been dedicated to discrete action spaces. We propose a potential solution to the problem of a continuous action space with a constraint (i.e., the sum of the portfolio weights is equal to 1). Second, we explore different feature extraction networks (i.e., convolutional neural network [CNN], long short‐term memory [LSTM] network, and convolutional LSTM network) combined with our system, and we conduct extensive experiments on the six kinds of assets, including 16 features. The empirical results show that the CNN performs best in the test set. Last, we discuss the effect of the trading frequency on our trading system and find that the monthly trading frequency has a higher Sharpe ratio in the test set than other trading frequencies.

Enhancing Oil–Water Flow Prediction in Heterogeneous Porous Media Using Machine Learning

The rapid and accurate forecasting of two-phase flow in porous media is a critical challenge in oil field development, exerting a substantial impact on optimization and decision-making processes. Although the Convolutional Long Short-Term Memory (ConvLSTM) network effectively captures spatiotemporal dynamics, its generalization in predicting complex engineering problems remains limited. Similarly, although the Fourier Neural Operator (FNO) demonstrates adeptness at learning operators for solving partial differential equations (PDEs), it struggles with three-dimensional, long-term prediction. In response to these limitations, we introduce an innovative hybrid model, the Convolutional Long Short-Term Memory-Fourier Neural Operator (CL-FNO), specifically designed for the long-term prediction of three-dimensional two-phase flows. This model integrates a 3D convolutional encoder–decoder structure to extract and generate hierarchical spatial features of the flow fields. It incorporates physical constraints to enhance the model’s forecasts with robustness through the infusion of prior knowledge. Additionally, a temporal function, constructed using gated memory-forgetting mechanisms, augments the model’s capacity to analyze time series data. The efficacy and practicality of the CL-FNO model are validated using a synthetic three-dimensional case study and application to an actual reservoir model.

Operational Forecasting of Global Ionospheric TEC Maps 1-, 2-, and 3-Day in Advance by ConvLSTM Model

In this paper, we propose a global ionospheric total electron content (TEC) maps (GIM) prediction model based on deep learning methods that is both straightforward and practical, meeting the requirements of various applications. The proposed model utilizes an encoder-decoder structure with a Convolution Long Short-Term Memory (ConvLSTM) network and has a spatial resolution of 5° longitude and 2.5° latitude, with a time resolution of 1 h. We utilized the Center for Orbit Determination in Europe (CODE) GIM dataset for 18 years from 2002 to 2019, without requiring any other external input parameters, to train the ConvLSTM models for forecasting GIM 1, 2, and 3 days in advance. Using the CODE GIM data from 1 January 2020 to 31 December 2023 as the test dataset, the performance evaluation results show that the average root mean square errors (RMSE) for 1, 2 and 3 days of forecasts are 2.81 TECU, 3.16 TECU, and 3.41 TECU, respectively. These results show improved performance compared to the IRI-Plas model and CODE’s 1-day forecast product c1pg, and comparable to CODE’s 2-day forecast c2pg. The model’s predictions get worse as the intensity of the storm increases, and the prediction error of the model increases with the lead time.

Short-term multi-step-ahead sector-based traffic flow prediction based on the attention-enhanced graph convolutional LSTM network (AGC-LSTM)

AbstractAccurate sector-based air traffic flow predictions are essential for ensuring the safety and efficiency of the air traffic management (ATM) system. However, due to the inherent spatial and temporal dependencies of air traffic flow, it is still a challenging problem. To solve this problem, some methods are proposed considering the relationship between sectors, while the complicated spatiotemporal dynamics and interdependencies between traffic flow of route segments related to the sector are not taken into account. To address this challenge, the attention-enhanced graph convolutional long short-term memory network (AGC-LSTM) model is applied to improve the short-term sector-based traffic flow prediction, in which spatial structures of route segments related to the sector are considered for the first time. Specifically, the graph convolutional networks (GCN)-LSTM network model was employed to capture spatiotemporal dependencies of the flight data, and the attention mechanism is designed to concentrate on the informative features from key nodes at each layer of the AGC-LSTM model. The proposed model is evaluated through a case study of the typical enroute sector in the central–southern region of China. The prediction results show that MAE reduces by 14.4% compared to the best performing GCN-LSTM model among the other five models. Furthermore, the study involves comparative analyses to assess the influence of route segment range, input and output sequence lengths, and time granularities on prediction performance. This study helps air traffic managers predict flight situations more accurately and avoid implementing overly conservative or excessively aggressive flow management measures for the sectors.

Unsupervised detecting anomalies in multivariate time series by Robust Convolutional LSTM Encoder–Decoder (RCLED)

Monitoring modern industrial systems generates a large amount of multivariate time series data. One of the critical tasks of monitoring these systems is anomaly detection. The auto-encoder has emerged as a promising solution for detecting anomalies in systems lacking explicit anomaly data in their historical records; however, its performance can be sensitive to noisy data. This work aims at developing a robust anomaly detection model based on the autoencoder for multivariate time series. The autoencoder architecture consists of convolution, long short-term memory, and self-attention layers for better extraction of complex features from multivariate time series. In addition, a novel training technique, inherited from the idea of robust principal component analysis, was developed to efficiently train the proposed model when the input data is affected by noise. To evaluate the model’s performance, we tested it on two data sets: synthetic, real-industrial, and three public data. We also compare the performance of our model to that of the other state-of-the-art models. The results show that the proposed model outperforms all the latest models, especially when the noise level is considerable.

Efficient river hydrodynamics modelling in realistic river systems using a Fourier neural operator-based network

Effective river management relies heavily on the accurate simulation of river flow dynamics to develop scenario-based strategies and inform decision making. Deep learning (DL) models, as alternatives to conventional simulation techniques, offer a compelling blend of precision and efficiency. This study introduces FNOCL (Fourier Neural Operator with ConvLSTM units), a novel DL neural network for modelling unsteady natural river flows. The network is adept at handling the complexities of spatial and temporal variations. FNOCL builds upon the Fourier neural operator (FNO) by incorporating convolutional long short-term memory (ConvLSTM) layers, combining robust performance with high proficiency in terms of capturing spatiotemporal patterns. Notably, FNOCL accommodates complex river cross-section profiles, thus enabling the accurate modelling of flow dynamics within intricate river morphologies. Our study applies FNOCL to two distinct river networks with varying morphology complexity levels. With reference to the HEC-RAS model, FNOCL showcases its ability to attain high accuracy in model prediction tasks and superior generalization with regard to addressing the uncertainty associated with inflow variations and cross-sectional changes. In comparative analyses with benchmark models (i.e., FNO and ConvLSTM), FNOCL performs most efficiently during the training process and obtains the most accurate water level and river flow predictions in the spatiotemporal domain. Furthermore, FNOCL exhibits a substantial speedup over HEC-RAS, notably reaching up to 50 times in a complex river network. These results highlight FNOCL as an efficient surrogate for conventional simulators, facilitating precise analyses across various scenarios. It provides efficient computations for system responses within complex river flow networks, thereby enhancing applications in river management, such as flood risk assessment, flood control, and the planning and design of hydraulic control structures.

ENFformer: Long-short term representation of electric network frequency for digital audio tampering detection

Detecting digital audio tampering is essential for ensuring judicial fairness and societal security. Traditional methods, primarily based on Electric Network Frequency (ENF), have been limited by their reliance on singular, static features, which overlook critical temporal dynamics inherent in ENF data. This results in suboptimal detection accuracy. Addressing these limitations, this paper introduces a novel transformer model, ENFformer, designed for the detection of digital audio tampering by leveraging both short and long-term temporal features of ENF data. Initially, the ENFformer model extracts traditional zero-order and first-order phase features using discrete Fourier transforms (DFT0 and DFT1), along with frequency features obtained through the Hilbert transform. These features are processed using a frame-based algorithm to develop their temporal counterparts. To enhance feature extraction, the model employs a two-layer one-dimensional Convolutional Long Short-Term Memory (ConvLSTM) network to assimilate short-term temporal features, followed by a Bidirectional Long Short-Term Memory (BiLSTM) network for long-term feature integration. A branch attention mechanism then synergizes these long-term features, which are further refined by a transformer module for accurate tampered audio identification. Our empirical evaluations on the Carioca and ENF-EDIT1 databases demonstrate that ENFformer achieves detection accuracies of 97.33% and 93.50% respectively, surpassing existing state-of-the-art methods. These results confirm the effectiveness of our approach, which significantly advances the field of digital audio tampering detection by incorporating a comprehensive analysis of temporal information in ENF features. The source code of this study is publicly available at https://github.com/CCNUZFW/ENFformer.

Convolutional LSTM–LSTM model for predicting the daily number of influenza patients in South Korea using satellite images

ObjectivesInfluenza affects a considerable proportion of the global population each year, and meteorological conditions may have a significant impact on its transmission. In this study, we aimed to develop a prediction model for the number of influenza patients at the national level using satellite images and provide a basis for predicting influenza through satellite image data. Study designWe developed an influenza incidence prediction model using satellite images and influenza patient data. MethodsWe collected satellite images and daily influenza patient data from July 2014 to June 2019 and developed a convolutional long short-term memory (LSTM)–LSTM neural network model. The model with the lowest average of mean absolute error (MAE) was selected. ResultsThe final model showed a high correlation between the predicted and actual number of influenza patients, with an average MAE of 5.9010 per million population. The model performed best with a 2-week time sequence. ConclusionsWe developed a national-level prediction model using satellite images to predict influenza incidence. The model offers the advantage of nationwide analysis. These results may reduce the burden of influenza by enabling timely public health interventions.

ConvLSTM-based Sound Source Localization in a manufacturing workplace

In this paper, Sound Source Localization (SSL) is explored as an approach to localize both human operators and machines emitting sound signals in a manufacturing workplace. In particular, a comprehensive analysis of the source localization ability of a state-of-the-art deep learning architecture in environments of increasing complexity is presented. Scenarios including single, dual, and multiple sound sources, in the form of both human and Computerized Numerical Control (CNC) machines, are investigated, as well as configurations with a mix of stationary and moving sources. Our work contributes to the extant literature by enriching previous research findings primarily devoted to single stationary sources. Furthermore, by focusing on the simultaneous and centralized detection of sources of different nature and type, it diverges from traditional SSL studies in manufacturing, which emphasize the localization of humans by robots in human–robot interaction, and presents a localization approach which enables a broader control over the workspace. For the localization task, a Convolutional LSTM architecture able to capture both spatial and temporal sound characteristics is also proposed, with each source assigned a dedicated model.Extensive experiments were carried out for each scenario in a simulated environment, where different levels of noise were also applied. The results showed the remarkable accuracy and robustness of the deep learning models when it comes to localizing single and dual stationary sources, as well as single moving sources. For multiple stationary and moving sources a general decline in the detection performance was observed, alongside a heightened sensitivity to noise.

Daily Streamflow Forecasting Using Networks of Real-Time Monitoring Stations and Hybrid Machine Learning Methods

Considering the increased risk of urban flooding and drought due to global climate change and rapid urbanization, the imperative for more accurate methods for streamflow forecasting has intensified. This study introduces a pioneering approach leveraging the available network of real-time monitoring stations and advanced machine learning algorithms that can accurately simulate spatial–temporal problems. The Spatio-Temporal Attention Gated Recurrent Unit (STA-GRU) model is renowned for its computational efficacy in forecasting streamflow events with a forecast horizon of 7 days. The novel integration of the groundwater level, precipitation, and river discharge as predictive variables offers a holistic view of the hydrological cycle, enhancing the model’s accuracy. Our findings reveal that for a 7-day forecasting period, the STA-GRU model demonstrates superior performance, with a notable improvement in mean absolute percentage error (MAPE) values and R-square (R2) alongside reductions in the root mean squared error (RMSE) and mean absolute error (MAE) metrics, underscoring the model’s generalizability and reliability. Comparative analysis with seven conventional deep learning models, including the Long Short-Term Memory (LSTM), the Convolutional Neural Network LSTM (CNNLSTM), the Convolutional LSTM (ConvLSTM), the Spatio-Temporal Attention LSTM (STA-LSTM), the Gated Recurrent Unit (GRU), the Convolutional Neural Network GRU (CNNGRU), and the STA-GRU, confirms the superior predictive power of the STA-LSTM and STA-GRU models when faced with long-term prediction. This research marks a significant shift towards an integrated network of real-time monitoring stations with advanced deep-learning algorithms for streamflow forecasting, emphasizing the importance of spatially and temporally encompassing streamflow variability within an urban watershed’s stream network.

How good are different machine and deep learning models in forecasting the future price of metals? Full sample versus sub-sample

This study aims to forecast metal futures in commodity markets, including gold, silver, copper, platinum, palladium, and aluminium, using different machine and deep learning models. Prevalent models such as Stacked Long-Short Term Memory, Convolutional LSTM, Bidirectional LSTM, Support Vector Regressor, Extreme Gradient Boosting, and Gated Recurrent Unit are utilized. The model performance is assessed by multiple factors such as Root Mean Squared Error, Mean Absolute Error, and Mean Absolute Percentage Error. The study stands out by considering multiple metal commodity futures simultaneously, incorporating both Machine Learning and Deep Learning models, and conducting two sets of experiments with a full sample and subsample analysis. In addition, it uses different inputs of 30- and 60-days periods for robustness checks. Mean Absolute Percentage Error values suggest that different machine and deep learning models are efficient on prediction the future metal prices. However, the model performance varies significantly with the influence of metal choice, sample period, and inputs on prediction performance. Therefore, it suggests that constructing a theory based on machine and deep learning models is challenging.

СУПУТНИКОВИЙ МОНІТОРИНГУ ОЦІНКИ СТАНУ ЗАБРУДНЕННЯ ВОДНИХ ОБ’ЄКТІВ НА ОСНОВІ МАШИННОГО НАВЧАННЯ

Satellite monitoring is a reliable tool for obtaining operational information about the state of water bodies and their changes with a certain periodicity, thanks to the use of a large set of heterogeneous data. And the availability of archival data becomes a significant advantage, opening up the possibility of comparing the obtained results and analyzing factors affecting the state of water bodies and the environment. The paper describes the main methods and datasets that can be used to apply remote monitoring methods based on machine learning. The use of medium spatial resolution data is proposed as the main data set. Examples of the application of the remote sensing approach to the assessment of water pollution in large areas are analyzed. Possible machine learning methods for solving an applied problem are presented, the new artificial intelligence (AI) approach of convolutional long-short-term memory (ConvLSTM) model significantly outperforms classical methods.

Convolutional Long Short-Term Memory (ConvLSTM)-Based Prediction of Voltage Stability in a Microgrid

The maintenance of an uninterrupted electricity supply to meet demand is of paramount importance for maintaining the stable operation of an electrical power system. Machine learning and deep learning play a crucial role in maintaining that stable operation. These algorithms have the ability to acquire knowledge from past data, enabling them to efficiently identify and forecast potential scenarios of instability in the future. This work presents a hybrid convolutional long short-term memory (ConvLSTM) technique for training and predicting nodal voltage stability in an IEEE 14-bus microgrid. Analysis of the findings shows that the suggested ConvLSTM model exhibits the highest level of precision, reaching a value of 97.65%. Furthermore, the ConvLSTM model has been shown to perform better compared to alternative machine learning and deep learning models such as convolutional neural networks, k-nearest neighbors, and support vector machine models, specifically in terms of accurately forecasting voltage stability. The IEEE 14-bus system tests indicate that the suggested method can quickly and accurately determine the stability status of the system. The comparative analysis obtained the results and further justified the efficiency and voltage stability of the proposed model.

A Prolonged Artificial Nighttime-light Dataset of China (1984-2020)

Nighttime light remote sensing has been an increasingly important proxy for human activities. Despite an urgent need for long-term products and pilot explorations in synthesizing them, the publicly available long-term products are limited. A Night-Time Light convolutional LSTM network is proposed and applied the network to produce a 1-km annual Prolonged Artificial Nighttime-light DAtaset of China (PANDA-China) from 1984 to 2020. Assessments between modeled and original images show that on average the RMSE reaches 0.73, the coefficient of determination (R2) reaches 0.95, and the linear slope is 0.99 at the pixel level, indicating a high confidence in the quality of generated data products. Quantitative and visual comparisons witness PANDA-China’s superiority against other NTL datasets in its significantly longer NTL dynamics, higher temporal consistency, and better correlations with socioeconomics (built-up areas, gross domestic product, population) characterizing the most relevant indicator in different development phases. The PANDA-China product provides an unprecedented opportunity to trace nighttime light dynamics in the past four decades.

MEBAL2 database and benchmark: Image-based multispectral eyeblink detection

This work introduces a new multispectral database and framework to train and evaluate eyeblink detection in RGB and Near-Infrared (NIR). Our contributed dataset (mEBAL2, multimodal EyeBlink and Attention Level estimation, Version 2) is the largest existing eyeblink database, representing a great opportunity to improve data-driven multispectral approaches for blink detection and related applications (e.g., attention level estimation). mEBAL2 includes 21,100 image sequences from 180 different students (more than 2 million labeled images in total) while conducting a number of e-learning tasks of varying difficulty or taking a real course on HTML initiation through the edX MOOC platform. mEBAL2 uses multiple sensors, including two Near-Infrared (NIR) and one RGB camera to capture facial gestures during the execution of the tasks, as well as an Electroencephalogram (EEG) band to get the cognitive activity of the user and blinking events. Furthermore, this work proposes 3 data-driven approaches as benchmarks for blink detection on mEBAL2, where the architecture based on Convolutional Long Short-Term Memory (ConvLSTM) achieved performances of up to 99%. The experiments explored whether combining RGB and NIR spectrum data improves blink detection in training and architectures that merge both types of data. Experiments showed that the NIR spectrum enhances results, even when only RGB images are available during inference. Finally, the generalization capacity of the proposed eyeblink detectors, along with state-of-the-art eyeblink detection implementations, is validated in wilder and more challenging environments like the HUST-LEBW dataset to show the usefulness of mEBAL2 to train a new generation of data-driven approaches for eyeblink detection.

Incorporating physical constraints in a deep learning framework for short-term daily prediction of sea ice concentration

Grasping the rapidly changing sea ice patterns has become the key to Arctic navigation. Starting from meteorology and oceanography, this study combines the empirical orthogonal function (EOF) analysis with convolutional long short-term memory (ConvLSTM) model to realize short-term sea ice prediction. To further improve the effectiveness and predictability of the ConvLSTM model, we adopt the EOF analysis to incorporate the spatial and temporal patterns of sea ice changes into the ConvLSTM model in three ways, namely, model input factors, model internal network structure, and model loss function. Through the mean absolute error (MAE), anomaly correlation coefficient (ACC), and anomaly metrics, we find that embedding the EOF physical constraints into the model's internal structure fully employs the spatial pattern and time series information, affording an optimal prediction effect. Additionally, the impact of the weight proportion between the physical constraint and the base loss and the effect of combining the three ways are discussed. The experimental results reveal that tuning the weight proportion in the loss function can improve the model's training effect. However, multiple combinations of the same physical law make it difficult to introduce new effective information.

Energy Expenditure Prediction from Accelerometry Data Using Long Short-Term Memory Recurrent Neural Networks.

The accurate estimation of energy expenditure from simple objective accelerometry measurements provides a valuable method for investigating the effect of physical activity (PA) interventions or population surveillance. Methods have been evaluated previously, but none utilize the temporal aspects of the accelerometry data. In this study, we investigated the energy expenditure prediction from acceleration measured at the subjects' hip, wrist, thigh, and back using recurrent neural networks utilizing temporal elements of the data. The acceleration was measured in children (N = 33) performing a standardized activity protocol in their natural environment. The energy expenditure was modelled using Multiple Linear Regression (MLR), stacked long short-term memory (LSTM) networks, and combined convolutional neural networks (CNN) and LSTM. The correlation and mean absolute percentage error (MAPE) were 0.76 and 19.9% for the MLR, 0.882 and 0.879 and 14.22% for the LSTM, and, with the combined LSTM-CNN, the best performance of 0.883 and 13.9% was achieved. The prediction error for vigorous intensities was significantly different (p < 0.01) from those of the other intensity domains: sedentary, light, and moderate. Utilizing the temporal elements of movement significantly improves energy expenditure prediction accuracy compared to other conventional approaches, but the prediction error for vigorous intensities requires further investigation.

PDED-ConvLSTM: Pyramid Dilated Deeper Encoder–Decoder Convolutional LSTM for Arctic Sea Ice Concentration Prediction

Arctic sea ice concentration plays a key role in the global ecosystem. However, accurate prediction of Arctic sea ice concentration remains a challenging task due to its inherent nonlinearity and complex spatiotemporal correlations. To address these challenges, we propose an innovative encoder–decoder pyramid dilated convolutional long short-term memory network (DED-ConvLSTM). The model is constructed based on the convolutional long short-term memory network (ConvLSTM) and, for the first time, integrates the encoder–decoder architecture of ConvLSTM (ED-ConvLSTM) with a pyramidal dilated convolution strategy. This approach aims to efficiently capture the spatiotemporal properties of the sea ice concentration and to enhance the identification of its nonlinear relationships. By applying convolutional layers with different dilation rates, the PDED-ConvLSTM model can capture spatial features at multiple scales and increase the receptive field without losing resolution. Further, the integration of the pyramid convolution module significantly enhances the model’s ability to understand complex spatiotemporal relationships, resulting in notable improvements in prediction accuracy and generalization ability. The experimental results show that the sea ice concentration distribution predicted by the PDED-ConvLSTM model is in high agreement with ground-based observations, with the residuals between the predictions and observations maintained within a range from −20% to 20%. PDED-ConvLSTM outperforms other models in terms of prediction performance, reducing the RMSE by 3.6% compared to the traditional ConvLSTM model and also performing well over a five-month prediction period. These experiments demonstrate the potential of PDED-ConvLSTM in predicting Arctic sea ice concentrations, making it a viable tool to meet the requirements for accurate prediction and provide technical support for safe and efficient operations in the Arctic region.

The Prediction of the Remaining Useful Life of Rotating Machinery Based on an Adaptive Maximum Second-Order Cyclostationarity Blind Deconvolution and a Convolutional LSTM Autoencoder.

The prediction of the remaining useful life (RUL) is important for the conditions of rotating machinery to maintain reliability and decrease losses. This study proposes an efficient approach based on an adaptive maximum second-order cyclostationarity blind deconvolution (ACYCBD) and a convolutional LSTM autoencoder to achieve the feature extraction, health index analysis, and RUL prediction for rotating machinery. First, the ACYCBD is used to filter noise from the vibration signals. Second, based on the peak value properties, a novel health index (HI) is designed to analyze the health conditions for the denoising signal, showing a high sensitivity for the degradation of bearings. Finally, for better prognostics and health management of the rotating machinery, based on convolutional layers and LSTM, an autoencoder can achieve a transform convolutional LSTM network to develop a convolutional LSTM autoencoder (ALSTM) model that can be applied to forecast the health trend for rotating machinery. Compared with the SVM, CNN, LSTM, GRU, and DTGRU methods, our experiments demonstrate that the proposed approach has the greatest performance for the prediction of the remaining useful life of rotating machinery.

Global Precipitation Nowcasting of Integrated Multi-satellitE Retrievals for GPM: A U-Net Convolutional LSTM Architecture

Abstract This paper presents a deep supervised learning architecture for 30 min global precipitation nowcasts with a 4-hour lead time. The architecture follows a U-Net structure with convolutional long short-term memory (ConvLSTM) cells empowered by ConvLSTM-based skip connections to reduce information loss due to the pooling operation. The training uses data from the Integrated MultisatellitE Retrievals for GPM (IMERG) and a few key drivers of precipitation from the Global Forecast System (GFS). The impacts of different training loss functions, including the mean-squared error (regression) and the focal-loss (classification), on the quality of precipitation nowcasts are studied. The results indicate that the regression network performs well in capturing light precipitation (&lt;1.6 mm hr−1) while the classification network can outperform the regression counterpart for nowcasting of high-intensity precipitation (&gt;8 mm hr−1), in terms of the critical success index (CSI). It is uncovered that including the forecast variables can improve precipitation nowcasting, especially at longer lead times in both networks. Taking IMERG as a relative reference, a multi-scale analysis, in terms of fractions skill score (FSS), shows that the nowcasting machine remains skillful for precipitation rate above 1 mm hr−1 at the resolution of 10 km compared to 50 km for GFS. For precipitation rates greater than 4 mm hr−1, only the classification network remains FSS-skillful on scales greater than 50 km within a 2-hour lead time.