Long Short-term Memory Approach Research Articles

Field Data Forecasting Using LSTM and Bi-LSTM Approaches

Water, an essential resource for crop production, is becoming increasingly scarce, while cropland continues to expand due to the world’s population growth. Proper irrigation scheduling has been shown to help farmers improve crop yield and quality, resulting in more sustainable water consumption. Soil Moisture (SM), which indicates the amount of water in the soil, is one of the most important crop irrigation parameters. In terms of water usage optimization and crop yield, estimating future soil moisture (forecasting) is an essentially valuable task for crop irrigation. As a result, farmers can base crop irrigation decisions on this parameter. Sensors can be used to estimate this value in real time, which may assist farmers in deciding whether or not to irrigate. The soil moisture value provided by the sensors, on the other hand, is instantaneous and cannot be used to directly compute irrigation parameters such as the best timing or the required water quantity to irrigate. The soil moisture value can, in fact, vary greatly depending on factors such as humidity, weather, and time. Using machine learning methods, these parameters can be used to predict soil moisture levels in the near future. This paper proposes a new Long-Short Term Memory (LSTM)-based model to forecast soil moisture values in the future based on parameters collected from various sensors as a potential solution. To train and validate this model, a real-world dataset containing a set of parameters related to weather forecasting, soil moisture, and other related parameters was collected using smart sensors installed in a greenhouse in Chiang Mai province, Thailand. Preliminary results show that our LSTM-based model performs well in predicting soil moisture with a 0.72% RMSE error and a 0.52% cross-validation error (LSTM), and our Bi-LSTM model with a 0.76% RMSE error and a 0.57% cross-validation error. In the future, we aim to test and validate this model on other similar datasets.

Prediction models for urban flood evolution for satellite remote sensing

Accurate and timely flood forecasts are critical for protecting people and infrastructure in a changing climate. Satellite remote sensing provides the necessary wide area coverage and period revisits to measure episodic heavy precipitation and resultant urban floods. We propose two methods to assimilate satellite-observed precipitation into hydrologic models in real time to update flood forecasts, bypassing two previous barriers in this technology: infrequent satellite overpasses and long model run times. Constellations of small satellites overcomes the first barrier by providing frequent flights over an area of interest; however, these constellations require coordination and planning to capture precipitation data where it is most needed to inform flood forecasts. The primary purpose of this paper is to address the second barrier – that is high computational costs that make it infeasible to run flood forecast models on-board these satellites so that they can re-orient to measure where most needed. We develop a simple regression-based approach and a machine learning framework (Long Short-Term Memory (LSTM) models) to provide reliable flood forecasts using satellite-observed precipitation at a fraction of the computational cost of physics-based hydrologic models. We apply these approaches to a test case for the Atlanta metropolitan region using the Weather Research and Forecasting model hydrologic modeling system (WRF-Hydro) to simulate flooding across the model domain for several precipitation events. We compare the accuracy of our proposed approaches to the WRF-Hydro model using different spatial extents and temporal frequencies of precipitation observations to examine different plausible satellite constellation scenarios. The LSTM approach trades performance accuracy and adaptability for computational efficiency, which can be important in a time and resource constrained scenario. The LSTM model reduces total error up to 38% from an initial flood forecast. Additionally, this approach correctly classified flooding to within one flood magnitude category in ∼90% of cases. These new forecasting algorithms can be used onboard constellations of small satellites to observe ongoing flood events, update short term predictions, and schedule observations to maximize useful measurements and thereby improve flood warning systems for protecting residents and properties.

Forecasting Monthly Rainfall using Bio-Inspired Artificial Algae Deep Learning Network

Rainfall forecasting is the most critical and challenging task because of its dependence on different climatic and weather parameters. Hence, robust and accurate rainfall forecasting models need to be created by applying various machine learning and deep learning approaches. Several automatic systems were created to predict the weather, but it depends on the type of weather pattern, season and location, which leads in maximizing the processing time. Therefore, in this work, significant artificial algae long short-term memory (LSTM) deep learning network is introduced to forecast the monthly rainfall. During this process, Homogeneous Indian Monthly Rainfall Data Set (1871–2016) is utilized to collect the rainfall information. The gathered information is computed with the help of an LSTM approach, which is able to process the time series data and predict the dependency between the data effectively. The most challenging phase of LSTM training process is finding optimal network parameters such as weight and bias. For obtaining the optimal parameters, one of the Meta heuristic bio-inspired algorithms called Artificial Algae Algorithm (AAA) is used. The forecasted rainfall for the testing dataset is compared with the existing models. The forecasted results exhibit superiority of our model over the state-of-the-art models for forecasting Indian Monsoon rainfall. The LSTM model combined with AAA predicts the monsoon from June–September accurately.

An improved learning-based LSTM approach for lane change intention prediction subject to imbalanced data

Lane change intention prediction is an essential component for motion planning of Autonomous Vehicles (AVs). In this work, we aim to achieve this task by using Long-Short Term Memory (LSTM) network. One critical challenge on this task is that the dataset used for training such a network is usually highly imbalanced due to the fact that the size of left/right lane change data is much smaller than that of the lane keeping data. The imbalanced dateset would lead to trivial output of LSTM model. To deal with this problem, we propose a hierarchical over-sampling bagging method based on Grey Wolf Optimizer (GWO) algorithm and Synthetic Minority Over-sampling Technique (SMOTE). With the proposed method, more diverse and informative instances of minority classes can be generated for training LSTM model. Furthermore, we also propose a sampling technique to keep the temporal information and make the proposed method applicable to sequential data. Moreover, to further improve the prediction performance, we also take the interactions between neighboring vehicles into account by concatenating their trajectories when constructing features. We evaluate our method against several baseline algorithms over two benchmark datasets and the empirical results validate the effectiveness and efficiency of our method in terms of the indexes of prediction time, F1, and G-mean.

A Real-Time Data Analysis Platform for Short-Term Water Consumption Forecasting with Machine Learning

This article presents a real-time data analysis platform to forecast water consumption with Machine-Learning (ML) techniques. The strategy fully relies on a web-oriented architecture to ensure better management and optimized monitoring of water consumption. This monitoring is carried out through a communicating system for collecting data in the form of unevenly spaced time series. The platform is completed by learning capabilities to analyze and forecast water consumption. The analysis consists of checking the data integrity and inconsistency, in looking for missing data, and in detecting abnormal consumption. Forecasting is based on the Long Short-Term Memory (LSTM) and the Back-Propagation Neural Network (BPNN). After evaluation, results show that the ML approaches can predict water consumption without having prior knowledge about the data and the users. The LSTM approach, by being able to grab the long-term dependencies between time steps of water consumption, allows the prediction of the amount of consumed water in the next hour with an error of some liters and the instants of the 5 next consumed liters in some milliseconds.

Unscented Kalman Filter-Aided Long Short-Term Memory Approach for Wind Nowcasting

Obtaining reliable wind information is critical for efficiently managing air traffic and airport operations. Wind forecasting has been considered one of the most challenging tasks in the aviation industry. Recently, with the advent of artificial intelligence, many machine learning techniques have been widely used to address a variety of complex phenomena in wind predictions. In this paper, we propose a hybrid framework that combines a machine learning model with Kalman filtering for a wind nowcasting problem in the aviation industry. More specifically, this study has three objectives as follows: (1) compare the performance of the machine learning models (i.e., Gaussian process, multi-layer perceptron, and long short-term memory (LSTM) network) to identify the most appropriate model for wind predictions, (2) combine the machine learning model selected in step (1) with an unscented Kalman filter (UKF) to improve the fidelity of the model, and (3) perform Monte Carlo simulations to quantify uncertainties arising from the modeling process. Results show that short-term time-series wind datasets are best predicted by the LSTM network compared to the other machine learning models and the UKF-aided LSTM (UKF-LSTM) approach outperforms the LSTM network only, especially when long-term wind forecasting needs to be considered.

A novel hybrid framework for wind speed forecasting using autoencoder‐based convolutional long short‐term memory network

A precise forecast of wind speed is a fundamental requirement of wind power integration. The nonlinear and intermittent nature of the wind makes wind speed forecasting (WSF) complicated for linear approaches. Addressing the complications faced by the linear approaches, this paper proposed a novel and robust approach using long short-term memory (LSTM) autoencoder, convolutional neural network (CNN), and LSTM model for enhanced WSF. The proposed hybrid approach is divided into two main components: feature encoding, dimensionality reduction using LSTM autoencoder and forecasting using convolutional LSTM. In the first stage, the LSTM autoencoder eliminates the uncertainties present in raw wind speed data and also reduces the computational load on the forecasting convolutional LSTM approach. Then, in the second stage, CNN is used to extract the optimum features, and the LSTM network is used to forecast the wind speed. Five different benchmark forecasting models are used to evaluate and study the proposed hybrid approach's performance. The experiment is performed with real-time wind speed data from the Garden city wind farm, USA. The proposed hybrid approach performance is verified using various performance metrics. The experimental results demonstrate that the proposed approach improved by 40% over the second best benchmark forecasting approach.

VASP: An autoencoder-based approach for multivariate anomaly detection and robust time series prediction with application in motorsport

The aim is to provide a framework for robust time series prediction in the presence of anomalies. The framework is developed based on a data set from motorsport but is not limited to this specific area. In motorsport, the usage of sensors during races is generally restricted. Estimating the outputs of these missing sensors therefore provides an advantage over the competition. Deep learning approaches such as long short-term memory (LSTM) neural networks have proven to be useful for that task, however, their accuracy decreases significantly if anomalies occur in the input signals. To overcome this problem, we propose the variational autoencoder based selective prediction (VASP) framework which combines the tasks of anomaly detection and time series prediction. VASP consists of a variational autoencoder (VAE), an anomaly detector and LSTM predictors. Depending on the anomaly detector, a subset of the inputs may be replaced by the VAE, allowing a more robust prediction. To the best of our knowledge the approach of using a VAE to only selectively replace anomalous input data before prediction has not yet been published. Our contributions are clear implementation guidelines and a comparison to other VAE-based methods and a LSTM approach as baseline. We simulate anomalies with three approaches and show that VASP outperforms other methods by having no trade-off between accuracy and robustness. VASP is as accurate as the baseline for regular data, but for anomalous inputs the error is reduced by 13% to 33% on average and up to 70% in special cases.

Intelligent Online Monitoring of Rolling Bearing: Diagnosis and Prognosis.

This paper suggests a new method to predict the Remaining Useful Life (RUL) of rolling bearings based on Long Short Term Memory (LSTM), in order to obtain the degradation condition of the rolling bearings and realize the predictive maintenance. The approach is divided into three parts: the first part is the clustering to detect the damage state by the density-based spatial clustering of applications with noise. The second one is the health indicator construction which could give a better reflection of the bearing degradation tendency and is selected as the input for the prediction model. In the third part of the RUL prediction, the LSTM approach is employed to improve the accuracy of the prediction. The rationale of this work is to combine the two methods—the density-based spatial clustering of applications with noise and LSTM—to identify the abnormal state in rolling bearings, then estimate the RUL. The suggested method is confirmed by experimental data of bearing life cycle, and the RUL prediction results of the model LSTM are compared with the nonlinear au-regressive model with exogenous input model. In addition, the constructed health indicator is compared with the spectral kurtosis feature. The results demonstrated that the suggested method is more appropriate than the nonlinear au-regressive model with exogenous input model for the prediction of bearing RUL.

Overview, modeling and forecasting the effects of COVID-19 pandemic on energy market and electricity demand: a case study on Turkey

ABSTRACT In this study, the overall energy status of Turkey in the early stage of the COVID-19 pandemic was analyzed. The situation in the energy market and electricity demand in the pandemic period as global issues were investigated in Turkey as a developing country. In addition to this general overview using limited data for the early period of the pandemic, energy demand and energy generation values were modeled utilizing machine learning approaches. Daily energy demand values were modeled and forecasted by utilizing Nonlinear Autoregression Neural Network (NARNN), Auto-Regressive Integrated Moving Average (ARIMA), and Long-Short Term Memory (LSTM) techniques. According to the results of the first stage of the modeling process, the LSTM approach was found as the most accurate model. In the second step of the modeling and forecasting analysis, monthly electricity generation values from natural gas and coal were predicted. In the energy generation forecasting analysis for April-December 2020 period, contraction in energy generation by natural gas and coal were obtained as 6.27% and 7.19%, respectively in comparison to 2019. Finally, the energy market during and after the pandemic was evaluated and the strategies to be implemented in the post-pandemic process were discussed.

Car Parking Availability Prediction: A Comparative Study of LSTM and Random Forest Regression Approaches

Drivers spend an enormous amount of time searching for parking spots every year. Waste of time, emission of carbon and air pollution have been issues in hunting for parking spots without proper prediction. In this paper, we have proposed to build a framework based on Recurrent Neural Network (RNN) using Long Short Term Memory (LSTM) and Random Forest Regression model to provide prediction of parking availability and compared results afterwards. A real-world case of parking spots availability consisting of 5,500 parking spots in Kuala Lumpur City Centre (KLCC), Malaysia, has been used for regression implementation in this comparative analysis. The results showed that random forest outperformed LSTM approach based on performance metrics.

Depressed People Detection from Bangla Social Media Status using LSTM and CNN Approach

At present, depression is the main reason for suicidal death. Depression also causes different kinds of diseases. Nowadays, people are deeply involved in social media and like to share their feelings on social media. So, it becomes easy to analyze depression through social media. In this paper, a combination of two CNN (Convolutional Neural Network) and LSTM (Long Short-Term Memory) models has been proposed to make a hybrid CNN-LSTM model, CNN has performed for the image to create a matrix, and LSTM has given the result from the given matrix. In this paper, datasets are prepared based on depression and non-depression-related status. The proposed method has been applied to that dataset. The best result has been obtained using a hybrid neural network with the word embedding technique using the Bengali Facebook status dataset. We have used the SVM (Support Vector Machine) model to predict a small dataset of Bengali Facebook status and count vectorizer to count the word in the document. Finally, this paper has built up a model that makes strength and support for deep learning architecture.

Data-driven approaches for runoff prediction using distributed data

Accurate rainfall-runoff modelling is particularly challenging due to complex nonlinear relationships between various factors such as rainfall characteristics, soil properties, land use, and temporal lags. Recently, with improvements to computation systems and resources, data-driven models have shown good performances for runoff forecasting. However, the relative performance of common data-driven models using small temporal resolutions is still unclear. This study presents an application of data-driven models using artificial neural network, support vector regression and long-short term memory approaches and distributed forcing data for runoff predictions between 2010 and 2019 in the Russian River basin, California, USA. These models were used to predict hourly runoff with 1–6 h of lead time using precipitation, soil moisture, baseflow and land surface temperature datasets provided from the North American Land Data Assimilation System. The predicted results were evaluated in terms of seasonal and event-based performance using various statistical metrics. The results showed that the long-short term memory and support vector regression models outperforms artificial neural network model for hourly runoff forecasting, and the predictive performance of the models was greater during the wet seasons compared to the dry seasons. In addition, a comparison of the data-driven model results with the National Water Model, a fully distributed physical-based hydrologic model, showed that the long-short term memory and support vector regression models provide comparable performance. The results demonstrate that data-driven models for hourly runoff forecasting are sufficiently predictive and useful in areas where observation systems are not available.

Daily Runoff Forecasting Using a Cascade Long Short-Term Memory Model that Considers Different Variables

Accurate forecasts of daily runoff are essential for facilitating efficient resource planning and management of a hydrological system. In practice, daily runoff is needed for various practical applications and can be predicted using precipitation and evapotranspiration data. To this end, a long short-term memory (LSTM) under a cascade framework (C-LSTM) approach is proposed for forecasting daily runoff. This C-LSTM model is composed of a 2-level forecasting process. (1) In the first level, an LSTM is established to learn the relationship between the precipitation and evapotranspiration at present and to learn several meteorological variables one day in advance. (2) In the second level, an LSTM is constructed to forecast the daily runoff using the historical and simulated precipitation and evapotranspiration data produced by the first LSTM. Through cascade modeling, the complex features of the numerous targets in the different stages can be sufficiently extracted and learned by multiple models in a single framework. In order to evaluate the performance of the C-LSTM approach, four mesoscale sub-catchments of the Ljubljanica River in Slovenia were investigated. The results indicate that based on the root-mean-square error, the Pearson correlation coefficient, and the Nash-Sutcliffe model efficiency coefficient, the proposed model yields better results than two other tested models, including the normal LSTM and other neural network approaches. Based on the results of this study, we conclude that the LSTM under the cascade architecture is a valuable approach and can be regarded as a promising model for forecasting daily runoff.

A Machine Learning-Based Early Warning System for the Housing and Stock Markets

This study analyzes the relationship between the housing and stock markets, focusing on housing market bubbles. Stock market dynamics generally have a more significant impact on housing price movements than housing market dynamics have on stock dynamics. However, if housing market information is provided as a signal, housing price movements can predict stock market volatility. Accordingly, we build a machine learning-based early warning system (EWS) for the housing market using a long short-term memory (LSTM) neural network. Applying the generalized supremum augmented Dickey-Fuller test to extract the bubble signal in the housing market, we find that the signal simultaneously detects future changes in the housing market prices and future stock market volatility, and our EWS effectively detects the bubble signal. We confirm that the LSTM approach performs better than other benchmark models, the random forest and support vector machine models.

Short-Term Wind Power Forecasting Using R-LSTM

Renewable energies such as wind and solar begin receiving remarkable popularity in accordance with the energy demand, expeditious expansion of solar and wind energy generation involves acute forecasting of wind and solar power, so in past and recent years it has become an intensive research area. An accurate forecast of wind power to maintain an affordable, secure, and economical power supply is most significant. Numerous investigations and research have been performed in this area in recent years. This research article aims to develop a short-term wind power forecasting model to improve the accuracy of the prediction. Therefore, a novel approach based on LSTM (Long Short-Term Memory) is proposed to forecast from 1 to 6 hours ahead of wind power. A recursive strategy is used when predicting short-term wind power, unlike the conventional LSTM approach. The proposed model is implemented using historical generated wind power data for Gujarat state. A comparative analysis is performed between the proposed and existing approach presented in the literature, from the analysis, it is noted that the proposed R-LSTM (Rolling-LSTM) model outperformed with minimal error and better accuracy.

Deep Learning for Land Cover Change Detection

Land cover and its change are crucial for many environmental applications. This study focuses on the land cover classification and change detection with multitemporal and multispectral Sentinel-2 satellite data. To address the challenging land cover change detection task, we rely on two different deep learning architectures and selected pre-processing steps. For example, we define an excluded class and deal with temporal water shoreline changes in the pre-processing. We employ a fully convolutional neural network (FCN), and we combine the FCN with long short-term memory (LSTM) networks. The FCN can only handle monotemporal input data, while the FCN combined with LSTM can use sequential information (multitemporal). Besides, we provided fixed and variable sequences as training sequences for the combined FCN and LSTM approach. The former refers to using six defined satellite images, while the latter consists of image sequences from an extended training pool of ten images. Further, we propose measures for the robustness concerning the selection of Sentinel-2 image data as evaluation metrics. We can distinguish between actual land cover changes and misclassifications of the deep learning approaches with these metrics. According to the provided metrics, both multitemporal LSTM approaches outperform the monotemporal FCN approach, about 3 to 5 percentage points (p.p.). The LSTM approach trained on the variable sequences detects 3 p.p. more land cover changes than the LSTM approach trained on the fixed sequences. Besides, applying our selected pre-processing improves the water classification and avoids reducing the dataset effectively by 17.6%. The presented LSTM approaches can be modified to provide applicability for a variable number of image sequences since we published the code of the deep learning models. The Sentinel-2 data and the ground truth are also freely available.

A fractal interpolation approach to improve neural network predictions for difficult time series data

Deep Learning methods, such as Long Short-Term Memory (LSTM) neural networks prove capable of predicting real-life time series data. Crucial for this technique to work is a sufficient amount of data. This can either be very long time-series data or fine-grained time-series data. If the data is insufficient, in terms of length or complexity, LSTM approaches perform poorly. We propose a fractal interpolation approach to generate a more fine-grained time series out of insufficient data sets. The interpolation is dynamically adapted to the time series using a time-dependent complexity feature so that the complexity properties of the interpolated time series are related to that of the original one. Also we perform a linear interpolation with the same number of interpolation points to compare results.This paper shows that predictions of fractal interpolated and linear interpolated time series clearly outperform the ones of the original data for a test fit on unknown data. Though predictions of linear and fractal interpolated time series data perform very similar, the fractal interpolated ones outperform the linear interpolated ones on difficult time series data. Also, though the complexities of sub-intervals are tailored to match the one from the original data, the interpolated time series shows a much higher degree of persistency and (in terms of the Hurst exponent) a much higher degree of long term memory.

Gait Trajectory and Gait Phase Prediction Based on an LSTM Network.

Lower body segment trajectory and gait phase prediction is crucial for the control of assistance-as-needed robotic devices, such as exoskeletons. In order for a powered exoskeleton with phase-based control to determine and provide proper assistance to the wearer during gait, we propose an approach to predict segment trajectories up to 200 ms ahead (angular velocity of the thigh, shank and foot segments) and five gait phases (loading response, mid-stance, terminal stance, preswing and swing), based on collected data from inertial measurement units placed on the thighs, shanks, and feet. The approach we propose is a long-short term memory (LSTM)-based network, a modified version of recurrent neural networks, which can learn order dependence in sequence prediction problems. The algorithm proposed has a weighted discount loss function that places more weight in predicting the next three to five time frames but also contributes to an overall prediction performance for up to 10 time frames. The LSTM model was designed to learn lower limb segment trajectories using training samples and was tested for generalization across participants. All predicted trajectories were strongly correlated with the measured trajectories, with correlation coefficients greater than 0.98. The proposed LSTM approach can also accurately predict the five gait phases, particularly swing phase with 95% accuracy in inter-subject implementation. The ability of the LSTM network to predict future gait trajectories and gait phases can be applied in designing exoskeleton controllers that can better compensate for system delays to smooth the transition between gait phases.

A Decision Support System for Water Optimization in Anti-Frost Techniques by Sprinklers.

Precision agriculture is a growing sector that improves traditional agricultural processes through the use of new technologies. In southeast Spain, farmers are continuously fighting against harsh conditions caused by the effects of climate change. Among these problems, the great variability of temperatures (up to 20 C in the same day) stands out. This causes the stone fruit trees to flower prematurely and the low winter temperatures freeze the flower causing the loss of the crop. Farmers use anti-freeze techniques to prevent crop loss and the most widely used techniques are those that use water irrigation as they are cheaper than other techniques. However, these techniques waste too much water and it is a scarce resource, especially in this area. In this article, we propose a novel intelligent Internet of Things (IoT) monitoring system to optimize the use of water in these anti-frost techniques while minimizing crop loss. The intelligent component of the IoT system is designed using an approach based on a multivariate Long Short-Term Memory (LSTM) model, designed to predict low temperatures. We compare the proposed approach of multivariate model with the univariate counterpart version to figure out which model obtains better accuracy to predict low temperatures. An accurate prediction of low temperatures would translate into significant water savings, as anti-frost techniques would not be activated without being necessary. Our experimental results show that the proposed multivariate LSTM approach improves the univariate counterpart version, obtaining an average quadratic error no greater than 0.65 C and a coefficient of determination R greater than 0.97. The proposed system has been deployed and is currently operating in a real environment obtained satisfactory performance.