Long Short-term Memory Neural Network Model Research Articles

Utilizing single light-emitting-diode (LED) lamp and silicon solar-cells visible light positioning (VLP) based on angle-of-arrival (AOA) and long-short-term-memory-neural-network (LSTMNN)

We propose and present the first proof-of-concept demonstration of an angle-of-arrival (AOA)-based visible-light-positioning (VLP) system using long-short-term-memory neural-network (LSTMNN) model. Only a single LED lamp and silicon based solar cell Rx are needed. The LSTM cell can manipulate and analyze time-serial sequential data, reducing the influence of time-dependent fluctuation and noises during the AOA data acquisition; hence, enhancing the positioning accuracy. We compare different ML models including the linear regression (LR) and artificial neural network (ANN), as well as different AOA data sequence lengths and additional LSTM layers. The experimental result reveals that the LSTMNN model outperforms the other models under evaluation. When the sequence length of 20 and one LSTM layer are employed, the mean positioning error is 1.78 cm and the 90% of the experimental data has positioning error within ∼2.9 cm. Compared to the LR model and ANN model, the mean positioning errors are reduced by 49% and 40% respectively; and the 90% cumulative distribution function (CDF) values are reduced by 3 cm and 1.3 cm respectively by utilizing the proposed LSTMNN model.

A new application of deep neural network (LSTM) and RUSLE models in soil erosion prediction

Rainfall variation causes frequent unexpected disasters all over the world. Increasing rainfall intensity significantly escalates soil erosion and soil erosion related hazards. Forecasting accurate rainfall helps early detection of soil erosion vulnerability and can minimise the damages by taking appropriate measures caused by severe storms, droughts and floods. This study aims to predict soil erosion probability using the deep learning approach: long short-term memory neural network model (LSTM) and revised universal soil loss equation (RUSLE) model. Daily rainfall data were gathered from five agro-meteorological stations in the Central Highlands of Sri Lanka from 1990 to 2021 and fed into the LSTM model simulation. The LSTM model was forecasted with the time-series monthly rainfall data for a long lead time period, rainfall values for next 36 months in each station. Geo-informatics tools were used to create the rainfall erosivity map layer for the year 2024. The RUSLE model prediction indicates the average annual soil erosion over the Highlands will be 11.92 t/ha/yr. Soil erosion susceptibility map suggests around 30 % of the land area will be categorised as moderate to very-high soil erosion susceptible classes. The resulted map layer was validated using past soil erosion map layers developed for 2000, 2010 and 2019. The soil erosion susceptibility map indicates an accuracy of 0.93 with the area under the receiver operator characteristic curve (AUC-ROC), showing a satisfactory prediction performance. These findings will be helpful in policy-level decision making and researchers can further tested different deep learning models with the RUSLE model to enhance the prediction capability of soil erosion probability.

Sustainable Reuse Strategies of Enterprise Financial Management Model Following Deep Learning Under Big Data

The study aims to help enterprises to formulate a financial sustainable development strategy. A financial crisis forecast system based on deep learning (DL) is proposed to assist enterprises in checking their financial bills in time, knowing about their financial situations, formulating corresponding strategies, and realizing financially sustainable development. First, the relevant theories of financially sustainable development and DL are reviewed. Second, a long short-term memory (LSTM) neural network model based on DL is implemented and the normal sample data are compared with the unbalanced sample data. Finally, the performance of the model is analyzed according to the experimental results. The experiments show that the performance of the financial crisis forecast system is the best when the time step is T-3. The accuracy rate of the LSTM model is more than 93%, and the highest value of AUC (area under the curve) is 93.67%. The AUC value of the LSTM neural network model is compared with that of the fully connected neural network model and logistic regression model.

Joint forecasting of multi-energy loads for a university based on copula theory and improved LSTM network

In this study, a multi-energy loads forecasting model based on the artificial intelligence method is proposed. Firstly, based on Copula theory, the nonlinear relationship among different load types themselves, as well as other influence factors affecting the load demand such as ambient temperature etc. are analyzed. Following which, the input factors for load forecasting are selected according to the influence degree. Secondly, based on LSTM (Long Short-Term Memory) neural network model and its improved forms (Stack LSTM and Bidirectional LSTM), the multi-type load forecasting model is developed. A university campus is selected to verify the feasibility and validity of the model compared with the actual data. According to the simulation results, the Stack LSTM illustrates better prediction performance and generalization ability compared with other methods. In addition, the prediction accuracy of cooling and heating demands can be highly improved when considering the affection of electrical load. Moreover, the reduction of load time-scale may obviously increase the prediction accuracy.

HUMAN ACTIVITY RECOGNITION USING LSTM MODEL

Detecting the physical body movements of a human is called Human Activity Recognition . HAR is a time series classifying work, means it can be used in different fields, domains and varieties of application. Sensor data are used. The Movements here are often regular activities like walking, jogging, standing and sitting. In general deep learning architecture manages HAR job with a distinct result. Long Short Term Memory (LSTM) neural architecture which is a deep learning model can be designate as a distinct sort of recurrent neural network model this is able to gaining knowledge of long time dependencies in data. The LSTM neural network model comprises a merger of four layers that interact with each other. We have used Long Short Term Memory neural architecture in UCI HAR dataset to identify the type of activity or movement the person is doing. We saw that after the network completes training, optimization and accuracy is found to be 87% to 90% also a graph and confusion matrix has been used to show the losses, accuracies, training iteration, etc.

USING LONG SHORT-TERM MEMORY MODEL FOR CLOUD FOREST VEGETATION GROWTH STATUS PREDICTION – A CASE STUDY IN SHEI-PA NATIONAL PARK

Abstract. Cloud Forests (CFs) are characterized by their persistent foggy environment, in which fog can save two times the amount of precipitation in the dry season and increase water storage by 10% in the rainy season. CFs play an important role in ecosystems as high biodiversity and abundant endemic species live within CFs. However, CFs are sensitive to environmental changes, especially in current global climate warming conditions. Therefore, a typical cloud forest in Taiwan, Shei-Pa National park, was chosen as the study area. Specifically, the Normalized Difference Vegetation Index (NDVI) with meteorological factors including rainfall, average temperature, maximum temperature, and minimum temperature were obtained to assess the overall CFs trend from 2001 to 2017. Moreover, the Long Short-Term Memory neural network model (LSTM) was implemented to predict the future vegetation status. Preliminary results have shown that vegetation condition in Shei-Pa National park was getting better; rainfall, average temperature, and minimum temperature represented an upward trend while maximum temperature showed a downward trend. Furthermore, the LSTM- maximum temperature model displayed the highest prediction power with the MAPE index of 4.84%. The results provide a valuable reference for forest resource conservation and future climate adaptation strategies in Taiwan.

Evaluation of Dynamic Tensions of Single Point Mooring System under Random Waves with Artificial Neural Network

Real-time monitoring of the mooring safety of floating structures is of great significance to their production operations. A deep learning model is proposed here, based on the long short-term memory (LSTM) artificial neural network. Firstly, the numerical simulation is carried out with the single-point mooring system of a Floating Production Storage and Offloading (FPSO) as the training data of LSTM. Then the proposed LSTM is performed. Finally, taking the motion of FPSO which is not encountered by LSTM neural network model as input, we predict the mooring line tension with this model. Here, one FPSO in the South China Sea is taken as a research case, hydrodynamic and mooring models are established, and the network structure and hyper-parameters of the LSTM model are determined. The prediction results of the LSTM under different combinations of wind, wave, and current are compared with the calculation results of AQWA software. The model constructed here can well predict the mooring line tension of different combinations of wind, wave and current.

Reservoir production prediction with optimized artificial neural network and time series approaches

Numerical simulation of oil reservoirs is one of the most commonly used methods for reservoir production prediction, but its accuracy is based on accurate geological modeling and high-quality history matching. Therefore, numerical simulation is time-consuming and costly and requires extensive information. Traditional back-propagation neural networks and their improved algorithms are widely used for production prediction, but they are not suitable for time-series prediction problems. Based on variations in oil production, this study proposes a reservoir production prediction model based on a combined convolutional neural network (CNN) and a long short-term memory (LSTM) neural network model optimized by the particle swarm optimization (PSO) algorithm. First, the model extracts important temporal data features through the upper CNN, which is next imported to the lower LSTM network to further extract correlation features in the time dimension; then, it iteratively optimizes the key hyperparameters in the CNN-LSTM model through the PSO algorithm; finally, it uses the trained model for reservoir prediction. Compared with the training results of the LSTM neural network and CNN model, the PSO–CNN-LSTM model has higher prediction accuracy in time-series production prediction. Our proposed hybrid model is a data-driven method and is based on routinely available production data. Quick and accurate production prediction can lead to better informed operational decisions and optimization of recovery and economics.

Research on Stock Price Time Series Prediction Based on Deep Learning and Autoregressive Integrated Moving Average

Different from traditional algorithms and model, machine learning is a systematic and comprehensive application of computer algorithms and statistical models, and it has been widely used in many fields. In the field of finance, machine learning is mainly used to study the future trend of capital market price. In this paper, to predict the time-series data of stock, we applied the traditional models and machine learning models for forecasting the linear and non-linear problem, respectively. First, stock samples that occurred from year 2010 to 2019 at the New York Stock Exchange are collected. Next, the ARIMA (autoregressive integrated moving average model) model and LSTM (long short-term memory) neural network model are applied to train and predict stock price and stock price subcorrelation. Finally, we evaluate the proposed model by several indicators, and the experiment results show that: (1) Stock price and stock price correlation are accurately predicted by the ARIMA model and LSTM model; (2) compared with ARIMA, the LSTM model performance better in prediction; and (3) the ensemble model of ARIMA-LSTM significantly outperforms other benchmark methods. Therefore, our proposed method provides theoretical support and method reference for investors about stock trading in China stock market.

Temporal Prediction of Paralytic Shellfish Toxins in the Mussel Mytilus galloprovincialis Using a LSTM Neural Network Model from Environmental Data.

Paralytic shellfish toxins (PSTs) are produced mainly by Alexandrium catenella (formerly A. tamarense). Since 2000, the National Institute of Fisheries Science (NIFS) has been providing information on PST outbreaks in Korean coastal waters at one- or two-week intervals. However, a daily forecast is essential for immediate responses to PST outbreaks. This study aimed to predict the outbreak timing of PSTs in the mussel Mytilus galloprovincialis in Jinhae Bay and along the Geoje coast in the southern coast of the Korea Peninsula. We used a long-short-term memory (LSTM) neural network model for temporal prediction of PST outbreaks from environmental data, such as water temperature (WT), tidal height, and salinity, measured at the Geojedo, Gadeokdo, and Masan tidal stations from 2006 to 2020. We found that PST outbreaks is gradually accelerated during the three years from 2018 to 2020. Because the in-situ environmental measurements had many missing data throughout the time span, we applied LSTM for gap-filling of the environmental measurements. We trained and tested the LSTM models with different combinations of environmental factors and the ground truth timing data of PST outbreaks for 5479 days as input and output. The LSTM model trained from only WT had the highest accuracy (0.9) and lowest false-alarm rate. The LSTM-based temporal prediction model may be useful as a monitoring system of PSP outbreaks in the coastal waters of southern Korean.

Predict the Price Change over Time with LSTM Neural Network

Time series analysis can be used to analyze the price. The LSTM Neural Network model is used to predict the price. Moreover, the correlation analysis between variables and price is conducted. By improving and optimizing the mathematical model, the RMSE values of LSTM (Long Short Term Memory) with a single function of weight price are 8.719 and 13.759, and the RESM values of prediction results are larger than those of other models, so the accuracy of prediction results is higher.

Time-To-Recovery Prediction in a Disrupted Three-Echelon Supply Chain Using LSTM*

This paper introduces a Long-Short Term Memory (LSTM) neural network-based method for Time-To-Recovery (TTR) prediction within a cognitive digital supply chain twin framework to enhance supply chain resilience and improve decision-making under disruption. The introduced method is applied to a virtual three-echelon supply chain network modelled using discrete event simulation. The virtual system was used to train an LSTM neural network model to predict TTR under several disruption scenarios. Results show that predicted TTR values tend to be relatively lower than the actual values at early disruption stages, then improve throughout the progression of the disruption effect on the supply chain network.

Determining the invoicing dates for raw material order and finish product dispatch using neural networks under exchange rate volatility

ABSTRACT The gains from international supply chains are highly affected by the exchange rate fluctuations in the foreign exchange market. Traditional forecasting methods have not been very useful, and as a result, business firms tend to use hedging or forward contracts to mitigate the exchange rate risk. This research focuses on using machine learning models to forecast the exchange rate for future decision-making in business. This paper uses both time-series data and the categorical data with the LSTM (Long Short-Term Memory) Neural Network Model to tackle both linear and non-linear data on monetary fundamentals and derives the best dates for invoicing in the international transaction using data of a manufacturing firm. Results show that using the predictions of the LSTM model to decide the invoicing dates for international transactions delivers foreign exchange gain with a better success rate than selecting random dates for both import and export.

Data driven covid-19 spread prediction based on mobility and mask mandate information.

COVID-19 is one of the largest spreading pandemic diseases faced in the documented history of mankind. Human to human interaction is the most prolific method of transmission of this virus. Nations all across the globe started to issue stay at home orders and mandating to wear masks or a form of face-covering in public to minimize the transmission by reducing contact between majority of the populace. The epidemiological models used in the literature have considerable drawbacks in the assumption of homogeneous mixing among the populace. Moreover, the effect of mitigation strategies such as mask mandate and stay at home orders cannot be efficiently accounted for in these models. In this work, we propose a novel data driven approach using LSTM (Long Short Term Memory) neural network model to form a functional mapping of daily new confirmed cases with mobility data which has been quantified from cell phone traffic information and mask mandate information. With this approach no pre-defined equations are used to predict the spread, no homogeneous mixing assumption is made, and the effect of mitigation strategies can be accounted for. The model learns the spread of the virus based on factual data from verified resources. A study of the number of cases for the state of New York (NY) and state of Florida (FL) in the USA are performed using the model. The model correctly predicts that with higher mobility the cases would increase and vice-versa. It further predicts the rate of new cases would see a decline if a mask mandate is administered. Both these predictions are in agreement with the opinions of leading medical and immunological experts. The model also predicts that with the mask mandate option even a higher mobility would reduce the daily cases than lower mobility without masks. We additionally generate results and provide RMSE (Root Mean Square Error) comparison with ARIMA based model of other published work for Italy, Turkey, Australia, Brazil, Canada, Egypt, Japan, and the UK. Our model reports lower RMSE than the ARIMA based work for all eight countries which were tested. The proposed model would provide administrations with a quantifiable basis of how mobility, mask mandates are related to new confirmed cases; so far no epidemiological models provide that information. It gives fast and relatively accurate prediction of the number of cases and would enable the administrations to make informed decisions and make plans for mitigation strategies and changes in hospital resources.

METHOD OF MODELING TEMPERATURE-DISPLACEMENT CORRELATION FOR LONG-SPAN ARCH BRIDGES BASED ON LONG SHORT-TERM MEMORY NEURAL NETWORKS

The establishment of temperature-displacement correlation is essential for the displacement-based performance evaluation of long-span bridges. We propose a new method based on long short-term memory (LSTM) neural network for modeling multiple temperature-displacement correlation. The LSTM neural network, which can describe the time-lag effect and is suitable for processing ultralong data sequences, is adopted as the basic neural network. We use the adaptive moment estimation method to optimize the LSTM neural network, and introduce the dropout regularization technique to improve the generalization ability of the model. The predominant thermal variables that affect the vertical displacement in the main girder of the bridge are extracted using long-term synchronous monitoring data of temperatures and displacements obtained from a three-span continuous tied arch bridge. An LSTM neural network between multiple temperature variables and displacements is established. A back propagation (BP) neural network is also modeled for comparison. The results show that the structural effective temperature has a significant nonlinear relationship with the vertical displacement in the main girder, while the temperature difference among structural components and the temperature gradient in the main arch rib have a linear correlation with the vertical displacement in the main girder. The effective temperature in the main arch rib and the temperature difference between the main girder and the main arch rib are the predominant thermal variables that result in the vertical displacement in the main girder. Compared with the BP neural network model, the LSTM neural network model proposed in this paper can dramatically reduce the reproduction error and prediction error of the thermal displacement.

Clock bias prediction algorithm for navigation satellites based on a supervised learning long short-term memory neural network

In a satellite navigation system, high-precision prediction of satellite clock bias directly determines the accuracy of navigation, positioning, and time synchronization and is the key to realizing autonomous navigation. To further improve satellite clock bias prediction accuracy, we establish a satellite clock bias prediction model by using long short-term memory (LSTM) that can accurately express the nonlinear characteristics of the navigation satellite clock bias. Outliers in the original clock bias should be preprocessed before using the clock bias for prediction. By analyzing the working principle of the traditional median absolute deviations method, the ambiguity of the mathematical model of that method was improved. Experimental results show that the improved method is better than the traditional method at detecting gross errors. The single difference sequence of the preprocessed satellite clock bias was taken as the research object. First, a quadratic polynomial model was fit to the trend term of the single difference sequence. Second, based on the LSTM neural network model and the basic principles of supervised learning, a supervised learning LSTM network model (SL-LSTM) was proposed that models cyclic and random terms. Finally, the prediction function of the satellite clock bias was realized by extrapolating the model by adding a trend term. We adopt the GPS precision satellite clock bias of International GNSS Service data forecast experiments and apply wavelet neural network (WNN), autoregressive integrated moving average (ARIMA), and quadratic polynomial (QP) models to compare their prediction effects. The average prediction RMSE for 3 h, 6 h, 12 h, 1 d, and 3 d based on the SL-LSTM improved by approximately −21.80, −1.85, 8.57, 2.27, and 40.79%, respectively, compared with the results of the WNN. The average prediction RMSE based on the SL-LSTM improved by approximately 38.23, 65.48, 80.22, 85.18, and 94.51% compared with the ARIMA results. The average prediction RMSE based on the SL-LSTM improved by approximately 82.37, 75.88, 67.24, 45.71, and 58.22% compared with the QP results. Compared with the WNN, the SL-LSTM method has no obvious advantages in the prediction accuracy and stability in short-term prediction but achieves a better long-term prediction accuracy and stability. With an increased prediction duration, the SL-LSTM method is clearly better than the other three methods in terms of the prediction accuracy and stability. The results indicated that the quality of satellite clock bias prediction by the SL-LSTM method is better than that of the above three methods and is more suitable for the middle- and long-term prediction of satellite clock bias.

Prediction of dam horizontal displacement based on CNN-LSTM and attention mechanism

Aiming at the characteristics of dam safety monitoring data sequence with few samples, short sequence and nonlinearity, a dam horizontal displacement prediction method based on attention mechanism, convolutional neural network (CNN) and long short-term memory (LSTM) is proposed. This method can reduce the loss of historical information and improve the prediction accuracy. First, the missing values are supplemented by linear interpolation to improve the integrity of the data. Then the abstract feature data extracted by CNN is mapped to the predicted value of LSTM, and then optimized through attention mechanism. Finally, the model is trained and verified with the monitoring data of a concrete gravity dam in Chongqing as a sample. Experimental results show that the root mean square error (RMSE), mean absolute percentage error (MAPE) and fit (R2) of the CNN-LSTM hybrid model based on attention mechanism are 0.3882, 0.7121% and 0.9543, respectively. The prediction accuracy of the new model is better than the CNN-LSTM model and the LSTM neural network model.

Remote NOx emission prediction model based on LSTM neural network

Test results from many researchers show that NOx emission from many on-broad heavy-duty diesel vehicles is higher than which been registered. Therefore, CN_VI emission regulations clearly proposes that the heavy-duty diesel vehicles should be supervised by a T-BOX which can transmit CAN message from vehicle OBD interface to the remote monitoring platform. Based on the formation mechanism of NOx emission and the variety of OBD data flow, the LSTM (Long Short-Term Memory) neural network model inputs such as engine speed, torque, atmospheric pressure, coolant temperature, fuel consumption rate and intake air mass flow are selected by using partial least square method (PLS). 19877 groups of data from engine test results were used for model training and verification, the root mean square error of training and test are RTR = 29.7 × 10-6 and RTE = 19.9 × 10-6,with a high prediction accuracy which can fully meet the requirements of the SCR system DeNOx performance diagnosis module in the OBD remote monitoring system.

High-Accuracy Remote Sensing Water Depth Retrieval for Coral Islands and Reefs Based on LSTM Neural Network

Li, X.M.; Ma, Y.; Leng, Z.H.; Zhang, J., and Lu, X.X., 2020. High-accuracy remote sensing water depth retrieval for coral islands and reefs based on LSTM neural network. In: Jung, H.-S.; Lee, S.; Ryu, J.-H., and Cui, T. (eds.), Advances in Geospatial Research of Coastal Environments. Journal of Coastal Research, Special Issue No. 102, pp. 21-32. Coconut Creek (Florida), ISSN 0749-0208.Accurate water depth data are essential to ensure navigational safety of ships operating in regions with coral islands and reefs; however, it is often difficult or impossible to conduct in situ bathymetric surveys in such areas. In this study, a Long-Short Term Memory (LSTM) neural network in deep learning was introduced for multispectral remote sensing detection of water depth, and an LSTM neural network model suitable for remote sensing water depth retrieval for coral islands and reefs was developed. The LSTM model retrieval result was optimal when using an Adam optimizer, batch size of 10 %, 2000 epochs and a 50/100 network structure. Compared with the classical Log-linear, Stumpf and improved Stumpf models, the LSTM model demonstrated better water depth retrieval capability with minimum mean absolute errors (MAEs) and mean relative errors (MREs) irrespective of whether using 300 or 2000 training points. The retrieval accuracy of the LSTM model with only 300 training points was better than the other three models with 2000 training points. Compared with SVR model, MRE of LSTM model reduced 19.03 % and 4.14 % respectively when using 300 and 2000 training points. Analysis of the Dong Island case showed that the water depth retrieval results based on the LSTM model clearly reflected the geomorphic units of the entire reef such as the reef flat, front slope (seaward slope), and patch reef. It also revealed subtle geomorphologic features of the reef flat surface and the valley system. The LSTM model demonstrated satisfactory performance for water depths of 5–25 m, with average MREs of 7 % and 9 % (minimum: 4.02 %, maximum: 14.62 %). These findings verify the application performance of the LSTM remote sensing water depth retrieval model both qualitatively and quantitatively.

Realtime prediction of dynamic mooring lines responses with LSTM neural network model

A Long Short-Term Memory (LSTM) neural network model is developed to provide a real-time calculation tool for monitoring the mooring line responses under the operating condition by using the vessel motion as the only input. A feature extraction method based on first order moment and second order center moment is proposed in the data pre-processing to improve the characteristics of fluctuation information of input data. The effective judging standard of the scope of prediction accuracy regarding the proposed neural network model is defined. The impact of normalization method, neural number, and training sets length on the predicting accuracy are studied. The feasibility of the established LSTM neural network model is verified considering different data relevance between the training sets and validating sets. The results indicate that the mooring line responses can be predicted with high precision based on the vessel motion as input by using the established LSTM neural network model.