Short-term Memory Method Research Articles

Time series analysis model for forecasting unsteady electric load in buildings

Accurate and reliable load forecasting is crucial for ensuring the security and stability of the power grid. This paper proposes a combined prediction method based on Empirical Wavelet Transform (EWT) and Autoformer time series prediction model for the non-stationary and non-linear time series of electric load. The original sequence is first decomposed by EWT to obtain a set of stable subsequences, and then the Autoformer time series prediction model is used to predict each subsequence. Finally, the prediction results of each subsequence are combined to obtain the final prediction results. The proposed EWT-Autoformer prediction model is applied to an electric load example, and the experimental results are compared with the Recurrent Neural Network (RNN) method, Long Short-Term Memory (LSTM) method, and Informer method under the same conditions. The experimental results indicate that compared to LSTM, the method proposed in the paper has an R2 improvement of 9–20 percentage points, an improvement of 6–8 percentage points compared to RNN, an improvement of 3–7 percentage points compared to Informer, and an improvement of 2–3 percentage points compared to Autoformer. In addition, the RMSE and MAE are also significantly lower than other models.

Network traffic classification model based on attention mechanism and spatiotemporal features

Traffic classification is widely used in network security and network management. Early studies have mainly focused on mapping network traffic to different unencrypted applications, but little research has been done on network traffic classification of encrypted applications, especially the underlying traffic of encrypted applications. To address the above issues, this paper proposes a network encryption traffic classification model that combines attention mechanisms and spatiotemporal features. The model firstly uses the long short-term memory (LSTM) method to analyze continuous network flows and find the temporal correlation features between these network flows. Secondly, the convolutional neural network (CNN) method is used to extract the high-order spatial features of the network flow, and then, the squeeze and excitation (SE) module is used to weight and redistribute the high-order spatial features to obtain the key spatial features of the network flow. Finally, through the above three stages of training and learning, fast classification of network flows is achieved. The main advantages of this model are as follows: (1) the mapping relationship between network flow and label is automatically constructed by the model without manual intervention and decision by network features, (2) it has strong generalization ability and can quickly adapt to different network traffic datasets, and (3) it can handle encrypted applications and their underlying traffic with high accuracy. The experimental results show that the model can be applied to classify network traffic of encrypted and unencrypted applications at the same time, especially the classification accuracy of the underlying traffic of encrypted applications is improved. In most cases, the accuracy generally exceeds 90%.

Improving solar radiation source efficiency using adaptive dynamic squirrel search optimization algorithm and long short-term memory

Artificial intelligence and machine learning are used to optimize the design parameters of renewable energy sources, which are now regarded as vital components in current clean energy sources. As a result, system requirements can be reduced, and a well-designed system can improve performance. Artificial intelligence approaches in renewable energy sources and system design would significantly cut optimization time while maintaining high modeling accuracy and optimum performance. This study examines machine learning in depth, emphasizing how it can be used in developing renewable energy sources because of the vast range of technologies it can use. This paper approximates the hourly tilted solar irradiation using climate factors. The irradiance is estimated using a hybrid ensemble-learning approach. This approach combines a proposed adaptive dynamic squirrel search optimization algorithm (ADSSOA) with long short-term memory (LSTM) methods. To the best of our knowledge, this combination has not been used for solar radiation. The results are analyzed and contrasted with the outcomes of several recent swarm intelligence algorithms, such as the genetic algorithm, particle swarm optimization, and gray wolf optimizer. The binary ADSSOA approach performed as expected, with an average error of 0.1801 and a standard deviation of 0.0656. The ADSSOA–LSTM model had the lowest root mean square error (0.000388) compared to LSTM’s (0.001221). In addition, the statistical analysis uses 10 iterations of each presented and evaluated method to provide accurate comparisons and reliable results.

The Utilization of Deep Learning in Forecasting The Inflation Rate of Education Costs in Malang

The public needs information about the predicted inflation rate for education costs to manage family finances and prepare education funds. This information is also beneficial for the government to create policies in education. Malang is one of the educational cities in Indonesia, but research on the prediction of the inflation rate of education costs in the city still needs to be made available. Besides, the researchers have yet to find previous studies on forecasting that used the specific inflation rate for education costs in Indonesia by applying the Deep Learning method, especially those using the Consumer Price Index (CPI) data for the Education Expenditure Group. This research aims to develop a model to forecast the inflation of education costs in Malang using the Deep Learning Method. This research was conducted using Consumer Price Index (CPI) data for the Education Expenditure Group in Malang during 1996-2021s taken from the Central Bureau of Statistics (BPS) Malang. The forecasting method used is the Long and Short-Term Memory (LSTM) method, which is a development of the Recurrent Neural Network (RNN). The results showed that it achieved the best accuracy by a model with one hidden layer and four hidden nodes, namely MAPE=2.8765% and RMSE=8.37.

A STOCK PREDICTION SYSTEM USING TEKNIKAL INDICATORS WITH THE LSTM METHOD

The capital market industry in Indonesia is developing in a better direction so that the growth of new investors is also increasing. Until the end of February 2021, operational data from the Indonesian Stock Exchange (IDX) and data from the Indonesian Central Securities Depository (KSEI) recorded that the number of new capital market investors had increased by 16.35% or 634,350 investors, from the previous 3,880,753 investors. to 4,515,103 investors. The development of the capital market industry in Indonesia, which has increased investor interest in investing, is expected to mobilize public funds to support national economic development. Some companies that are familiar to the community are BCA, BNI, BRI and MANDIRI. This study attempts to forecast banking stock prices on the LQ45 index, using the Long Short-Term Memory (LSTM) method. LSTM is one of the Recurrent Neural Networks (RNN) which has good accuracy in predictions. The identified fields are Close, Open, RSI, MACD and MA. The evaluation method used in this prediction system is MAPE in the form of percent output.

Analyze News Effect on Trend Stock Price in Indonesia Based on Bidirectional-Long Short Term Memory

News platforms such as BBC, UN News, and CNN are news sites that are very global, both globally and nationally. With this site, someone can find information in other countries or their country. The news contained on the BBC website can be analyzed using sentiment analysis. Sentiment analysis is carried out to see whether the news tends to be positive, negative, or neutral so that researchers or institutions can find out how the response of the news is to other sectors such as stocks in Indonesia. With the IDX website as a list of company shares in Indonesia, sentiment analysis can be carried out on news on the BBC website that can affect the rise or fall of stock prices in Indonesia using a combination of Word2Vec and the Bidirectional- Long Short Term Memory (BiLSTM) method. The BiLSTM method is an algorithm that has a function to process text data to predict the value of stock price trends by utilizing Word2Vec for word embedding of news. In this study, the dataset used is international news on the BBC website and historical stock prices of several companies on the IDX website. This study utilizes both methods to be able to predict stock price trends. By using 15.674 data, this study shows that the BiLSTM method has an average accuracy rate of 80.03%.

Penerapan Metode Principal Component Analysis (PCA) dan Long Short-Term Memory (LSTM) dalam Memprediksi Prediksi Curah Hujan Harian

Since the last three years North Luwu has experienced frequent hydrological disasters in the form of floods and landslides. The disaster had a negative impact on the availability of clean water, failed to plant and even tended to reduce the quality of the harvest. Cocoa is one of the leading commodities of North Luwu Regency whose productivity has decreased due to the impact of climate change so that it will affect the sustainability of the local population's income. Therefore, the purpose of this research is to anticipate rainfall that will occur to prevent or reduce the risk of failure and loss. Principal Component Analysis (PCA) Method is used as feature extraction to find out the most influential variables and the Long Short-Term Memory (LSTM) method is used as a prediction method. Future rainfall is predicted using meteorological variables such as pressure, evaporation, maximum temperature, average humidity, and sunshine duration from 1 January 2017 to 30 September 2022. Based on the PCA results, 4 variables are obtained that have the most influence on rainfall, namely: variable evaporation, maximum temperature, average humidity, and length of sunlight. These variables are used as input to predict rainfall using LSTM. In this study using trial parameters, namely the number of hidden, batch size, and learn rate drop period. The best prediction results were obtained for MAPE of 0.0018 with the number of hidden, batch size and learn rate drop periods of 100, 32, and 50 respectively. The prediction results show very heavy rainfall occurring on August 28, 2021 of 101.9734 mm, 21 September 2021 of 108.6528 mm, and 5 April 2022 of 116.5510 mm. In this study PCA was able to increase accuracy in considering all parameters and choosing the most effective.

Enhancing deep learning for the comprehensive forecast model in flue gas desulfurization systems

The sulfur dioxide (SO2) emissions contribute to severe environmental pollution and respiratory problems, which make the limestone–gypsum wet flue gas desulfurization (WFGD) an increasingly important issue. The WFGD system consists of the flue gas system, the limestone slurry supply system, the oxidation air supply system, and the gypsum slurry discharge system, which have the characteristics of multivariable coupling, high dimension, nonlinear, large delay, and difficulty for parameters measurement, bringing great challenges and limitations for the traditional mechanism modeling of the WFGD system. To solve those issues above, in this study, an enhancing modeling method that integrates feature fusion and deep learning is proposed to predict the dynamics of SO2 emission in a WFGD system. Aiming at the problem that most of the existing modeling methods cannot fully and accurately extract the hidden features in WFGD system data, the convolutional neural network (CNN) method is responsible for extracting multiple spatial features including the local and global spatial features from WFGD system data, and the long short-term memory (LSTM) method mines the temporal relationship from the previous and current state of the WFGD system. Finally, the attention mechanism (AM) method is enhanced for the effective improvement of crucial information. Taking an actual coal-fired boiler as an illustration, the SO2 emissions at different time steps are predicted. The experiment results reveal that the proposed CNN-LSTM-AM model can fully and accurately extract system features and outperform other approaches in terms of prediction accuracy.

Predicting biomechanical properties of additively manufactured polydopamine coated poly lactic acid bone plates using deep learning

Fused Filament Fabrication based bone plates lack mechanical strength, resulting in premature failure. Biocompatible Polydopamine (PDM) coating forms covalent bonds with Poly Lactic Acid (PLA), resulting in enhanced mechanical characteristics. Infill density, submersion time, shaker speed, and coating solution concentration, have a significant effect on the mechanical properties of coated bone plates. Monitoring the mechanical strength experimentally for each set of process parameters is a time-consuming and tedious procedure. Before resorting to experimental tests, it is important to model and predict the mechanical strength in order to increase the mechanical strength of bone plates. Predictive modeling using machine learning approaches has proven to be the best alternative to traditional statistical tools. In this study, various machine learning algorithms, including Random Forest, k-Nearest Neighbors, AdaBoost, and Decision Trees, and Long Short-Term Memory (LSTM) method, were utilized to predict tensile and flexural strengths. Error metrics consisting of Mean Square Error (MSE), Root Mean Square Error (RMSE), Correlation Coefficient (R2), Mean Absolute Percentage Error (MAPE) and RRMSE (Root Relative Mean Squared Error) have been utilized to assess the performance of various models. Owing to the prediction findings, the LSTM model beat all ML-based models by demonstrating the best MSE, RMSE, and R2 values for tensile and flexural strength, which were 5.96 MPa, 2.44 MPa, and 0.9169, respectively, and 11.14 MPa, 3.34 MPa, and 0.9242, respectively. Consequently, the results of this study suggest that LSTM is the best model for predicting the tensile and flexural strength of PDM-coated PLA bone plates without doing experiments.

A new deep learning method for meteorological drought estimation based-on standard precipitation evapotranspiration index

In this study, we studied drought parameter estimation based on the standard precipitation evapotranspiration index (SPEI), for four stations located at different places in Türkiye. We proposed a new hybrid deep structure of convolutional neural network (CNN) and long short-term memory (LSTM) methods and compared it with several other existing methods. Our extensive experimental results obtained in different cases of the station, time scale, and estimator show that better estimation results are often obtained with 12-month time scale for all methods. For instance using SPEI-9 values for Şanlıurfa station, the CNN method achieves 0.57 RMSE, 0.24 MADE, 0.11 MAE, and 99.38% R2, whereas the proposed adaptive CNN-LSTM obtains values of 0.42, 0.18, 0.04, and 99.69% for the same metrics, respectively. Also, using SPEI-6 values for Mardin station, the bidirectional-LSTM method achieves 0.72 RMSE, 0.35 MADE, 0.22 MAE, and 98.94% R2, while the proposed adaptive CNN-LSTM obtains values of 0.32, 0.12, 0.02, and 99.53%, respectively In this means, the proposed method is superior to existing methods in all cases in terms of almost all performance metrics. It is also robust to changes in given input time series data coming from different stations. We also investigated the percentage occurrence of drought categories at different time scales. The results of this analysis also show that the highest percentages are seen in the wet category for all stations. The percentages occurrence of drought wet category are 48.3, 50.07, 50.14, and 50.79 at SPEI-3, SPEI-6, SPEI-9, and SPEI-12 time scales, respectively.

Quantification of the meteorological and hydrological droughts links over various regions of Iran using gridded datasets.

Drought is a gradual phenomenon that occurs slowly and directly impacts human life and agricultural products. Due to its significant damage, comprehensive studies must be conducted on drought events. This research employs precipitation and temperature from a satellite-based gridded dataset (i.e., NASA-POWER) and runoff from an observation-based gridded dataset (i.e., GRUN) to calculate hydrological and meteorological gical droughts in Iran during 1981-2014 based on the Standardised Precipitation-Evapotranspiration Index (SPEI) and Hydrological Drought Index (SSI) indices, respectively. In addition, the relationship between the meteorological and hydrological droughts is assessed over various regions of Iran. Afterward, this study employed the Long Short-Term Memory (LSTM) method to predict the hydrological drought based on the meteorological drought over the northwest region of Iran. Results show that hydrological droughts are less dependent on precipitation in the northern regions and the coastal strip of the Caspian Sea. These regions also have a poor correlation between meteorological and hydrological droughts. The correlation between hydrological and meteorological drought in this region is 0.44, the lowest value among the studied regions. Also, on the margins of the Persian Gulf and southwestern Iran, meteorological droughts affect hydrological droughts for 4months. Besides, except the central plateau, most regions experienced meteorological and hydrological droughts in the spring. The correlation between droughts in the center of the Iranian plateau, which has a hot climate, is less than 0.2. The correlation between these two droughts in the spring is stronger than in other seasons (CC = 0.6). Also, this season is more prone to drought than other seasons. In general, hydrological droughts occurred one to two months after the meteorological drought in most regions of Iran. LSTM model for northwest Iran showed that the predicted values had a high correlation with the observed values, and their RMSE was less than 1 in this region. CC, RMSE, NSE, and R-square of the LSTM model are 0.7, 0.55, 0.44, and 0.6, respectively. Overall, these results can be used to manage water resources and allocate water downstream to deal with hydrological droughts.

South African inflation modelling using bootstrapped long short-term memory methods.

Inflation is a critical economic series, and proper targeting is required for a stable economy. With the current economic conditions that the world has faced as a result of COVID-19, understanding the effects of this on economies is critical because it will guide policies. Recent research on South African inflation has focused on statistical modelling, specifically the ARFIMA, GARCH, and GJR-GARCH models. In this study, we extend this into deep learning and use the MSE, RMSE, RSMPE, MAE, and MAPE to assess performance. To test which model has better forecasts, we use the Diebold-Mariano test. According to the findings of this study, clustered bootstrap LSTM models outperform the previously used ARFIMA-GARCH and ARFIMA-GJR-GARCH models.

Sentiment analysis based offensive language identification system for code-mixed data

Hate speech identification is the process of categorising textual information into hate and non- hate speech with the goal of identifying hate speech's targeted features. The objective of this research work is to take the Dataset from FIRE 2021 shared task code mixed data that includes YouTube comments and Twitter conversations and to detect whether the messages represents the offensive or non-offensive category. To detect the offensive language sentences, various deep learning models like Long Short-Term Memory, Bidirectional Long Short-Term Memory, Convolutional Neural Network, Gated Recurrent Unit, and hybrid model like Convolutional Neural Network with Bidirectional Long Short-Term Memory methods were utilised in this research work. The performance of all the mentioned models is evaluated using precision, recall, F1-score, and accuracy. Out of all the models, both LSTM and GRU models perform better with the accuracy of 0.81, precision of 0.85 and recall of 0.95.

Using sequence-to-sequence LSTM to predict RNA virus mutations

Infection with viruses is one of the main causes of human illness and even death in today's society. Vaccination can help people fight the virus. However, virus mutation will always cause vaccines to fail. Predicting the mutations of viral could help detect the mutated virus early and develop new vaccines so to reduce the rate of infection and death from the virus. The common strategy used to predict virus mutation is determining the important components of the virus, like amino acids and proteins, then using deep learning or machine learning methods to construct models. It is an effective strategy. However, using this strategy always cause people to spend lots of time and money studying the component of the virus, like amino acids, proteins, nucleic acid and so on. To increase efficiency and reduce the cost of research, a new method that can predict virus mutation based on nucleotide sequence alone is something people looking forward to. In recent years, in natural language processing, building a sequence-to-sequence model is becoming a popular and effective method to deal with textual data. As a type of text data, using the idea of sequence to sequence should be good to deal with the RNA sequence. In addition, in terms of the past study of predicting virus mutation, RNA sequence as a kind of time sequence, people generally use long short-term memory (LSTM) methods to handle it. Thus, in this study, we would combine the idea of the sequence-to-sequence model with LSTM method to predict the possible mutation of the virus. This experiment studies the mutation of two typical influenza viruses and achieves encouraging results.

Improving Stock Market Predictions: An Equity Forecasting Scanner Using Long Short-Term Memory Method with Dynamic Indicators for Malaysia Stock Market

Stock market predictions are a challenging problem due to the dynamic and complex nature of financial data. This study proposes an approach that integrates the domain knowledge of investors with a long-short-term memory (LSTM) algorithm for predicting stock prices. The proposed approach involves collecting data from investors in the form of technical indicators and using them as input for the LSTM model. The model is then trained and tested using a dataset of 100 stocks. The accuracy of the model is evaluated using various metrics, including the average prediction accuracy, average cumulative return, Sharpe ratio, and maximum drawdown. The results are compared to the performance of other strategies, including the random selection of technical indicators. The simulation results demonstrate that the proposed model outperforms the other strategies in terms of accuracy and performance in a 100-stock investment simulation, highlighting the potential of integrating investor domain knowledge with machine learning algorithms for stock price prediction.

Attention-mechanism based DiPLS-LSTM and its application in industrial process time series big data prediction

Big data and time series are typical features of modern industrial process data. Effective time series modeling methods are required for ensuring the normal and stable operation of industrial processes. However, the existing time series prediction methods usually cannot take important historical data features long ago into consideration. In this paper, we proposed a new attention-mechanism based dynamic-inner partial least squares long short-term memory (ADiPLS-LSTM) method to predict the time series data. First, the DiPLS method is applied to extract the dynamic features in the selected data group, and the attention-mechanism is used then to define the importance of the responded features. The product of the extracted dynamic features and the attention-mechanism results is regarded as the input of LSTM to predict the future time information. The proposed method uses both the recent data information and important data information in a long run, which helps to obtain more accurate prediction results. The effectiveness of the ADiPLS-LSTM method is verified by the data of a 660MW coal-fired boiler. Compared with BPNN, LSTM, and DiPLS-LSTM, the proposed method also advances in time series data prediction and interpretability.

Decoding the physiological response of plants to stress using deep learning for forecasting crop loss due to abiotic, biotic, and climatic variables

This paper presents a simple method for detecting both biotic and abiotic stress in plants. Stress levels are measured based on the increase in nutrient uptake by plants as a mechanism of self-defense when under stress. A continuous electrical resistance measurement was used to estimate the rate of change of nutrients in agarose as the growth medium for Cicer arietinum (Chickpea) seeds. To determine the concentration of charge carriers in the growth medium, Drude’s model was used. For identifying anomalies and forecasting plant stress, two experiments were conducted and outliers were found in electrical resistance and relative changes in carrier concentration. Anomaly in the first iteration was detected by applying k-Nearest Neighbour, One Class Support Vector Machine and Local Outlier Factor in unsupervised mode on electrical resistance data. In the second iteration, the neural network-based Long Short Term Memory method was used on the relative change in the carrier concentration data. As a result of the change in resistance of growth media during stress, nutrient concentrations shifted by 35%, as previously reported. Farmers who cater to small communities around them and are most affected by local and global stress factors can use this method of forecasting.

An applicability test of the conventional and neural network methods to map the overall water quality of the Caspian Sea

This study investigates the water quality of the Caspian Sea by examining the presence of nutrients and heavy metals in the water. Water samples were collected from 22 stations and analyzed for nutrient and heavy metal levels. The study used the fuzzy method to prepare water quality maps and employed ANNs methods to predict microbial contamination for future years. The results revealed that the western and northwestern parts of the region had higher nutrient levels (about 40.2 % of the region), while the eastern and northeastern shores were highly polluted due to increased urbanization (about 70.1 % of the region). The long short-term memory (LSTM) method was found to have the highest accuracy compared to other ANNs methods and indicated a recent increase in pollution (RWater quality2=0.940, ROECD2=0.950, RTRIX2=0.840). The study recommends targeted research to identify the causes and means of controlling pollution in light of the predicted increase in pollution in the Caspian Sea.

Prediksi Indeks Harga Produsen Pertanian Karet Di Indonesia Menggunakan Metode LSTM

Natural rubber is one of the plantation commodities that has a fairly wide market in international trade because it is needed as a raw material for various industries. Rubber producer prices need to be predicted because producer prices are the first price in the lead from other price levels. So that information about price changes at the producer level is very important as an early warning system against price fluctuations at the next price level. The Long Short-Term Memory (LSTM) algorithm was chosen because it is considered capable of accommodating the problem of predicting the price index of producers in the rubber agriculture sector being faced because, LSTM itself is one of the developments of a neural network, which can be used for time series data modeling and is capable of continuous learning. . Parameter analysis carried out in this study is the number of hidden neurons, epochs and batch size. The best combination of parameters produced in this study is 50 hidden neurons, 25 epochs and batch size 10. The best values ​​generated in this study are the RMSE value of training data 384.20 and the value of RMSE testing 306.01 and the value of MAPE training 1.25% and the value of MAPE testing 1.09% The best MAPE error calculation in this study is "Predicting the Price Index of Agricultural Rubber Producers in Indonesia Using the Long Short Term Memory Method" which produces the best MAPE. These results indicate that the MAPE error can be said to be very good because the best MAPE value produced is below 10%.

Predicting the 25th and 26th solar cycles using the long short-term memory method

Abstract Solar activities directly or indirectly affect space missions, geophysical environment, space climate, and human activities. We used the long short-term memory (LSTM) deep learning method to predict the amplitude and peak time of solar cycles (SCs) 25 and 26 by using the monthly relative sunspot number data taken from the National Astronomical Observatory of Japan (NAOJ). The dataset is divided into eight schemes of two to nine slices for training, showing that the five-slice LSTM model with root mean square error of 11.38 is the optimal model. According to the prediction, SC 25 will be about 21$\%$ stronger than SC 24, with a peak of 135.2 occurring in 2024 April. SC 26 will be similar to SC 25 and reach its peak of 135.0 in 2035 January. Our analysis results indicate that the sunspot data from NAOJ is highly credible and comparable.