LSTM RNN Research Articles

Developing Semantic Textual Similarity for Guragigna Language Using Deep Learning Approach

Semantic Similarity is one of the highest levels of NLP. STS has significant advantages in NLP applications like information retrieval, information extraction, text summarization, data mining, machine translation, and other tasks. This research aims to present a deep learning approach for capturing semantic textual similarity (STS) in the Guragigna. The methodology involves collecting a Guragigna language corpus and preprocessing the text data and text representation is done using the Universal Sentence Encoder (USE), along with word embedding techniques including Word2Vec and GloVe and mean Square Error (MSE) is used to measure the performance. In the experimentation phase, models like LSTM, Bidirectional RNN, GRU, and Stacked RNN are trained and evaluated using different embedding techniques. The results demonstrate the efficacy of the developed models in capturing semantic textual similarity in the Guragigna language. Across different embedding techniques, including Word2Vec, GloVe, and USE, the Bidirectional RNN model with USE embedding achieves the lowest MSE of 0.0950 and the highest accuracy of 0.9244. GloVe and Word2Vec embedding also show competitive performance with slightly higher MSE and lower accuracy. The Universal Sentence Encoder consistently emerges as the top-performing embedding across all RNN architectures. The research results demonstrate the effectiveness of LSTM, GRU, Bi RNN, and Stacked RNN models in measuring semantic textual similarity in the Guragigna language.

Temperature and precipitation projection in the lower Mahanadi Basin through machine learning methods

This study examined climate change dynamics in the lower Mahanadi River basin by integrating observed and climate model data. Historical precipitation and temperature data (1979–2020) from the India Meteorological Department (IMD) and monthly climate model data from the CORDEX-SMHI-MIROC model via the Earth System Grid Federation (ESGF) are utilized. Four machine learning models (Fbprophet, Holt-Winters, LSTM RNN, and SARIMAX) are applied to forecast precipitation, Tmax, and Tmin, and are compared across different representative concentration pathway (RCP 2.6, 4.5, and 8.5) scenarios. Diverse trajectories emerge, highlighting potential shifts in precipitation and temperature dynamics over near, mid, and far-term intervals. Fbprophet and SARIMAX are identified as superior models through performance evaluation metrics (R2, RMSE, r, P-bias, and NSE). Spatial analysis using ArcGIS and IDW interpolation reveals spatial variations in climate projections, aiding in visualizing future climate trends within the Mahanadi Basin. This study acknowledges limitations such as historical data uncertainties, socio-economic indicators, and unpredictable RCP trajectories, introducing a novel method to integrate machine learning with climate model data for assessing reliability. It also explores anticipated shifts in monthly precipitation and temperature patterns, providing insights into future climate variations.

Short term road network Macroscopic Fundamental Diagram parameters and traffic state prediction based on LSTM

Macroscopic Fundamental Diagram (MFD) is widely used in traffic state evaluation due to its description of the macro level of urban road network. This study focuses on the discrimination and short-term prediction of macro traffic states in urban road networks, using MFD combined with FCM clustering for state partitioning to characterize different macro states of the road network. To predict the MFD state, this paper builds two LSTMs to perform short-term predictions on two important parameters in MFD: road network weighted flow qw and road network weighted density kw. The parameters of the first three statistical intervals and the predicted time period are used as inputs to output the MFD parameters for the predicted time period. To ensure that the two LSTM structures and hyper-parameters settings can achieve the best prediction performance for MFD, the parameter optimization process of both should be included in the same search framework for hyper-parameters search. Therefore, this paper uses GA algorithm combined with multi-objective particle swarm optimization algorithm as the solving algorithm, with the accuracy of solving two MFD parameters and the accuracy of MFD point positioning as the hyper-parameters solving objectives. The study was validated using actual road network data from Hong Kong, and the results showed that the method proposed in this paper has an MRE prediction error of less than 7.8% for the two parameters of MFD, and can predict the future temporal trend of the two parameters, demonstrating the feasibility of MFD related predictions. The model's prediction of the overall shape and change trajectory trend of MFD is consistent with reality, and some test sets in MFD state prediction show high accuracy, the overall accuracy is 81.45%. To verify the effectiveness of the multi-objective search algorithm, typical LSTM models and RNN models were used for comparison. The experiment proved that the model used in this study performed better in error control and state prediction. This study explores and practices a short-term prediction method for road network MFD parameters, MFD status, and their changing trends. Provided path reference for urban road network prediction, traffic control status, and MFD related research.

Modeling of the effect of transportation system accessibility on residential real estate prices: The case of Washington metropolitan area, USA

Deep learning accurate predictions of house prices are essential for prospective homeowners, investors, appraisers, and insurers. However, some studies lack accuracy as they overlook critical factors like accessibility and economic attributes that influence house prices. This paper aims to predict house prices by considering structural, locational, accessibility, and economic attributes, while also exploring the effect of accessibility on housing prices. The dataset contains 2,019,663 real estate transaction records from 1975 to 2018 in the Washington metropolitan area, obtained from the Zillow website. In this study, the accessibility index is calculated using Distance, Cumulative Opportunities, and Gravity measures, with the gravity measure surpassing others due to its consideration of both land use and transportation aspects. Economic attributes are then utilized to predict the average monthly house price using deep learning algorithms such as LSTM, GRU, and Simple RNN, with the Simple RNN demonstrating superior performance. Following the amalgamation of structural and locational attributes with the accessibility index and average house prices, various machine learning algorithms—including Linear Regression, Lasso, Ridge, Random Forest, GBM, LightGBM, XGBoost, Decision Tree, AdaBoost, Artificial Neural Network, and Stacked Generalization—are employed for prediction. Subsequent evaluation reveals that Stacked Generalization (ANN + LightGBM) provides the best performance, with an R-squared value of 0.96 and RMSE of $23,290. Moreover, this paper identifies accessibility index thresholds (80,003 for large buildings and 160,103 for small buildings) and demonstrates that a higher accessibility index leads to lower housing prices, attributed to noise pollution, decreased privacy, and increased supply responses.

Deep Learning for Dynamic Modeling and Coded Information Storage of Vector‐Soliton Pulsations in Mode‐Locked Fiber Lasers

AbstractSoliton pulsations are ubiquitous feature of non‐stationary soliton dynamics in mode‐locked lasers and many other physical systems. To overcome difficulties related to a huge amount of necessary computations and low efficiency of traditional numerical methods in modeling the evolution of non‐stationary solitons, a two‐parallel bidirectional long short‐term memory recurrent neural network (TP‐Bi_LSTM RNN) is proposed, with the main objective to predict dynamics of vector‐soliton pulsations (VSPs) in various complex states, whose real‐time dynamics is verified by experiments. For two examples, viz., single‐ and bi‐periodic VSPs, with period‐21 and a combination of period‐3 and period‐43, the prediction results are better than provided by direct simulations – namely, deviations produced by the TP‐Bi_LSTM RNN results are 36% and 18% less than those provided by the simulations, respectively. This means that predicted results provided by the neural network are better than numerical simulations. Moreover, the prediction results for unstable VSP state with period‐9 indicate that the optimization of training sets and the number of training iterations are particularly important for the predictability. Besides, the scheme of coded information storage based on the TP‐Bi_LSTM RNN, instead of actual pulse signals, is realized too. The findings offer new applications of deep learning to ultrafast optics and information storage.

LSTM RNN-based excitation force prediction for the real-time control of wave energy converters

Wave energy is a type of abundant and dense renewable energy. Wave force prediction is a critical technology that influences power absorption efficiency in the real-time control of wave energy converter (WEC). Could wave elevation be used to predict wave excitation force directly by training artificial neural network? This method results in rapid and suitable prediction for real-time control. A long short-term memory recurrent neural network (LSTM RNN) algorithm is introduced to identify characteristics of wave excitation forces based on wave elevations. In this method, the wave elevations in front of the structure are measured to obtain sufficient time to actuate the control manipulation. A total of 180 regular wave and 12 irregular wave tests are conducted, and the LSTM RNN model is trained based on the experimental results. The performance of the LSTM algorithm is verified. According to the regular cases in the study, the LSTM prediction can identify high-order harmonic loads, and the anti-noise capability of the LSTM algorithm can filter random noises from the measure signals. In the irregular cases, the LSTM RNN algorithm performs effectively to predict the wave force excited on the structure using wave elevations measured by wave probes. The best combinations of the test setting parameters are determined to guide experimental tests and WEC prototypes.

Sentiment-Driven Exchange Rate Forecasting: Integrating Twitter Analysis with Economic Indicators

Abstract This study focuses on predicting the USD/TL exchange rate by integrating sentiment analysis from Twitter with traditional economic indicators. With the dynamic nature of global finance, accurate exchange rate forecasting is crucial for financial planning and risk management. While economic indicators have traditionally been used for this purpose, the increasing influence of public sentiment, particularly on digital platforms like Twitter, has prompted the exploration of sentiment analysis as a complementary tool. Our research aims to evaluate the effectiveness of combining sentiment analysis with economic indicators in predicting the USD/TL exchange rate. We employ machine learning techniques, including LSTM Neural Network, xgboost, and RNN, to analyze Twitter data containing keywords related to the Turkish economy alongside TL/USD exchange rate data. Our findings demonstrate that integrating sentiment analysis from Twitter enhances the predictive accuracy of exchange rate movements. This study contributes to the evolving landscape of financial forecasting by highlighting the significance of sentiment analysis in exchange rate prediction and providing insights into its potential applications in financial decision-making processes. JEL classification numbers: C53, F31, E60. Keywords: Twitter narratives, LSTM, XGBoost, RNN, USD/TL FX rate, Narrative economics.

Analysis of Website Traffic Time Series Forecasting using ARIMA, Prophet, and LSTM RNN

Analysis of Website Traffic Time Series Forecasting using ARIMA, Prophet, and LSTM RNN

Traditional Chinese medicine diagnostic prediction model for holistic syndrome differentiation based on deep learning

BackgroundWith the development of traditional Chinese medicine (TCM) syndrome knowledge accumulation and artificial intelligence (AI), this study proposes a holistic TCM syndrome differentiation model for the classification prediction of multiple TCM syndromes based on deep learning and accelerates the construction of modern foundational TCM equipment. MethodsWe searched publicly available TCM guidelines and textbooks for expert knowledge and validated these sources using ten-fold cross-validation. Based on the Bidirectional Encoder Representations from Transformers (BERT) and Convolutional Neural Networks (CNN) models, with the classification constraints from TCM holistic syndrome differentiation, the TCM-BERT-CNN model was constructed, which completes the end-to-end TCM holistic syndrome text classification task through symptom input and syndrome output. We assessed the performance of the model using precision, recall, and F1 scores as evaluation metrics. ResultsThe TCM-BERT-CNN model had a higher precision (0.926), recall (0.9238), and F1 score (0.9247) than the BERT, TextCNN, Long Short-Term Memory (LSTM) of Recurrent Neural Network (RNN), and attention mechanism-based LSTM (LSTM-attention) models and achieved superior results in model performance and predictive classification of most TCM syndromes. Symptom feature visualization demonstrated that the TCM-BERT-CNN model can effectively identify the correlation and characteristics of symptoms in different syndromes with a strong correlation, which conforms to the diagnostic characteristics of TCM syndromes. ConclusionsThe TCM-BERT-CNN model proposed in this study is in accordance with the TCM diagnostic characteristics of holistic syndrome differentiation and can effectively complete diagnostic prediction tasks for various TCM syndromes. The results of this study provide new insights into the development of deep learning models for holistic syndrome differentiation in TCM.

Employing LSTMs for Music Remixing

Abstract: The Main Idea was to generate a mixed music from solo musical instrument. For carrying out this task there are various subtask to be carried from data collection to preprocessing the data to make it compatible with the Long Short Term Memory Recurrent Neural Network (LSTM RNNs) model. So, Music mp3 media was collected from melody loops and each one was trimmed down to 30 seconds. Conversion tool was used for segregating the Music into four different mp3 media file i.e. Bass.mp3, Drums.mp3, piano.mp3 and others.mp3. We prepared different models for accomplishing the task to get good results; therefore we preprocessed and transformed the data every time for the new Model architecture. We started with developing the most basic model architecture using Deep neural network (DNNs) and LSTM RNN, and this model architecture was so basic that it did not captures the dependencies within the music , although results was fine but music it generated does not make any sense. Than we tried experimenting by developing different model architectures which generates sensible music and we developed more complex architectures using LSTM RNNs which have the potential to make good results. But there is one disadvantage of it that it takes very long time to get fully trained. In order to reduce the training time and utilizing our resources properly we divided our model into 4 sub-models each for generating music of different musical instruments , These sub-models were trained in parallel on different Google Colaboratory notebook and their best model weights were stored for Reusing them for inference. These all were put into single notebook for predictions and each of these predictions are then combined to make the desired result

VHXLA: A post-earthquake damage prediction method for high-speed railway track-bridge system using VMD and hybrid neural network

The timely and accurate prediction of post-earthquake damage is critically important for ensuring the safety of high-speed railway track-bridge systems. The study introduces a novel method known as the VHXLA model. This innovative model blends variational mode decomposition (VMD) with a hybrid neural network for predicting non-linear multi-component post-earthquake damage in high-speed railways. The VHXLA model comprises a signal processing module and a deep learning module. The signal processing module utilises VMD decomposition, and the Hilbert transform (HT) to transform two-dimensional seismic signals into four-dimensional complex time–frequency signals, thus simplifying the task of identifying seismic time–frequency characteristics via the neural network. The deep learning module integrates diverse types of neural network components, such as the Xception CNN feature extraction submodule, the LSTM RNN temporal learning submodule, and the multi-head attention mechanism submodule. A Bayesian self-optimisation method is implemented to determine the number of decomposition layers in VMD and select essential hyperparameters. The model's effectiveness is evaluated by predicting the damage of an experimentally validated finite element model, subjecting it to seismic loads, and thus gauging its performance. The results indicated that the signal processing module, based on VMD decomposition, significantly improves the neural network's signal processing capability. In addition, the synergistic integration of different modules in the VHXLA model provides superior prediction accuracy compared to pre-existing damage prediction models. Notably, the prediction accuracy is consistent across different positions of the same predicted component in the high-speed railway track-bridge system.

Machine learning-based optimal crop selection system in smart agriculture

The cultivation of most crops depends upon the regional weather conditions. So, the analysis of the agro-climatic conditions of a zone contributes significantly to deciding the right crop for the right land in the right season to obtain a better yield. Machine learning algorithms facilitate this process to a great extent for better results. In this paper, the authors proposed an ML-based crop selection model based on the weather conditions and soil parameters, collectively. Weather analysis is done using LSTM RNN and the process of crop selection is completed using Random Forest Classifier. This model gives better results for weather prediction in comparison to ANN. With LSTM RNN, the RMSE observed in Min. Temp. prediction is 5.023%, Max. Temp. Prediction is 7.28%, and Rainfall Prediction is 8.24%. In the second phase, the Random Forest Classifier showed 97.235% accuracy for crop selection, 96.437% accuracy in predicting resource dependency, and 97.647 accuracies in giving the appropriate sowing time for the crop. The model construction time taken with a random forest classifier using mentioned data size is 5.34 s. The authors also suggested the future research direction to further improve this work.

Structural-fire responses forecasting via modular AI

This study analyses the structural response of an aluminium reticulated roof structure that is constructed at Sichuan Fire Research Institute (Sichuan, China), and to be tested in fire. The structural fire behaviour under 960 localised fire scenarios is considered first, and then used to construct a database for training a modular artificial intelligence (AI) system for real-time forecasting. The system consists of several AI models, each of which predicts the displacement at a specific monitoring point. These individual predictions are then combined to generate a comprehensive forecast of the global structural-fire behaviour. The individual AI model utilized is a Long Short-Term Memory Recurrent Neural Network (LSTM RNN). The modular design allows different models to be modified or added as needed, making the system flexible and adaptable, and improving the accuracy and reliability of the predictions. The results demonstrate the effectiveness of the modular AI approach in accurately forecasting fire-induced structural collapses as indicated by the sensitivity the local models can have. The key objective of this research is to help to make informed decisions and prioritize efforts to minimize the risk of structural collapse in fire.

Detection of Rice Blast Disease (Magnaporthe grisea) Using Different Machine Learning Techniques

Rice is one of the most important staple food crops in the world. Most Asian countries are dependent on rice and huge quantities of rice are grown every year. However, there are many categories of diseases (e.g., blast) which affect rice production and can ultimately lead to huge financial loss to rice growers. Yield loss due to rice blast disease about 10 to 30 percent annually and under favourable condition, this disease can destroy the rice plant within 15 to 20 days and cause yield loss up to 100%.Therefore to ensure better quality, quantity and better productivity early disease detection should be done so that the right amount of pesticides can be as administered at right time to curb the infection. Nowadays Machine Learning has been integrated into the agriculture sector. The aim of this review paper is to identify which Machine Learning algorithms work best in rice blast disease detection. The algorithms reviewed here include Naive Bayes, LSTM RNN, Random Forest Classifiers, Support Vector Machines, K Means, Decision Tree and Convolutional Neural Networks. This review paper also covers the future scope of improvement of some Machine Learning algorithms like Naive Bayes and Recurrent Neural Networks.

Energy Management Strategy Based on Deep Reinforcement Learning and Speed Prediction for Power‐Split Hybrid Electric Vehicle with Multidimensional Continuous Control

An efficient energy management strategy (EMS) is significant to improve the economy of hybrid electric vehicles (HEVs). Herein, a power‐split HEV model is built and validated against test results, and then the EMS is proposed for this model based on vehicle speed prediction and deep reinforcement learning (DRL) algorithms. The rule‐based local controller and global optimal empirical knowledge are introduced to enhance the convergence speed. It is shown in the results that the twin delayed deep deterministic policy gradient algorithm (TD3) achieves more satisfactory performance on converge speed and energy efficiency. The networks of the DRL algorithm with continuous control update more robustly during iterations, in contrast to the discrete ones. Although the power‐split HEV with lower control dimension can reduce the learning burden for DRL EMS; however, the multidimensional control space shows greater optimization potential. As a result, the equivalent fuel consumption of TD3‐based EMS with multidimensional continuous control differences from the global optimal algorithm only by 4.92%. Herein, it is demonstrated in the results that long short‐term memory recurrent neural network (LSTM RNN) performs better for vehicle speed prediction than classical RNN and BP neural network, and the predictive vehicle speed feature helps improve fuel economy by 0.55%.

A New Software Defined Networks (SDN) in IoTs Based Deep Learning Techniques

In this study, a new software defined networks (SDN) in IoTs based on deep learning techniques was implemented using various types of classifiers such as DNN, CNN, GRU, LSTM RNN and SDN Ryu controller. The system was able to handle high-dimensional and complex data by using NSL-KDD dataset, and was able to detect unknown intrusions that traditional methods may miss. The effectiveness of the proposed model was evaluated by accuracy, precision, recall, F-score, and confusion matrix. Python 3.10 has been used to implementation our system. The proposed system was able to achieve good performance, however, the system's efficacy will be determined by the kind of the data feed and the scale of the issue that is attemped to address. This study highlights the potential of DL-based NIDS with SDN and IoT to detect network intrusions, but also highlights the need for continuous monitoring and updating to ensure that the system remains effective.

Predictive Control of a Heaving Compensation System Based on Machine Learning Prediction Algorithm

Floating structures have become a major part of offshore structure communities as offshore engineering moves from shallow waters to deeper ones. Floating installation ships or platforms are widely used in these engineering operations. Unexpected wave-induced motions affect floating structures, especially in harsh sea conditions. Horizontal motions on the sea surface can be offset by a dynamic positioning system, and heave motions can be controlled by a heave compensation system. Active heave compensation (AHC) systems are applied to control vertical heave motions and improve safety and efficiency. Predictive control based on machine learning prediction algorithms further improves the performance of active heave compensation control systems. This study proposes a predictive control strategy for an active heave compensation system with a machine learning prediction algorithm to minimise the heave motion of crane payload. A predictive active compensation model is presented to verify the proposed predictive control strategy, and proportion–integration–differentiation control with predictive control is adopted. The reliability of back propagation neural network (BPNN) and long short-term memory recurrent neural network (LSTM RNN) prediction algorithms is proven. The influence of the predictive error on compensation performance is analysed by comparing predictive feedforward cases with actual-data feedforward cases. Predictive feedforward control with regular and irregular wave conditions is discussed, and the possible strategies are examined. After implementing the proposed predictive control strategy based on a machine learning algorithm in an active heave compensation system, the heave motion of the payload is reduced considerably. This investigation is expected to contribute to the motion control strategy of floating structures.

Exploration of English speech translation recognition based on the LSTM RNN algorithm

Exploration of English speech translation recognition based on the LSTM RNN algorithm

Classification of benign and malignant subtypes of breast cancer histopathology imaging using hybrid CNN-LSTM based transfer learning

BackgroundGrading of cancer histopathology slides requires more pathologists and expert clinicians as well as it is time consuming to look manually into whole-slide images. Hence, an automated classification of histopathological breast cancer sub-type is useful for clinical diagnosis and therapeutic responses. Recent deep learning methods for medical image analysis suggest the utility of automated radiologic imaging classification for relating disease characteristics or diagnosis and patient stratification.MethodsTo develop a hybrid model using the convolutional neural network (CNN) and the long short-term memory recurrent neural network (LSTM RNN) to classify four benign and four malignant breast cancer subtypes. The proposed CNN-LSTM leveraging on ImageNet uses a transfer learning approach in classifying and predicting four subtypes of each. The proposed model was evaluated on the BreakHis dataset comprises 2480 benign and 5429 malignant cancer images acquired at magnifications of 40×, 100×, 200× and 400×.ResultsThe proposed hybrid CNN-LSTM model was compared with the existing CNN models used for breast histopathological image classification such as VGG-16, ResNet50, and Inception models. All the models were built using three different optimizers such as adaptive moment estimator (Adam), root mean square propagation (RMSProp), and stochastic gradient descent (SGD) optimizers by varying numbers of epochs. From the results, we noticed that the Adam optimizer was the best optimizer with maximum accuracy and minimum model loss for both the training and validation sets. The proposed hybrid CNN-LSTM model showed the highest overall accuracy of 99% for binary classification of benign and malignant cancer, and, whereas, 92.5% for multi-class classifier of benign and malignant cancer subtypes, respectively.ConclusionTo conclude, the proposed transfer learning approach outperformed the state-of-the-art machine and deep learning models in classifying benign and malignant cancer subtypes. The proposed method is feasible in classification of other cancers as well as diseases.

Time-Series Forecasting of Chlorophyll-a in Coastal Areas Using LSTM, GRU and Attention-Based RNN Models

The chlorophyll-a (Chl-a) concentration is commonly considered as the main indicator of phytoplankton biomass in coastal waters. Forecasting and understanding the status of Chl-a is beneficial to coastal ecosystem management and is an important emergency management measure for algae blooms. To obtain accurate predictions, the long short-term memory neural network (LSTM) and gated recurrent unit neural network (GRU) were implemented for Chl-a forecasting, and based on the LSTM and GRU units, two simplified attention-based encoder-decoder recurrent neural network (AEDRNN) models were also developed for time series predictions. The performance of the proposed models was compared with that of the auto-regressive integrated moving average (ARIMA), multilayer perceptron (MLP) and Elman recurrent neural network (ERNN) models by experimentally generating multi-step-ahead predictions using a dataset in the Zhejiang coastal areas of China. The results demonstrated that the LSTM, GRU and AEDRNN models significantly outperformed the ARIMA, MLP and ERNN models according to multiple statistical indicators. Moreover, the AEDRNN models were superior to the LSTM and GRU models, especially for middle-term predictions. In addition, the AEDRNN model with LSTM units was more robust than the AEDRNN model with GRU units in terms of accuracy and stability; therefore, it was considered to be the best model for Chl-a forecasting.