LSTM Recurrent Neural Network Research Articles

Automatic diagnosis of epileptic seizures using entropy-based features and multimodel deep learning approaches

Epilepsy is one of the most common brain diseases, characterised by repeated seizures that occur on a regular basis. During a seizure, a patient's muscles flex uncontrollably, causing a loss of mobility and balance, which can be harmful or even fatal. Developing an automatic approach for warning patients of oncoming seizures necessitates substantial research. Analyzing the electroencephalogram (EEG) output from the human brain's scalp region can help predict seizures. EEG data were analyzed to extract time domain features such as Hurst exponent (Hur), Tsallis entropy (TsEn), enhanced permutation entropy (impe), and amplitude-aware permutation entropy (AAPE). In order to automatically diagnose epileptic seizure in children from normal children, this study conducted two sessions. In the first session, the extracted features from the EEG dataset were classified using three machine learning (ML)-based models, including support vector machine (SVM), K nearest neighbor (KNN), or decision tree (DT), and in the second session, the dataset was classified using three deep learning (DL)-based recurrent neural network (RNN) classifiers in The EEG dataset was obtained from the Neurology Clinic of the Ibn Rushd Training Hospital. In this regard, extensive explanations and research from the time domain and entropy characteristics demonstrate that employing GRU, LSTM, and BiLSTM RNN deep learning classifiers on the All−time−entropy fusion feature improves the final classification results.

Identification of Counterfeit Videos using Deep Learning Methodology

The rise of deep learning has ushered in a proliferation of deep fake videos, posing significant challenges to the credibility of visual content. Our research introduces a groundbreaking approach by merging Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) to enhance the accuracy of deepfake prediction. This unique integration, which has not been previously implemented, significantly boosts the model's capability to discern deepfakes. The synergy of CNNs and RNNs in our methodology represents an advancement, contributing to increased accuracy in detecting synthetic content. We leverage CNNs and RNNs for an efficient solution. First, we employ a Res-Next CNN to extract distinctive features from individual video frames, effectively encoding spatial information. These features are then used in the subsequent phase, where a LSTM-RNN models temporal dynamics within the videodata.The temporal aspect is crucial in differentiating deep fake videos due to subtle inconsistencies over time. The LSTM RNN processes the feature sequence, enabling the model to identify temporal patterns unique to deep fakes. This holistic approach, combining spatial and temporal analysis, enhances the model's ability to detect even highly convincing synthetic content. Our model is trained on a comprehensive dataset with rigorous evaluations, demonstrating competitive performance through standard metrics such as accuracy, precision. Practically, our model offers real- time video analysis, automatically identifying deep fake content and mitigating potential risks. Importantly, our approach is simple and robust, suitable for deployment across diverse scenarios. In summary, our research provides an effective solution to the critical issue of deepfake detection. By synergizing CNNs and LSTM-based RNNs, we offer a practical means to uphold the integrity of visual content in an era where digital information authenticity is paramount. Keywords— Deep Learning, CNN, RNN, Deepfake, LSTM, accuracy, precision, visual content, digital information

FORECASTING THE CONSUMPTION OF ELECTRICAL ENERGY ON THE CEB ELECTRICAL NETWORK IN LOME IN TOGO BY APPROACHES: SIMPLE LINEAR REGRESSION, RECURRENT NEURAL NETWORKS AND GENETIC ALGORITHMS

Forecasting the electrical power to be consumed requires planning production on several levels. In this work we used data from the Electric Community of Benin. Temperature, relative humidity, wind speed, normal direct irradiance, precipitation and diffuse radiation are the meteorological variables that made it possible to analyze the forecasts. The objective is to do learning with genetic algorithms, LSTM recurrent neural networks and simple linear regression after a characterization and a correlation study and then to submit the results to performance evaluation criteria. The results of the characterization made it possible to understand that certain variables are significant and influence the consumption of electrical energy. The study of the correlation gives 94% between direct normal irradiance and diffuse irradiance. Both give with the temperature 67% for one and 68% for the other. Regarding modeling, the results are bad with genetic algorithms if we take into account the correlation coefficient (R² = 28.84%), good with simple linear regression (R² = 69.08%) and very interesting for networks of recurrent neurons where we find: MAE = 0.11 MSE = 0.02 MAPE = 18.50% RMSE = 13.09% RRMSE = 18.25% and R² = 96.11%. Given these results, we deduce that short- and long-term memory recurrent neural networks (LSTM) are very well suited to predicting the electrical power consumed on the CEB electrical network.

Метод интеллектуального анализа текстовой информации для психиатрической диагностики

The automated depression detection system is a progressive technique in terms of improving clinical diagnosis and early medical intervention in cases where depression can have the most serious consequences, including self-harm or suicide. An innovative method of automated detection of depression based on textual data of patients is proposed. The developed method includes modern technologies such as the architecture of the recurrent neural network LSTM and various methods of text vectorization. Experiments conducted on publicly available datasets have confirmed the high efficiency and accuracy of the proposed method compared to the approaches used today. A unique feature of the method is the use of textual characteristics, which ensures the safety of the data provided by patients and eliminates their distortion. This approach not only increases the reliability of the results, but also avoids potential distortion of information in the analysis process. The developed method of automatic assessment of depression has high accuracy and does not require the presence of a doctor, which significantly increases the effectiveness of the process of identifying and assessing the level of depression. This approach can become a promising direction in the development of automated mental health support systems, reducing reaction time and providing more prompt assistance. In the future, the research will include training the model on data in Russian and further tuning of methods, as well as expanding the use of GloVe vectorization to improve contextual understanding of textual data. These steps are aimed at creating a more adapted and effective system for detecting depression in various linguistic contexts.

Prediction of Degraded Infrastructure Conditions for Railway Operation.

In the railway sector, rolling stock and infrastructure must be maintained in perfect condition to ensure reliable and safe operation for passengers. Climate change is affecting the urban and regional infrastructure through sea level rise, water accumulations, river flooding, and other increased-frequency extreme natural situations (heavy rains or snows) which pose a challenge to maintenance. In this paper, the use of artificial intelligence based on predictive maintenance implementation is proposed for the early detection of degraded conditions of a bridge due to extreme climatic conditions. For this prediction, continuous monitoring is proposed, with the aim of establishing alarm thresholds to detect dangerous situations, so restrictions could be determined to mitigate the risk. However, one of the main challenges for railway infrastructure managers nowadays is the high cost of monitoring large infrastructures. In this work, a methodology for monitoring railway infrastructures to define the optimal number of transductors that are economically viable and the thresholds according to which infrastructure managers can make decisions concerning traffic safety is proposed. The methodology consists of three phases that use the application of machine learning (Random Forest) and artificial cognitive systems (LSTM recurrent neural networks).

Modelling Hysteresis in Shape Memory Alloys Using LSTM Recurrent Neural Network

The complex behavior of shape memory alloys (SMAs), characterized by hysteresis and nonlinear dynamics, results in complex constitutive equations. To circumvent the complexity of solving these equations, a black box neural network (NN) has been employed in this research to model a rotary actuator actuated by an SMA wire. Considering the historical dependence of the pulley’s rotational angle on the applied voltage, a recurrent neural network (RNN) is suitable for capturing past information. Specifically, a long short-term memory (LSTM) neural network is selected due to its ability to address issues encountered in standard recurrent networks. There are major drawbacks with modelling hysteresis with NNs that do not account for historical behavior. Traditional NNs, characterized by a one-to-one mapping, struggle to capture hysteresis loops wherein system behavior varies during loading and unloading cycles. Therefore, a single-tag data is used to determine the loading or unloading state, but tag signal causes discontinuity in network and omits various aspects of hysteresis in SMA, particularly within minor loops. In contrast, NNs incorporating past data to predict hysteresis behavior alleviate the need for tag data. However, such networks tend to have complex structures with a substantial number of neurons to effectively capture the inherent nonlinearity in SMAs. The long short-term memory (LSTM) neural network employed in this research, characterized by a simpler structure, achieves high accuracy in predicting hysteresis in SMAs without the need for tag data. In the proposed LSTM model, data related to the pulley’s rotational angle and the wire’s applied voltage from the current moment and the two previous moments serve as input. The data passes through a layer comprising three LSTM cells, and the output from the last LSTM cell is fed into a fully connected layer to predict the pulley’s rotational angle for the next moment. Training data are obtained by applying voltage at various frequencies and formats to the SMA wire while simultaneously recording the pulley’s angle with an encoder. Evaluation of the LSTM model is conducted in two configurations: online prediction (one-step ahead) and offline prediction (multistep ahead). In the online configuration where the model uses encoder data as angular inputs, the root mean square error (RMSE) of predictions for various input voltages is significantly low at about 0.1 degrees where the maximum rotational angle of pulley is 8 degrees. In the offline configuration when using the model’s predictions as angular inputs instead of encoder data, the RMSE rises to 0.3 degrees. To provide a clear demonstration of the LSTM model’s ability in this particular configuration, a comparison has been conducted between LSTM model and a rate-dependent Prandtl-Ishlinskii (RDPI) hysteresis model for predicting the pulley’s angle. The LSTM model outperforms the RDPI model by 70% in terms of accuracy. Overall, the LSTM model demonstrates capability in effectively modeling SMA hysteresis in both online and offline configurations.

Classifying Human Manual Control Behavior Using LSTM Recurrent Neural Networks

Classifying Human Manual Control Behavior Using LSTM Recurrent Neural Networks

Forecasting superstore fruit and vegetable sales based on ARIMA-LSTM

In the context of a fresh food supermarket, managing perishable vegetables, which have a notably limited shelf life, maintaining optimal stock levels becomes imperative for safeguarding profitability. This research paper intricately explores the analysis of numerous datasets using Power Query and Power BI, with a pivotal focus on devising a succinct coding system. To ensure data quality, outliers are meticulously eliminated using advanced techniques like linear interpolation and Z-Score processing. The ARIMA time series model is employed to predict the sales volume for each category in July, having been trained on data from June 2022 to June 2023, adeptly capturing historical sales patterns and emerging trends. Additionally, a parameter-tuned LSTM recurrent neural network, optimized with Adam's algorithm, is utilized to refine predictions, considering its inherent memory capabilities. A thoughtfully designed weighted average approach combines outputs from both models, strategically minimizing overfitting and enhancing prediction accuracy. The resultant research fortifies inventory management optimization, bolstering the supermarket's profitability while effectively meeting customer demands.

Multi-radar Interference Mitigation in Photonics-based Radar with Sliding Window LSTM Recurrent Neural Network

Multi-radar Interference Mitigation in Photonics-based Radar with Sliding Window LSTM Recurrent Neural Network

Biomechanical analysis of the danger of the whip-leg striking maneuver in martial arts sparring bouts

Abstract This paper recognizes the human motion in martial arts movements based on the LSTM recurrent neural network algorithm, which uses signal statistics to extract the action time series features through the data collection from the relevant human motion sensors in order to effectively recognize the athletes’ movements. The parameter settings of the LSTM neural network model are optimized using an improved particle swarm algorithm. The experimental process is then created. After filming the video, it is divided into two documents, A and B, and saved. The experimenter’s actions undergo technical analysis using LSTM. Three groups of athletes were invited to carry out the actual test after the design was completed, and biomechanical analysis was performed based on the experimental data. The experiments showed that the activation of the left medial femoral muscle of the athletes was greater than that of the left lateral femoral muscle, and the root mean square of electromyography was 25.38±11 and 27.54+11, respectively. The angle of the left leg was 69±2.5, and that of the right leg was 71±9.2, which was more than 80% of the safe rotational amplitude. It is easy to cause sports injury if there is too much difference in the activation level of leg muscles during whiplash.

Daily-Scale Prediction of Arctic Sea Ice Concentration Based on Recurrent Neural Network Models

Arctic sea ice prediction is of great practical significance in facilitating Arctic route planning, optimizing fisheries management, and advancing the field of sea ice dynamics research. While various deep learning models have been developed for sea ice prediction, they predominantly operate at the seasonal or sub-seasonal scale, often focusing on localized areas, and few cater to full-region daily-scale prediction. This study introduces the use of spatiotemporal sequence data prediction models, namely, the convolutional LSTM (ConvLSTM) and predictive recurrent neural network (PredRNN), for the prediction of sea ice concentration (SIC). Our analysis reveals that, when solely utilizing SIC historical data as the input, the ConvLSTM model outperforms the PredRNN model in SIC prediction. To enhance the models’ capacity to capture spatiotemporal relationships between multiple variables, we expanded the range of input data types to form the ConvLSTM-multi and PredRNN-multi models. Experimental findings demonstrate that the prediction accuracy of the four models significantly surpasses the CMIP6 model in three prospective climate scenarios (SSP126, SSP245, and SSP585). Of the four models, the ConvLSTM-multi model excels in assimilating the influence of reanalysis data on sea ice within the sea ice edge region, thus exhibiting superior performance than the PredRNN-multi model in predicting daily Arctic SIC over the subsequent 10 days. Furthermore, sensitivity tests on various model parameters highlight the substantial impact of sea surface temperature and prediction date on the accuracy of daily sea ice prediction, and meteorological and oceanographic parameters primarily affect the prediction accuracy of the thin-ice region at the edge of the sea ice.

Short-Term Load Forecasting Using an LSTM Neural Network for a Grid Operator

Electricity is crucial for daily life due to the number of activities that depend on it. To forecast future electric load, which changes over time and depends on various factors, grid operators (GOs) must create forecasting models for various time horizons with a high degree of accuracy because the results have a huge impact on their decision-making regarding, for example, the scheduling of power units to supply user consumption in the short or long term or the installation of new power plants. This has led to the exploration of multiple techniques like statistical models and Artificial Intelligence (AI), with Machine-Learning and Deep-Learning algorithms being the most popular in this latter field. This paper proposes a neural network-based model to forecast short-term load for a Colombian grid operator, considering a seven-day time horizon and using an LSTM recurrent neural network with historical load values from a region in Colombia and calendar features such as holidays and the current month corresponding to the target week. Unlike other LSTM implementations found in the literature, in this work, the LSTM cells read multiple load measurements at once, and the additional information (holidays and current month) is concatenated to the output of the LSTM. The result is used to feed a fully connected neural network to obtain the desired forecast. Due to social problems in the country, the load data presents a strange behavior, which, in principle, affects the prediction capacity of the model. Still, it is eventually able to adjust its forecasts accordingly. The regression metric MAPE measures the model performance, with the best predicted week having an error of 1.65% and the worst week having an error of 26.22%. Additionally, prediction intervals are estimated using bootstrapping.

Enhancing Intrusion Detection with LSTM Recurrent Neural Network Optimized by Emperor Penguin Algorithm

Intrusion detection systems (IDS) have been developed to identify and classify these attacks in order to prevent them from occurring. However, the accuracy and efficiency of these systems are still not satisfactory. In previous research, most of the methods used were based on ordinary neural networks, which had low accuracy. Therefore, this thesis, with the aim of presenting a new approach to intrusion detection and improving its accuracy and efficiency, uses long-term memory (LSTM) optimized with the Penguin optimization algorithm (EPO). In the proposed approach, first, the features were pre-processed by normalization, cleaning, and formatting in number format. In the next step, the linear discriminant analysis (LDA) method was used to reduce the dimensions of the processed features, and after that, the EPO algorithm was used to optimize the size of the hidden unit of the LSTM network. Finally, the optimized network was evaluated using the NSL-KDD dataset, which is a widely used benchmark dataset in the field of intrusion detection. The results obtained for the training and test datasets were 99.4 and 98.8%, respectively. These results show that the proposed approach can accurately identify and classify network intrusions and outperform many existing approaches. Keywords: Intrusion Detection Systems, Penguin Meta-Heuristic Algorithm, Long-Term Memory Neural Network, Linear Detection Analysis.

Evaluation of effectiveness and resources consumption of water mist spray systems in Mediterranean areas by predictions based on LSTM Recurrent Neural Networks

To counter the increasing urban overheating, climate adaptation solutions are proposed. Among them, water mist spray recently acquired particular attention, due to its efficiency, cost-effectiveness, and versatility. However, spray devices require a large amount of water and energy to cool even limited areas, thus their environmental costs/benefits ratio should be carefully evaluated. This study analyses cooling benefits and resources consumption of mist devices in 11 cities within 3 climate contexts, through Recurrent Neural Networks (RNNs) trained with experimental data. RNNs predict the expected time series of thermal benefits and of energy and water consumptions, also considering different design solutions of devices. Results show that when sun/wind shielding is used in the sprayed area, or the height of nozzles is limited, higher cooling results are obtained. However, energy and water consumption are extremely high if misting systems are perennially active during the day. Considering all simulated conditions, the predicted average daily energy to obtain a unitary variation of the Mediterranean Outdoor Comfort Index is 4,17 Wh/m2, while the corresponding average daily volume of water is 0,56 l*h/m2. These results confirm the need for applications managed by control logics based on the acquisition of real-time climatic data to reduce the environmental loads.

A hybrid model based on LSTM neural networks with attention mechanism for short-term wind power forecasting

Wind power plants have gained prominence in recent decades owing to their positive environmental and economic impact. However, the unpredictability of wind resources poses significant challenges to the secure and stable operation of the power grid. To address this challenge, numerous computational and statistical methods have been proposed in the literature to forecast short-term wind power generation. However, the demand for more accurate and reliable methodologies to tackle this problem remains. In this context, this paper proposes a new hybrid framework that combines a statistical pre-processing stage with an attention-based deep learning approach to overcome the shortcomings of existing forecasting strategies in accurately predicting multi-seasonal wind power time series. The proposed ensemble model involves a data transformation stage that normalizes the data distribution, along with modeling and removing multiple seasonal patterns from the historical time-series. Considering these results, the proposed model further incorporates an LSTM Recurrent Neural Network (RNN) model with an attention mechanism, for each month of the year, to better capture the relevant temporal dependencies in the input residuals sequence. The model was trained and evaluated on hourly wind power data obtained from the Spanish electricity market, spanning the period from 2008 to 2019. Experimental results show that the proposed model outperforms well-established DL-based models, achieving lower error metrics. These findings have potential applications in energy trading, grid planning, and renewable energy management.

Retracted: Design and Implementation of Human Motion Recognition Information Processing System Based on LSTM Recurrent Neural Network Algorithm.

[This retracts the article DOI: 10.1155/2021/3669204.].

Improving the understanding of web user behaviors through machine learning analysis of eye-tracking data

Eye-tracking techniques are widely used to analyze user behavior. While eye-trackers collect valuable quantitative data, the results are often described in a qualitative manner due to the lack of a model that interprets the gaze trajectories generated by routine tasks, such as reading or comparing two products. The aim of this work is to propose a new quantitative way to analyze gaze trajectories (scanpaths) using machine learning. We conducted a within-subjects study (N = 30) testing six different tasks that simulated specific user behaviors in web sites (attentional, comparing two images, reading in different contexts, and free surfing). We evaluated the scanpath results with three different classifiers (long short-term memory recurrent neural network—LSTM, random forest, and multilayer perceptron neural network—MLP) to discriminate between tasks. The results revealed that it is possible to classify and distinguish between the 6 different web behaviors proposed in this study based on the user’s scanpath. The classifier that achieved the best results was the LSTM, with a 95.7% accuracy. To the best of our knowledge, this is the first study to provide insight about MLP and LSTM classifiers to discriminate between tasks. In the discussion, we propose practical implications of the study results.

Recurrent neural network modeling of multivariate time series and its application in temperature forecasting.

Temperature forecasting plays an important role in human production and operational activities. Traditional temperature forecasting mainly relies on numerical forecasting models to operate, which takes a long time and has higher requirements for the computing power and storage capacity of computers. In order to reduce computation time and improve forecast accuracy, deep learning-based temperature forecasting has received more and more attention. Based on the atmospheric temperature, dew point temperature, relative humidity, air pressure, and cumulative wind speed data of five cities in China from 2010 to 2015 in the UCI database, multivariate time series atmospheric temperature forecast models based on recurrent neural networks (RNN) are established. Firstly, the temperature forecast modeling of five cities in China is established by RNN for five different model configurations; secondly, the neural network training process is controlled by using the Ridge Regularizer (L2) to avoid overfitting and underfitting; and finally, the Bayesian optimization method is used to adjust the hyper-parameters such as network nodes, regularization parameters, and batch size to obtain better model performance. The experimental results show that the atmospheric temperature prediction error based on LSTM RNN obtained a minimum error compared to using the base models, and these five models obtained are the best models for atmospheric temperature prediction in the corresponding cities. In addition, the feature selection method is applied to the established models, resulting in simplified models with higher prediction accuracy.

Predicting the Price of Bitcoin using LSTM Recurrent Neural Network

In this paper, we tried to estimate the Bitcoin price precisely taking into consideration various parameters that affect the Bitcoin value. In our work, we pointed to understand and identify daily changes in the Bitcoin market while obtaining insight into most appropriate features surrounding Bitcoin price. We will predict the daily price change with highest possible accuracy. The market capitalization of publicly traded cryptocurrencies is currently above $230 billion. Bitcoin, the most valuable cryptocurrency, serves primarily as a digital store of value, and its price predictability has been well-studied. For the first phase of our investigation, we aim to understand and identify daily trends in the Bitcoin market while gaining insight into optimal features surrounding Bitcoin price. Our data set consists of various features relating to the Bitcoin price and payment network over the course of five years, recorded daily. For the second phase of our investigation, using the available information, we will predict the sign of the daily price change with highest possible accuracy with deep learning algorithm such as long short term memory for greater accuracy. Compared with benchmark results for daily price prediction, we achieve a better performance, with the highest accuracies of the statistical methods and deep learning algorithms. Deep Learning models includes Long Short-Term Memory in RNN for Bitcoin price prediction are superior to statistical methods

Status Quo and Influence of Martial Arts Fitness in Pursuit of Health Using LSTM Recurrent Neural Network Algorithm

With the acceleration of the socialist modernization process, people’s living standards continue to improve, and the modernization of cities is also developing rapidly. In the process of urban construction, a series of problems such as how to deal with the relationship between citizens and the surrounding environment, how to carry out citizens' health activities, and how to organize citizens to exercise healthily are becoming more and more obvious. With the introduction of the National Sports Outline, as an important sports activity, Chinese martial arts has a long history and profound cultural heritage, and it still plays an irreplaceable role in promoting the national spirit and building a harmonious society today. Therefore, this paper introduced the LSTM recurrent neural network algorithm to collect and extract the current data on martial arts fitness in the pursuit of health. By constructing the LSTM neuron structure and building the LSTM recurrent neural network structure, a state estimation based on LSTM feature extraction was proposed. The algorithm used the memory ability of LSTM to extract the relevant features of the data stream, which significantly improved the accuracy of the data stream collection. The experimental results showed that the time required by the method in this paper were 12.7, 10.2, 12.4 s and 11.8 min respectively, and the accuracy rates were 98.78, 98.26, 99.03 and 97.89%, respectively, which were greatly improved compared with the existing methods.