Kind Of Recurrent Neural Network Research Articles

Improved Plasma Etch Endpoint Detection Using Attention-Based Long Short-Term Memory Machine Learning

Existing etch endpoint detection (EPD) methods, primarily based on single wavelengths, have limitations, such as low signal-to-noise ratios and the inability to consider the long-term dependencies of time series data. To address these issues, this study proposes a context of time series data using long short-term memory (LSTM), a kind of recurrent neural network (RNN). The proposed method is based on the time series data collected through optical emission spectroscopy (OES) data during the SiO2 etching process. After training the LSTM model, the proposed method demonstrated the ability to detect the etch endpoint more accurately than existing methods by considering the entire time series. The LSTM model achieved an accuracy of 97.1% in a given condition, which shows that considering the flow and context of time series data can significantly reduce the false detection rate. To improve the performance of the proposed LSTM model, we created an attention-based LSTM model and confirmed that the model accuracy is 98.2%, and the performance is improved compared to that of the existing LSTM model.

An algorithm for two-dimensional pattern detection by combining Echo State Network-based weak classifiers

Pattern detection is one of the essential technologies in computer vision. To solve pattern detection problems, the system needs a vast amount of computational resources. To train a multilayer perceptron or convolutional neural network, the gradient descent method is commonly used. The method consumes computational resources. To reduce the amount of computation, we propose a two-dimensional pattern detection algorithm based on Echo State Network (ESN). The training rule of ESN is based on one-shot ridge regression, which enables us to avoid the gradient descent. ESN is a kind of recurrent neural network (RNN), which is often used to embed temporal signals inside the network, rarely used for the embedding of static patterns. In our prior work (Kage, 2023), we found that static patterns can be embedded in an ESN network by associating the training patterns with its stable states, or attractors. By using the same training procedure as our prior work, we made sure that we can associate each training patch image with the desired output vector. The resulting performance of a single ESN classifier is, however, relatively poor. To overcome this poor performance, we introduced an ensemble learning framework by combining multiple ESN weak classifiers. To evaluate the performance, we used CMU-MIT frontal face images (CMU DB). We trained eleven ESN-based classifiers by using six CMU DB training images and evaluated the performance by using a CMU DB test image. We succeeded in reducing false positives in the CMU DB test image down to 0.0515 %.

Sign Language Recognition for Real-time Communication

Abstract: Sign language is an essential communication tool for India's Deaf and Hard of Hearing people. This study introduces a novel approach for recognising and synthesising Indian Sign Language (ISL) using Long Short-Term Memory (LSTM) networks. LSTM, a kind of recurrent neural network (RNN), has demonstrated promising performance in sequential data processing. In this study, we leverage LSTM to develop a robust ISL recognition system, which can accurately interpret sign gestures in real-time. Additionally, we employ LSTM-based models for ISL synthesis, enabling the conversion of spoken language into sign language for improved inclusivity and accessibility. We evaluate the proposed approach on a diverse dataset of ISL signs, achieving high recognition accuracy and natural sign synthesis. The integration of LSTM in ISL technology holds significant potential for breaking down communication barriers and improving the quality of life for India's deaf and hard of hearing people

Wind power deviation charge reduction using long short term memory network

High penetration of variable generation like wind in modern power systems results in frequent load-generation imbalances. Additional spinning reserves and balancing services are deployed to reduce or avoid such imbalances. However, these additional balancing services result in increased total operation costs and it is generally shifted to Wind Power Generation Companies (WPGC) as Deviation Charges (DCs). The high DCs diminish the profitability of WPGCs. DCs are directly proportional to forecasting errors and can be reduced by enhancing forecasting accuracy. This paper proposes a Long Short-Term Memory (LSTM) network to reduce DCs. LSTM is a special kind of recurrent neural network and it is capable of learning long-term and short-term dependencies effectively. Therefore, the LSTM network can produce minimal error wind power forecasts and this helps to reduce DCs. The performance of the proposed LSTM network is realized using the real system data. Results obtained from the proposed LSTM model are compared with various forecasting models like ARIMA, ANN, and SVR. The comparison shows that the proposed LSTM model shows the least total DCs on most of the days followed by ANN, SVR, and ARIMA. The proposed LSTM, ANN, SVR, and ARIMA models show an average total DC of 6194.25$, 8459.32$, 15822.22$, and 38269.89$ respectively. Thus, the proposed LSTM model is capable of reducing 517.82%, 155.43%, and 36.56% DCs from ARIMA, SVR, and ANN models respectively, and helps the WPGCs to enhance their profitability.

Improving deep-learning methods for area-based traffic demand prediction via hierarchical reconciliation

Mobility services require accurate demand prediction in both space and time to effectively manage fleet rebalancing, provide quick on-demand responses, and enable advanced ride-sharing with minimum fleet size. Although the optimization of mobility services is a widely studied topic, the demand prediction side has received relatively less attention. In this paper, we aim to develop an efficient method for traffic demand forecasting by means of deep learning and hierarchical reconciliation approaches. The concepts, as well as the theories behind the proposed approach, are founded on Hierarchical Time Series (HTS), which also adopts Long Short-term Memory (LSTM) as a special kind of Recurrent Neural Network (RNN) to learn from the associated time series and produce reliable demand predictions. The proposed approach relies on the proper design of the HTS structure to get accurate forecasts for the number of trip departures and their associated uncertainty within predefined zones and aggregated collections of these zones. Moreover, the error analysis is essential for accomplishing the reconciliation in the HTS structure optimally. The proposed approach, consisting of deep learning, error analysis, and optimal reconciliation stages, has a remarkable ability to predict demand and control the forecasting quality at all levels of the hierarchical structure. We evaluate the proposed approach using a large-scale GPS tracking dataset from Lyon, France. The proposed method reduces the root mean square error (RMSE) by 13.92% and 14.77% for predefined and aggregated zones, respectively, compared with the LSTM using the historical demand and external features of time at the quarter-hour time step. Similarly, the corresponding improvements in mean absolute percentage error (MAPE) are 14.87% and 19.23%, respectively.

A Hybrid Deep Learning GRU based Approach for Text Classification using Word Embedding

Text categorization has become an increasingly important issue for businesses that handle massive volumes of data generated online, and it has found substantial use in the field of NLP. The capacity to group texts into separate categories is crucial for users to effectively retain and utilize important information. Our goal is to improve upon existing recurrent neural network (RNN) techniques for text classification by creating a deep learning strategy through our study. Raising the quality of the classifications made is the main difficulty in text classification, nevertheless, as the overall efficacy of text classification is often hampered by the data semantics' inadequate context sensitivity. Our study presents a unified approach to examine the effects of word embedding and the GRU on text classification to address this difficulty. In this study, we use the TREC standard dataset. RCNN has four convolution layers, four LSTM levels, and two GRU layers. RNN, on the other hand, has four GRU layers and four LSTM levels. One kind of recurrent neural network (RNN) that is well-known for its comprehension of sequential data is the gated recurrent unit (GRU). We found in our tests that words with comparable meanings are typically found near each other in embedding spaces. The trials' findings demonstrate that our hybrid GRU model is capable of efficiently picking up word usage patterns from the provided training set. Remember that the depth and breadth of the training data greatly influence the model's effectiveness. Our suggested method performs remarkably well when compared to other well-known recurrent algorithms such as RNN, MV-RNN, and LSTM on a single benchmark dataset. In comparison to the hybrid GRU's F-measure 0.952, the proposed model's F-measure is 0.982%. We compared the performance of the proposed method to that of the three most popular recurrent neural network designs at the moment RNNs, MV-RNNs, and LSTMs, and found that the new method achieved better results on two benchmark datasets, both in terms of accuracy and error rate.

Constraining chaos: Enforcing dynamical invariants in the training of reservoir computers.

Drawing on ergodic theory, we introduce a novel training method for machine learning based forecasting methods for chaotic dynamical systems. The training enforces dynamical invariants-such as the Lyapunov exponent spectrum and the fractal dimension-in the systems of interest, enabling longer and more stable forecasts when operating with limited data. The technique is demonstrated in detail using reservoir computing, a specific kind of recurrent neural network. Results are given for the Lorenz 1996 chaotic dynamical system and a spectral quasi-geostrophic model of the atmosphere, both typical test cases for numerical weather prediction.

Rainfall forecasting in the Barak river basin, India using a LSTM network based on various climate indices

The proposed study employs a long short-term memory (LSTM) neural network (NN) to forecast monthly rainfall in the Barak river basin in the northeastern region of India for a prediction horizon up to 4 months in advance. Out of nine significant climate variables, sea surface temperature (SST), sea level pressure (SLP), Nino 3.4 index, the Indian summer monsoon rainfall (ISMR) anomalies and dipole mode index (DMI) were identified to be the best-suited predictors and were introduced as the inputs in the NN. The LSTM is a special kind of recurrent neural network (RNN) which specializes in feature extraction and storing memory in its cell state cumulatively. The model results display strong correlations between the potential predictor sets and the rainfall distribution across the basin. The obtained forecast results were scrutinized in terms of various statistical measures and the predictions were found to be at par with the real time observations (correlations greater than 0.90 and hit score greater than 85%). The testing phase of model produced root mean square errors in the range of 12.45% to 15.65% highlighting satisfactory model performance. The proposed method of incorporating different climate indices form a novel approach to forecast rainfall in the region which may lead to timely and effective management of water resources.

A CNN and LSTM-based Model for Creating Captions for Photos

Can a machine interpret an image's meaning with the same speed as the human brain when it is seen? This problem was heavily researched by computer vision specialists, who believed it to be unsolvable until recently. It is now possible to develop models that can generate captions for pictures because of advancements in deep learning techniques, accessibility to large datasets, and processing power. This will be accomplished by the Python-based implementation of the article's deep learning convolutional neural network technique and a particular kind of recurrent neural network. Here the proposed model uses CNN and LSTM methods to achieve desired task

Sign Language Recognition Using Landmark Detection, GRU and LSTM

Speech impairment is a kind of disability, affects individual's ability to communicate with each other. People with this problem use sign language for their communication. Though communication through sign language has been taken care of, there exists communication gap between signed and non-signed people. To overcome this type of complexity researchers are trying to develop systems using deep learning approach. The main objective of this paper is subject to implement a vision-based application that offers translation of sign language to voice message and text to reduce the gap between two kinds of people mentioned above. The proposed model extracts temporal and spatial features after taking video sequences. To extract the spatial features, MediaPipe Holistic has been used that consists of several solutions for the detecting face, had and pose landmarks. Different kind of RNN (Recurrent Neural Network) like LSTM (Long Short-Term Memory) and GRU (Gated Recurrent Unit) have been used is to train on temporal features. By using both models and American Signed Language, 99% accuracy has been achieved. The experimental result shows that the recognition method with MediaPipe Holistic followed by GRU or LSTM can achieve a high recognition rate that meets the need of a Sign Language Recognition system that on the real-time basis. Based on the expectation, this analysis will facilitate creation of intelligent- based Sign Language Recognition systems and knowledge accumulation and provide direction to guide to the correct path.

A nonconvex function activated noise-tolerant neurodynamic model aided with Fischer-Burmeister function for time-varying quadratic programming in the presence of noises

Time-varying quadratic programming problems (TVQPPs) with equality and inequality constraints often arise in the fields of scientific computation and engineering application. Zeroing neural network (ZNN), being a special kind of recurrent neural network, has shown powerful capabilities to compute a variety of zeroing finding problems with monotonically increasing odd activation function. However, the projection sets of nonconvex activation function are obviously excluded, which means a general conclusion remain unexplored. In addition, noises are always ubiquitous in actual applications. Nevertheless, most existing ZNN-based models usually assume that the solving process is free of noise before the calculation. In this paper, a general zeroing neural network model with nonconvex activated function (GZNNM-NAF) and a general noise-tolerant zeroing neural network model with nonconvex activated function (GNTZNNM-NAF), which are also viewed as ZNN-type models, are developed by the inspiration of the traditional ZNN model from a control-based perspective. The ZNN-type models break the limitation of the traditional ZNN models of activation function, which allows nonconvex sets for projection operations and combines nonlinear complementary function for dealing with inequality constraints arising in TVQPPs. Moreover, theoretical results indicate that the ZNN-type models globally converge to time-varying optimal solution of TVQPPs with equality and inequality constraints under the noise circumstance. According to the different cases of nonconvex activation function, robustness analyses are demonstrated in detail for TVQPPs. It may enlarge the scope of the proposed method, especially in the field of practical application. Finally, a numerical example and an application example to manipulator motion generation are analyzed to verify the superiority and robustness of the developed ZNN-type models for TVQPPs with different measurement noises.

Individualized Short-Term Electric Load Forecasting Using Data-Driven Meta-Heuristic Method Based on LSTM Network.

Short-term load forecasting is viewed as one promising technology for demand prediction under the most critical inputs for the promising arrangement of power plant units. Thus, it is imperative to present new incentive methods to motivate such power system operations for electricity management. This paper proposes an approach for short-term electric load forecasting using long short-term memory networks and an improved sine cosine algorithm called MetaREC. First, using long short-term memory networks for a special kind of recurrent neural network, the dispatching commands have the characteristics of storing and transmitting both long-term and short-term memories. Next, four important parameters are determined using the sine cosine algorithm base on a logistic chaos operator and multilevel modulation factor to overcome the inaccuracy of long short-term memory networks prediction, in terms of the manual selection of parameter values. Moreover, the performance of the MetaREC method outperforms others with regard to convergence accuracy and convergence speed on a variety of test functions. Finally, our analysis is extended to the scenario of the MetaREC_long short-term memory with back propagation neural network, long short-term memory networks with default parameters, long short-term memory networks with the conventional sine-cosine algorithm, and long short-term memory networks with whale optimization for power load forecasting on a real electric load dataset. Simulation results demonstrate that the multiple forecasts with MetaREC_long short-term memory can effectively incentivize the high accuracy and stability for short-term power load forecasting.

BITCOIN PRICE PREDICTION USING MACHINE LEARNING

lSTM(Long Short-Term Memory) is a kind of Recurrent Neural Network which is used in the field of Machine Learning. Traditional neural networks can’t remember previous inputs But Recurrent Neural Networks enable us to learn from previous sequence input data. A LSTM unit is composed of a cell, an input gate, an output gate and a forget gate. this repository was written a Bitcoin Price Prediction project based on Google Trend keywords by using LSTM algorithm and Python 3.6 version. Here we tried to determine, “ Does LSTM algorithm predict Bitcoin Close price by adding keywords volume from Google trends”. Bitcoin price dataset was downloaded hourly using coinapi.io API and Google Trends keywords were downloaded hourly using python pytrend library. Finally chosen Bitcoin, BTC, Blockchain, Crypto currency LSTM algorithm predicted Bitcoin Close prices better than we expected by improving its learning in every epoch.

Analysis of environmental factors using AI and ML methods

The main goal of this research paper is to apply a deep neural network model for time series forecasting of environmental variables. Accurate forecasting of snow cover and NDVI are important issues for the reliable and efficient hydrological models and prediction of the spread of forest. Long Short Term Memory (LSTM) model for the time series forecasting of snow cover, temperature, and normalized difference vegetation index (NDVI) are studied in this research work. Artificial neural networks (ANN) are widely used for forecasting time series due to their adaptive computing nature. LSTM and Recurrent neural networks (RNN) are some of the several architectures provided in a class of ANN. LSTM is a kind of RNN that has the capability of learning long-term dependencies. We followed a coarse-to-fine strategy, providing reviews of various related research materials and supporting it with the LSTM analysis on the dataset of Himachal Pradesh, as gathered. Environmental factors of the Himachal Pradesh region are forecasted using the dataset, consisting of temperature, snow cover, and vegetation index as parameters from the year 2001–2017. Currently, available tools and techniques make the presented system more efficient to quickly assess, adjust, and improve the environment-related factors analysis.

A new approach for physical human activity recognition from sensor signals based on motif patterns and long-short term memory

Numerous studies have been carried out in recent years on the recognition, tracking, and discrimination of human activities. Automatic recognition of physical activities is often referred to as human activity recognition (HAR). There are generally vision-based and sensor-based approaches for activity recognition. The computer vision-based approach generally works well in laboratory conditions, but it can fail in real-world problems due to clutter, variable light intensity, and contrast. Sensor-based HAR systems are realized by continuously monitoring and analyzing physiological signals measured from heterogeneous sensors connected to the person's body. In this study, the Motif Patterns (MP) approach, which extracts features from sensor signals, is proposed for HAR. The success of the HAR systems depends on the effectiveness of the features extracted from the signals. The LSTM network is a special kind of recurrent neural network that has been used to make very successful predictions on time series data where long-term dependencies are. The LSTM network type offers a successful solution approach to solving long-term dependencies problems such as human activity recognition. The classification process was carried out with Long-Short Term Memory (LSTM) using MP features extracted from accelerometer, gyroscope, and magnetometer sensor signals. A large dataset of 9120 signals was used to test the proposed approach. A high success rate of 98.42 % was achieved with the proposed MP + LSTM method. As a result, it has been seen that the proposed approach has been obtained with a high success rate for HAR using sensor signals.

STOCK PRICE PREDICTION

Machine learning has many important applications in the stock price prediction. Here, we will discuss about predicting the returns on stocks. This has uncertainties and it is a very complex task. This project will be developed into two parts: First, we will learn how to predict stock price using the Long Short-Term Memory neural networks. Predicting stock market prices involves human-computer interaction. For stock market analysis, conventional batch processing methods cannot be utilized efficiently due to the correlated nature of stock prices. We suggest an algorithm that utilizes a kind of recurrent neural network (RNN) called Long Short-Term Memory (LSTM), where using stochastic gradient descent the weights are adjusted for individual data points

Echo state network model for analyzing solar-wind effects on the AU and AL indices

Abstract. The properties of the auroral electrojets are examined on the basis of a trained machine-learning model. The relationships between solar-wind parameters and the AU and AL indices are modeled with an echo state network (ESN), a kind of recurrent neural network. We can consider this trained ESN model to represent nonlinear effects of the solar-wind inputs on the auroral electrojets. To identify the properties of auroral electrojets, we obtain various synthetic AU and AL data by using various artificial inputs with the trained ESN. The analyses of various synthetic data show that the AU and AL indices are mainly controlled by the solar-wind speed in addition to Bz of the interplanetary magnetic field (IMF) as suggested by the literature. The results also indicate that the solar-wind density effect is emphasized when solar-wind speed is high and when IMF Bz is near zero. This suggests some nonlinear effects of the solar-wind density.

Cabin Load Prediction Using Time Series Forecasting in Long-haul Trucks for Optimal Energy Management

Predicting the electrical loads experienced by a battery pack during the 10 hour hotel period of a long haul class 8 mild hybrid truck with a sleeper cab, and using the information to achieve an optimal energy management strategy and controlling the State of Charge (SOC) of battery pack can help in improving it's freight efficiency. In this work, Machine Learning (ML) based algorithm has been proposed to predict the driver activity during the hotel period. Hence, the power load demanded from the auxiliaries can be predicted. A special kind of Recurrent Neural Network (RNN) called Long and Short Term Memory (LSTM) is used for the prediction task because of its ability to store recurrent information of a small and a large time horizon. To train the LSTM algorithm, the synthetic load profles are synthesized using rules and observations derived from the existing baseline electrical power load profile of the hotel period. This paper entails the whole process of data synthesis to training the neural network on the synthesized data and the prediction and validation of the power load. The input to the network is a time series of 600 time steps. Dynamic Time Warping (DTW) is used to manipulate the time axis and point wise euclidean distance between the forecast and the test data is used to quantify the accuracy of the model. Then by performing hyper-parameter optimization we find the best combination for number of hidden units and the number of training days for the algorithm.

Automated Stock Price Prediction using LSTM Recurrent Neural Network

In Stock Market Prediction, the aim is to predict the future value of the financial stocks of a company. The recent trend in stock market prediction technologies is the use of machine learning which makes predictions based on the values of current stock market indices by training on their previous values. Machine learning itself employs different models to make prediction easier and authentic. The paper focuses on the use of Regression and LSTM based Machine learning to predict stock values. Factors considered are open, close, low, high and volume. A correct prediction of stocks can lead to huge profits for the seller and the broker. Frequently, it is brought out that prediction is chaotic rather than random, which means it can be predicted by carefully analyzing the history of the respective stock market. Machine learning is an efficient way to represent such processes. It predicts a market value close to the tangible value, thereby increasing the accuracy. The introduction of machine learning to the area of stock prediction has appealed to many researchers because of its efficient and accurate measurements. In Stock Market Prediction, the aim is to predict the future value of the financial stocks of a company. The recent trend in stock market prediction technologies is the use of machine learning which makes predictions based on the values of current stock market indices by training on their previous values. Machine learning itself employs different models to make prediction easier and more authentic. The paper focuses on the use of Regression and LSTM based Machine learning to predict stock values. Factors considered are open, close, low, high, and volume. Stock market prediction is a major exertion in the field of finance and establishing businesses. The stock market is totally uncertain as the prices of stocks keep fluctuating on a daily basis because of numerous factors that influence it. One of the traditional ways of predicting stock prices was by using only historical data. But with time it was observed that other factors such as peoples' sentiments and other news events occurring in and around the country affect the stock market, for e.g.national elections, natural calamities etc. Investors in the stock market seek to maximize their profits for which they require tools to analyze the prices and trends of various stocks. Machine learning algorithms have been used to devise new techniques to build prediction models that can forecast the prices of stock and tell about the market trend with good accuracy. Many prediction models have been proposed to incorporate all the major factors affecting the price of stocks. Due to the correlated nature of stock prices, conventional batch processing methods cannot be utilized efficiently for stock market analysis. We propose an online learning algorithm that utilizes a kind of recurrent neural network (RNN) called Long Short Term Memory (LSTM), where the weights are adjusted for individual data points using stochastic gradient descent. This will provide more accurate results when compared to existing stock price prediction algorithms. The network is trained and evaluated for accuracy with various sizes of data, and the results are tabulated. A comparison with respect to accuracy is then performed against an Artificial Neural Network.

A new target tracking filter based on deep learning

At present, current filters can basically solve the filtering problem in target tracking, but there are still many problems such as too many filtering variants, too many filtering forms, loosely coupled with the target motion model, and so on. To solve the above problems, we carry out cross-application research of artificial intelligence theory and methods in the field of tracking filters. We firstly analyze the computation graphs of typical α-β and Kalman. Through analysis, it is concluded that α-β and Kalman have the same computation structures analogous to a typical recurrent neural network and can be considered as a kind of recurrent neural network with constrained weights. Then, given this and considering that a recurrent neural network has the recognition capability for target motion patterns, a new filter is developed in a unified neural network architecture and specifically constructed using feedforward neural network, recurrent neural network, and attention mechanism. And the unified tracking filter proposed in this paper can generate three aspects of unity: a unified target motion model, an adaptive filter method, and an overall track filtering framework. Finally, Simulation results show that the proposed filter is effective and useful, of which the overall performance is superior to those of compared filters.