RNN Model Research Articles

Optimizing Recurrent Neural Network with Bayesian Algorithm for Behavioural Authentication System

Current studies on behavioural biometrics authentication have been focused on the use of deep learning and keystroke dynamics but the aspect of conscious optimization of the algorithm in order to obtain best outcome has not been considered. This study employed and incorporated Bayesian algorithm into Recurrent Neural Network to build a Keystroke Behavioural Biometric (KBB) authentication model used against social engineering attacks. The model begins with importing the keylogging dataset for data pre-processing, feature extraction, and RNN algorithm was used to build the KBB model. Hyperparameter tuning was done to achieve optimal results. A traditional optimizer called Adaptive Momentum Estimation (Adam) was used and evaluated so as to estimate the impact of optimization in model inferencing. RNN model result with Bayesian optimization technique shows a better performance than the result of RNN model with ADAM optimization. The essence of incorporating and evaluating the best optimization technique is to come up with an effective and accurate model for behavioural biometric authentication, that could mitigate effectively against social engineering attacks.

Optimization of electrical discharge machining parameters for enhanced performance on inconel 718 using Cu-Ni-B4C nanocomposite electrodes and advanced modeling techniques

Abstract This paper investigate into the complex field of electrical discharge machining (EDM) to improve material removal rate (MRR), electrode wear rate (EWR), and surface roughness (SR) for the machining of Inconel 718, a difficult-to-machine superalloy. The effects of discharge current, pulse duration, and pulse interval on machining performance were assessed through experiments. Response surface methodology (RSM) and artificial neural network (ANN) models, such as RNN, LSTM, and CNN, were used to optimize. Twenty runs of confirmation experiments were used to confirm the optimal process parameters found by the created models for better machining. For Inconel 718, the novel Cu-Ni-B4C nanocomposite electrode greatly enhanced EDM performance. The ideal configuration increased MRR while decreasing wear and surface roughness. Machined surfaces were inspected using SEM and EDAX analysis. With optimal settings of 50 μs pulse duration and 90 μs pulse interval, increasing current to 8 Amps increased MRR to 0.0118 g min−1, reducing EWR to 0.001 g min−1 and SR to 3.108 μm. Compared to the RNN, LSTM, and RSM models, the CNN model had the greatest R-squared (R2) score of 0.9999, suggesting greater MRR, EWR, and SR prediction.

"Enhancing Lung Cancer Detection: A Comparative Analysis of CNN and RNN Models on X-Ray Image Data"

"Enhancing Lung Cancer Detection: A Comparative Analysis of CNN and RNN Models on X-Ray Image Data"

Performance of Deep Learning Models on Imputed Time Series Data: A Simulation Study and Application to Leading Airline Companies' Stock Price

In this study, the validity of imputation techniques for deep learning methods in time series analysis is investigated using datasets based on daily closing data in the stock market. Datasets of daily closing stock prices for Turkish Airlines, Deutsche Lufthansa AG, and Delta Airlines, as well as a simulated dataset, are used. LSTM, GRU, RNN, and Transformer models, which are deep learning models, are employed. The original dataset and datasets with 5%, 15% and 25% missing data are analyzed imputing Linear, Spline, Stineman, Mean and Random imputation techniques. The results show that model performance varies depending on the imputation technique and the rate of missing data. GRU and Transformer models are favored for their robustness and excellent performance. For handling missing data, using spline and Stineman imputations is advisable to maintain high model accuracy. This study emphasizes the usability of various imputation techniques and deep learning models in time series analysis. It assesses model performance using both MAPE and RMSE to gain a comprehensive understanding of predictive accuracy and reliability, aiming to guide future research by comparing these methods.

A predictive SD‐WAN traffic management method for IoT networks in multi‐datacenters using deep RNN

AbstractDeploying the Internet of Things (IoT) in integrated edge‐cloud environments exposes the IoT traffic data to performance issues such as delay, bandwidth limitation etc. Recently, Software‐Defined Wide Area Network (SD‐WAN) has emerged as an architecture that originates from the Software‐Defined Network (SDN) paradigm and provides solutions for networking multiple data centers by allowing network administrators to manage and control network layers. In this article, an SDWAN‐based policy for traffic management in IoT is introduced in which the Quality of Service (QoS) metrics such as end‐to‐end delay and bandwidth utilization are optimized. The proposed method implements the traffic management policy in the SDWAN controller. When the IoT traffic flows reach the SDWAN infrastructure network, graph search algorithms are performed to find the near‐optimal paths that affect the end‐to‐end delay of traffic flows. Because of the ability of deep learning to process complex data, a deep RNN model is used to predict the network state information, such as link latency and available bandwidth, before the traffic flows reach the infrastructure network. The proposed method consists of four key modules to predict the routes for future time intervals: (a) an SD‐WAN topology updater unit that checks the link changes and availability, (b) the network state information collector, which collects the network state information to create a dataset, (c) the learning unit, which trains a deep RNN model using the created dataset, and (d) the route predictor unit, which uses the trained model to predict the network state information using a heuristic algorithm to determine the routes. The simulation results showed that the deep RNN model can achieve high accuracy and low Mean Absolute Error (MAE), and the proposed method outperforms shortest‐path algorithms in terms of latency. At the same time, the available bandwidth is almost fairly distributed among all network links.

Daily Runoff Prediction Based on FA-LSTM Model

Accurate and reliable short-term runoff prediction plays a pivotal role in water resource management, agriculture, and flood control, enabling decision-makers to implement timely and effective measures to enhance water use efficiency and minimize losses. To further enhance the accuracy of runoff prediction, this study proposes a FA-LSTM model that integrates the Firefly algorithm (FA) with the long short-term memory neural network (LSTM). The research focuses on historical daily runoff data from the Dahuangjiangkou and Wuzhou Hydrology Stations in the Xijiang River Basin. The FA-LSTM model is compared with RNN, LSTM, GRU, SVM, and RF models. The FA-LSTM model was used to carry out the generalization experiment in Qianjiang, Wuxuan, and Guigang hydrology stations. Additionally, the study analyzes the performance of the FA-LSTM model across different forecasting horizons (1–5 days). Four quantitative evaluation metrics—mean absolute error (MAE), root mean square error (RMSE), coefficient of determination (R2), and Kling–Gupta efficiency coefficient (KGE)—are utilized in the evaluation process. The results indicate that: (1) Compared to RNN, LSTM, GRU, SVM, and RF models, the FA-LSTM model exhibits the best prediction performance, with daily runoff prediction determination coefficients (R2) reaching as high as 0.966 and 0.971 at the Dahuangjiangkou and Wuzhou Stations, respectively, and the KGE is as high as 0.965 and 0.960, respectively. (2) FA-LSTM model was used to conduct generalization tests at Qianjiang, Wuxuan and Guigang hydrology stations, and its R2 and KGE are 0.96 or above, indicating that the model has good adaptability in different hydrology stations and strong robustness. (3) As the prediction period extends, the R2 and KGE of the FA-LSTM model show a decreasing trend, but the whole model still showed feasible forecasting ability. The FA-LSTM model introduced in this study presents an effective new approach for daily runoff prediction.

Neural network analysis in time series forecasting

Objectives. To build neural network models of time series (LSTM, GRU, RNN) and compare the results of forecasting with their mutual help and the results of standard models (ARIMA, ETS), in order to ascertain in which cases a certain group of models should be used.Methods. The paper provides a review of neural network models and considers the structure of RNN, LSTM, and GRU models. They are used for modeling time series in Russian macroeconomic statistics. The quality of model adjustment to the data and the quality of forecasts are compared experimentally. Neural network and standard models can be used both for the entire series and for its parts (trend and seasonality). When building a forecast for several time intervals in the future, two approaches are considered: building a forecast for the entire interval at once, and step-by-step forecasting. In this way there are several combinations of models that can be used for forecasting. These approaches are analyzed in the computational experiment.Results. Several experiments have been conducted in which standard (ARIMA, ETS, LOESS) and neural network models (LSTM, GRU, RNN) are built and compared in terms of proximity of the forecast to the series data in the test period.Conclusions. In the case of seasonal time series, models based on neural networks surpassed the standard ARIMA and ETS models in terms of forecast accuracy for the test period. The single-step forecast is computationally less efficient than the integral forecast for the entire target period. However, it is not possible to accurately indicate which approach is the best in terms of quality for a given series. Combined models (neural networks for trend, ARIMA for seasonality) almost always give good results. When forecasting a non-seasonal heteroskedastic series of share price, the standard approaches (LOESS method and ETS model) showed the best results.

Design and Implementation for Research Paper Classification Based on CNN and RNN Models

Design and Implementation for Research Paper Classification Based on CNN and RNN Models

Hybrid RNN Based Text Classification Model for Unstructured Data

The volume of social media posts is on the rise as the number of social media users expands. It is imperative that these data be analyzed using cutting-edge algorithms. This goal is handled by the many techniques used in text categorization. There are a variety of text categorization techniques available, ranging from machine learning to deep learning. Numerical crunching has become easier with less processing time since the emergence of high-end computer facilities. This has led to the development of sophisticated network architectures that can be trained to achieve higher precision and recall. The performance of neural network models which was evaluated by the F1 score is affected by cumulative performance in precision and recall. The current study intends to analyze and compare the performance of the neural network proposed, A Hybrid RNN model that has two layers of BiLSTM and two layers of GRU to that of previous hybrid models. GloVE dataset is used to train the models and their accuracy, precision, recall, and F1 score are used to assess performance. Except for the RNN + GRU model, the RNN + BILSTM + GRU model has a precision of 0.767, a recall of 0.759, and an F1-score of 0.7585. This hybrid model outperforms the others.

House price prediction

In recent years, more and more facts have proven that neural network models have good performance in machine learning, and researchers always use RNN and CNN models to solve complex problems, like how RNN can utilize historical information to predict further weather, identify human notes, and analyze text. The CNN model is always used to identify signatures, process picture images, and classify items. How I find that in the linear prediction field, people always use linear regression to predict results. For example, in house price prediction, people will use linear regression or the random forest model to predict house price. However, the author think neural network model can also have good performance in house prices. So, this paper is about building a better neural network model to predict house prices. This paper describes how to process data before training two models and compare their prediction accuracy. After that, this paper describes new neural network and train this model, which, after training, will output its accuracy. Luckily, after a long-time training, the author gets a neural network model to predict house price, it has a higher accuracy in predicting house price than two tradition models that we used.

Shifts in attention drive context-dependent subspace encoding in anterior cingulate cortex in mice during decision making

Attention supports decision making by selecting the features that are relevant for decisions. Selective enhancement of the relevant features and inhibition of distractors has been proposed as potential neural mechanisms driving this selection process. Yet, how attention operates when relevance cannot be directly determined, and the attention signal needs to be internally constructed is less understood. Here we recorded from populations of neurons in the anterior cingulate cortex (ACC) of mice in an attention-shifting task where relevance of stimulus modalities changed across blocks of trials. In contrast with V1 recordings, decoding of the irrelevant modality gradually declined in ACC after an initial transient. Our analytical proof and a recurrent neural network model of the task revealed mutually inhibiting connections that produced context-gated suppression as observed in mice. Using this RNN model we predicted a correlation between contextual modulation of individual neurons and their stimulus drive, which we confirmed in ACC but not in V1.

Design of Tool Wear Monitoring System in Bone Material Drilling Process

Biological bone materials, complex and anisotropic, require precise machining in surgeries. Bone drilling, a key technique, is susceptible to increased friction from tool wear, leading to excessive forces and high temperatures that can damage bone and surrounding tissues, affecting recovery. This study develops a monitoring platform to assess tool wear during bone drilling, employing an experimental setup that gathers triaxial force and vibration data. A recognition model using a bidirectional long short-term memory network (BI-LSTM) with a multi-head attention mechanism identified wear levels. This model, termed ABI-LSTM, was optimized and benchmarked against SVR, RNN, and CNN models. The results from implementing the ABI-LSTM-based monitoring system demonstrated its efficacy in detecting tool wear, thereby potentially reducing surgical risks such as osteonecrosis and drill breakage, and enhancing surgical outcomes.

Investigation of Visual Language Landscape of Tourist Attractions from Multimodal Perspective

With the development of economic globalization, the tourism industry has been welcomed by the public. The visual language landscape of tourist attractions can not only assist tourists to play and watch the project, but if it is properly planned, the language landscape can also become a major feature and highlight of the scenic spot. Therefore, how to set up and construct the visual language landscape of tourist attractions is a problem that needs to be considered in each region. In response to the above problems, on the basis of understanding the concept types of the visual language landscape of tourist attractions, this article conducts in-depth research and investigation on the visual language landscape of tourist attractions, combining the evaluation dataset in the multimodal perspective and the Convolutional Neural Network (CNN) –Recurrent Neural Network (RNN) model based on semantic regularization. This article conducted a comparative experiment on each model on the NUS-WIDE dataset and the MS-COCO dataset. The experimental results showed that it was crucial to give full play to the expressive power of the CNN. Compared to the NUS-WIDE dataset, the MS-COCO dataset brought less additional boost by leveraging social media tags. The CIDEr score of the CNN-RNN model based on semantic regularization was improved by 11.4%, which placed the foundation for the investigation and analysis of the linguistic landscape of tourist attractions.

Comparative Analysis of Recurrent Neural Network Models Performance in Predicting Bitcoin Prices

The recurring neural network is a deep learning algorithm that is commonly used to develop prediction systems. There are many variants of RNN such as RNN itself, long-short-term memory (LSTM), and gated recurring unit, so it is frequently debatable which algorithm from the RNN family has the most optimal efficiency and computation time. When developing a prediction system, sequential or time series data is required so that an accurate prediction can be made. Sequential or time series data involve data arranged in a time sequence, such as weather data, financial data, carbon emission data, and traffic data recorded over time. This research will be carried out by predicting the three RNN models against historical Bitcoin value data. The research method used is Experimental Design by comparing the performance between the three models on bitcoin value time series data, testing is done by involving hyperparameters such as Tanh, Sigmoid, and ReLU activation functions, batch size, and epochs. The aim of this research is to find out which RNN model can produce the most optimal performance and find out what performance measures can be used to evaluate and compare the performance between the three models. The results of the study show that LSTM is the most effective model with RMSE 0.012441 and MSE 0.000155 but inefficient because it takes 3 minutes 24 seconds to run the computation; in the meantime, the Tanh activation function gives the most optimal prediction than Sigmoid and RelU and therefore should be the main candidate to be used with RNN models when predicting Bitcoin prices.

A Spatio-temporal Graph Transformer driven model for recognizing fine-grained data human activity

Human activity recognition(HAR) is an important research focus in ubiquitous computing. It has been widely applied in various domains, such as smart homes, healthcare assistance, and sports training. Accurate human activity recognition directly impacts the performance of downstream tasks. Existing methods for human activity recognition primarily rely on extracting temporal or spatial features from the data. These features are used for the task of human activity recognition. The existing methods mainly use CNN or RNN models to model the Euclidean correlations among spatially adjacent sensors or channels. However, non-Euclidean pairwise correlations among all sensors or channels are even critical for accurate classification, which has been ignored by the existing methods. In this paper, we incorporate fine-grained spatial structural data into the model to overcome these limitations. A novel deep learning model for human activity recognition is proposed, which is called the fine-grained data-oriented Spatial-Temporal Graph Transformer network (STGT). The introduction of the STGT model can eliminate the limitations of existing spatial feature extraction methods by leveraging a novel data organization approach proposed in this study. This model enhances the spatial features within the time feature extraction module for effective spatio-temporal feature extraction. We conducted experiments on four large-scale real-world HAR datasets to evaluate its performance. The experimental results demonstrate the superiority of our method over state-of-the-art approaches.

A hybrid CNN–RNN model for rainfall–runoff modeling in the Potteruvagu watershed of India

AbstractAccurate rainfall‐runoff analysis is essential for water resource management, with artificial intelligence (AI) increasingly used in this and other hydrological areas. The need for precise modelling has driven substantial advancements in recent decades. This study employed six AI models. These were the support vector regression model (SVR), the multilinear regression model (MLR), the extreme gradient boosting model (XGBoost), the long‐short‐term memory (LSTM) model, the convolutional neural network (CNN) model, and the convolutional recurrent neural network (CNN‐RNN) hybrid model. It covered 1998–2006, with 1998–2004 for calibration/training and 2005–2006 for validation/testing. Five metrics were used to measure model performance: coefficient of determination (R2), Nash‐Sutcliffe efficiency (NSE), mean absolute error (MAE), root‐mean square error (RMSE), and RMSE‐observations standard deviation ratio (RSR). The hybrid CNN‐RNN model performed best in both training and testing periods (training: R2 is 0.92, NSE is 0.91, MAE is 10.37 m3s−1, RMSE is 13.13 m3s−1, and RSR is 0.30; testing: R2 is 0.95, NSE is 0.94, MAE is 12.18 m3s−1, RMSE is 15.86 m3s−1, and RSR is 0.25). These results suggest the hybrid CNN‐RNN model is highly effective for rainfall‐runoff analysis in the Potteruvagu watershed.

Active convolutional neural networks sign language (ActiveCNN-SL) framework: a paradigm shift in deaf-mute communication

Real-time speech-to-text and text-to-speech technologies have significantly influenced the accessibility of communication for individuals who are deaf or mute. This research aims to assess the efficacy of these technologies in facilitating communication between deaf or mute individuals and those who are neither deaf nor mute. A mixed-method approach will incorporate qualitative and quantitative data collection and analysis techniques. The study will involve participants from deaf or mute and non-deaf or non-mute communities. The research will scrutinize the precision and efficiency of communication using these technologies and evaluate user experience and satisfaction. Furthermore, the study intends to pinpoint potential obstacles and limitations of these technologies and offer suggestions for enhancing their effectiveness in fostering inclusivity. The study proposes an active learning framework for sign language gesture recognition, termed Active Convolutional Neural Networks—Sign Language (ActiveCNN-SL). ActiveCNN-SL aims to minimize the labeled data required for training and augment the accuracy of sign language gesture recognition through iterative human feedback. This proposed framework holds the potential to enhance communication accessibility for deaf and mute individuals and encourage inclusivity across various environments. The proposed framework is trained using two primary datasets: (i) the Sign Language Gesture Images Dataset and (ii) the American Sign Language Letters (ASL)—v1. The framework employs Resnet50 and YoloV.8 to train the datasets. It has demonstrated high performance in terms of precision and accuracy. The ResNet model achieved a remarkable accuracy rate of 99.98% during training, and it also exhibited a validation accuracy of 100%, surpassing the baseline CNN and RNN models. The YOLOv8 model outperformed previous methods on the ASL alphabet dataset, achieving an overall mean average accuracy for all classes of 97.8%.

Predicting tidal level in tropical Eastern Bintan waters using residual long short-term memory

<span lang="EN-US">The sea brings many benefits for society, especially for a maritime country such as Indonesia. The potential in various sectors is limited only by the willingness of a party to invest in it. One such investment is in learning the knowledge and information that can be gathered from the sea, and even predicting its behavior with enough data. Using a residual LSTM algorithm, we will predict the tidal level in eastern Bintan island, a tropical island on the tip of Malay peninsula. The dataset is acquired from two sensor points in eastern Bintan coast from July 2018 to June 2019 for a span of one year, giving a total of 7,961 data points. The residual LSTM model consists of a residual wrapper with two consecutive LSTM layers and one dense layer. The model is also compared with variations of LSTM and RNN models. The result of the residual LSTM model has an MAE value of 0.1495 cm and an RMSE value of 0.3353 cm, compared to the baseline model’s 1.1148 cm and 1.4107 cm respectively. The model also has an RMSE value improvement of 76.23% compared to the base model.</span>

Content-based Filtering Movie Recommender System Using Semantic Approach with Recurrent Neural Network Classification and SGD

The application of recommendation systems has been applied in various types of platforms, especially applications for watching movies such as Netflix and Disney+. The recommendation system is purposed to make it easier for users, especially in choosing a movie because currently the number of movie productions is increasing every day. This research proposed a CBF movie recommendation system by comparing the performance of several semantic methods to be able to get the best rating prediction results. In order to improve the performance quality to get the best rating prediction results, this research utilized semantic feature methods by comparing the performance of the evaluation results produced by the TF-IDF method and word embedding applications, such as BERT, GPT-2, RoBERTa, and implemented RNN model to classify the results of rating prediction. The data were used to generate the recommendation system by involving 854 data movie and 39 accounts with a total of 34,056 movie reviews on Twitter. This research has succeeded in getting a method that produced rating predictions, namely RoBERTa. In the classification process with the RNN model and SGD optimization, the measurement results with confusion matrix by classifying the RoBERTA rating prediction obtained an evaluation value of 0.6514 loss, 95.59% accuracy, and 0.6514 precision.

Exploring the influence of crystallization fouling on microscale heat exchangers through machine learning analysis

Crystallization deposition within micro-scale heat exchangers poses significant challenges to their efficiency and functionality, stemming from the accumulation of crystalline residues on heat transfer surfaces. This study employs advanced machine learning methodologies, including GRU, LSTM, RNN, and CNN models, to explore the underlying factors influencing micro heat exchanger performance. Through meticulous analysis of key parameters such as Reynolds number, sedimentation coefficient, flow rate, and channel dimensions, the study aims to delineate the foundational factors shaping heat exchanger performance at the microscopic level. Results reveal the exceptional accuracy of CNN model in forecasting experimental outcomes, surpassing 99% accuracy and demonstrating superior performance compared to traditional numerical methods. Temperature emerges as a pivotal determinant, profoundly influencing crystallization dynamics, with its intricate interplay with solute solubility elucidated through rigorous analysis. Furthermore, comparative assessment of training times highlights the CNN model’s efficiency, attributed to its specialized architecture suited for spatial data processing. This study provides valuable insights into the impact of crystallization deposition on microscale heat exchangers, showcasing the transformative potential of machine learning in optimizing heat exchanger performance and addressing operational challenges in industrial applications.