Class Of Recurrent Neural Networks Research Articles

Hybrid deep learning approach for sentiment analysis using text and emojis.

Sentiment Analysis (SA) is a technique for categorizing texts based on the sentimental polarity of people's opinions. This paper introduces a sentiment analysis (SA) model with text and emojis. The two preprocessed data's are data with text and emojis and text without emojis. Feature extraction consists text features and text with emojis features. The text features are features like N-grams, modified Term Frequency-Inverse Document Frequency (TF-IDF), and Bag-of-Words (BoW) features extracted from the text. In classification, CNN (Conventional Neural Network) and MLP (Multi-Layer Perception) use emojis and text-based SA. The CNN weight is optimized by a new Electric fish Customized Shark Smell Optimization (ECSSO) Algorithm. Similarly, the text-based SA is carried out by hybrid Long Short-Term Memory (LSTM) and Recurrent Neural Network (RNN) classifiers. The bagged data are given as input to the classification process via RNN and LSTM. Here, the weight of LSTM is optimized by the suggested ECSSO algorithm. Then, the mean of LSTM and RNN determines the final output. The specificity of the developed scheme is 29.01%, 42.75%, 23.88%,22.07%, 25.31%, 18.42%, 5.68%, 10.34%, 6.20%, 6.64%, and 6.84% better for 70% than other models. The efficiency of the proposed scheme is computed and evaluated.

Spike-based computation using classical recurrent neural networks

Spiking neural networks (SNNs) are a type of artificial neural networks in which communication between neurons is only made of events, also called spikes. This property allows neural networks to make asynchronous and sparse computations and therefore drastically decrease energy consumption when run on specialized hardware. However, training such networks is known to be difficult, mainly due to the non-differentiability of the spike activation, which prevents the use of classical backpropagation. This is because state-of-the-art SNNs are usually derived from biologically-inspired neuron models, to which are applied machine learning methods for training. Nowadays, research about SNNs focuses on the design of training algorithms whose goal is to obtain networks that compete with their non-spiking version on specific tasks. In this paper, we attempt the symmetrical approach: we modify the dynamics of a well-known, easily trainable type of recurrent neural network (RNN) to make it event-based. This new RNN cell, called the spiking recurrent cell, therefore communicates using events, i.e. spikes, while being completely differen-tiable. Vanilla backpropagation can thus be used to train any network made of such RNN cell. We show that this new network can achieve performance comparable to other types of spiking networks in the MNIST benchmark and its variants, the Fashion-MNIST and the Neuromorphic-MNIST. Moreover, we show that this new cell makes the training of deep spiking networks achievable.

A novel transformer-based DL model enhanced by position-sensitive attention and gated hierarchical LSTM for aero-engine RUL prediction

Accurate prediction of remaining useful life (RUL) for aircraft engines is essential for proactive maintenance and safety assurance. However, existing methods such as physics-based models, classical recurrent neural networks, and convolutional neural networks face limitations in capturing long-term dependencies and modeling complex degradation patterns. In this study, we propose a novel deep-learning model based on the Transformer architecture to address these limitations. Specifically, to address the issue of insensitivity to local context in the attention mechanism employed by the Transformer encoder, we introduce a position-sensitive self-attention (PSA) unit to enhance the model's ability to incorporate local context by attending to the positional relationships of the input data at each time step. Additionally, a gated hierarchical long short-term memory network (GHLSTM) is designed to perform regression prediction at different time scales on the latent features, thereby improving the accuracy of RUL estimation for mechanical equipment. Experiments on the C-MAPSS dataset demonstrate that the proposed model outperforms existing methods in RUL prediction, showcasing its effectiveness in modeling complex degradation patterns and long-term dependencies.

An adaptive Marine Predator Optimization Algorithm (MPOA) integrated Gated Recurrent Neural Network (GRNN) classifier model for arrhythmia detection

Cardiovascular disorders are typically diagnosed using an Electrocardiogram (ECG). It is a painless method that mimics the cyclical contraction and relaxation of the heart's muscles. By monitoring the electrical activity of the heart, an ECG can be used to identify irregular heartbeats, strokes, cardiac illnesses, or enlarged hearts. Numerous studies and analyses of ECG signals to identify heart problems have been conducted during the past few years. The disadvantages of the current and hybrid approaches include low accuracy, prolonged time, and a dearth of high-quality datasets. The proposed work aims to design a new detection tool for the diagnosis of Arrhythmia from ECG signals. In this study, the wavelet transformation technique is used for preprocessing after collecting the input ECG signal in order to create the quality-improved and normalized signals. The most pertinent characteristics are then selected from the preprocessed output using the cutting-edge Marine Predator Optimization Algorithm (MPOA), which accelerates classifier learning. In addition, the hyper tuned Gated Recurrent Neural Network (Hyp-GRNN) method is used to learn the most effective features and properly classify the normal and arrhythmia-affected signals. The Ebola Optimization Algorithm (EOSA) is used to execute the hyper parameter tweaking in order to enhance the accuracy and training time of the classifier. At the time of evaluation, performance is assessed and compared using a variety of metrics, including accuracy, precision, recall, F-measure and etc.

An Incremental Input-to-State Stability Condition for a Class of Recurrent Neural Networks

An Incremental Input-to-State Stability Condition for a Class of Recurrent Neural Networks

Optimizing dynamic keystroke pattern recognition with hybrid deep learning technique and multiple soft biometric factors

In this work, we propose an optimization approach for dynamic keystroke pattern recognition by leveraging a hybrid deep learning technique and multiple soft biometric factors. Our methodology begins with the introduction of a novel algorithm called dynamic drone squadron optimization (DDSO) to optimize the selection of optimal features from a pool of multiple keystroke features. We then present an enhanced version of the improved sperm swarm optimization (ISSO) algorithm, which effectively combines the optimal weight features derived from multiple biometric responses. Furthermore, we introduce the multi-stage recurrent neural network (MS-RNN) classifier to accurately recognize and classify keystroke patterns. The performance of our proposed ISSO+MS-RNN technique is evaluated using the benchmark KBOC dataset to validate its effectiveness. Comparative analysis is conducted against existing state-of-the-art techniques, employing various evaluation measures, to demonstrate the superior performance of proposed approach.`

Global inverse optimality for a class of recurrent neural networks with multiple proportional delays

This paper formulates two novel theoretical designs of input-to-state stabilizing control for a class of recurrent neural networks with multiple proportional delays. The analysis tool developed in this paper is based on Lyapunov function and inverse optimality method, which does not require solving Hamilton-Jacobi-Bellman equations. Two inverse optimal feedback laws are constructed via the dimensions of state and input, which ensure the input-state stability for the considered system. When the dimensions of state and input are different, we establish a scalar function and give one of the control laws by Sontag's formula. Furthermore, the designs of inverse optimal control reach both global inverse optimality and global asymptotic stability of the system for some meaningful cost functional. Four numerical examples are provided to show the effectiveness of the inverse optimal control.

Determination of railway switch state using a recurrent neural network classifier

Determination of railway switch state using a recurrent neural network classifier

First/second-order predefined-time convergent ZNN models for time-varying quadratic programming and robotic manipulator application

Zeroing neural network (ZNN) model, an important class of recurrent neural network, has been widely applied in the field of computation and optimization. In this paper, two ZNN models with predefined-time convergence are proposed for the time-varying quadratic programming (TVQP) problem. First, in the framework of the traditional ZNN model, the first-order predefined-time convergent ZNN (FPTZNN) model is proposed in combination with a predefined-time controller. Compared with the existing ZNN models, the proposed ZNN model is error vector combined with sliding mode control technique. Then, the FPTZNN model is further extended and the second-order predefined-time convergent ZNN (SPTZNN) model is developed. Combined with the Lyapunov method and the concept of predefined-time stability, it is shown that the proposed FPTZNN and SPTZNN models have the properties of predefined-time convergence, and their convergence time can be flexibly adjusted by predefined-time control parameters. Finally, the proposed FPTZNN and SPTZNN models are compared with the existing ZNN models for the TVQP problem in simulation experiment, and the simulation experiment results verify the effectiveness and superior performance of the proposed FPTZNN and SPTZNN models. In addition, the proposed FPTZNN model for robot motion planning problem is applied and successfully implemented to verify the practicality of the model.

Stable fixed points of combinatorial threshold-linear networks

Combinatorial threshold-linear networks (CTLNs) are a special class of recurrent neural networks whose dynamics are tightly controlled by an underlying directed graph. Recurrent networks have long been used as models for associative memory and pattern completion, with stable fixed points playing the role of stored memory patterns in the network. In prior work, we showed that target-free cliques of the graph correspond to stable fixed points of the dynamics, and we conjectured that these are the only stable fixed points possible [19,8]. In this paper, we prove that the conjecture holds in a variety of special cases, including for networks with very strong inhibition and graphs of size n≤4. We also provide further evidence for the conjecture by showing that sparse graphs and graphs that are nearly cliques can never support stable fixed points. Finally, we translate some results from extremal combinatorics to obtain an upper bound on the number of stable fixed points of CTLNs in cases where the conjecture holds.

Performance enhancement of wind power forecast model using novel pre‐processing strategy and hybrid optimization approach

SummaryDue to the energy crisis and environmental concerns, wind power has seen a considerable increase in use over the past 10 years as a source of renewable energy. Since wind is intermittent and variable, it is obvious that as penetration levels rise, the influence of wind power generation on the electric power system must be considered. Wind power forecasting is essential because large‐scale wind power integration will make it more difficult to plan, operate, and control the power system. An accurate forecast is an efficient way to deal with the operational problems brought on by wind variability. To better utilize wind energy resources, it is essential to increase prediction accuracy. Frequently, the noise in the dataset causes the ramp events to be misclassified or overestimated. The main emphasis of this study is the pre‐processing of wind power data that produces precise time‐series data while reducing noise or artifact content and maintaining the swing feature of the original data. For this task, the data augmentation approach is proposed, where the augmented data (synthetic data) will be added up with the training data, in such a way as to make the strategy useful for real‐time applications. As the next step, the significant features, such as higher‐order statistical features and lower‐order statistical features are extracted. The extracted features act as the input to the recurrent neural network (RNN) classifier, the weights of which are tuned using the proposed honey badger crow (HBCro) optimization algorithm. The proposed HBCro optimization algorithm acts as the major contribution of the proposed model, and it is modeled with the integration of the concepts of the crow search optimization (CSO) algorithm and the honey badger optimization algorithm (HBA). The proposed system is simulated in MATLAB and the effectiveness of the proposed method is validated by comparison with other conventional methods in terms of Error measures. Furthermore, the developed HBCro‐based RNN obtained efficient performance in terms of MSE, MAE, RMSE, RMPSE, MAPE, MARE, MSRE, and RMSRE of 0.1578, 0.0442, 0.2102, 123.238, 124.72, 0.9944, 1.1799, and 1.0732, respectively.

Finite time convergent recurrent neural network for variational inequality problems subject to equality constraints

This paper aims to present a new class of recurrent neural networks based on projection operators for solving variational inequalities and related optimization problems subject to linear equalities and bound constraints. In the proposed approach, the neural network structure is based on the Karush–Kuhn–Tucker (KKT) optimality conditions. Instead of commonly used activation functions, the KKT multipliers are considered control inputs and implemented with finite time stabilizing terms based on unit control. The output variables of the neural network are proven to be stable in the sense of Lyapunov and convergent to optimal solutions in finite time. In addition, a methodology based on the particle swarm optimization (PSO) algorithm is presented for the optimal selection of network design parameters. The main advantage of the proposed neural network is its fixed number of parameters regardless of problem dimension, enabling the network to be easily scalable from a lower to a higher dimension. Finally, simulation results of numerical examples are provided to illustrate the effectiveness and performance of the proposed neural network.

Time-warping invariant quantum recurrent neural networks via quantum-classical adaptive gating

Adaptive gating plays a key role in temporal data processing via classical recurrent neural networks (RNNs), as it facilitates retention of past information necessary to predict the future, providing a mechanism that preserves invariance to time warping transformations. This paper builds on quantum RNNs (QRNNs), a dynamic model with quantum memory, to introduce a novel class of temporal data processing quantum models that preserve invariance to time-warping transformations of the (classical) input-output sequences. The model, referred to as time warping-invariant QRNN (TWI-QRNN), augments a QRNN with a quantum–classical adaptive gating mechanism that chooses whether to apply a parameterized unitary transformation at each time step as a function of the past samples of the input sequence via a classical recurrent model. The TWI-QRNN model class is derived from first principles, and its capacity to successfully implement time-warping transformations is experimentally demonstrated on examples with classical or quantum dynamics.

Time series classification models based on nonlinear spiking neural P systems

Reservoir computing (RC) is a novel class of recurrent neural networks (RNN) models. Nonlinear spiking neural P (NSNP) systems are neural-like computing models with nonlinear spiking mechanisms. By introducing NSNP systems as the reservoir, we propose a new RC model for time series classification task, termed TSC-NSNP model. However, due to the high-dimensional nature of the reservoir state space, the TSC-NSNP model, like existing RC models, will encounter some challenges. To address the challenges. we utilize the reservoir model space representation and dimensionality reduction method to propose two improved models, termed TSC-DR-NSNP model and TSC-RMS-NSNP model. The three RC models can be easily realized and learnt in the RC framework. The proposed three RC models are evaluated on 21 benchmark time series classification data sets, and are compared with 20 classification models. The comparisons demonstrate the effectiveness of the presented three RC models for time series classification tasks.

Bi-Level Intrusion Detection in IoT Networks Using Ensemble Method and A-GRU-RNN Classifier

For minimizing security vulnerabilities and attacks, which affect Internet of Things (IoT) applications’ performance, it is essential to design an efficient authentication protocol owing to the rapid deployment of IoT. This paper proposes a bi-level Intrusion Detection (ID) in IoT using an ensemble and Arctan-based Gated Recurrent Unit-Recurrent Neural Network (A-GRU-RNN) classifier. This work mainly concentrates on both major and minor attacks. Allowing the trusted nodes to join the network is the other motive of this work. Here, the nodes are registered to the network and then initialized. Then, to verify the trust levels of the nodes, test packet transmission is performed. By employing the Deterministic Initialization Method-centric K-Means algorithm (DIM-K-Means) algorithm, the trusted nodes are formed into the cluster. Next, the respective CH is selected by Multiplex-Valued Encoding Sea Lion Optimization (MVE-SLO). Afterwards, by employing Multi-Point Relays-Optimized Link State Routing (MPR-OLSR), routing is taken place. The steps, namely preprocessing, attribute extraction, attribute reduction, and classification are taken to detect the confidentiality of the sensed data. The classification phase significantly determines whether the data is attacked or not. If the data is attacked, then it detects the attack type. Here, the publicly available dataset is used. As per the experimental outcome, the proposed method withstands high-security levels when analogized to the prevailing methodologies.

Improving biosensor accuracy and speed using dynamic signal change and theory-guided deep learning

False results and time delay are longstanding challenges in biosensing. While classification models and deep learning may provide new opportunities for improving biosensor performance, such as measurement confidence and speed, it remains a challenge to ensure that predictions are explainable and consistent with domain knowledge. Here, we show that consistency of deep learning classification model predictions with domain knowledge in biosensing can be achieved by cost function supervision and enables rapid and accurate biosensing using the biosensor dynamic response. The impact and utility of the methodology were validated by rapid and accurate quantification of microRNA (let-7a) across the nanomolar (nM) to femtomolar (fM) concentration range using the dynamic response of cantilever biosensors. Data augmentation and cost function supervision based on the consistency of model predictions and experimental observations with the theory of surface-based biosensors improved the F1 score, precision, and recall of a recurrent neural network (RNN) classifier by an average of 13.8%. The theory-guided RNN (TGRNN) classifier enabled quantification of target analyte concentration and false results with an average prediction accuracy, precision, and recall of 98.5% using the initial transient or entire dynamic response, which is indicative of high prediction accuracy and low probability of false-negative and false-positive results. Classification scores were used to establish new relationships among biosensor performance characteristics (e.g., measurement confidence) and design parameters (e.g., inputs and hyperparameters of classification models and data acquisition parameters) that may be used for characterizing biosensor performance.

Systematic Bias Correction in Ocean Mesoscale Forecasting Using Machine Learning

AbstractThe ocean circulation is modulated by meandering currents and eddies. Forecasting their evolution is a key target of operational models, but their forecast skill remains limited. We propose a machine learning approach that improves the output of an ocean circulation model by learning and predicting its systematic biases. This method can be applied a priori to any region, and is tested in the Gulf of Mexico, where the Loop Current (LC) and the large anticyclonic eddies that detach from it are major forecasting targets. The LC dynamics are recurrent and lie on a low‐dimensional dynamical attractor. Building upon the information gained analyzing this low dimensional attractor, we improve the representation of sea surface anomalies in model outputs through information from satellite altimeter data using a Sequence‐to‐Sequence model, which is a special class of Recurrent Neural Network. Building upon the HYCOM‐NCODA analysis system, we deliver a correction to the forecast at the observation resolution. For at least 15 days the proposed method learns to forecast the systematic bias in the HYCOM‐NCODA, outperforming persistence, and improving the forecast. This data‐driven approach is fast and can be implemented as an added step to any dynamical hindcasting or forecasting model. It offers an interesting avenue for further developing hybrid modeling tools. In these tools, fundamental physical conservations are preserved through the integration of partial differential equations which obey them. In addition, the method highlights specific deficiencies of the hindcast system that deserve further investigation in the future.

Improved Criteria for Stability of a Class of Recurrent Neural Networks With Generalized Piecewise Constant Argument

Improved Criteria for Stability of a Class of Recurrent Neural Networks With Generalized Piecewise Constant Argument

A comparative study of the neural network models for the stock market data classification—A multicriteria optimization approach

The aim of this paper is to explore the potential of the class of recurrent neural networks in developing a classification model for the stock market. For this research, the data that replicates the entire Nasdaq stock market limit order book is used. After extracting order book attributes from the raw dataset, feature selection approaches based on conditional entropy and Stochastic Universal Sampling are proposed, in order to highlight potentially informative features for our data classification task. Furthermore, the performances of nine presented recurrent neural network models for classifying the stock market data vector into one of the labels from the set S = {buy, sell, idle} are examined. In this study, the performances of the models based on different networks, namely the gated recurrent unit, the long short-term memory, and the recurrent neural network, are compared. Using a multi-criteria approach it is concluded that the model based on the GRU topology fed with the features selected using the newly proposed feature selection method outperforms other examined models.

Improved Multiresolution Neural Network for Mobility-Aware Security and Content Caching for Internet of Vehicles

With the emerging communication and computation technologies, the Internet of Vehicles (IoV) has become a new paradigm that enables vehicles to communicate with Road Side Units (RSUs) as well as with other vehicles to collect or exchange information. However, the inherent characteristics of IoV, such as the high mobility of the vehicle and the limited storage capacity of the edge nodes, cause numerous difficulties in developing a caching scheme. Therefore, in this paper, a novel improved Multinomial Recurrent Neural Network (MRNN) classifier and 2CK-ECC algorithm are proposed to predict the vehicular mobility, security, and content caching for the IoV. Initially, the vehicles are checked for registration. Afterwards, vehicle login and vehicle authentication take place. Then, mobility prediction is carried out for the authenticated vehicles using the MRNN Classifier. After that, vehicle clustering is done via the novel Variance Balanced Iterative Reducing and Clustering Using Hierarchies (VBIRCH) technique. Next, the Relay Vehicles (RVs) are selected using the novel Harris Self Avoiding Hawks Optimization (HSAHO). Further, the data contents are divided into chunks and saved in the cache memory. From the cache memory, data is transferred to the RSU. Finally, a novel Caesar Combined Key-based Elliptic Curve Cryptography (2CK-ECC) algorithm is investigated for secure data transmission. The experimental outcomes demonstrate that the proposed technique outperforms the existing base line approaches.