Recurrent Neural Network System Research Articles

Recurrent neural networks-guided vector-valued synchronized key exchange for secure and privacy-preserving communication in Industrial Internet of Things

This paper presents an innovative approach to the problem of key exchange in the Industrial Internet of Things (IIoT) implementation. Recurrent Neural Networks (RNNs) and vector-valued neural synchronization are the key components of the proposed secure transmission method. Drive-response methods are integrated to improve speed in important applications, meeting the continuous need for effective cryptographic key exchange between IIoT devices. Conventional algorithms face drawn-out assessment procedures that jeopardize neural synchronization concealment. This work examines how random input vector generation and synchronization issues in RNNs with drive-response are affected by proportional and non-proportional delays. Furthermore, it delves into an unexplored domain, investigating the synchronization of response-based RNN systems without delays and drive-response-based RNN systems with various proportional delays. It also suggests a simplified analysis of Artificial Neural Networks (ANNs) synchronization, organizing ANNs for session key switch-over using an RNN system. To achieve polynomial and non-polynomial synchronization in the proposed driver response systems, this method offers several benefits: it introduces a polynomial synchronization theory for RNNs to generate synchronized input vectors for ANN synchronization; it uses inequality assessment techniques and Lyapunov formulas to derive relevant control inputs and time-dependent conditions; it establishes the relationship between polynomial and non-polynomial synchronization; and it provides numerical examples demonstrating its effectiveness. It also builds a neural network for generating session keys throughout the IIoT network by aligning vector-valued ANNs in a reciprocal manner. This strategy outperforms previous approaches in the literature, validated through simulations, balances resilience against attacks with minimal computational load, resulting in more effective and resilient industrial applications.

An improved intelligent technique for maximum power point tracking under partial shading conditions of photo voltaic system

A novel technique is presented for Maximum Power Point Tracking (MPPT) based photovoltaic (PV) system in partial shadow conditions for harvesting maximum power. In this paper, a hybrid technique is developed, which combines Black Widow Optimization (BWO) with Recurrent Neural Network (RNN). To train the data set and provide a control signal for the converter, an RNN is used. After fitting the training data sets, the suggested method achieved maximum power by utilizing BWO based on the control parameters. This proposed method minimizes the difference between actual and average power. Using an optimization technique, the main goal of this proposed strategy is to obtain peak power harvest under various conditions, including partial shading, while minimizing error function, With the help of MATLAB/Simulink software, the conclusions are revealed under various partial shading conditions. For each category, the observed results are evaluated at various time intervals. The proposed method is also compared to other techniques such as the Ant Colony Optimization (ACO)-RNN system, Particle Swarm Optimization (PSO)-RNN system, and Gravitational Search Algorithm (GSA)-RNN system. The proposed system is 36.11% faster than GSA with RNN, 39.47% faster than PSO, and 42.5% faster than ACO with RNN in terms of tracking speed. Significantly, the proposed work is 0.87% more efficient than the other models in terms of obtaining maximum power. In terms of obtaining maximum power, the proposed work BWOA-RNN is more effective than other methods.

Sun Position Forecasting Menggunakan Metode RNN – LSTM Sebagai Referensi Pengendalian Daya Solar Cell

Solar energy is used as electrical energy for daily life by using solar cell. There are several models for maximizing the absorption of solar cell power by using reflectors and solar tracking. Reflector focuses sunlight on the solar cell to get maximum light. Solar tracking real-time tracking the sun by moving two motors and sensors to read the direction of the sun's angle. Both have the disadvantage of tracking the sun continuously. So it is necessary to optimize the system to determine the tilt angle of the actuator and be able to streamline the intensity of sunlight that can be absorbed by solar cells with the identification system that can be done in realtime or directly to minimize the occurrence of power wastage. Prediction is one of the most important elements for decision-makers of the problems that occur above. In this paper, the method used in predicting the sun's angle is by using the Recurrent Neural Network (RNN) method which has a Long-Short Term Memory (LSTM) structure in the RNN system. In this reseacrh, the data used to predict the sun angle from the sunscalc.org site in the area of the University of Muhammadiyah Malang campus 3 with a period of one year. Testing with the RNN-LSTM structure is done with two different prediction models, namely weekly, monthly, and annual data for daily data results, and hourly data for daily data for one week. The test results on weekly data detect Root Mean Square Error of 0.12%, the monthly data is 0.1% and the annual data is 0.24%. The monthly data model has the fewest error values, so predictive data has a high degree of accuracy.

Deep Learning Prediction of Polycyclic Aromatic Hydrocarbons in the High Arctic.

Given the lack of understanding of the complex physiochemical and environmental processes of persistent organic pollutants (POPs) in the Arctic and around the globe, atmospheric models often yield large errors in the predicted atmospheric concentrations of POPs. Here, we developed a recurrent neural network (RNN) method based on nonparametric deep learning algorithms. The RNN model was implemented to predict monthly air concentrations of polycyclic aromatic hydrocarbons (PAHs) at the high Arctic monitoring station Alert. To train the RNN system, we used MODIS satellite remotely sensed forest fire data, air emissions, meteorological data, sea ice cover area, and sampled PAH concentration data from 1996 to 2012. The system was applied to forecast monthly PAH concentrations from 2012 to 2014 at the Alert station. The results were compared with monitored PAHs and an atmospheric transport model (CanMETOP) for POPs. We show that the RNN significantly improved PHE and BaP predictions from 2012 to 2014 by 62.5 and 91.1%, respectively, compared to CanMETOP predictions. The sensitivity analysis using the Shapley value reveals that air emissions determined the magnitude of PAH levels in the high Arctic, whereas forest fires played a significant role in the changes in PAH concentrations in the high Arctic, followed by air temperature and meridional wind fields.

Recognizing online handwritten Chinese characters using RNNs with new computing architectures

Although online handwritten Chinese characters recognition has been explored for decades, it is still a challenging task to recognize handwritten Chinese characters accurately. In this paper, we propose an end-to-end recognizer for online handwritten Chinese characters recognition based on a new recurrent neural network (RNN). In the system, two new computing architectures are proposed based on traditional RNN system. One is the variance constraint, and the other is attention weight vector. The variance constraint is used to increase the representation ability of RNN, and the attention weight vector is used to describe the importance of hidden layer states at different time steps. Benefited from the two innovations, the recognition system obtains higher recognition accuracy with fewer parameters. Experiments are carried out on two handwritten Chinese character datasets, IAHCC-UCAS2016 dataset and ICDAR-2013 competition database. The experimental results show that the two innovations are effective, and the proposed end-to-end recognizer obtains better performance than the state-of-the-art methods.

Drawing and Recognizing Chinese Characters with Recurrent Neural Network.

Recent deep learning based approaches have achieved great success on handwriting recognition. Chinese characters are among the most widely adopted writing systems in the world. Previous research has mainly focused on recognizing handwritten Chinese characters. However, recognition is only one aspect for understanding a language, another challenging and interesting task is to teach a machine to automatically write (pictographic) Chinese characters. In this paper, we propose a framework by using the recurrent neural network (RNN) as both a discriminative model for recognizing Chinese characters and a generative model for drawing (generating) Chinese characters. To recognize Chinese characters, previous methods usually adopt the convolutional neural network (CNN) models which require transforming the online handwriting trajectory into image-like representations. Instead, our RNN based approach is an end-to-end system which directly deals with the sequential structure and does not require any domain-specific knowledge. With the RNN system (combining an LSTM and GRU), state-of-the-art performance can be achieved on the ICDAR-2013 competition database. Furthermore, under the RNN framework, a conditional generative model with character embedding is proposed for automatically drawing recognizable Chinese characters. The generated characters (in vector format) are human-readable and also can be recognized by the discriminative RNN model with high accuracy. Experimental results verify the effectiveness of using RNNs as both generative and discriminative models for the tasks of drawing and recognizing Chinese characters.

A system of recurrent neural networks for modularising, parameterising and dynamic analysis of cell signalling networks

In this paper, we show how to extend our previously proposed novel continuous time Recurrent Neural Networks (RNN) approach that retains the advantage of continuous dynamics offered by Ordinary Differential Equations (ODE) while enabling parameter estimation through adaptation, to larger signalling networks using a modular approach. Specifically, the signalling network is decomposed into several sub-models based on important temporal events in the network. Each sub-model is represented by the proposed RNN and trained using data generated from the corresponding ODE model. Trained sub-models are assembled into a whole system RNN which is then subjected to systems dynamics and sensitivity analyses. The concept is illustrated by application to G1/S transition in cell cycle using Iwamoto et al. (2008) ODE model. We decomposed the G1/S network into 3 sub-models: (i) E2F transcription factor release; (ii) E2F and CycE positive feedback loop for elevating cyclin levels; and (iii) E2F and CycA negative feedback to degrade E2F. The trained sub-models accurately represented system dynamics and parameters were in good agreement with the ODE model. The whole system RNN however revealed couple of parameters contributing to compounding errors due to feedback and required refinement to sub-model 2. These related to the reversible reaction between CycE/CDK2 and p27, its inhibitor. The revised whole system RNN model very accurately matched dynamics of the ODE system. Local sensitivity analysis of the whole system model further revealed the most dominant influence of the above two parameters in perturbing G1/S transition, giving support to a recent hypothesis that the release of inhibitor p27 from Cyc/CDK complex triggers cell cycle stage transition. To make the model useful in a practical setting, we modified each RNN sub-model with a time relay switch to facilitate larger interval input data (≈20min) (original model used data for 30s or less) and retrained them that produced parameters and protein concentrations similar to the original RNN system. Results thus demonstrated the reliability of the proposed RNN method for modelling relatively large networks by modularisation for practical settings. Advantages of the method are its ability to represent accurate continuous system dynamics and ease of: parameter estimation through training with data from a practical setting, model analysis (40% faster than ODE), fine tuning parameters when more data are available, sub-model extension when new elements and/or interactions come to light and model expansion with addition of sub-models.

Language Identification in Short Utterances Using Long Short-Term Memory (LSTM) Recurrent Neural Networks.

Long Short Term Memory (LSTM) Recurrent Neural Networks (RNNs) have recently outperformed other state-of-the-art approaches, such as i-vector and Deep Neural Networks (DNNs), in automatic Language Identification (LID), particularly when dealing with very short utterances (∼3s). In this contribution we present an open-source, end-to-end, LSTM RNN system running on limited computational resources (a single GPU) that outperforms a reference i-vector system on a subset of the NIST Language Recognition Evaluation (8 target languages, 3s task) by up to a 26%. This result is in line with previously published research using proprietary LSTM implementations and huge computational resources, which made these former results hardly reproducible. Further, we extend those previous experiments modeling unseen languages (out of set, OOS, modeling), which is crucial in real applications. Results show that a LSTM RNN with OOS modeling is able to detect these languages and generalizes robustly to unseen OOS languages. Finally, we also analyze the effect of even more limited test data (from 2.25s to 0.1s) proving that with as little as 0.5s an accuracy of over 50% can be achieved.

Stability Analysis of Discrete-Time Recurrent Neural Networks With Stochastic Delay

This paper is concerned with the stability analysis of discrete-time recurrent neural networks (RNNs) with time delays as random variables drawn from some probability distribution. By introducing the variation probability of the time delay, a common delayed discrete-time RNN system is transformed into one with stochastic parameters. Improved conditions for the mean square stability of these systems are obtained by employing new Lyapunov functions and novel techniques are used to achieve delay dependence. The merit of the proposed conditions lies in its reduced conservatism, which is made possible by considering not only the range of the time delays, but also the variation probability distribution. A numerical example is provided to show the advantages of the proposed conditions.

Absolute Exponential Stability of Recurrent Neural Networks With Generalized Activation Function

In this paper, the recurrent neural networks (RNNs) with a generalized activation function class is proposed. In this proposed model, every component of the neuron's activation function belongs to a convex hull which is bounded by two odd symmetric piecewise linear functions that are convex or concave over the real space. All of the convex hulls are composed of generalized activation function classes. The novel activation function class is not only with a more flexible and more specific description of the activation functions than other function classes but it also generalizes some traditional activation function classes. The absolute exponential stability (AEST) of the RNN with a generalized activation function class is studied through three steps. The first step is to demonstrate the global exponential stability (GES) of the equilibrium point of original RNN with a generalized activation function being equivalent to that of RNN under all vertex functions of convex hull. The second step transforms the RNN under every vertex activation function into neural networks under an array of saturated linear activation functions. Because the GES of the equilibrium point of three systems are equivalent, the next stability analysis focuses on the GES of the equilibrium point of RNN system under an array of saturated linear activation functions. The last step is to study both the existence of equilibrium point and the GES of the RNN under saturated linear activation functions using the theory of M-matrix. In the end, a two-neuron RNN with a generalized activation function is constructed to show the effectiveness of our results.

On the analysis of a recurrent neural network for solving nonlinear monotone variational inequality problems

We investigate the qualitative properties of a recurrent neural network (RNN) for solving the general monotone variational inequality problems (VIPs), defined over a nonempty closed convex subset, which are assumed to have a nonempty solution set but need not be symmetric. The equilibrium equation of the RNN system simply coincides with the nonlinear projection equation of the VIP to be solved. We prove that the RNN system has a global and bounded solution trajectory starting at any given initial point in the above closed convex subset which is positive invariant for the RNN system. For general monotone VIPs, we show by an example that the trajectory of the RNN system can converge to a limit cycle rather than an equilibrium in the case that the monotone VIPs are not symmetric. Contrary to this, for the strictly monotone VIPs, it is shown that every solution trajectory of the RNN system starting from the above closed convex subset converges to the unique equilibrium which is also locally asymptotically stable in the sense of Lyapunov, no matter whether the VIPs are symmetric or nonsymmetric. For the uniformly monotone VIPs, the aforementioned solution trajectory of the RNN system converges to the unique equilibrium exponentially.

A recurrent neural network for nonlinear continuously differentiable optimization over a compact convex subset

We propose a general recurrent neural-network (RNN) model for nonlinear optimization over a nonempty compact convex subset which includes the bound subset and spheroid subset as special cases. It is shown that the compact convex subset is a positive invariant and attractive set of the RNN system and that all the network trajectories starting from the compact convex subset converge to the equilibrium set of the RNN system. The above equilibrium set of the RNN system coincides with the optimum set of the minimization problem over the compact convex subset when the objective function is convex. The analysis of these qualitative properties for the RNN model is conducted by employing the properties of the projection operator of Euclidean space onto the general nonempty closed convex subset. A numerical simulation example is also given to illustrate the qualitative properties of the proposed general RNN model for solving an optimization problem over various compact convex subsets.