Discrete Recurrent Neural Networks Research Articles

An Optimized Recurrent Neural Network for re-modernize food dining bowls and estimating food capacity from images

Maintaining health and preventing chronic diseases requires awareness of the energy and nutrient content of one's diet. Due to the rapid development of computing and Artificial intelligence (AI) technology, nutritional evaluation can be performed using smartphones or wearable food images.Thechallenges of this technology include estimating the visual estimation of the amount of food in a bowl. We describe an innovative approach for determining a bowl's dimensions by sticking a paper ruler throughout the bottom and edges yet photographing the outcome. We used a variety of colored liquids, bowlsand crystallized foods to evaluate the precision of our technique for measuring food volume employing spherical bowls as boxes. Different food-related datasets can have other numbers of images, types of food dining bowls and estimated food capacity from the images represented. Image preprocessing using a wiener filter enhances the estimating of food dining bowl capacity from images. Following preprocessing, a threshold-based approach to image segmentation is used for the resulting data; finally, we used the Gradient Lion Swarm Optimized Discrete Recurrent Neural Network (GLSO-DRNN) model to present an AI-based multi-dish food identification approach. An excellent mAP = 0.96 was found between the results. The suggested model shows great promise as a means of improving dish reporting because of its high classification performance.The GLSO-DRNN performed better toproving its flexibility and efficient discrete parameter optimization. In addition to offering accurate food quantity estimates, it demonstrated significant potential for other uses in the field of smart dining technology. The results highlight aneffective ofthis advanced computational method that is improving food dining bowlsand interactions with components associated with food. In the future, we plan to improve the techniques' performance and incorporate our technology into a working Smartphone based on the cloud environment.

Hidden Hypergraphs, Error-Correcting Codes, and Critical Learning in Hopfield Networks.

In 1943, McCulloch and Pitts introduced a discrete recurrent neural network as a model for computation in brains. The work inspired breakthroughs such as the first computer design and the theory of finite automata. We focus on learning in Hopfield networks, a special case with symmetric weights and fixed-point attractor dynamics. Specifically, we explore minimum energy flow (MEF) as a scalable convex objective for determining network parameters. We catalog various properties of MEF, such as biological plausibility, and then compare to classical approaches in the theory of learning. Trained Hopfield networks can perform unsupervised clustering and define novel error-correcting coding schemes. They also efficiently find hidden structures (cliques) in graph theory. We extend this known connection from graphs to hypergraphs and discover n-node networks with robust storage of memories for any . In the case of graphs, we also determine a critical ratio of training samples at which networks generalize completely.

Event-triggered H∞ state estimation for time-varying neural networks with variance-constraint and fading measurements

This paper addresses the event-triggered H∞ state estimation problem for a class of discrete recurrent neural networks subject to variance-constraint and fading measurements. The phenomena of fading measurements are described by introducing a set of mutually independent random variables, which reflect that each sensor has individual missing probability. In addition, for the purpose of energy saving, an event-triggered H∞ state estimation scheme is used for time-varying neural networks to determine whether the measurement output is transmitted to the estimator or not. Some sufficient conditions are obtained to guarantee that the estimation error system satisfies both estimation error variance constraint and prescribed H∞ performance requirement. Finally, the feasibility of the proposed event-triggered H∞ state estimation method is verified by a numerical example.

Neural-Impulsive Pinning Control for Complex Networks Based on V-Stability

In this work, a neural impulsive pinning controller for a twenty-node dynamical discrete complex network is presented. The node dynamics of the network are all different types of discrete versions of chaotic attractors of three dimensions. Using the V-stability method, we propose a criterion for selecting nodes to design pinning control, in which only a small fraction of the nodes is locally controlled in order to stabilize the network states at zero. A discrete recurrent high order neural network (RHONN) trained with extended Kalman filter (EKF) is used to identify the dynamics of controlled nodes and synthesize the control law.

State and fault estimation for nonlinear recurrent neural network systems: Experimental testing on a three‐tank system

AbstractAn observer is presented for the simultaneous estimation of the system state and actuator and sensor faults of a discrete recurrent neural network (RNN) system. The presented approach enables disturbance attenuation and guarantees observer convergence. First, the discrete RNN is converted to a discrete linear parameter varying (LPV) model. Then, the LPV model is further transformed into a descriptor system by extending the system state and sensor fault. Next, an H∞ observer is presented for the simultaneous estimation of the extended state and actuator fault of the descriptor system. Finally, the problem of observer design is translated into solving a linear matrix inequality. Experimental tests on a three‐tank system have validated the effectiveness and correctness of the presented method.

Variance-constrained resilient H_{infty } state estimation for time-varying neural networks with randomly varying nonlinearities and missing measurements

This paper addresses the resilient H_{infty } state estimation problem under variance constraint for discrete uncertain time-varying recurrent neural networks with randomly varying nonlinearities and missing measurements. The phenomena of missing measurements and randomly varying nonlinearities are described by introducing some Bernoulli distributed random variables, in which the occurrence probabilities are known a priori. Besides, the multiplicative noise is employed to characterize the estimator gain perturbation. Our main purpose is to design a time-varying state estimator such that, for all missing measurements, randomly varying nonlinearities and estimator gain perturbation, both the estimation error variance constraint and the prescribed H_{infty } performance requirement are met simultaneously by providing some sufficient criteria. Finally, the feasibility of the proposed variance-constrained resilient H_{infty } state estimation method is verified by some simulations.

A New Local Bipolar Autoassociative Memory Based on External Inputs of Discrete Recurrent Neural Networks With Time Delay.

In this paper, local bipolar auto-associative memories are presented based on discrete recurrent neural networks with a class of gain type activation function. The weight parameters of neural networks are acquired by a set of inequalities without the learning procedure. The global exponential stability criteria are established to ensure the accuracy of the restored patterns by considering time delays and external inputs. The proposed methodology is capable of effectively overcoming spurious memory patterns and achieving memory capacity. The effectiveness, robustness, and fault-tolerant capability are validated by simulated experiments.In this paper, local bipolar auto-associative memories are presented based on discrete recurrent neural networks with a class of gain type activation function. The weight parameters of neural networks are acquired by a set of inequalities without the learning procedure. The global exponential stability criteria are established to ensure the accuracy of the restored patterns by considering time delays and external inputs. The proposed methodology is capable of effectively overcoming spurious memory patterns and achieving memory capacity. The effectiveness, robustness, and fault-tolerant capability are validated by simulated experiments.

Delay Dependent Robust Stability of A Discrete Time Recurrent Neural Network with Time Varying Delays

In this paper, the robust stability analysis of a problem is investigated for a class of discrete recurrent neural networks with distributed time varying delays for delay dependent case. The problem is to determine the robust stability by employing Lyapunov–Krasovskii stability theory. The class of neural network under some consideration is globally asymptotically stable if the quadratic matrix inequality involving several parameters is less than zero. Furthermore, a Linear Matrix Inequality (LMI) approach is provided to show the stability analysis. The numerical examples are given to show the usefulness of the proposed robust stability conditions. The numerical simulation is proved using MATLAB.

A resilience approach to state estimation for discrete neural networks subject to multiple missing measurements and mixed time-delays

In this paper, the resilient state estimation problem is investigated for a class of discrete recurrent neural networks (RNNs) subject to mixed time-delays, missing measurements and stochastic disturbance. The mixed time-delays consist of randomly occurring time-delay and distributed sensor delays, where a random variable obeying the Bernoulli distribution is employed to characterize the phenomenon of randomly occurring time-delay. In addition, the phenomena of the multiple missing measurements are characterized by introducing a set of mutually independent random variables, which reflect that each sensor could have individual missing probability. Meanwhile, the additive variation of the estimator gain is considered to reflect the possible parameter deviations when implementing the state estimation algorithm. Our main purpose is to design a resilient state estimator such that, in the presence of multiple missing measurements, randomly occurring time-delay and distributed sensor delays, the estimation error dynamics is exponentially stable in the mean square. A sufficient condition is established to guarantee the existence of the resilient state estimator and the explicit expression of the desired estimator gain is given based on the solutions to some matrix inequalities. Finally, we use a numerical example to verify the validity of the presented resilient state estimation method.

A generalized bipolar auto-associative memory model based on discrete recurrent neural networks

This paper presents a novel method for designing associative memories based on discrete recurrent neural networks to accurately memorize the networks׳ external inputs. In the method, a generalized model is proposed for bipolar auto-associative memory and establishing an exponential stable criteria of the networks. The model is of generality with considering time delay and introducing a tunable slope activation function, and can robustly recall the memorized external input patterns in an auto-associative way. Experimental verification demonstrates that the proposed method is more effective and generalized than other existing ones.

Convergence Theory for Unified Discrete-Time RNNs with Quasi-Symmetric Connection

This paper presents the global convergence theory of the discrete-time uniform pseudo projection anti-monotone network with the quasi–symmetric matrix, which removes the connection matrix constraints. The theory widens the range of applications of the discrete–time uniform pseudo projection anti–monotone network and is valid for many kinds of discrete recurrent neural network models.

Stability of Discrete Recurrent Neural Networks with Interval Delays

A global exponential stability method for a class of discrete time recurrent neural networks with interval time-varying delays and norm-bounded time-varying parameter uncertainties is developed in this paper. The method is derived based on a new Lyapunov-Krasovskii functional to exhibit the delay-range-dependent dynamics and to compensate for the enlarged time-span. In addition, it eliminates the need for over bounding and utilizes smaller number of LMI decision variables. Effective solutions to the global stability problem are provided in terms of feasibility-testing of parameterized linear matrix inequalities (LMIs). Numerical examples are presented to demonstrate the potential of the developed technique.

이진 삼차 재귀 신경망과 유전자 알고리즘을 이용한 문맥-자유 문법의 추론

이 논문은 이진 삼차 재귀 신경망(Binary Third-order Recurrent Neural Networks: BTRNN)에 유전자 알고리즘을 적용하여 문맥-자유 문법을 추론하는 방법을 제안한다. BTRNN은 각 입력심볼에 대응되는 재귀 신경망들의 다층적 구조이고 외부의 스택과 결합된다. BTRNN의 매개변수들은 모두 이진수로 표현되며 상태 전이와 동시에 스택의 한 동작이 실행된다. 염색체로 표현된 BTRNN들에 유전자 알고리즘을 적용하여 긍정과 부정의 입력 패턴들의 문맥-자유 문법을 추론하는 최적의 BTRNN를 얻는다. 이 방법은 기존의 신경망 이용방법보다 적은 학습량과 적은 학습회수로 작거나 같은 상태 수를 갖는 BTRNN을 추론한다. 또한 문법 표현의 염색체 이용방법보다 parsing과정에서 결정적인 상태전이와 스택동작이 실행되므로 입력 패턴에 대한 인식처리 시간복잡도가 우수하다. 문맥-자유 문법의 비단말 심볼의 개수 p, 단말 심볼의 개수 q, 그리고 길이가 k인 문자열이 입력이 될 때, BTRNN의 최대 상태수가 m이라고 하면, BTRNN의 인식처리 병렬처리 시간은 O(k)이고 순차처리 시간은 O(km)이다. We present the method to infer Context-Free Grammars by applying genetic algorithm to the Binary Third-order Recurrent Neural Networks(BTRNN). BTRNN is a multiple-layered architecture of recurrent neural networks, each of which is corresponding to an input symbol, and is combined with external stack. All parameters of BTRNN are represented as binary numbers and each state transition is performed with any stack operation simultaneously. We apply Genetic Algorithm to BTRNN chromosomes and obtain the optimal BTRNN inferring context-free grammar of positive and negative input patterns. This proposed method infers BTRNN, which includes the number of its states equal to or less than those of existing methods of Discrete Recurrent Neural Networks, with less examples and less learning trials. Also BTRNN is superior to the recent method of chromosomes representing grammars at recognition time complexity because of performing deterministic state transitions and stack operations at parsing process. If the number of non-terminals is p, the number of terminals q, the length of an input string k, and the max number of BTRNN states m, the parallel processing time is O(k) and the sequential processing time is O(km).

Global stability results of discrete recurrent neural networks with interval delays

Global stability results of discrete recurrent neural networks with interval delays

Stability of Quasi-Periodic Orbits in Recurrent Neural Networks

A simple discrete recurrent neural network model is considered. The local stability is analyzed with the associated characteristic model. In order to study the dynamic behavior of the quasi-periodic orbit, it is necessary to determine the Neimark-Sacker bifurcation. In the case of two neurons, one necessary condition that yields the Neimark-Sacker bifurcation is found. In addition to this, the stability and direction of the Neimark-Sacker bifurcation are determined by applying normal form theory and the center manifold theorem. An example is given and a numerical simulation is performed to illustrate the results. The phase-locking phenomena are analyzed for certain experimental results with Arnold Tongues in a particular weight configuration.

Improved Delay-Dependent Stability Condition of Discrete Recurrent Neural Networks With Time-Varying Delays

This brief investigates the problem of global exponential stability analysis for discrete recurrent neural networks with time-varying delays. In terms of linear matrix inequality (LMI) approach, a novel delay-dependent stability criterion is established for the considered recurrent neural networks via a new Lyapunov function. The obtained condition has less conservativeness and less number of variables than the existing ones. Numerical example is given to demonstrate the effectiveness of the proposed method.

New results on robust exponential stability for discrete recurrent neural networks with time-varying delays

This paper is concerned with the problem of robust exponential stability analysis for uncertain discrete recurrent neural networks with time-varying delays. In terms of linear matrix inequality (LMI) approach, some novel stability conditions are proposed via a new Lyapunov function. Neither any model transformation nor free-weighting matrices are employed in our theoretical derivation. The established stability criteria significantly improve and simplify some existing stability conditions. Numerical examples are given to demonstrate the effectiveness of the proposed methods.

Uncertainty Reduction of the Flood Stage Forecasting Using Neural Networks Model1

Abstract: The Elman Discrete Recurrent Neural Networks Model (EDRNNM), which is one of the special types of neural networks model, is developed and applied for the flood stage forecasting at the Musung station (No. 1) of the Wi‐stream catchment, which is one of the International Hydrological Program representative basins, Korea. A total of 135 different training patterns, which involve hidden nodes, standardization process, data length, and lead‐time, are selected for the minimization of the architectural uncertainty. The model parameters, such as optimal connection weights and biases, are estimated during the training performance of the EDRNNM, and we apply them to evaluate the validation performance of the EDRNNM. Sensitivity analysis is used to reduce the uncertainty of input data information of the EDRNNM. As the results of sensitivity analysis, the Improved EDRNNM consists of four input nodes resulting from the exclusion of Dongkok station (No.5) in initial five input nodes group of the EDRNNM. The accuracy of flood stage forecasting during the training and validation performances of the Improved EDRNNM remains the same as that of the EDRNNM. The Improved EDRNNM, therefore, gives highly reliable flood stage forecasting. The best optimal EDRNNM, so called the Improved EDRNNM, is determined by elimination of the uncertainties of architectural and input data information in this study. Consequently, we can avoid unnecessary data collection and operate the flood stage forecasting system economically.

Training of a discrete recurrent neural network for sequence classification by using a helper FNN

This research is concerned with a gradient descent training algorithm for a target network that makes use of a helper feed-forward network (FFN) to represent the cost function required for training the target network. A helper FFN is trained because the cost relation for the target is not differentiable. The transfer function of the trained helper FFN provides a differentiable cost function of the parameter vector for the target network allowing gradient search methods for finding the optimum values of the parameters. The method is applied to the training of discrete recurrent networks (DRNNs) that are used as a tool for classification of temporal sequences of characters from some alphabet and identification of a finite state machine (FSM) that may have produced all the sequences. Classification of sequences that are input to the DRNN is based on the terminal state of the network after the last element in the input sequence has been processed. If the DRNN is to be used for classifying sequences the terminal states for class 0 sequences must be distinct from the terminal states for class 1 sequences. The cost value to be used in training must therefore be a function of this disjointedness and no more. The outcome of this is a cost relationship that is not continuous but discrete and therefore derivative free methods have to be used or alternatively the method suggested in this paper. In the latter case the transform function of the helper FFN that is trained using the cost function is a differentiable function that can be used in the training of the DRNN using gradient descent.

Winner-take-all discrete recurrent neural networks

This paper proposes a discrete recurrent neural network has simple organizations and clear dynamic behaviors. The dynamic properties of the proposed winner-take-all networks are studied detail. Simulation results are given to show network performance. Since the network model is formulated as discrete time systems, it has advantages for computer simulations over digital simulations of a continuous time neural network model. Thus they can be easily implemented in digital hardware.