Standard Cellular Neural Networks Research Articles

Learning for Cellular Neural Networks Using the GARPLA Algorithm

The main content of this article is to build a hybrid algorithm between genetic algorithm and Recurrent Perceptron Learning Algorithm to determine the weight set for standard cellular neural networks. In particular, the Recurrent Perceptron Learning Algorithm was announced by Gukzelis in 1998 to determine the weight set for CeNNs, however, the nature of the algorithm built according to LMS learning rules means there is still a possibility of local convergence problems of algorithm. Meanwhile, the genetic algorithm originates from natural selection problems and is capable of determining the local optimal domain of the problem. Therefore, the GARPLA hybrid algorithm will fully determine the optimal weight set for the CeNNs network, reducing potential local optimization problems encountered in the RPLA algorithm.

Multistability of Dynamic Memristor Delayed Cellular Neural Networks With Application to Associative Memories.

Recently, dynamic memristor (DM)-cellular neural networks (CNNs) have received widespread attention due to their advantage of low power consumption. The previous works showed that DM-CNNs have at most 318 equilibrium points (EPs) with n=16 cells. Since time delay is unavoidable during the process of information transmission, the goal of this article is to research the multistability of DM-CNNs with time delay, and, meanwhile, to increase the storage capacity of DM-delay (D)CNNs. Depending on the different constitutive relations of memristors, two cases of the multistability for DM-DCNNs are discussed. After determining the constitutive relations, the number of EPs of DM-DCNNs is increased to 3n with n cells by means of the appropriate state-space decomposition and the Brouwer's fixed point theorem. Furthermore, the enlarged attraction domains of EPs can be obtained, and 2n of these EPs are locally exponentially stable in two cases. Compared with standard CNNs, the dynamic behavior of DM-DCNNs shows an outstanding merit. That is, the value of voltage and current approach to zero when the system becomes stable, and the memristor provides a nonvolatile memory to store the computation results. Finally, two numerical simulations are presented to illustrate the effectiveness of the theoretical results, and the applications of associative memories are shown at the end of this article.

Memristor standard cellular neural networks computing in the flux–charge domain

The paper introduces a class of memristor neural networks (NNs) that are characterized by the following salient features. (a) The processing of signals takes place in the flux–charge domain and is based on the time evolution of memristor charges. The processing result is given by the constant asymptotic values of charges that are stored in the memristors acting as non-volatile memories in steady state. (b) The dynamic equations describing the memristor NNs in the flux–charge domain are analogous to those describing, in the traditional voltage–current domain, the dynamics of a standard (S) cellular (C) NN, and are implemented by using a realistic model of memristors as that proposed by HP. This analogy makes it possible to use the bulk of results in the SCNN literature for designing memristor NNs to solve processing tasks in real time. Convergence of memristor NNs in the presence of multiple asymptotically stable equilibrium points is addressed and some applications to image processing tasks are presented to illustrate the real-time processing capabilities. Computing in the flux–charge domain is shown to have significant advantages with respect to computing in the voltage–current domain. One advantage is that, when a steady state is reached, currents, voltages and hence power in a memristor NN vanish, whereas memristors keep in memory the processing result. This is basically different from SCNNs for which currents, voltages and power do not vanish at a steady state, and batteries are needed to keep in memory the processing result.

New Conditions for Global Asymptotic Stability of Memristor Neural Networks.

Recent papers in the literature introduced a class of neural networks (NNs) with memristors, named dynamic-memristor (DM) NNs, such that the analog processing takes place in the charge-flux domain, instead of the typical current-voltage domain as it happens for Hopfield NNs and standard cellular NNs. One key advantage is that, when a steady state is reached, all currents, voltages, and power of a DM-NN drop off, whereas the memristors act as nonvolatile memories that store the processing result. Previous work in the literature addressed multistability of DM-NNs, i.e., convergence of solutions in the presence of multiple asymptotically stable equilibrium points (EPs). The goal of this paper is to study a basically different dynamical property of DM-NNs, namely, to thoroughly investigate the fundamental issue of global asymptotic stability (GAS) of the unique EP of a DM-NN in the general case of nonsymmetric neuron interconnections. A basic result on GAS of DM-NNs is established using Lyapunov method and the concept of Lyapunov diagonally stable matrices. On this basis, some relevant classes of nonsymmetric DM-NNs enjoying the property of GAS are highlighted.

Speeding Up Cellular Neural Network Processing Ability by Embodying Memristors.

Cellular neural networks (CNNs) are an efficient tool for image analysis and pattern recognition. Based on elementary cells connected to neighboring units, they are easy to install in hardware, carrying out massively parallel processes. This brief presents a new model of CNN with memory devices, which enhances further CNN performance. By introducing a memristive element in basic cells, we carry out different experiments, allowing the analysis of the functions traditionally carried out by the standard CNN. Without modifying the templates considered by the scientific literature, this simple variation originates a significant improvement in ∼30 % of performances in pattern recognition and image processing. These progresses were experimentally calculated on the time the system requires to reach a fixed point. Moreover, the different role that each parameter has in the developed method was also analyzed to better understand the complex processing ability of these systems.

Global Robust Stability Criteria for Interval Delayed Full-Range Cellular Neural Networks

This brief considers a class of delayed full-range (FR) cellular neural networks (CNNs) with uncertain interconnections between neurons modeled by means of intervalized matrices. Using mathematical tools from the theory of differential inclusions, a fundamental result on global robust stability of standard (S) CNNs is extended to prove global robust exponential stability for the corresponding class (same interconnection weights and inputs) of FR-CNNs. The result is of theoretical interest since, in general, the equivalence between the dynamical behavior of FR-CNNs and S-CNNs is not guaranteed.

Comparison of convergence and stability properties for the state and output solutions of neural networks

A typical neuron cell is characterized by the state variable and the neuron output, which is obtained by passing the state through a nonlinear active device implementing the neuron activation. The paper introduces a wide class of neural networks for which the state solutions and the output solutions enjoy the same convergence and stability properties. The class, which includes as a special case the standard cellular neural networks, is characterized by piecewise-linear Lipschitz continuous neuron activations, Lipschitz continuous (possibly) high-order interconnections between neurons and asymptotically stable isolated neuron cells. The paper also shows that if we relax any of the assumptions on the smoothness of the neuron activations or interconnecting structure, or on the stability of the isolated neuron cells, then the equivalence between the convergence properties of the state solutions and the output solutions is in general no longer guaranteed. To this end, three relevant classes of neural networks in the literature are considered, where each class violates one of the assumptions made in the paper, and it is shown that the state solutions of the networks enjoy stronger convergence properties with respect to the output solutions or viceversa. Copyright © 2010 John Wiley & Sons, Ltd.

LIMIT SET DICHOTOMY AND CONVERGENCE OF SEMIFLOWS DEFINED BY COOPERATIVE STANDARD CNNs

The paper analyzes some fundamental properties of the solution semiflow of nonsymmetric cooperative standard (S) cellular neural networks (CNNs) with a typical three-segment piecewise-linear (pwl) neuron activation. Two relevant subclasses of SCNNs, corresponding to one-dimensional circular SCNNs with two-sided or single-sided positive interconnections between nearest neighboring neurons only, are considered. For these subclasses it is shown that the associated solution semiflow satisfies the fundamental properties of the CONVERGENCE CRITERION, the NONORDERING OF LIMIT SETS and the LIMIT SET DICHOTOMY, and that this is true although the semiflow is not eventually strongly monotone. As a consequence such CNNs are almost convergent, i.e., almost all solutions converge toward an equilibrium point as time tends to infinity. To the authors' knowledge the paper is the first rigorous investigation on the geometry of limit sets and convergence properties of cooperative SCNNs with a pwl neuron activation. All available convergence results in the literature indeed concern a modified cooperative CNN model where the original pwl activation of the SCNN model is replaced by a continuously differentiable strictly increasing sigmoid function. The main results in the paper are established by conducting a deep analysis of the properties of the omega-limit sets of the solution semiflow defined by the considered subclasses of SCNNs. In doing so the paper exploits and extends some mathematical tools for monotone systems in order that they can be applied to pwl vector fields that govern the dynamics of SCNNs. By using some transformations and referring to specific examples it is also shown that the treatment in the paper can be extended to other subclasses of SCNNs.

Extended LaSalle's Invariance Principle for Full-Range Cellular Neural Networks

The paper develops a Lyapunov method, which is based on a generalized version of LaSallepsilas invariance principle, for studying convergence and stability of the differential inclusions modeling the dynamics of the full-range (FR) model of cellular neural networks (CNNs). The method is applied to yield a rigorous proof of convergence for symmetric FR-CNNs. The proof, which is a direct consequence of the fact that a symmetric FR-CNN admits a strict Lyapunov function, is much more simple than the corresponding proof of convergence for symmetric standard CNNs.

On global exponential stability of standard and full‐range CNNs

AbstractThis paper compares the dynamical behaviour of the standard (S) cellular neural networks (CNNs) and the full‐range (FR) CNNs, when the two CNN models are characterized by the same set of parameters (interconnections and inputs). The FR‐CNNs are assumed to be characterized by ideal hard‐limiter nonlinearities with two vertical segments in the i–v characteristic. The main result is that some basic conditions ensuring global exponential stability (GES) of the unique equilibrium point of S‐CNNs, with or without delay, continue to ensure the same property for FR‐CNNs for the same set of parameters. The significance of this result is discussed with respect to the results in a paper by Corinto and Gilli addressing the similarity of the qualitative behaviour of S‐CNNs and FR‐CNNs. FR‐CNNs are analysed in this paper from a rigorous mathematical viewpoint by means of theoretical tools from set‐valued analysis and differential inclusions. In particular, GES is investigated via an extended Lyapunov approach that is applicable to the differential inclusion describing the dynamics of FR‐CNNs. Copyright © 2007 John Wiley & Sons, Ltd.

GLOBAL CONSISTENCY OF DECISIONS AND CONVERGENCE OF COMPETITIVE CELLULAR NEURAL NETWORKS

In a series of papers published in the seventies, Grossberg had developed a geometric approach for analyzing the global dynamical behavior and convergence properties of a class of competitive dynamical systems. The approach is based on the property that it is possible to associate a decision scheme with each competitive system in that class, and that global consistency of the decision scheme implies convergence of each solution toward some stationary state. In this paper, the Grossberg approach is extended to the class of competitive standard Cellular Neural Networks (CNNs), and it is used to investigate convergence under the hypothesis that the competitive CNN has a globally consistent decision scheme. The extension is nonobvious and requires to deal with the set-valued vector field describing the dynamics of the CNN output solutions. It is also stressed that the extended approach does not require the existence of a Lyapunov function, hence it is applicable to address convergence in the general case where the CNN neuron interconnections are not necessarily symmetric. By means of the extended approach, a number of classes of third-order nonsymmetric competitive CNNs are discovered, which have a globally consistent decision scheme and are convergent. Moreover, global consistency and convergence hold for interconnection parameters belonging to sets with non-empty interior, and thus they represent physically robust properties. The paper also shows that when the dimension is higher than three, there are fundamental differences between the convergence properties of competitive CNNs implied by a globally consistent decision scheme, and those of the class of competitive dynamical systems considered by Grossberg. These differences lead to the need to introduce a stronger notion of global consistency of decisions, with respect to that proposed by Grossberg, in order to guarantee convergence of competitive CNNs with more than three neurons.

Dynamical Analysis of Full-Range Cellular Neural Networks by Exploiting Differential Variational Inequalities

The paper considers the full-range (FR) model of cellular neural networks (CNNs) in the case where the neuron nonlinearities are ideal hard-comparator functions with two vertical straight segments. The dynamics of FR-CNNs, which is described by a differential inclusion, is rigorously analyzed by means of theoretical tools from set-valued analysis and differential inclusions. The fundamental property proved in the paper is that FR-CNNs are equivalent to a special class of differential inclusions termed differential variational inequalities. A sound foundation to the dynamics of FR-CNNs is then given by establishing the existence and uniqueness of the solution starting at a given point, and the existence of equilibrium points. Moreover, a fundamental result on trajectory convergence towards equilibrium points (complete stability) for reciprocal standard CNNs is extended to reciprocal FR-CNNs by using a generalized Lyapunov approach. As a consequence, it is shown that the study of the ideal case with vertical straight segments in the neuron nonlinearities is able to give a clear picture and analytic characterization of the salient features of motion, such as the sliding modes along the boundary of the hypercube defined by the hard-comparator nonlinearities. Finally, it is proved that the solutions of the ideal FR model are the uniform limit as the slope tends to infinity of the solutions of a model where the vertical segments in the nonlinearities are approximated by segments with finite slope.

ROBUSTNESS OF COMPLETE STABILITY FOR 1-D CIRCULAR CNNs

This paper investigates the issue of robustness of complete stability of standard Cellular Neural Networks (CNNs) with respect to small perturbations of the nominally symmetric interconnections. More specifically, a class of circular one-dimensional (1-D) CNNs with nearest-neighbor interconnections only, is considered. The class has sparse interconnections and is subject to perturbations which preserve the interconnecting structure. Conditions assuring that the perturbed CNN has a unique equilibrium point at the origin, which is unstable, are provided in terms of relative magnitude of the perturbations with respect to the nominal interconnection weights. These conditions allow one to characterize regions in the perturbation parameter space where there is loss of stability for the perturbed CNN. In turn, this shows that even for sparse interconnections and structure preserving perturbations, robustness of complete stability is not guaranteed in the general case.

FOURTH-ORDER NEARLY-SYMMETRIC CNNS EXHIBITING COMPLEX DYNAMICS

In this paper, the possible presence of complex dynamics in nearly-symmetric standard Cellular Neural Networks (CNNs), is investigated. A one-parameter family of fourth-order CNNs is presented, which exhibits a cascade of period-doubling bifurcations leading to the birth of a complex attractor, close to some nominal symmetric CNN. Different from previous work on this topic, the bifurcations and complex dynamics are obtained for small relative errors with respect to the nominal interconnections. The fourth-order CNNs have negative (inhibitory) interconnections between distinct neurons, and are designed by a variant of a technique proposed by Smale to embed a given dynamical system within a competitive dynamical system of larger order.

Global convergence of neural networks with discontinuous neuron activations

The paper introduces a general class of neural networks where the neuron activations are modeled by discontinuous functions. The neural networks have an additive interconnecting structure and they include as particular cases the Hopfield neural networks (HNNs), and the standard cellular neural networks (CNNs), in the limiting situation where the HNNs and CNNs possess neurons with infinite gain. Conditions are derived which ensure the existence of a unique equilibrium point, and a unique output equilibrium point, which are globally attractive for the state and the output trajectories of the neural network, respectively. These conditions, which are applicable to general nonsymmetric neural networks, are based on the concept of Lyapunov diagonally-stable neuron interconnection matrices, and they can be thought of as a generalization to the discontinuous case of previous results established for neural networks possessing smooth neuron activations. Moreover, by suitably exploiting the presence of sliding modes, entirely new conditions are obtained which ensure global convergence in finite time, where the convergence time can be easily estimated on the basis of the relevant neural-network parameters. The analysis in the paper employs results from the theory of differential equations with discontinuous right-hand side as introduced by Filippov. In particular, global convergence is addressed by using a Lyapunov-like approach based on the concept of monotone trajectories of a differential inclusion.

HARMONIC BALANCE APPROACH TO PREDICT PERIOD-DOUBLING BIFURCATIONS IN NEARLY SYMMETRIC CNNs

This paper further investigates a basic issue that has received attention in the recent literature, namely, the robustness of complete stability of standard Cellular Neural Networks (CNNs) with respect to small perturbations of the nominal symmetric interconnections. More specifically, a class of third-order CNNs with a nominal symmetric interconnection matrix is considered, and the Harmonic Balance (HB) method is exploited for addressing the possible existence of period-doubling bifurcations, and complex dynamics, for small perturbations of the nominal interconnections. The main result is that there are indeed parameter sets close to symmetry for which period-doubling bifurcations are predicted by the HB method. Moreover, the predictions are found to be reliable and accurate by means of computer simulations.

Implementation of CNN neuron activations

A new implementation in CMOS technology for the neuron cell of the standard cellular neural networks (CNNs) is introduced. The proposed cell activation closely approximates a three-segment piecewise linear function, also at the comers of the technological process. This feature permits a generalised design procedure for the CNN which accurately accounts for the errors in the electronic implementation of the cell.

ON THE SPATIAL ENTROPY AND PATTERNS OF TWO-DIMENSIONAL CELLULAR NEURAL NETWORKS

This work investigates binary pattern formations of two-dimensional standard cellular neural networks (CNN) as well as the complexity of the binary patterns. The complexity is measured by the exponential growth rate in which the patterns grow as the size of the lattice increases, i.e. spatial entropy. We propose an algorithm to generate the patterns in the finite lattice for general two-dimensional CNN. For the simplest two-dimensional template, the parameter space is split up into finitely many regions which give rise to different binary patterns. Qualitatively, the global patterns are classified for each region. Quantitatively, the upper bound of the spatial entropy is estimated by computing the number of patterns in the finite lattice, and the lower bound is given by observing a maximal set of patterns of a suitable size which can be adjacent to each other.

A NEW METHOD TO ANALYZE COMPLETE STABILITY OF PWL CELLULAR NEURAL NETWORKS

In recent years, the standard Cellular Neural Networks (CNN's) introduced by Chua and Yang [1988] have been one of the most investigated paradigms for neural information processing. In a wide range of applications, the CNN's are required to be completely stable, i.e. each trajectory should converge toward some stationary state. However, a rigorous proof of complete stability, even in the simplest original setting of piecewise-linear (PWL) neuron activations and symmetric interconnections [Chua & Yang, 1988], is still lacking. This paper aims primarily at filling this gap, in order to give a sound analytical foundation to the CNN paradigm. To this end, a novel approach for studying complete stability is proposed. This is based on a fundamental limit theorem for the length of the CNN trajectories. The method differs substantially from the classic approach using LaSalle invariance principle, and permits to overcome difficulties encountered when using LaSalle approach to analyze complete stability of PWL CNN's. The main result obtained, is that a symmetric PWL CNN is completely stable for any choice of the network parameters, i.e. it possesses the Absolute Stability property of global pattern formation. This result is really general and shows that complete stability holds under hypotheses weaker than those considered in [Chua & Yang, 1988]. The result does not require, for example, that the CNN has binary stable equilibrium points only. It is valid even in degenerate situations where the CNN has infinite nonisolated equilibrium points. These features significantly extend the potential application fields of the standard CNN's.

COMPLETE STABILITY FOR A CLASS OF CELLULAR NEURAL NETWORKS

This work investigates a class of lattice dynamical systems originated from cellular neural networks. In the vector field of this class, each component of the state vector and the output vector is related through a sigmoidal nonlinear output function. For two types of sigmoidal output functions, Liapunov functions have been constructed in the literatures. Complete stability has been studied for these systems using LaSalle's invariant principle on the Liapunov functions. The purpose of this presentation is two folds. The first one is to construct Liapunov functions for more general sigmoidal output functions. The second is to extend the interaction parameters into a more general class, using an approach by Fiedler and Gedeon. This presentation also emphasizes the complete stability when the equilibrium is not isolated for the standard cellular neural networks.