Cellular Neural Network Implementation Research Articles

Multi-stable states and synchronicity of a cellular neural network with memristive activation function

The cellular neural network (CNN) is an implementable solution for fully connected neural networks. Using nanoscale memristor to realize its nonlinear activation function can simplify the circuit implementation of CNN effectively. This paper presents a paradigm of the basic CNN cell by introducing a voltage-controlled memristor as the activating module of its output circuit. A three-cell memristor-based CNN (mCNN) is constructed to demonstrate the parameter- and initial condition-influenced dynamical behaviors induced by the activating memristor. Furtherly, two identical three-cell mCNNs are chosen as the subnets to construct a memristor-coupled mCNN, based on which the multi-stable states and the synchronous behaviors are investigated. Numerical results show that the multi-stable states of the memristor-coupled mCNN are flexibly switched by adjusting the coupling strength and initial conditions. Under the control of the memristor coupler, the two subnets can achieve complete synchronization, lag synchronization and phase synchronization. Finally, the FPGA-based hardware experiments are executed to verify the numerical results.

Efficient Hardware Implementation of Cellular Neural Networks with Incremental Quantization and Early Exit

Cellular neural networks (CeNNs) have been widely adopted in image processing tasks. Recently, various hardware implementations of CeNNs have emerged in the literature, with Field Programmable Gate Array (FPGA) being one of the most popular choices due to its high flexibility and low time-to-market. However, CeNNs typically involve extensive computations in a recursive manner. As an example, to simply process an image of 1,920 × 1,080 pixels requires 4--8 Giga floating point multiplications (for 3 × 3 templates and 50–100 iterations), which needs to be done in a timely manner for real-time applications. To address this issue, in this article, we propose a compressed CeNN framework for efficient FPGA implementations. It involves various techniques, such as incremental quantization and early exit, which significantly reduces computation demands while maintaining an acceptable performance. Particularly, incremental quantization quantizes the numbers in CeNN templates to powers of two, so that complex and expensive multiplications can be converted to simple and cheap shift operations, which only require a minimum number of registers and logical elements (LEs). While a similar concept has been explored in hardware implementations of Convolutional Neural Networks (CNNs), CeNNs have completely different computation patterns, which require different quantization and implementation strategies. Experimental results on FPGAs show that incremental quantization and early exit can achieve a speedup of up to 7.8× and 8.3×, respectively, compared with the state-of-the-art implementations, while with almost no performance loss with four widely adopted applications. We also discover that different from CNNs, the optimal quantization strategies of CeNNs depend heavily on the applications. We hope that our work can serve as a pioneer in the hardware optimization of CeNNs.

A Multi-Level-Optimization Framework for FPGA-Based Cellular Neural Network Implementation

Cellular Neural Network (CeNN) is considered as a powerful paradigm for embedded devices. Its analog and mix-signal hardware implementations are proved to be applicable to high-speed image processing, video analysis, and medical signal processing with its efficiency and popularity limited by smaller implementation size and lower precision. Recently, digital implementations of CeNNs on FPGA have attracted researchers from both academia and industry due to its high flexibility and short time-to-market. However, most existing implementations are not well optimized to fully utilize the advantages of FPGA platform with unnecessary design and computational redundancy that prevents speedup. We propose a multi-level-optimization framework for energy-efficient CeNN implementations on FPGAs. In particular, the optimization framework is featured with three level optimizations: system-, module-, and design-space-level, with focus on computational redundancy and attainable performance, respectively. Experimental results show that with various configurations our framework can achieve an energy-efficiency improvement of 3.54× and up to 3.88× speedup compared with existing implementations with similar accuracy.

Color image edge extraction using memristor-based CNN

Because of the locally connected lattice structure and high-speed parallel processing, cellular neural network (CNN) have been widely used in image processing. Traditional processing methods typically employ fixed templates, which impose significant limitations on practical complex image processing. However, the hardware implementation of large-scale CNNs becomes impossible due to the bottleneck of traditional CMOS technology. In this paper, a new threshold-adaptive algorithm is proposed by considering pixel space distributions based on human visual perception, which can overcome the aforementioned limitation. Then, the memristor, a two-terminal nonlinear device with unique high-speed switching, nonvolatility, and nanometer scale is used to solve the circuit realization problem. Specifically, we design a spintronic memristor-based CNN (SMCNN) to facilitate the proposed threshold-adaptive algorithm. Finally, by using the example of color image processing, the effectiveness of the proposed SMCNN is demonstrated by means of numerical simulations and comparative analysis.

Non-Boolean Computing Benchmarking for Beyond-CMOS Devices Based on Cellular Neural Network

This paper presents a uniform benchmarking methodology for non-Boolean computation based on the cellular neural network (CNN) for a variety of beyond-CMOS device technologies, including charge-based and spintronic devices. Three types of CNN implementations are investigated using analog, digital, and spintronic circuits. Monte Carlo simulations are performed to quantify the impact of the input noise, thermal noise, and the number of bits representing the weights of synapses on the overall recall probability and delay. The results demonstrate that the recall probability improves significantly as the number of synapses increase. Using a 4-b resolution for synapse weights provides the best tradeoff between the required numbers of synapses and synapse bits for a target recall rate. Finally, three types of CNN implementations with various device technologies are benchmarked for a given input noise and recall accuracy target. It is shown that spintronic devices are promising candidates to implement CNNs, where up to 3× energy-delay product improvement is predicted in domain wall devices compared to its conventional CMOS counterpart.

A novel memristive cellular neural network with time-variant templates

Summary A cellular neural network (CNN) is a massively parallel analog array processor capable of solving various complex processing problems by using specific templates that characterize the synaptic connections. The hardware implementation and applications of CNN have attracted a great deal of attention. Recently, memristors with nanometer-scale and variable gradual conductance have been exploited to make compact and programmable electric synapses. This paper proposes and studies a novel memristive CNN (Mt-CNN) with time-variant templates realized by memristor crossbar synaptic circuits. The template parameters are estimated analytically. The Mt-CNN provides a promising solution to hardware realization of real-time template updating processes, which can be used to effectively deal with various complicated problems of cascaded processing. Its effectiveness and advantages are demonstrated by practical examples of edge detection on noisy images.

Hybrid memristor/RTD structure-based cellular neural networks with applications in image processing

Cellular neural network (CNN) has been acted as a high-speed parallel analog signal processor gradually. However, recently, since the decrease in the size of transistor is going to approach the utmost, the transistor-based integrated circuit technology hits a bottleneck. As a result, the advantage of very large scale integration implementation of CNN becomes hard to really present, and further development of this era faces severe challenges unavoidably. In this study, two types of memristor-based cellular neural networks have been proposed. One type uses a memristor to replace the linear resistor in a conventional CNN cell circuit. And the other places a resonant tunneling diode (RTD) in this position and uses memristive synaptic connections to structure a hybrid memristor RTD CNN model. The excellent performances of the proposed CNNs are verified by conventional means of, for instance, stability analysis and efficient applications in image processing. Since both the memristor and the resonant tunneling diode are nanoscale, the size of the network circuits can be greatly reduced, and the integration density of the system will be significantly improved.

Image-based motion planning for two mobile robots in an environment with obstacles by using cellular neural networks

Abstract The paper presents an image-based algorithm for motion-planning of two mobile robots moving to the same target in an environment with obstacles. Due to parallel computing, the Cellular Neural Networks (CNN) techniques ensure the images processing in real-time and represent an advantageous solution for autonomous mobile robots guidance. The path planning algorithm can be improved increasing the speed of image processing, using advanced type of the CNN implementation and it can be extended for three or more robots.

A Homogeneous Algorithm for Motion Estimation and Compensation by Using Cellular Neural Networks

In this paper we present an original implementation of a homogeneous algorithm for motion estimation and compensation in image sequences, by using Cellular Neural Networks (CNN). The CNN has been proven their efficiency in real-time image processing, because they can be implemented on a CNN chip or they can be emulated on Field Programmable Gate Array (FPGA). The motion information is obtained by using a CNN implementation of the well-known Horn & Schunck method. This information is further used in a CNN implementation of a motion-compensation method. Through our algorithm we obtain a homogeneous implementation for real-time applications in artificial vision or medical imaging. The algorithm is illustrated on some classical sequences and the results confirm the validity of our algorithm.

Artificial neural networks in hardware: A survey of two decades of progress

This article presents a comprehensive overview of the hardware realizations of artificial neural network (ANN) models, known as hardware neural networks (HNN), appearing in academic studies as prototypes as well as in commercial use. HNN research has witnessed a steady progress for more than last two decades, though commercial adoption of the technology has been relatively slower. We study the overall progress in the field across all major ANN models, hardware design approaches, and applications. We outline underlying design approaches for mapping an ANN model onto a compact, reliable, and energy efficient hardware entailing computation and communication and survey a wide range of illustrative examples. Chip design approaches (digital, analog, hybrid, and FPGA based) at neuronal level and as neurochips realizing complete ANN models are studied. We specifically discuss, in detail, neuromorphic designs including spiking neural network hardware, cellular neural network implementations, reconfigurable FPGA based implementations, in particular, for stochastic ANN models, and optical implementations. Parallel digital implementations employing bit-slice, systolic, and SIMD architectures, implementations for associative neural memories, and RAM based implementations are also outlined. We trace the recent trends and explore potential future research directions.

Cellular neural network with hybrid single-electron and MOS transistors architecture and its application

Based on both the cell equivalent circuit of cellular neural network and the electrical characteristic model of cellular neural networks（CNN） cell，the cell circuit of cellular neural networks is implemented. The activation function of cell circuit is made of two cascaded SET-MOS inverter, which is proposed previously by the author. The CNN cloning template is built by coupling capacitance of input terminal. Then the CNN and its application in image processing are built and studied. The computer simulation results show that the designed circuits is suitable for CNN implementation because of its simple structure, low power dissipation and fast response. The designed circuit can be used to form CNN of various scales so as to further satisfy the need of large-scale signal processing and improve the density of integrated circuit.

Implementation and application of cellular neural networks based on single electron device

This paper realizes cellular neural networks using the characteristic of negative differential resistance of hybrid single electron transistor and complementary metallic oxide semiconductor field effect transistor structure. The main building blocks consisting of cell core circuit, A and B template circuits are designed. Then a cellular neural network (CNN) is built and its application in image processing is studied. The computer simulation shows that the designed circuits are suitable for CNN implementation owing to its simple structure, low power dissipation and fast response. It could be used to form CNN of various scales so as to further increase the density of integrated circuits.

BIFURCATIONS AND CHAOS IN CELLULAR NEURAL NETWORKS

The dynamic behavior of first-order autonomous space invariant cellular neural networks (CNNs) is investigated. It is shown that complex dynamics may occur in very simple CNN structures, described by two-dimensional templates that present only vertical and horizontal couplings. The bifurcation processes are analyzed through the computation of the limit cycle Floquet's multipliers, the evaluation of the Lyapunov exponents and of the signal spectra. As a main result a detailed and accurate two-dimensional bifurcation diagram is reported. The diagram allows one to distinguish several regions in the parameter space of a single CNN. They correspond to stable, periodic, quasi-periodic, and chaotic behavior, respectively. In particular it is shown that chaotic regions can be reached through two different routes: period doubling and torus breakdown. We remark that most practical CNN implementations exploit first order cells and space-invariant templates: so far only a few examples of complex dynamics and no complete bifurcation analysis have been presented for such networks.

Simplicial rtd-based cellular nonlinear networks

Recently, a novel structure called the simplicial cellular neural network (CNN) has been introduced , which permits one to implement any Boolean/Gray-level function of any number of variables. This paper is devoted to explore novel circuit architectures for the implementation of the simplicial CNN based on resonant tunneling diodes. The final objective is to implement a fully programmable CNN in a hardware platform based on nanoelectronic devices.

Implementation of Neural Network for Image Processing on the FPGA Programmable Systems

In this paper the implementation of fragment digital Cellular Neural Network (CNN) for image processing on the Field Programmable Gate Array (FPGA) and it's experiment results are present. The high processing speed of the network is use to provide real time processing. Results shows that the architecture CNN and FPGA and implementation has good corespondent.

Complex dynamic phenomena in space-invariant cellular neural networks

It is shown that first-order autonomous space-invariant cellular neural networks (CNNs) may exhibit a complex dynamic behavior (i.e., equilibrium point and limit cycle bifurcation, strange and chaotic attractors). The most significant limit cycle bifurcation processes, leading to chaos, are investigated through the computation of the corresponding Floquet's multipliers and Lyapunov exponents. It is worth noting that most practical CNN implementations exploit first-order cells and space-invariant templates: so far no example of complex dynamics has been shown in first-order autonomous space-invariant CNNs.

IMPLEMENTING BACK-PROPAGATION- THROUGH-TIME LEARNING ALGORITHM USING CELLULAR NEURAL NETWORKS

In a programmable (multistage) cellular neural network (CNN) structure, the CPU is a CNN universal chip which supports massively parallel computations on patterns and images, including videos. In this paper, we decompose the structure of a class of simultaneous recurrent networks (SRN) into a CNN program and run it on a von Neumann-like stored program CNN structure. To train the SRN, we map the back-propagation-through-time (BTT) learning algorithm into a sequence of CNN subroutines to achieve real-time performance via a CNN universal chip. By computing in parallel, the CNN universal chip can be programmed to implement in real time the BTT learning algorithm, which has a very high time complexity. An estimate of the time complexity of the BTT learning algorithm based on the CNN universal chip is presented. For small-scale problems, our simulation results show that a CNN implementation of the BTT learning algorithm for a two-dimensional SRN is at least 10,000 times faster than that based on state-of-the-art sequential workstations. For the few large-scale problems which we have so far simulated, the CNN implemented BTT learning algorithm maintained virtually the same time complexity with a learning time of a few seconds, while those implemented on state-of-the-art sequential workstations dramatically increased their time complexity, often requiring several days of running time. Several examples are presented to demonstrate how efficiently a CNN universal chip can speed up the learning algorithm for both off-line and on-line applications.

Cellular neural network implementation using a phase-only joint transform correlator

A phase-only joint transform correlator (JTC) is used to realize cellular neural networks (CNNs). The operation of summing cross-correlations of bipolar data in CNNs can be realized in parallel by phase-encoding bipolar data. Compared to other optical systems for implementing CNNs, the proposed method offers the advantages of easier implementation and robustness in terms of system alignment, and requires neither electronic precalculation nor data rearrangement. Simulation results of the proposed optical CNNs for edge detection are provided.

Optical implementation of cellular neural networks based on bias and joint correlation

A new bias method for the optical implementation of cellular neural networks is proposed to reduce electronic precalculation and increase processing speed. A multiple-object joint transform correlator is then used to realize the summation of multiple correlations resulting from the bias method. Compared with other optical systems for cellular neural networks, the proposed method offers the advantages of higher processing speed, easy implementation, and robustness. Computer simulations of the optical cellular neural networks for edge detection and corner and horizontal line extraction are also presented.

A Nonlinear Wave Metric and its CNN Implementation for Object Classification

In this paper a nonlinear wave metric is introduced for object classification. It is shown that the choice of a metric is a nontrivial problem since it is easy to give examples when well-known distance measures, such as Hamming, Hausdorff, and Nonlinear Hausdorff metrics are completely inadequate for this classification. As an alternative a generalized theorem is proposed that includes the previous metrics as special cases. It is based on nonlinear wave propagation and defines a computational framework that is well-suited for parallel array processors. In this study we investigate different Cellular Neural Network (CNN) architectures and solutions for the proposed metric and analyze its VLSI implementation complexity.