Cellular Neural Network Architecture Research Articles

Multi-Scale Spatial–Spectral Attention-Based Neural Architecture Search for Hyperspectral Image Classification

Convolutional neural networks (CNNs) are indeed commonly employed for hyperspectral image classification. However, the architecture of cellular neural networks typically requires manual design and fine-tuning, which can be quite laborious. Fortunately, there have been recent advancements in the field of Neural Architecture Search (NAS) that enable the automatic design of networks. These NAS techniques have significantly improved the accuracy of HSI classification, pushing it to new levels. This article proposes a Multi-Scale Spatial–Spectral Attention-based NAS, MS3ANAS) framework for HSI classification to automatically design a neural network structure for HSI classifiers. First, this paper constructs a multi-scale attention mechanism extended search space, which considers multi-scale filters to reduce parameters while maintaining large-scale receptive field and enhanced multi-scale spectral–spatial feature extraction to increase network sensitivity towards hyperspectral information. Then, we combined the slow–fast learning architecture update paradigm to optimize and iteratively update the architecture vector and effectively improve the model’s generalization ability. Finally, we introduced the Lion optimizer to track only momentum and use symbol operations to calculate updates, thereby reducing memory overhead and effectively reducing training time. The proposed NAS method demonstrates impressive classification performance and effectively improves accuracy across three HSI datasets (University of Pavia, Xuzhou, and WHU-Hi-Hanchuan).

Deep Cellular Recurrent Network for Efficient Analysis of Time-Series Data With Spatial Information.

Efficient processing of large-scale time-series data is an intricate problem in machine learning. Conventional sensor signal processing pipelines with hand-engineered feature extraction often involve huge computational costs with high dimensional data. Deep recurrent neural networks have shown promise in automated feature learning for improved time-series processing. However, generic deep recurrent models grow in scale and depth with the increased complexity of the data. This is particularly challenging in presence of high dimensional data with temporal and spatial characteristics. Consequently, this work proposes a novel deep cellular recurrent neural network (DCRNN) architecture to efficiently process complex multidimensional time-series data with spatial information. The cellular recurrent architecture in the proposed model allows for location-aware synchronous processing of time-series data from spatially distributed sensor signal sources. Extensive trainable parameter sharing due to cellularity in the proposed architecture ensures efficiency in the use of recurrent processing units with high-dimensional inputs. This study also investigates the versatility of the proposed DCRNN model for the classification of multiclass time-series data from different application domains. Consequently, the proposed DCRNN architecture is evaluated using two time-series data sets: a multichannel scalp electroencephalogram (EEG) data set for seizure detection, and a machine fault detection data set obtained in-house. The results suggest that the proposed architecture achieves state-of-the-art performance while utilizing substantially less trainable parameters when compared to comparable methods in the literature.

Computer Medical Image Segmentation Based on Neural Network

Image segmentation in medical imaging has long been a problem in radiological image processing. Most of the image segmentation methods in traditional vision algorithms are difficult to achieve high-resolution image segmentation due to the complexity of the algorithm. This article proposes an image segmentation method based on an optimized cellular neural network. This method introduces a non-linear template and data quantization on the basis of a basic network model, which greatly reduces the computational complexity while maintaining the accuracy of image segmentation. We then applied the method to a computer-aided system to classify tumor lesions in mammograms. Finally, we propose an FPGA-based multilevel optimization architecture for energy-efficient cellular neural networks. The optimization scheme includes three levels: system level, module level, and design space. This solution improves computing performance by increasing system parallelism, using data reuse technology to fully utilize loading bandwidth, and using data quantization to reduce computational redundancy. It also introduces pipeline and dual cache structures to optimize memory access, and analyzes the limited resources through the Roofline model. System for best performance. The experimental results show that the FPGA accelerator in this article can improve unit performance by 34% compared with other existing research work. The nonlinear quantified cellular neural network proposed in this article can reduce LUT resource consumption by 74% and energy of 48.2%. Compared with the original network, in the two projection position segmentation results of the mammogram, only 1.5% and 0.6% of the accuracy loss, respectively.

7 TOPS/W Cellular Neural Network Processor Core for Intelligent Internet-of-Things

We discuss the architecture, implementation and testing of a simplicial Cellular Neural Network (CNN) vector processor core aimed at vision oriented intelligent Internet-of-Things (IoT) devices. The architecture comprises a linear array of 64 processing elements (PE), each connected to a 4 neighbor clique operating on 8-bit input and state data. A 3-bit simplicial parameter, allows multilevel function approximation and extends the functionality over previously reported chips. Input data vectors are stored in two $64 \times 64 \times 8$ -bit data caches. The chip is synthesized from a custom designed ultra low voltage CMOS library and fabricated in a 55nm CMOS technology. Dynamic voltage/frequency scaling allows operation at power supplies between 0.5 and 1.2 Volts allowing for a tradeoff between speed and power. The fabricated chip achieves an overall performance of 7.05 TOPS/W at 732fps, with a dynamic energy efficiency of 12.2fJ per operation (OP) at 1.2 Volts.

Prediction of the moderator temperature field in a heavy water reactor based on a cellular neural network

Reactors with heavy water coolants and moderators have been used extensively in today's power industry. Monitoring of the moderator condition plays an important role in ensuring normal operation of a power plant. A cellular neural network, the architecture of which has been adapted for hardware implementation, is proposed for use in a system for prediction of the heavy water moderator temperature. A reactor model composed in accordance with the CANDU Darlington heavy water reactor design was used to form the training sample collection and to control correct operation of the neural network structure. The sample components for the adjustment and configuration of the network topology include key parameters that characterize the energy generation process in the core. The paper considers the feasibility of the temperature prediction only for the calandria's central cross-section. To solve this problem, the cellular neural network architecture has been designed, and major parts of the digital computational element and methods for their implementation based on an FPLD have also been developed. The method is described for organizing an optical coupling between individual neural modules within the network, which enables not only the restructuring of the topology in the training process, but also the assignment of priorities for the propagation of the information signals of neurons depending on the activity in a situation analysis at the neural network structure inlet. Asynchronous activation of cells was used based on an oscillating fractal network, the basis for which was a modified ring oscillator. The efficiency of training the proposed architecture using stochastic diffusion search algorithms is evaluated. A comparative analysis of the model behavior and the results of the neural network operation have shown that the use of the neural network approach is effective in safety systems of power plants.

Architecture of a Fully Pipelined Real-Time Cellular Neural Network Emulator

In this paper, architecture of a Real-Time Cellular Neural Network (CNN) Processor (RTCNNP-v2) is given and the implementation results are discussed. The proposed architecture has a fully pipelined structure, capable of processing full-HD 1080p@60 (1920 × 1080 resolution at 60 Hz frame rate, 124.4 MHz visible pixel rate) video streams, which is implemented on both high-end and low-cost FPGA devices, Altera Stratix IV GX 230, and Cyclone III C 25, respectively. Many features of the architecture are designed to be either pre-synthesis configurable or runtime programmable, which makes the processor extremely flexible, reusable, scalable, and practical.

A scalable CNN architecture and its application to short exposure stellar images processing on a HPRC

A CNN-based algorithm for short exposure image processing and an application-specific computing architecture developed to accelerate its execution are presented. Algorithm is based on a flexible and scalable Cellular Neural Networks (CNN) architecture specifically designed to optimize the projection of CNN kernels on a programmable circuit. The objective of the proposed algorithm is to minimize the adverse effect that atmospheric disturbance has on the images obtained by terrestrial telescopes. Algorithm main features are that it can be adapted to the detection of several astronomical objects and it supports multi-stellar images. The implementation platform made use of a High Performance Reconfigurable Computer (HPRC) combining general purpose standard microprocessors with custom hardware accelerators based on FPGAs, to speed up execution time. The hardware/software partitioning and co-design process have been carried out using high level design tools, instead of traditional Hardware Description Languages (HDLs). Results are presented in terms of circuit area/speed, processing performance and output quality.

Decentralized asynchronous learning in cellular neural networks.

Cellular neural networks (CNNs), as previously described, consist of identical units called cells that are connected to their adjacent neighbors. These cells interact with each other in order to fulfill a common goal. The current methods involved in learning of CNNs are usually centralized (cells are trained in one location) and synchronous (all cells are trained simultaneously either sequentially or in parallel depending on the available hardware/software platform). In this paper, a generic architecture of CNNs is presented and a special case of supervised learning is demonstrated explaining the internal components of a cell. A decentralized asynchronous learning (DAL) framework for CNNs is developed in which each cell of the CNN learns in a spatially and temporally distributed environment. An application of DAL framework is demonstrated by developing a CNN-based wide-area monitoring system for power systems. The results obtained are compared against equivalent traditional methods and shown to be better in terms of accuracy and speed.

Square-Root-Domain Realization of Single-Cell Architecture of Complex TDCNN

Square-root-domain (SRD) CMOS analog realization of a single cell architecture of the complex Temporal Derivative Cellular Neural Networks (TDCNNs) is introduced in this paper. TDCNN initiates time derivative ‘diffusion’ between CNN cells for non-separable spatiotemporal filtering applications, where the input to the CNN is an image that changes over time. The evaluation of the performance of the complex SRD TDCNN cell has been done using the Cadence Orcad software with TSMC 0.18-μm CMOS process model parameters. The provided simulated results confirm the validity of the theory.

A Novel Cloning Template Designing Method by Using an Artificial Bee Colony Algorithm for Edge Detection of CNN Based Imaging Sensors

Cellular Neural Networks (CNNs) have been widely used recently in applications such as edge detection, noise reduction and object detection, which are among the main computer imaging processes. They can also be realized as hardware based imaging sensors. The fact that hardware CNN models produce robust and effective results has attracted the attention of researchers using these structures within image sensors. Realization of desired CNN behavior such as edge detection can be achieved by correctly setting a cloning template without changing the structure of the CNN. To achieve different behaviors effectively, designing a cloning template is one of the most important research topics in this field. In this study, the edge detecting process that is used as a preliminary process for segmentation, identification and coding applications is conducted by using CNN structures. In order to design the cloning template of goal-oriented CNN architecture, an Artificial Bee Colony (ABC) algorithm which is inspired from the foraging behavior of honeybees is used and the performance analysis of ABC for this application is examined with multiple runs. The CNN template generated by the ABC algorithm is tested by using artificial and real test images. The results are subjectively and quantitatively compared with well-known classical edge detection methods, and other CNN based edge detector cloning templates available in the imaging literature. The results show that the proposed method is more successful than other methods.

RKGHeM(4,4) technique with error control for connected component detector

The aim of this paper is focused on proposing a new versatile Runge-Kutta embedded fourth order four stage geometric heronian mean algorithm for simulating connected component detector (CCD) under cellular neural network (CNN) architecture. The mathematical expressions for local truncation error (LTE), global truncation error (GTE) and error estimates have been derived. Simulation results are presented for new RKGHeM(4,4) algorithm.

IMPLEMENTATION OF THE COMPLEX PROCESSING OF DYNAMIC PROGRAMMING WITH NONLINEAR TEMPLATES OF CELLULAR NEURAL NETWORKS

A complex processing of the dynamic programming is implemented with the parallel architecture of Cellular Neural Networks. Dynamic programming is an efficient algorithm to find optimal path and Cellular Neural Network is a parallel computation architecture composed of identical computation cell array and identical connections at each cell. Breaking down complex processing of the dynamic programming into a sequence of simple steps, the dynamic programming algorithm can be built with the nonlinear templates of Cellular Neural Networks. The procedure to breakdown the complex computation into the sequence of CNN building blocks is illustrated. To show the feasibility of the proposed method, the designed CNN-based dynamic programming is applied for detecting the traces of road boundaries. Edge information of road image is extracted and assigned as local distance value accordingly, then dynamic programming algorithm is implemented by a nonlinear CNN template. The proposed algorithm and its possible circuit structure are described, and simulation results are reported.

New Digital Architecture of CNN for Pattern Recognition

New Digital Architecture of CNN for Pattern RecognitionThe paper deals with the design of a new digital CNN (Cellular Neural Network) architecture for pattern recognition. The main parameters of the new design were the area consumption of the chip and the speed of calculation in one iteration. The CNN was designed as a digital synchronous circuit. The largest area of the chip belongs to the multiplication unit. In the new architecture we replaced the parallel multiplication unit by a simple AND gate performing serial multiplication. The natural property of this method of multiplication is rounding. We verified some basic properties of the proposed CNN such as edge detection, filling of the edges and noise removing. At the end we compared the designed network with other two CNNs. The new architecture allows to save till 86% gates in comparison with CNN with parallel multipliers.

An eight layer cellular neural network for spatio‐temporal image filtering

AbstractSpatio‐temporal filters are critical components of biologically inspired or neuromorphic algorithms for image motion analysis. In this paper, we describe eight layer cellular neural network architectures that can be used to implement these filters. Despite the apparently large number of layers, we describe how these architectures can be implemented efficiently using weak inversion transistor circuits. Integrating both spatial and temporal filtering into a single network reduces hardware complexity in comparison with an architecture that cascades separate spatial and temporal filtering stages. In addition, by considering spatial and temporal filtering jointly, we can obtain filters with enhanced velocity selectivity, as well as more robust population responses to moving image input. Copyright © 2006 John Wiley & Sons, Ltd.

NEW LEARNING METHOD FOR CELLULAR NEURAL NETWORKS TEMPLATE BASED ON COMBINATION BETWEEN ROUGH SETS AND GENETIC PROGRAMMING

ABSTRACT A new learning algorithm for space invariant Cellular Neural Network (CNN) is introduced. Learning is formulated as an optimization problem by combining rough sets and genetic programming. Rough Sets approach has been selected for creating priori knowledge about the actual effective cells, determining their significance in classifying the output, and discovering the optimal CNN structure. According to the lattice of CNN architecture and depending on the priori knowledge gained by rough sets, genetic programming will be used in deriving the cloning template. Exploration of any stable domain is possible by the current approach. Details of the algorithm are discussed and several application results are shown.

A CMOS Implementation of a 1-Bit Multi-Cell Encoded—Cellular Neural Network

A new Cellular Neural Network (CNN)-based system has been implemented and tested to demonstrate the ability of this novel system to process large digital images more rapidly than its conventional CNN counterpart. The multi-cell encoded CNN processes the data of multiple single-data CNN cells within each multi-cell encoded cell enabling the new architecture it's advantages in loading, processing, unloading speed and in layout. A one bit (1B) 4-cell-encoded CNN was implemented to illustrate this new system. In this CMOS implementation, data of four neighboring conventional 1B CNN cells are encoded for processing within encoded cells. The CMOS circuits and circuit networks of one encoded cell are presented. Due to area limitations, each test chip includes the hardware for only one one-dimensional encoded cell. Experimental results of one-chip and two-chip 1B, multi-cell encoded systems are presented for connected-component detection and edge detection test cases. These results demonstrate the correct response of this implementation due to variations in template values, boundary values, and initial conditions. Interactions between encoded cell components and encoded cells are demonstrated for both dynamic and static responses. This CMOS implementation validates this new CNN architecture and provides a template for implementations using more advanced device technologies and circuits.

An ON–OFF Orientation Selective Address Event Representation Image Transceiver Chip

This paper describes the electronic implementation of a four-layer cellular neural network architecture implementing two components of a functional model of neurons in the visual cortex: linear orientation selective filtering and half wave rectification. Separate ON and OFF layers represent the positive and negative outputs of two-phase quadrature Gabor-type filters, whose orientation and spatial-frequency tunings are electronically adjustable. To enable the construction of a multichip network to extract different orientations in parallel, the chip includes an address event representation (AER) transceiver that accepts and produces two-dimensional images that are rate encoded as spike trains. It also includes routing circuitry that facilitates point-to-point signal fan in and fan out. We present measured results from a 32/spl times/64 pixel prototype, which was fabricated in the TSMC0.25-/spl mu/m process on a 3.84 by 2.54 mm die. Quiescent power dissipation is 3 mW and is determined primarily by the spike activity on the AER bus. Settling times are on the order of a few milliseconds. In comparison with a two-layer network implementing the same filters, this network results in a more symmetric circuit design with lower quiescent power dissipation, albeit at the expense of twice as many transistors.

0.8 μm cmos implementation of weighted-order statistic image filter based on cellular neural network architecture

In this paper, a very large scale integration chip of an analog image weighted-order statistic (WOS) filter based on cellular neural network (CNN) architecture for real-time applications is described. The chip has been implemented in CMOS AMS 0.8 /spl mu/m technology. CNN-based filter consists of feedforward nonlinear template B operating within the window of 3 /spl times/ 3 pixels around the central pixel being filtered. The feedforward nonlinear CNN coefficients have been realized using programmable nonlinear coupler circuits. The WOS filter chip allows for processing of images with 300 pixels horizontal resolution. The resolution can be increased by cascading of the chips. Experimental results of basic circuit building blocks measurements are presented. Functional tests of the chip have been performed using a special test setup for PAL composite video signal processing. Using the setup real images have been filtered by WOS filter chip under test.

Video object segmentation for head–shoulder sequences in the cellular neural networks architecture

MPEG-4 introduces the concept of video object to support content-based functionalities. Video object segmentation is a key step in defining the contents of any video sequences. Head–shoulder sequence (HSS) is typical in video conferencing and surveillance systems, in which real-time performance is required. Since background information can be obtained in advance and pre-stored, video segmentation for HSS can use background information a priori. To avoid the critical selection of threshold for gradient-based method, and to overcome the insufficiency of monochrome intensity-based change detection, an efficient color edge-based change detection scheme (CECD) is utilized in this paper. In order to meet the real-time performance for HSS, it is implemented in the cellular neural networks (CNN) architecture. The algorithm is mainly based on 3 by 3, linear templates. Because of CNN's high parallelism and computational abilities, real-time performance is achieved. Experimental results on several test sequences show the robustness of this approach. It can achieve better spatial accuracy and temporal coherency than COST211 AM.

ON THE TEMPLATES CORRESPONDING TO CYCLE-SYMMETRIC CONNECTIVITY IN CELLULAR NEURAL NETWORKS

In the architecture of cellular neural networks (CNN), connections among cells are built on linear coupling laws. These laws are characterized by the so-called templates which express the local interaction weights among cells. Recently, the complete stability for CNN has been extended from symmetric connections to cycle-symmetric connections. In this presentation, we investigate a class of two-dimensional space-invariant templates. We find necessary and sufficient conditions for the class of templates to have cycle-symmetric connections. Complete stability for CNN with several interesting templates is thus concluded.