Neural Chip Research Articles

Compact holographic optical neural network system for real‐time pattern recognition

One of the important characteristics of artificial neural networks is their capability for massive interconnection and parallel processing. Recently, specialized electronic neural network processors and VLSI neural chips have been introduced in the commercial market. The number of parallel channels they can handle is limited because of the limited parallel interconnections that can be implemented with onedimensional electronic wires. High-resolution pattern recognition problems can require a large number of neurons for parallel processing of an image. This paper describes a holographic optical neural network (HONN) that is based on high-resolution volume holographic materials and is capable of performing massive 3-D parallel interconnection of tens of thousands of neurons. A HONN with more than 16,000 neurons packaged in an attache case has been developed. Rotation-shift-scaleinvariant pattern recognition operations have been demonstrated with this system. System parameters such as the signal-to-noise ratio, dynamic range, and processing speed are discussed.

Neural 2.00 — A program for neural net and statistical pattern recognition

A neural net program for pattern classification is presented, which includes: i) an improved version of Kohonen's learning vector quantization (LVQ with training count); ii) feed-forward neural networks with back-propagation training; iii) Gaussian (or Mahalanobis distance) classification; iv) Fisher linear discrimination. Back-prop trainings with emulations of Intel's ETANN and Siemens' MA16 neural chips are available as options. The program has been developed for high energy physics applications.

Results from a MA16-based neural trigger in an experiment looking for beauty

Results from a neural-network trigger based on the digital MA16 chip of Siemens are reported. The neural trigger has been applied to data from the WA92 experiment, looking for beauty particles, which have been collected during a run in which a neural trigger module based on Intel's analog neural chip ETANN operated, as already reported. The MA16 board hosting the chip has a 16-bit I O precision and a 53-bit precision for internal calculations. It operated at 50 MHz, yielding a response time for a 16 input-variable net of 3 μs for a Fisher discriminant (1-layer net) and of 6 μs for a 2-layer net. Results are compared with those previously obtained with the ETANN trigger.

Connection weights based on molecular mechanisms in Aplysia neuron synapses

Most views of neuro-cognitive function including the ANN analogy, assume that modification of efficacy or sensitivity of existing synapses — referred to as synaptic strength — is the brain's primary mechanism for information storage and transfer. Many factors influence biological strength, however in artificial neural networks the analogous interconnection weights represent pragmatic over-simplifications of biological synapses. This simplification has been useful and successful in order to implement ANN into integrated circuits but these neural chips are far away, in a biological sense, from representing the silicon implementation of real synapses. Based on the behavioral system of the marine snail Aplysia we show a biological neural network model where a theoretical synaptic strength value scaled from 0 to 1 results from the interplay of molecular and cellular mechanisms. Our simulation results show how synaptic weight values are related to the type of training paradigm, suggesting that real neural computation may only emerge inside a silicon chip as a consequence of a biologically inspired and more realistic definition of synaptic strength.

Design and characterisation of an optoelectronic neural chip

We present a prototype integrated circuit design suitable for use as a neural backplane in an optoelectronic artificial neural network. The device consists of a 4 × 4 array of optical input, electronic output processing elements. The processing elements can sample the temporally multiplexed grey scale generated by a binary mode ferroelectric liquid crystal spatial light modulator. They implement the post-synaptic summing, have a programmable thresholding function, and can respond to both positive and negative activations - a requirement of many artificial neural netork models.

Programmable-weight building blocks for analog VLSI neural network processors

Although the neural network paradigms have the intrinsic potential for parallel operations, a traditional computer cannot fully exploit it because of the serial hardware configuration. By using the analog circuit design approach, a large amount of parallel functional units can be realized in a small silicon area. In addition, appropriate accuracy requirements for neural operation can be satisfied. Components for a general-purpose neural chip have been designed and fabricated. Dynamically adjusted weight value storage provides programmable capability. Possible reconfigurable schemes for a general-purpose neural chip are also presented. Test of the prototype neural chip has been successfully conducted and an expected result has been achieved.

Capacity of autocorrelation associative memory with quantized synaptic weight

AbstractThe neural chips to speed up the process of a neural network have recently been developed actively. Since a conventional neural chip requires quantized synaptic weights, it is important to know the properties of the network with such weights. Devices for a 2‐layer network with the same number of input and output elements have been well developed, and such a device can be used as an autocorrelation associative memory by feeding its output to its input.This paper investigates theoretically the capacity of an autocorrelation associative memory when its synaptic weights are quantized with a finite number of bits. It also proposes an optimum quantization function. A system with a finite number of elements is computer simulated. The proposed method can be applied to the design of a neural chip for associative memory.

RESULTS FROM A NEURAL TRIGGER BASED ON THE MA16 MICROPROCESSOR

Results from a neural-network trigger based on the digital MA16 chip of Siemens are reported. The neural trigger has been applied to data from the WA92 experiment, looking for beauty particles, which have been collected during a run in which a neural trigger module based on Intel’s analog neural chip ETANN operated, as already reported. The MA16 has a precision of 16 bits for input variables, 16 bits for weights, 16 bits for scalar multipliers modifying the transfer function shape, 47 bits for thresholds, 53 bits for internal calculations, 38 bits for the output. Of the latter only 16 bits are used in the MA16 board, as input to the transfer function inplemented on 16-bit addressable EPROM’s. The MA16 board operated at 50 MHz, yielding a response time for a 16 input variable net of 3 μs for a Fisher discriminant (1-layer net) and of 6 μs for a 2-layer net. Results are compared with those previously obtained with the ETANN trigger.

Results from an on-line non-leptonic neural trigger implemented in an experiment looking for beauty

Results from a non-leptonic neural-network trigger hosted by experiment WA92, looking for beauty particle production from 350 GeV negative pions on a fixed Cu target, are presented. The neural trigger has been used to send events selected by means of a non-leptonic signature based on microvertex detector information to a special data stream, meant for early analysis. The non-leptonic signature, defined in a neural-network fashion, was devised so as to enrich the selected sample in the number of events containing C3 secondary vertices (i.e., vertices having three tracks with sum of electric charges equal to +1 or −1), which are sought for further analysis to identify charm and beauty non-leptonic decays. The neural trigger module consists of a VME crate hosting two MA16 digital neural chips from Siemens and two ETANN analog neural chips from Intel. During the experimental run, only the ETANN chips were operational. The neural trigger operated for two continuous weeks during the WA92 1993 run. For an acceptance of 15% for C3 events, the neural trigger yields a C3 enrichment factor of 6.6–7.1 (depending on the event sample considered), which multiplied by that already provided by the standard trigger leads to a global C3 enrichment factor of ∼ 150. In the event sample selected by the neural trigger, one every ∼ 7 events contains a C3 vertex. The response time of the neural trigger module is 5.8 μs.

Cintia: a neuro-fuzzy real-time controller for low-power embedded systems

Cintia, a neuro-fuzzy real-time controller based on pulse stream computation techniques, is designed for applications in low-power embedded systems. The system mixes two approaches: neuro-fuzzy controllers and finite-state automata. We implement the former by means of a custom neural chip, the latter as sequential code on a traditional microcontroller. Cintia demonstrates the advantages of mixing the two approaches and the feasibility of embedded neuro-fuzzy control systems. A low-power, single-chip version is also under design.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Image recognition with an analog neural net chip

We developed a machine vision system around an analog neural net chip and used it in several applications. Some of them were: locating the address blocks on mail pieces, finding the identification numbers on rail cars, and discriminating between handwritten and machine-printed characters. The chip, operating as a coprocessor of a workstation, provides a speed-up of a factor of 1000, compared with the workstation. The computation speed achieved lies between one and ten billion multiply-accumulates/s. The neural net chip is based on building blocks,neurons, that can be arranged in various network architectures. The dataflow is optimized for implementing large, structured neural nets, and is also suited for any task in which signals are to be convolved with many kernels. Some of the networks are trained on the neural net chip with a weight-perturbation learning algorithm that was adapted to work with the coarse quantization of the weights and the states in the chip.

From neural chip and engineered biomolecules to bioelectronic devices: An overview

At the first C.E.C. Workshop, in Brussels on 28–29 November 1991, attended by over 70 leading European scientists and industrialists, bioelectronics was defined as the ‘the use of biological materials and biological architectures for information processing systems and new devices’. At the end of the Frankfurt Workshop, bioelectronics, specifically bio-molecular electronics, was described as ‘the research and development of bio-inspired ( i.e. self-assembly) inorganic and organic materials and of bio-inspired ( i.e. massive parallelism) hardware architectures for the implementation of new information processing systems, sensors and actuators, and for molecular manufacturing down to the atomic scale’. The subject of this overview is to summarize some of the most significant progress in bio-molecular electronics from neural VLSI networks and bio-molecular engineering. As an example of one possible route, emphasis is placed on the results recently obtained within this laboratory.

Optical neural chips

An array of photodetectors, each having a variable sensitivity, forms the most important component of our two gallium arsenide chips. We designed the optical neurochip so that variable sensitivity photodiodes are monolithically integrated on top of an LED array, serving both as fast analog multipliers and as on-chip weight storage elements with learning capability. Our artificial retina device combines a VSPD array with a neural network for postprocessing, allowing us to perform fast, yet flexible, processing operations on projected images. >

Testing of programmable analog neural network chips

Artificial neural network chips can achieve high-speed performance in solving complex computational problems for signal and information processing applications. These chips contain regular circuit units such as synapse matrices that interconnect linear arrays of input and output neurons. The neurons and synapses may be implemented in an analog or digital design style. Although the neural processing has some degree of fault tolerance, a significant percentage of processing defects can result in catastrophic failure of the neural network processors. Systematic testing of these arrays of circuitry is of great importance in order to assure the quality and reliability of VLSI neural network processor chips. The proposed testing method consists of parametric test and behavioral test. Two programmable analog neural chips have been designed and fabricated. The systematic approach used to test the chips is described, and measurement results on parametric test are presented.

AN ON-LINE NON-LEPTONIC NEURAL TRIGGER APPLIED TO AN EXPERIMENT LOOKING FOR BEAUTY

Results from a non-leptonic neural-network trigger hosted by experiment WA92, looking for beauty particle production from 350 GeV π− on a Cu target, are presented. The neural trigger has been used to send on a special data stream (the Fast Stream) events to be analyzed with high priority. The non-leptonic signature uses microvertex detector data and was devised so as to enrich the fraction of events containing C3 secondary vertices (i.e, vertices having three tracks whith sum of electric charges equal to +1 or -1). The neural trigger module consists of a VME crate hosting two ETANN analog neural chips from Intel. The neural trigger operated for two continuous weeks during the WA92 1993 run. For an acceptance of 15% for C3 events, the neural trigger yields a C3 enrichment factor of 6.6–7.1 (depending on the event sample considered), which multiplied by that already provided by the standard non-leptonic trigger leads to a global C3 enrichment factor of ≈150. In the event sample selected by the neural trigger for the Fast Stream, 1 every ≈7 events contains a C3 vertex. The response time of the neural trigger module is 5.8 μs.

Pulse stream VLSI neural networks

EPSILON, a large, working, VLSI device, demonstrates pulse stream methods in the wider context of analog neural networks. EPSILON uses dynamic weight storage techniques, but a nonvolatile alternative is desirable. To that end, we have developed an amorphous silicon memory, which we present in experiments incorporating the device in a modest pulse stream neural chip. We have also developed a target-based training algorithm, which we demonstrate in a prototype learning device using a realistic problem. Finally, we explore system-level problems in experiments with a second version of EPSILON in a small, autonomous robot. >

Implementation of new superconducting neural circuits using coupled SQUIDs

A novel superconducting neuron circuit and two types of variable synapses, which are based on superconducting quantum interferometer devices (SQUIDs), are presented. A neuron circuit with good input-output isolation and steep threshold characteristics is accomplished using a combination of a single-junction SQUID coupled to a double-junction SQUID. The quantum state of the single-junction SQUID represents the neuron state, and the output voltage of the double-junction SQUID, which is operated in a nonlatching mode with shunt resistors, is a sigmoid-shaped function of the input. Both variable synapse circuits are composed of multiple shunted double-junction SQUIDs. The first type changes its conductance value by using both superconducting and voltage states. The second variable synapse circuit changes its output current digitally by switching its bias currents. Besides numerical simulations of the circuit characteristics, we have fabricated superconducting neural chips in a Nb/AlO/sub x//Nb Josephson junction technology. The fundamental operation of each element and a 2-bit neural-based A/D converter have been successfully tested. A learning system with a variable synapse is also discussed.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

PULSE STREAM VLSI CIRCUITS AND SYSTEMS: THE EPSILON NEURAL NETWORK CHIPSET

An analogue CMOS VLSI neural processing chip has been designed and fabricated. The device employs "pulse-stream" neural state signalling and is capable of computing some 360 million synaptic connections per second. In addition to basic characterisation results, the performance of the chip in solving "real-world" problems is also demonstrated. The experience gained from the development of this device has resulted in the design of a second "pulse-stream" chip with improved performance and features. It is anticipated that this second device will be integrated into a standard bus-based system and find early application in robotic control.

Kakadu--a low power analogue neural network classifier.

An analogue neural network VLSI chip designed for low power operation is presented. This chip consists of 84 synapse elements arranged as arrays of size 10 x 6 and 6 x 4 and was fabricated using a standard 1.2 micron double metal single poly CMOS process. The synapses are digitally programmable and static weight storage is provided. The chip has a typical power consumption of tens of microwatts. It has been successfully trained and tested on a range of classification problems including 4-bit parity, character recognition and morphological-based classification of intracardiac electrogram signals.

Neural accelerator for parallelization of back-propagation algorithm

The neural chip L-Neuro 1.0 from LEP labs is used to parallelize parts of the back-propagation algorithm to control real-time applications. One accelerator was built; it takes advantage from the coupling between two neural chips and one MC68030 processor. One three layer perceptron with 88 connections is calculated twice as fast as on a Sun Sparc 2 workstation. The robotics real-time applications are hence possible with only few chips.