Implementation Of Multilayer Neural Network Research Articles

Hybrid no-propagation learning for multilayer neural networks

Abstract A hybrid learning algorithm suitable for hardware implementation of multi-layer neural networks is proposed. Though backpropagation is a powerful learning method for multilayer neural networks, its hardware implementation is difficult due to complexities of the neural synapses and the operations involved in error backpropagation. We propose a learning algorithm with performance comparable to but easier than backpropagation to be implemented in hardware for on-chip learning of multi-layer neural networks. In the proposed learning algorithm, a multilayer neural network is trained with a hybrid of gradient-based delta rule and a stochastic algorithm, called Random Weight Change. The parameters of the output layer are learned using the delta rule, whereas the inner layer parameters are learned using Random Weight Change, thereby the overall multilayer neural network is trained without the need for error backpropagation. Experimental results showing better performance of the proposed hybrid learning rule than either of its constituent learning algorithms, and comparable to that of backpropagation on the benchmark MNIST dataset are presented. Hardware architecture illustrating the ease of implementation of the proposed learning rule in analog hardware vis-a-vis the backpropagation algorithm is also presented.

Fuzzy Method and Neural Network Model Parallel Implementation of Multi-Layer Neural Network Based on Cloud Computing for Real Time Data Transmission in Large Offshore Platform

Abstract With the rapid development of electronic technology, network technology and cloud computing technology, the current data is increasing in the way of mass, has entered the era of big data. Based on cloud computing clusters, this paper proposes a novel method of parallel implementation of multilayered neural networks based on Map-Reduce. Namely in order to meet the requirements of big data processing, this paper presents an efficient mapping scheme for a fully connected multi-layered neural network, which is trained by using error back propagation (BP) algorithm based on Map-Reduce on cloud computing clusters (MRBP). The batch-training (or epoch-training) regimes are used by effective segmentation of samples on the clusters, and are adopted in the separated training method, weight summary to achieve convergence by iterating. For a parallel BP algorithm on the clusters and a serial BP algorithm on uniprocessor, the required time for implementing the algorithms is derived. The performance parameters, such as speed-up, optimal number and minimum of data nodes are evaluated for the parallel BP algorithm on the clusters. Experiment results demonstrate that the proposed parallel BP algorithm in this paper has better speed-up, faster convergence rate, less iterations than that of the existed algorithms.

An application of multilayer neural network on hepatitis disease diagnosis using approximations of sigmoid activation function

Objective: Implementation of multilayer neural network (MLNN) with sigmoid activation function for the diagnosis of hepatitis disease. Methods: Artificial neural networks (ANNs) are efficient tools currently in common use for medical diagnosis. In hardware based architectures activation functions play an important role in ANN behavior. Sigmoid function is the most frequently used activation function because of its smooth response. Thus, sigmoid function and its close approximations were implemented as activation function. The dataset is taken from the UCI machine learning database. Results: For the diagnosis of hepatitis disease, MLNN structure was implemented and Levenberg Morquardt (LM) algorithm was used for learning. Our method of classifying hepatitis disease produced an accuracy of 91.9% to 93.8% via 10 fold cross validation. Conclusion: When compared to previous work that diagnosed hepatitis disease using artificial neural networks and the identical data set, our results are promising in order to reduce the size and cost of neural network based hardware. Thus, hardware based diagnosis systems can be developed effectively by using approximations of sigmoid function. Key words: Hepatitis disease diagnosis, multilayer neural network, 10-fold cross validation, approximations of sigmoid activation function

Reinforcement and backpropagation training for an optical neural network using self-lensing effects

The optical bench training of an optical feedforward neural network, developed by the authors, is presented. The network uses an optical nonlinear material for neuron processing and a trainable applied optical pattern as the network weights. The nonlinear material, with the applied weight pattern, modulates the phase front of a forward propagating information beam by dynamically altering the index of refraction profile of the material. To verify that the network can be trained in real time, six logic gates were trained using a reinforcement training paradigm. More importantly, to demonstrate optical backpropagation, three gates were trained via optical error backpropagation. The output error is optically backpropagated, detected with a CCD camera, and the weight pattern is updated and stored on a computer. The obtained results lay the ground work for the implementation of multilayer neural networks that are trained using optical error backpropagation and are able to solve more complex problems.

Implementation of multilayer neural network with threshold neurons and its analysis

In this paper, the implementation of new digital architecture for a multilayer neural network (MNN) with on-chip learning is discussed. The advantage of using the digital approach is that it can use state-of-the-art VLSI and ULSI implementation techniques. One of the major hard-ware problems in implementing a neural network is the activating function of the neurons. The proposed MNN uses a simple function as the neuron's activating function to reduce the circuit size. Moreover, the proposed MNN has an on-chip learning capability. As the learning algorithm, a backpropagation algorithm is modified for effective hard-wave implementation. The proposed MNN is implemented on a field-programmable gate array (FPGA) to evaluate the learning performance and circuit size.

Backpropagation in linear arrays-a performance analysis and optimization

Neural networks are valuable tools for the support of a wide range of image processing applications. For video-rate operation, special-purpose parallel hardware is often necessary. One of the most common architectures used for this purpose is the linear systolic array. The design and implementation of multi-layer neural networks in linear systolic arrays can be complex, however. This paper demonstrates that the smallest network is not necessarily the best in terms of learning or recall times. Furthermore, this paper shows that the manner in which networks are mapped into a particular hardware structure affects both the performance of the application and the efficiency with which the hardware resources are used. We analyze and identify how to best structure neural networks to optimize network performance for throughput, latency and the efficiency with which the hardware is used. We use the HANNIBAL neural network processor as a research vehicle for these investigations and demonstrate the value of the proposed techniques by a number of example applications.

Optoelectronic implementation of multilayer neural networks in a single photorefractive crystal

We present a novel, versatile optoelectronic neural network architecture for implementing supervised learning algorithms in photorefractive materials. The system is based on spatial multiplexing rather than the more commonly used angular multiplexing of the interconnect gratings. This simple, single-crystal architecture implements a variety of multilayer supervised learning algorithms including mean field theory, back-propagation, and Marr-Albus-Kanerva style algorithms. Extensive simulations show how beam depletion, rescattering, absorption, and decay effects of the crystal are compensated for by suitably modified supervised learning algorithms.