Binary Feedforward Neural Networks Research Articles

Sensitivity study of Binary Feedforward Neural Networks

This paper presents a novel and effective approach for establishing a quantified output sensitivity of Binary Feedforward Neural Networks to weight and input perturbations. Firstly, analytical formulae are derived for computing a neuron׳s sensitivity by means of matrix and probability theories. Then, based on the neuron׳s sensitivity and the network׳s architecture feature, a bottom-up strategy is followed to compute the entire network׳s sensitivity. The proposed approach has the obvious advantages of higher generality, lower computational complexity, and yet much higher accuracy. Experimental results verify the correctness and effectiveness of the approach.

Sensitivity-based adaptive learning rules for binary feedforward neural networks.

This paper proposes a set of adaptive learning rules for binary feedforward neural networks (BFNNs) by means of the sensitivity measure that is established to investigate the effect of a BFNN's weight variation on its output. The rules are based on three basic adaptive learning principles: the benefit principle, the minimal disturbance principle, and the burden-sharing principle. In order to follow the benefit principle and the minimal disturbance principle, a neuron selection rule and a weight adaptation rule are developed. Besides, a learning control rule is developed to follow the burden-sharing principle. The advantage of the rules is that they can effectively guide the BFNN's learning to conduct constructive adaptations and avoid destructive ones. With these rules, a sensitivity-based adaptive learning (SBALR) algorithm for BFNNs is presented. Experimental results on a number of benchmark data demonstrate that the SBALR algorithm has better learning performance than the Madaline rule II and backpropagation algorithms.

On sequential construction of binary neural networks

A new technique called sequential window learning (SWL), for the construction of two-layer perceptrons with binary inputs is presented. It generates the number of hidden neurons together with the correct values for the weights, starting from any binary training set. The introduction of a new type of neuron, having a window-shaped activation function, considerably increases the convergence speed and the compactness of resulting networks. Furthermore, a preprocessing technique, called hamming clustering (HC), is proposed for improving the generalization ability of constructive algorithms for binary feedforward neural networks. Its insertion in the sequential window learning is straightforward. Tests on classical benchmarks show the good performances of the proposed techniques, both in terms of network complexity and recognition accuracy.

The Patch algorithm: fast design of binary feedforward neural networks

A new constructive learning algorithm to generate binary neural networks is presented. The networks that are constructed are layered feedforward neural networks. Each layer is constructed to reduce the number of internal representations under the constraint that it is faithful. This algorithm, the Patch algorithm, is able to handle analogue inputs and problems with multiple output states. The algorithm allows training sets up to several thousands of patterns with up to several thousands of coordinates per pattern.

A comparison study of binary feedforward neural networks and digital circuits

A comparison study was carried out between feedforward neural networks composed of binary linear threshold units and digital circuits. These networks were generated by the regular partitioning algorithm and a modified Quine-McCluskey algorithm, respectively. The size of both types of networks and their generalisation properties are compared as a function of the nearest-neighbour correlation in the binary input sets. The ratio of the number of components required by digital circuits and the number of neurons grows linearly for the input sets considered. The considered neural networks do not outperform digital circuits with respect to generalisation. Sensitivity analysis leads to a preference for digital circuits, especially for increasing number of inputs. In the case of analog input sets, hybrid networks of binary neurons and logic gates are of interest.

The Patch algorithm: fast design of binary feedforward neural networks

A new constructive learning algorithm to generate binary neural networks is presented. The networks that are constructed are layered feedforward neural networks. Each layer is constructed to reduce the number of internal representations under the constraint that it is faithful. This algorithm, the Patch algorithm, is able to handle analogue inputs and problems with multiple output states. The algorithm allows training sets up to several thousands of patterns with up to several thousands of coordinates per pattern.

A training algorithm for binary feedforward neural networks

The authors present a new training algorithm to be used on a four-layer perceptron-type feedforward neural network for the generation of binary-to-binary mappings. This algorithm is called the Boolean-like training algorithm (BLTA) and is derived from original principles of Boolean algebra followed by selected extensions. The algorithm can be implemented on analog hardware, using a four-layer binary feedforward neural network (BFNN). The BLTA does not constitute a traditional circuit building technique. Indeed, the rules which govern the BLTA allow for generalization of data in the face of incompletely specified Boolean functions. When compared with techniques which employ descent methods, training times are greatly reduced in the case of the BLTA. Also, when the BFNN is used in conjunction with A/D converters, the applicability of the present algorithm can be extended to accept real-valued inputs.