Multilayer Feed-forward Back-propagation Research Articles

Automatic Classification of Subdwarf Spectra using a Neural Network

We apply a multilayer feed-forward back propagation artificial neural network to a sample of 380 subdwarf spectra classified by Drilling et al. (Drilling, J.S., Moehler, S., Jeffery, C.S., Heber, U., and Napiwotzki, R.: in press in: R. Gray (ed.), Probing the Personalities of Stars and Galaxies), showing that it is possible to use this technique on large sets of spectra and obtain classifications in good agreement with the standard. We briefly investigate the impact of training set size, showing that large training sets do not necessarily perform significantly better than small sets.

Improving the inversion of ionograms by combining neural network and data fusion techniques

Data fusion (integration) techniques are combined with MultiLayer Feedforward (backpropagation) neural networks in order to improve the inversion —extraction of the key describing parameters — of oblique-incidence ionograms (plots of apparent height of reflection versus transmission frequency). Two separate investigations were undertaken: first, the incorporation of vertical ionogram data to improve inversion; and secondly, the fusion of ionogram data gathered from a 2D array of ionosondes (the ground-based radio frequency transmitters). With the former, the average percentage errors obtained by incorporating data fusion dropped by a factor of five when compared with single ionogram inversion. Moreover, gradients of ionospheric parameters (critical frequency, layer height and thickness) were also obtained. In the case of the latter, the error rate dropped by a similar factor, and by even more when vertical ionograms were incorporated. Better results were forthcoming when a hierarchical network was used to invert the ionograms prior to fusion, compared with directly fusing the ionogram array data.

Aspect-independent sonar recognition of cylinders using dolphin-like signals

Echoes from four different cylinders having the same outer diameter (3.8 cm) and length (10.2 cm) were collected at 1-deg increments from the broadside to the end aspect and used in a neural network recognition study. The water-filled stainless-steel, water-filled aluminum, and foam-filled aluminum cylinders had the same 0.64-cm wall thickness. The fourth cylinder consisted of coral rock pebbles embedded in degassed epoxy. A dolphinlike sonar signal with a peak frequency of 120 kHz and a duration of approximation 50 μs was used in a monostatic echo collection system. Fifty echoes digitized at a 1-MHz rate were collected for each target at each aspect. Time-frequency representation of the echoes along with the envelope of the matched filter response were fed to two parallel independent multilayered feedforward backpropagation neural networks and the results were fused together in another feedforward network. The network was able to achieve better than 90% correct recognition of the echoes when trained with echoes that were separated in increments of 4 deg. Performance dropped to approximately 80% when the network was trained on echoes separated in increments of 10 deg.

Adaptive resonance theory based neural network for supervised chemical pattern recognition (FuzzyARTMAP) Part 2: Classification of post-consumer plastics by remote NIR spectroscopy using an InGaAs diode array

The supervised working FuzzyARTMAP pattern recognition algorithm has been applied to automated identification of post-consumer plastics by near-infrared spectroscopy (NIRS). Experimentally, a remote operating parallel multisensor device, based on a rapid InGaAs diode array detector combined with new collimation optics, has been used. The laboratory setup allows on-line identification of more than 100 spectra per second. Internal parameter settings of FuzzyARTMAP were varied to explore their influence on the classifier's behavior. Discrimination results obtained were better than those from an optimized multilayer feedforward backpropagation artificial neural network (MLF-BP) and significantly better than those provided by the partial least squares method (PLS2). Additional advantages of FuzzyARTMAP compared to these two classifiers are a significantly higher speed of calibration, the chemical interpretability of network weight coefficients and a built-in detector against extrapolations.

Comparison of an adaptive resonance theory based neural network (ART-2a) against other classifiers for rapid sorting of post consumer plastics by remote near-infrared spectroscopic sensing using an InGaAs diode array

An Adaptive Resonance Theory Based Artificial Neural Network (ART-2a) has been compared with Multilayer Feedforward Backpropagation of Error Neural Networks (MLF-BP) and with the SIMCA classifier. All three classifiers were applied to achieve rapid sorting of post-consumer plastics by remote near-infrared (NIR) spectroscopy. A new semiconductor diode array detector based on InGaAs technology has been experimentally tested for measuring the NIR spectra. It has been found by a cross validation scheme that MLF-BP networks show a slightly better discrimination power than ART-2a networks. Both types of artificial neural networks perform significantly better than the SIMCA method. A median sorting purity of better than 98% can be guaranteed for non-black plastics. More than 75 samples per second can be identified by the combination InGaAs diode array/neural network. However, MLF-BP neural networks can definitely not extrapolate. Uninterpretable predictions were observed in case of test samples that truly belong to a particular class but that are located outside the subspace defined by training set.

Pruning Divide & Conquer networks

Determining an effective architecture for multi-layer feedforward backpropagation neural networks can be a time-consuming effort. In general it requires human intervention in determining the number of layers, number of hidden cells, the learning rule and the learning parameters. Over the past few years several approaches to dynamically configure neural networks have been proposed, which remove most of the responsibility for choosing the correct network configuration from the user. As important as finding a viable network architecture for some given learning problem, is the need to obtain a minimal configuration. The total time required to emulate or simulate neural networks is largely dependent on the number of connections present in a network. Therefore, it is essential to provide pruning methods to reduce network complexity. In this paper two approaches to network pruning will be investigated: single- and multi-pass pruning. Their effectiveness is emphasized by applying them to several real world problem domains and comparing them to a modification of Optimal Brain Damage. It will be shown that multi-pass pruning is not only fast, but also effective in reducing the network size when used in conjunction with Divide & Conquer Learning. Finally, it is general enough to be applied to other dynamically network configuring learning approaches.

Divide and Conquer Neural Networks

Determining an effective architecture far a multi-layer feedforward back propagation neural network can be a time-consuming effort. We describe an algorithm called Divide and Conquer Neural Networks (DCN), which creates a feedforward neural network architecture during training, based upon the training examples. The first cell introduced on any layer is trained on all examples. Further cells on a layer are trained primarily on examples not already correctly classified. The learning algorithm is shown to be able to use several different learning rules, including the delta rule and perceptron rule, to modify the link weights one level at a time in the spirit of a perceptron. Error is never propagated backwards through a hidden cell. Examples are shown of networks generated for the exdusive-or, 4 and 5-parity, 2-spirals problem, Iris plant classification, predicting party affiliation from voting records, and the real-valued fuzzy exclusive-or. The results show the algorithm effectively learns viable architectures that can generalize.

Pruning Divide & Conquer networks

Determining an effective architecture for multi-layer feedforward backpropagation neural networks can be a time-consuming effort. In general it requires human intervention in determining the number of layers, number of hidden cells, the learning rule and the learning parameters. Over the past few years several approaches to dynamically configure neural networks have been proposed, which remove most of the responsibility for choosing the correct network configuration from the user. As important as finding a viable network architecture for some given learning problem, is the need to obtain a minimal configuration. The total time required to emulate or simulate neural networks is largely dependent on the number of connections present in a network. Therefore, it is essential to provide pruning methods to reduce network complexity. In this paper two approaches to network pruning will be investigated: single- and multi-pass pruning. Their effectiveness is emphasized by applying them to several real world proble...

Modelling flow dynamics in water distribution networks using artificial neural networks - A leakage detection technique

Computational approaches can be used to detect leakages in water distribution networks. One such approach is the Artificial Neural Networks (ANNs) technique. The advantage of ANNs is that they are robust and can be used to model complex linear and non-linear systems without making implicit assumptions. ANNs can be trained to forecast flow dynamics in a water distribution network. Such flow dynamics can be compared with water demands in a particular district metered area. The objective of this study was to model flow dynamics in four district metered areas of the City of Harare, Zimbabwe using the ANNs technique in an effort to detect systems leakages. A multi-layer feed-forward back-propagation artificial neural network was used for modelling the flow and simulate water demand using a Matlab Neural Network Toolkit. The difference between actual water consumed (metered consumption) and the simulated water demand for a district metered area represents the water leakage in the water distribution network of the district metered area. It was discovered that an ANN could be trained and be used to forecast flow with up to 99% confidence. Thus, ANNs technique is a flexible and efficient approach to detection of leakages in water distribution networks.Keywords: Artificial neural network; Leakage detection technique; Water distribution; Leakages