Decrementing hamming and Bayesian neural networks: Analog implementations and relative performance

Abstract

A review of recent patent applications indicates that neural networks using Hamming-type algorithms with minimum-mismatch selection provide an optimal combination of implementational simplicity, information storage capacity and signal-noise characteristics. These networks can be adapted to implement Bayes' rule, by setting link gains to the negative logarithm of conditional or a priori probabilities. Where probability distributions and noise are not uniform or random, the performance of Bayesian classifiers may be significantly better than that of the corresponding Hamming network on the same vector set. We demonstrate this for the noisy digit classification task. We also generate biologically plausible curvature detectors for character recognition and compare the performances of Bayesian and Hamming networks at classifying the resultant vectors. Preliminary results suggest that Hamming networks may provide good approximations to the Bayes optimum for sparse natural vector sets under some conditions.