Probabilistic Decision-based Neural Networks Research Articles

Texture recognition by generalized probabilistic decision-based neural networks

Texture recognition have received tremendous attentions in the past decades, due to its wide applications in computer vision and pattern recognition. For various applications, formulating texture features in distributional forms can sometimes provide meaningful representation than in numerical forms. In this paper, a generalized probabilistic decision-based neural network (GPDNN), based on a novel methodology for the measurement of the difference between two distributions, is proposed for texture recognition. Based on a two-layer pyramid-type network structure, the proposed GPDNN receives texture data via 2-D grid input nodes, and outputs the classification and/or retrieval results at the top layer node. Our prototype system demonstrates a successful utilization of GPDNN to the texture recognition on 40 texture images selected from the MIT Vision Texture (VisTex) database. Regarding the performance, experiment results show that (1) based on the proposed distribution difference measurement method, the texture retrieval accuracy is improved from 77% to 82% by comparing with some recently published leading methods, and (2) the proposed GPDNN has significant improvements in classification accuracy from 82.2% to 90.1% and retrieval accuracy from 79.9% to 88.6% by comparing with traditional approaches.

A self-growing probabilistic decision-based neural network with automatic data clustering

In this paper, we propose a new clustering algorithm for a mixture of Gaussian-based neural network and self-growing probabilistic decision-based neural networks (SPDNN). The proposed self-growing cluster learning (SGCL) algorithm is able to find the natural number of prototypes based on a self-growing validity measure, Bayesian information criterion (BIC). The learning process starts from a single prototype randomly initialized in the feature space and grows adaptively during the learning process until most appropriate number of prototypes are found. We have conducted numerical and real-world experiments to demonstrate the effectiveness of the SGCL algorithm. In the results of using SGCL to train the SPDNN for data clustering and speaker identification problems, we have observed a noticeable improvement among various model-based or vector quantization-based classification schemes.

User adaptive handwriting recognition by self-growing probabilistic decision-based neural networks

It is generally agreed that, for a given handwriting recognition task, a user dependent system usually outperforms a user independent system, as long as a sufficient amount of training data is available. When the amount of user training data is limited, however, such a performance gain is not guaranteed. One way to improve the performance is to make use of existing knowledge, contained in a rich multiuser data base, so that a minimum amount of training data is sufficient to initialize a model for the new user.We mainly address the user adaption issues for a handwriting recognition system. Based on self-growing probabilistic decision-based neural networks (SPDNNs), user adaptation of the parameters of SPDNN is formulated as incremental reinforced and antireinforced learning procedures, which are easily integrated into the batched training procedures of the SPDNN. In this study, we developed 1) an SPDNN based handwriting recognition system; 2) a two-stage recognition structure; and 3) a three-phase training methodology for a) a global coarse classifier (stage 1); b) a user independent hand written character recognizer (stage 2); and c) a user adaptation module on a personal computer. With training and testing on a 600-word commonly used Chinese character set, the recognition results indicate that the user adaptation module significantly improved the recognition accuracy. The average recognition rate increased from 44.2% to 82.4% in five adapting cycles, and the performance could finally increase up to 90.2% in ten adapting cycles.

Gaussian Mixture Models and Probabilistic Decision-Based Neural Networks for Pattern Classification: A Comparative Study

Probabilistic Decision-Based Neural Networks (PDBNNs) can be considered as a special form of Gaussian Mixture Models (GMMs) with trainable decision thresholds. This paper provides detailed illustrations to compare the recognition accuracy and decision boundaries of PDBNNs with that of GMMs through two pattern recognition tasks, namely the noisy XOR problem and the classification of two-dimensional vowel data. The paper highlights the strengths of PDBNNs by demonstrating that their thresholding mechanism is very effective in detecting data not belonging to any known classes. The original PDBNNs use elliptical basis functions with diagonal covariance matrices, which may be inappropriate for modelling feature vectors with correlated components. This paper overcomes this limitation by using full covariance matrices, and showing that the matrices are effective in characterising non-spherical clusters.

Application of Neural Networks to Cache Replacement

Probabilistic Decision-Based Neural Networks (PDBNNs) can be considered as a special form of Gaussian Mixture Models (GMMs) with trainable decision thresholds. This paper provides detailed illustrations to compare the recognition accuracy and decision boundaries of PDBNNs with that of GMMs through two pattern recognition tasks, namely the noisy XOR problem and the classification of two-dimensional vowel data. The paper highlights the strengths of PDBNNs by demonstrating that their thresholding mechanism is very effective in detecting data not belonging to any known classes. The original PDBNNs use elliptical basis functions with diagonal covariance matrices, which may be inappropriate for modelling feature vectors with correlated components. This paper overcomes this limitation by using full covariance matrices, and showing that the matrices are effective in characterising non-spherical clusters.

Face recognition/detection by probabilistic decision-based neural network

This paper proposes a face recognition system, based on probabilistic decision-based neural networks (PDBNN). With technological advance on microelectronic and vision system, high performance automatic techniques on biometric recognition are now becoming economically feasible. Among all the biometric identification methods, face recognition has attracted much attention in recent years because it has potential to be most nonintrusive and user-friendly. The PDBNN face recognition system consists of three modules: First, a face detector finds the location of a human face in an image. Then an eye localizer determines the positions of both eyes in order to generate meaningful feature vectors. The facial region proposed contains eyebrows, eyes, and nose, but excluding mouth (eye-glasses will be allowed). Lastly, the third module is a face recognizer. The PDBNN can be effectively applied to all the three modules. It adopts a hierarchical network structures with nonlinear basis functions and a competitive credit-assignment scheme. The paper demonstrates a successful application of PDBNN to face recognition applications on two public (FERET and ORL) and one in-house (SCR) databases. Regarding the performance, experimental results on three different databases such as recognition accuracies as well as false rejection and false acceptance rates are elaborated. As to the processing speed, the whole recognition process (including PDBNN processing for eye localization, feature extraction, and classification) consumes approximately one second on Sparc10, without using hardware accelerator or co-processor.