Neural Network For Pattern Classification Research Articles

R-POPTVR: A Novel Reinforcement-Based POPTVR Fuzzy Neural Network for Pattern Classification

In general, a fuzzy neural network (FNN) is characterized by its learning algorithm and its linguistic knowledge representation. However, it does not necessarily interact with its environment when the training data is assumed to be an accurate description of the environment under consideration. In interactive problems, it would be more appropriate for an agent to learn from its own experience through interactions with the environment, i.e., reinforcement learning. In this paper, three clustering algorithms are developed based on the reinforcement learning paradigm. This allows a more accurate description of the clusters as the clustering process is influenced by the reinforcement signal. They are the REINFORCE clustering technique I (RCT-I), the REINFORCE clustering technique II (RCT-II), and the episodic REINFORCE clustering technique (ERCT). The integrations of the RCT-I, the RCT-II, and the ERCT within the pseudo-outer product truth value restriction (POPTVR), which is a fuzzy neural network integrated with the truth restriction value (TVR) inference scheme in its five layered feedforward neural network, form the RPOPTVR-I, the RPOPTVR-II, and the ERPOPTVR, respectively. The Iris, Phoneme, and Spiral data sets are used for benchmarking. For both Iris and Phoneme data, the RPOPTVR is able to yield better classification results which are higher than the original POPTVR and the modified POPTVR over the three test trials. For the Spiral data set, the RPOPTVR-II is able to outperform the others by at least a margin of 5.8% over multiple test trials. The three reinforcement-based clustering techniques applied to the POPTVR network are able to exhibit the trial-and-error search characteristic that yields higher qualitative performance.

Estimation of the knee joint angle from surface electromyographic signals for active control of leg prostheses

The surface electromyographic (SEMG) signal is very convenient for prosthesis control because it is non-invasively acquired and intrinsically related to the user's intention. This work presents a feature extraction and pattern classification algorithm for estimation of the intended knee joint angle from SEMG signals acquired using two sets of electrodes placed on the upper leg. The proposed algorithm uses a combination of time-domain and frequency-domain approaches for feature extraction (signal amplitude histogram and auto-regressive coefficients, respectively), a self-organizing map for feature projection and a Levenberg–Marquardt multi-layer perceptron neural network for pattern classification. The new algorithm was quantitatively compared with the method proposed by Wang et al (2006 Med. Biol. Eng. Comput. 44 865–72), which uses wavelet packet feature extraction, principal component analysis and a multi-layer perceptron neural classifier. The proposed method provided lower error-to-signal percentage and peak error amplitudes, higher correlation and fewer error events. The algorithm presented in this work may be useful as part of a myoelectric controller for active leg prostheses designed for transfemoral amputees.

Bio-inspired and gradient-based algorithms to train MLPs: The influence of diversity

This paper has three main goals: (i) to employ two classes of algorithms: bio-inspired and gradient-based to train multi-layer perceptron (MLP) neural networks for pattern classification; (ii) to combine the trained neural networks into ensembles of classifiers; and (iii) to investigate the influence of diversity in the classification performance of individual and ensembles of classifiers. The optimization version of an artificial immune network, named opt-aiNet, particle swarm optimization (PSO) and an evolutionary algorithm (EA) are used as bio-inspired methods to train MLP networks. Besides, the standard backpropagation with momentum (BPM), a quasi-Newton method called DFP and a modified scaled-conjugate gradient (SCGM) are the gradient-based algorithms used to train MLP networks in this work. Comparisons among all the training methods are presented in terms of classification accuracy and diversity of the solutions found. The results obtained suggest that most bio-inspired algorithms deteriorate the diversity of solutions during the search, while immune-based methods, like opt-aiNet, and multiple initializations of standard gradient-based algorithms provide diverse solutions that result in good classification accuracy for the ensembles.

An ant colony optimization algorithm for continuous optimization: application to feed-forward neural network training

Ant colony optimization (ACO) is an optimization technique that was inspired by the foraging behaviour of real ant colonies. Originally, the method was introduced for the application to discrete optimization problems. Recently we proposed a first ACO variant for continuous optimization. In this work we choose the training of feed-forward neural networks for pattern classification as a test case for this algorithm. In addition, we propose hybrid algorithm variants that incorporate short runs of classical gradient techniques such as backpropagation. For evaluating our algorithms we apply them to classification problems from the medical field, and compare the results to some basic algorithms from the literature. The results show, first, that the best of our algorithms are comparable to gradient-based algorithms for neural network training, and second, that our algorithms compare favorably with a basic genetic algorithm.

Support-vector-based fuzzy neural network for pattern classification

Fuzzy neural networks (FNNs) for pattern classification usually use the backpropagation or C-cluster type learning algorithms to learn the parameters of the fuzzy rules and membership functions from the training data. However, such kinds of learning algorithms usually cannot minimize the empirical risk (training error) and expected risk (testing error) simultaneously, and thus cannot reach a good classification performance in the testing phase. To tackle this drawback, a support-vector-based fuzzy neural network (SVFNN) is proposed for pattern classification in this paper. The SVFNN combines the superior classification power of support vector machine (SVM) in high dimensional data spaces and the efficient human-like reasoning of FNN in handling uncertainty information. A learning algorithm consisting of three learning phases is developed to construct the SVFNN and train its parameters. In the first phase, the fuzzy rules and membership functions are automatically determined by the clustering principle. In the second phase, the parameters of FNN are calculated by the SVM with the proposed adaptive fuzzy kernel function. In the third phase, the relevant fuzzy rules are selected by the proposed reducing fuzzy rule method. To investigate the effectiveness of the proposed SVFNN classification, it is applied to the Iris, Vehicle, Dna, Satimage, Ijcnn1 datasets from the UCI Repository, Statlog collection and IJCNN challenge 2001, respectively. Experimental results show that the proposed SVFNN for pattern classification can achieve good classification performance with drastically reduced number of fuzzy kernel functions.

Steel Bars Identification in Reinforced Concrete Structures by Using ANN and Magnetic Fields

This work proposes a methodology for non destructive testing (NDT) of reinforced concrete structures, using superficial magnetic fields and artificial neural networks, in order to identify the size and position of steel bars, embedded into the concrete. For the purposes of this paper, magnetic induction curves were obtained by using a finite element program. Perceptron Multilayered (PML) ANNs, with Levemberg-Marquardt training algorithm were used. The results presented very good agreement with the expect ones, encouraging the development of real systems based upon the proposed methodology. This work proposes a new methodology for the inspection of steel bar immersed in concrete structures, based in the analysis of magnetic induction curves, created at the region of the structure by electromagnetic devices specially designed for this purpose. The analysis of the magnetic induction curves is done by means of artificial neural networks. For the purpose of this paper, the magnetic inductions values were simulated using a finite element program (6). 1. The Proposed Methodology The methodology proposed in this paper for the localization and identification of steel bars in reinforced concrete structures consists in the utilization of artificial neural networks for pattern classification of electro- magnetic signatures presented by the steel bars embedded into the concrete, at the concrete surface, when static and/or quasi-static electromagnetic fields are generated in the region of the concrete structure. The presence of the steel bars within the concrete will slightly disturb the field distribution at the concrete surface. This field perturbation will depend of the size, position and number of bars within the concrete. Each bars configuration will produce an unique deviation curve, called magnetic signature of the bar (or bars). When a large number of samples are taken, artificial neural networks can be used to create an input/output relationship between the magnetic signature and the bar (or bars) configuration. In order to obtain a better comprehension of the proposed methodology, it will used to identify a steel bar embedded in the concrete. First of all, hundreds of curves of induction magnetic deviations must be obtained. For the purpose of this paper, they were generated using a finite element program (6). The curves represent variations of the position and size of the bar into the concrete. Figure (1a) illustrate the variations of the position of the bar in the horizontal axis, and figure 1b illustrate the variations in the vertical axis. They represent 5 variations in the horizontal position (stepped by 2.5 mm) and 30 variations in the vertical position (stepped by 2 mm). By this way, 150 variations in the position were considered. Beyond this, 7 sizes were considered for

Constructing and training feed-forward neural networks for pattern classification

A new approach of constructing and training neural networks for pattern classification is proposed. Data clusters are generated and trained sequentially based on distinct local subsets of the training data. Obtained clusters are then used to construct a feed-forward network, which is further trained using standard algorithms operating on the global training set. The network obtained using this approach effectively inherits the knowledge from the local training procedure before improving on its generalization ability through the subsequent global training. Various experiments demonstrate the superiority of this approach over competing methods.

Neural network classifiers applied to condition monitoring of a pneumatic process valve actuator

As modern process plants become more complex, the ability to detect and identify the faulty operation of pneumatic control valves is becoming increasingly important. In this work, a neural network pattern classifier is employed to carry out fault diagnosis and identification upon the actuator of a Fisher–Rosemount 667 industrial process valve. The network is trained with experimental data obtained directly from a software package that comes with the valve. This has eliminated the need for additional instrumentation of the valve. Using this software, tests are carried out to obtain experimental parameters associated with the valve performance for incorrect supply pressure, diaphragm leakage, and vent blockage faults. Specifically, the valve signature and dynamic error band tests are used to directly obtain lower and upper bench sets, minimum, maximum, and average dynamic errors, as well as the dynamic linearity. Additionally, valve deadband and hysteresis are measured graphically from the available valve signature plots for each faulty condition. The relationships between these parameters, for each fault, form signatures that are subsequently learned by a multilayer feedforward network trained by error back-propagation. The test results show that the resulting network has the ability to detect and identify various magnitudes of each fault. It is also observed that a smaller network with a shorter training time results when the valve deadband and hysteresis are included in the training data. Thus, the extra effort required to extract these parameters from the valve signature plots is justified.

A fuzzy neural network for pattern classification and feature selection

A fuzzy neural network with memory connections for classification, and weight connections for selection is introduced, thereby solving simultaneously two major problems in pattern recognition: pattern classification and feature selection. The proposed network attempts to select important features from among the originally given plausible features, while maintaining the maximum recognition rate. The resulting value of weight connection represents the degree of importance of feature. Moreover, the knowledge acquired by the network can be described as a set of interpretable rules. The effectiveness of this new method has been validated by using Anderson's IRIS data. The results are: first, the use of two features selected by our method from among the original four in the proposed network results in virtually identical classifier performance; and second, the constructed classifier is described by three simple rules that are of if–then form.

AUTOMATIC FROG CALLS MONITORING SYSTEM: A MACHINE LEARNING APPROACH

Automatic recognition of frog vocalization is considered a valuable tool for a variety of biological research and environmental monitoring applications. In this research an automatic monitoring system, which can recognize the vocalizations of four species of frogs and can identify different individuals within the species of interest, is proposed. For the desired monitoring system, species identification is performed first with the proposed filtering and grouping algorithm. Individual identification, which can estimate frog population within the specific species, is performed in the second stage. Digital signal pre-processing, feature extraction, dimensionality reduction, and neural network pattern classification are performed step by step in this stage. Wavelet Packet feature extraction together with two different dimension reduction algorithms are synergistically integrated to produce final feature vectors, which are to be fed into a neural network classifier. The simulation results show the promising future of deploying an array of continuous, on-line environmental monitoring systems based upon nonintrusive analysis of animal calls.

Pattern classification by a condensed neural network

Neural networks have come to the fore as potent pattern classifiers. More amenable to parallel computation, they are much faster than the nearest neighbor classifier (NN), which, however, has distinctly outperformed them in several applications. The purpose of this study is to investigate a condensed neural network that combines the classification speed of neural networks and the low error rate of the nearest neighbor classifier. This condensed network is a fast, accurate classifier of simple architecture and function: it consists of a set of generalized perceptrons that draw maximal hyperspherical boundaries centered on patterns of memory units, each circumscribing reference patterns of a single category. The generalized perceptrons carry out classification, assisted by sporadic nearest neighbor matching to patterns of a small reference set. We compare the condensed network to a high performance neural network pattern classifier (Kohonen) and to NN in experiments on hand-printed character recognition.

Learn++: an incremental learning algorithm for supervised neural networks

We introduce Learn++, an algorithm for incremental training of neural network (NN) pattern classifiers. The proposed algorithm enables supervised NN paradigms, such as the multilayer perceptron (MLP), to accommodate new data, including examples that correspond to previously unseen classes. Furthermore, the algorithm does not require access to previously used data during subsequent incremental learning sessions, yet at the same time, it does not forget previously acquired knowledge. Learn++ utilizes ensemble of classifiers by generating multiple hypotheses using training data sampled according to carefully tailored distributions. The outputs of the resulting classifiers are combined using a weighted majority voting procedure. We present simulation results on several benchmark datasets as well as a real-world classification task. Initial results indicate that the proposed algorithm works rather well in practice. A theoretical upper bound on the error of the classifiers constructed by Learn++ is also provided.

Gaussian Mixture Models and Probabilistic Decision-Based Neural Networks for Pattern Classification: A Comparative Study

Probabilistic Decision-Based Neural Networks (PDBNNs) can be considered as a special form of Gaussian Mixture Models (GMMs) with trainable decision thresholds. This paper provides detailed illustrations to compare the recognition accuracy and decision boundaries of PDBNNs with that of GMMs through two pattern recognition tasks, namely the noisy XOR problem and the classification of two-dimensional vowel data. The paper highlights the strengths of PDBNNs by demonstrating that their thresholding mechanism is very effective in detecting data not belonging to any known classes. The original PDBNNs use elliptical basis functions with diagonal covariance matrices, which may be inappropriate for modelling feature vectors with correlated components. This paper overcomes this limitation by using full covariance matrices, and showing that the matrices are effective in characterising non-spherical clusters.

Task decomposition and module combination based on class relations: a modular neural network for pattern classification

In this paper, we propose a new method for decomposing pattern classification problems based on the class relations among training data. By using this method, we can divide a K-class classification problem into a series of ((2)(K)) two-class problems. These two-class problems are to discriminate class Ci from class Cj for i=1, ..., K and j = i+1, while the existence of the training data belonging to the other K-2 classes is ignored. If the two-class problem of discriminating class Ci from class Cj is still hard to be learned, we can further break down it into a set of two-class subproblems as small as we expect. Since each of the two-class problems can be treated as a completely separate classification problem with the proposed learning framework, all of the two-class problems can be learned in parallel. We also propose two module combination principles which give practical guidelines in integrating individual trained network modules. After learning of each of the two-class problems with a network module, we can easily integrate all of the trained modules into a min-max modular (M3) network according to the module combination principles and obtain a solution to the original problem. Consequently, a large-scale and complex K-class classification problem can be solved effortlessly and efficiently by learning a series of smaller and simpler two-class problems in parallel.

Simulation of weight prunning process in backpropagation neural network for pattern classification: A self-running threshold approach

Neural network minimization is a process in which non-affecting elements (non-contributing processing elements and weights) are deleted. This results in the network with a minimum configuration and positively contributing elements. In general, the scope of the minimization process is to achieve a network which provides optimal solution for a given task without loss of performance. Therefore, study into network minimization becomes essential for complicated tasks. The present study investigates ways of developing automatic thresholding methods for weight deleting process to minimize the network resulting in enhanced performances for classification of six control chart patterns. To achieve this aim, two weight deleting processes are developed including dynamic and static weight deleting processes. The automatic weight deleting mechanism introduced in the present study is based on the optimal RMS error variation. In this case, once the RMS error drops to the desired level, weight deleting process starts, which then enables one to select the optimal threshold automatically, in this case, RMS error becomes minimum. The study is extended to include two sets of data structures, namely first and second data kinds. In the first data kind, random variables and coefficients in equations governing the chart patterns are varied while in the second data kind, only random variables are changed. These enable one to compare the generalization capacity of the resulting network which uses the first and second data kinds separately. It is found that dynamic deleting process results improved performances and the late start of deleting process is fruitful for improved learning rate. In addition, the use of first data kind provides the network with better generalization capacity.

The use of multilayer neural networks in material synthesis

This paper demonstrates the incorporation of a multilayer neural network in semiconductor thin film deposition processes. As a first step toward neural network-based process control, we present results from neural network pattern classification and beam analysis of reflection high energy electron diffraction RHEED images of GaAs/AlGaAs crystal surfaces during molecular beam epitaxy growth. For beam analysis, we used the neural network to detect and measure the intensity of the RHEED beam spots during the growth process and, through Fourier transformation, determined the thin film deposition rate. The neural network RHEED pattern classification and intensity analysis capability allows, powerful in situ real time monitoring of epitaxial thin film deposition processes. Our results show that a three layer network with sixteen hidden neurons and three output neurons had the highest correct classification rate with a success rate of 100% during testing and training on 13 examples.

Pathologic discrimination through acoustic analysis of voice

The acoustic analysis of voice from dysphonic patients has been studied, mainly for classification of voice quality in terms of roughness, breathiness, and hoarseness. By a data reduction of speech signal, researchers have produced several acoustic measures in order to relate them with a perceptual diagnosis. In this work, a proposal was made to modify acoustic parameters in order to support a higher sample quantity, instead of using only a data window by digitalized speech. Also, robust statistics is used to obtain realiable measures that estimate the abnormal condition of a voice signal as well as to discriminate several pathologics of the larynx and vocal cords, without any perceptual consideration. Using only seven measures from adaptative inverse filtering (Kalman and Wiener filters) and a neural network for pattern classification on 228 voice signals of speakers (20 groups of distinct types of pathologies and one of normal speakers) evaluated through videolaringoscopy, results of the pathologic discrimination showed 22% of uncertainty. Redrawing the patient sample in seven distinct groups (five with defined pathologies, one with all others pathologies, and one control), then the error in the pathologic discrimination is 18%. [Work supported by Capes.]

Mine detection using scattering parameters

The detection and disposal of antipersonnel land mines is one of the most difficult and intractable problems faced in ground conflict. This paper presents detection methods which use a separated-aperture microwave sensor and an artificial neural network pattern classifier. Several data-specific preprocessing methods are developed to enhance neural network learning. In addition, a generalized Karhunen-Loeve transform and the eigenspace separation transform are used to perform data reduction and reduce network complexity. Highly favorable results have been obtained using the above methods in conjunction with a feedforward neural network.

Classifying pilot-plant distillation column faults using neural networks

An approach to fault detection is described which uses neural-network pattern classifiers trained using data from a rigorous differential-equation-based simulation of a pilot-plant column. Two cases studies are presented, both considering only plant data. For two classes of process data, a neural network and a K-Means classifier both produced excellent diagnoses. Extending the study to include three additional classes of plant operation, a neural network again gave accurate classifications, while a K-Means classifier failed to correctly categorise the data. Principal components analysis is used to visualise data clusters. The robustness of the neural networks was found to be generally good.

Classification of faults in gearboxes ? pre-processing algorithms and neural networks

Classical signal processing techniques when combined with pattern classification analysis can provide an automated fault detection procedure for machinery diagnostics. Artificial neural networks have recently been established as a powerful method of pattern recognition. The neural networkbased fault detection approach usually requires preprocessing algorithms which enhance the fault features, reducing their number at the same time. Various timeinvariant and timevariant signal preprocessing algorithms are studied here. These include spectral analysis, time domain averaging, envelope detection, Wigner-Ville distributions and wavelet transforms. A neural network pattern classifier with preprocessing algorithms is applied to experimental data in the form of vibration records taken from a controlled tooth fault in a pair of meshing spur gears. The results show that faults can be detected and classified without errors.