Back Propagation Neural Network Classifier Research Articles

Assessment of the effectiveness of support vector machines for hyperspectral data

Support vector machines (SVMs) have recently been introduced into machine learning for pattern recognition. In this paper, a multi-class SVM is used for classification of DAIS hyperspectral remotely sensed data. Results show that the SVM performs better than maximum likelihood, univariate decision tree and backpropagation neural network classifiers, even with small training data sets, and is almost unaffected by the Hughes phenomenon [IEEE Trans. Inform. Theory IT-14 (1968) 55].

Minimizing the negative effect of the overlapping pixels on the classification accuracy of the error back-propagation neural network classifier using the ancillary and supplemental data

A procedure is presented for minimizing the problems associated with the overlapping land cover spectral signatures on the classification accuracy, which involves the use of ancillary and supplemental data in remote sensing classification. Multispectral remote sensing data from a section of Kananaskis Country in the Rocky Mountains in Canada were used as an application for this investigation. A technique was established to simplify the method of selecting the dataset of the spectral training sample that can be used to determine decision rules for classifying the application area and improving the sorting process between the overlapping pixels. A three-stage computer classifier model was developed based on the back-propagation artificial neural network method to enhance the sorting process among land cover types that have similar spectral signatures. The performance of this computer model was assessed by comparing the results with the results of three prepackaged computer classifiers using the same dataset: maximum likelihood, minimum distance to means, and parallelepiped. The investigation demonstrated that the developed computer model performed satisfactorily overall with the use of the proposed enhanced sorting procedure.

Detection of internal wheat seed infestation by Rhyzopertha dominica using X-ray imaging

Standardization for grain grades has been established in most countries to maintain the quality of a crop until it reaches consumers. Different methods have been investigated for their potential to detect insect infestations in grain destined for domestic and export markets. The potential of detecting infestations caused by Rhyzopertha dominica in wheat kernels using a real-time soft X-ray method was determined in this study. Artificially infested wheat kernels were incubated at 30°C and 70% relative humidity and X-rayed sequentially for larval, pupal, and adult stages of R. dominica. Algorithms were used to extract histogram features, textural features, and histogram and shape moments from the X-ray images of wheat kernels. A backpropagation neural network (BPNN) and statistical classifiers were used to identify uninfested and infested kernels using the 57 extracted features. The BPNN correctly identified all uninfested and infested kernels and more than 99% of kernels infested by R. dominica larvae. The classification accuracies determined by the BPNN were higher using all 57 features than when using the histogram and textural features separately. The BPNN performed better than the parametric and non-parametric classifiers in the identification of uninfested and infested kernels by different stages of R. dominica.

New fault diagnosis of circuit breakers

Wavelet packets and neural networks have been used to analyze the vibration data of circuit breakers (CBs) for the detection of incipient CB faults. Wavelet packets are used to convert measured vibration data from healthy and defective CBs into wavelet features. Selected features highlighting the differences between healthy and faulty condition are processed by a backpropagation neural network for classification. Testing has been done for three 66-kV CBs with simulated faults. Detection accuracy is shown to be far better than other classical techniques such as the windowed Fourier transform, stand alone artificial neural networks or expert system. The accuracy of detection for some faults can be as high as 100%.

An experimental study on spectral discrimination capability of a backpropagation neural network classifier

Data were generated for two classes in a simulated feature space, with the classes having a varying amount of spectral overlap. It was hypothesized that the backpropagation neural network would be able to distinguish the classes in the situation of no overlap. Our results confirmed that two non-overlapping classes can be discriminated with 100% overall accuracy by the backpropagation neural network. The backpropagation neural network classified the simulated data sets with a significantly higher accuracy than the maximum likelihood or the parallelepiped classifier. When the experiment was repeated using remotely sensed imagery with more complicated spectral overlap among classes (for a land cover classification at Lemeleberg in the eastern Netherlands), the neural network yielded again a significantly higher classification accuracy than the maximum likelihood classifier or the parallelepiped classifier. Differences between the map outputs imply that integrating the different classification algorithms may improve the overall mapping accuracy.

Automated assessment of the composition of breast tissue revealed on tissue-thickness-corrected mammography.

Variations in the thickness of a compressed breast and the resulting variations in mammographic densities confound current automated procedures for estimating tissue composition of breasts from digitized mammograms. We sought to determine whether adjusting mammographic data for tissue thickness before estimating tissue composition could improve the accuracy of the tissue estimates. We developed methods for locally estimating breast thickness from mammograms and then adjusting pixel values so that the values correlated with the tissue composition over the breast area. In our technique, the pixel values are corrected for the nonlinearity of the combined characteristic curve from the film and film digitizer; the approximate relative thickness as a function of distance from the skin line is measured; and the pixel values are adjusted to reflect their distance from the skin line. To estimate tissue composition, we created a backpropagation neural network classifier from features extracted from the histogram of pixel values, after the data had been adjusted for characteristic curve and tissue thickness. We used a 10-fold cross-validation method to evaluate the neural network. The averaged scores of three radiologists were our gold standard. The performance of the neural network was calculated as the percentage of correct classifications of images that were or were not corrected to reflect tissue thickness. With its parameters derived from the pixel-value histogram, the neural network based on corrected images performed better (71% accuracy) than that based on uncorrected images (67% accuracy) (p < 0.05). Our results show that adjusting tissue thickness before estimating tissue composition improved the performance of our estimation procedure in reproducing the tissue composition values determined by radiologists.

New Fault Diagnosis of Circuit Breakers

Wavelet packets and neural networks have been used to analyze the vibration data of circuit breakers for the detection of incipient circuit breaker faults. Wavelet packets are used to convert measured vibration data from healthy and defective circuit breakers into wavelet features. Selected features highlighting the differences between healthy and faulty condition are processed by a back-propagation neural network for classification. Testing has been done for three 66 kV circuit breakers with simulated faults. Detection accuracy is shown to be far better than other classical techniques such as the windowed Fourier transform, stand alone artificial neural networks or expert system. The accuracy of detection for some faults can be as high as 100%.

Neural and MTS Algorithms for Feature Selection

The relationships among multi‐dimensional data (such as medical examination data) with ambibuity and variation are difficult to explore. The traditional approach to building a data classification system requires the formulation of rules by which the input data can be analyzed. The formulation of such rules is very difficult with large sets of input data. This paper first describes two classification approaches using back‐propagation (BP) neural network and Mahalanobis distance (MD) classifier, and then proposes two classification approaches for multi‐dimensional feature selection. The first one proposed is a feature selection procedure from the trained back‐propagation (BP) neural network. The basic idea of this procedure is to compare the multiplication weights between input and hidden layer and hidden and output layer. In order to simplify the structure, only the multiplication weights of large absolute values are used. The second approach is Mahalanobis‐Taguchi system (MTS) originally suggested by Dr. Taguchi. The MTS performs Taguchi’s fractional factorial design based on the Mahalanobis distance as a performance metric. We combine the automatic thresholding with MD; it can deal with a reduced model, which is the focus of this paper. In this work, two case studies will be used as examples to compare and discuss the complete and reduced models employing BP neural network and MD classifier. The implementation results show that proposed approaches are effective and powerful for the classification.

On-line fault section estimation in power systems with radial basis function neural network

Fault section estimation is of great importance to the restoration of power systems. Many techniques have been used to solve this problem. In this paper, the application of radial basis function neural network (RBF NN) to fault section estimation is addressed. The orthogonal least square (OLS) algorithm has been extended to optimize the number of neurons in hidden layer and the connection weights of RBF NN. A classical back-propagation neural network (BP NN) has been developed to solve the same problem for comparison. Computer test is conducted on a four-bus test system and the test results show that the RBF NN is quite effective and superior to BP NN in fault section estimation.

SEPARATING SEPTICEMIC AND NORMAL CHICKEN LIVERS BY VISIBLE/NEAR–INFRARED SPECTROSCOPY AND BACK–PROPAGATION NEURAL NETWORKS

The visible/nearinfrared spectra of 300 chicken livers were analyzed to explore the feasibility of usingspectroscopy to separate septicemic livers from normal livers. Three strategies involving offset, second difference, andfunctional link methods were applied to preprocess the spectra, while principal component analysis (PCA) was utilized toreduce the input data dimensions. PCA scores were fed into a feedforward backpropagation neural network forclassification. The results showed no obvious difference in classification accuracy between offset and nonoffset data whenno other preprocessing method was applied. The full 4002498 nm wavelength region produced better results than the400700 nm, 4001098 nm, and 11022498 nm subregions when more than 30 PCA scores were used. In general, theclassification accuracy was improved by increasing the number of scores of input data, but too many scores diminishedperformance. The functional link test showed that using functionallink spectra selected at every third point with 60 scoresachieved the same classification accuracy as that obtained when using all the data points with 90 scores. The bestclassification model used offset correction followed by second difference (g = 31) and 60 scores. It achieved a classificationaccuracy of 98% for normal and 94% for septicemic livers.

Feature relevance in morphological galaxy classification

ABSTRA C T We investigate the utility of a variety of features in performing morphological galaxy classification using back-propagation neural network classifiers based on a sample of 805 galaxies classified by Naim et al. We derive a total of 22 features from each galaxy image and use these as inputs to a neural network trained using back-propagation. The morphological types are subdivided into two to seven groups, and the relevance of each of the features is examined for each grouping. We use the magnitude of the regularization parameter for each input to determine whether a feature can be eliminated. We then prune the input features of the network, typically down to four features. We examine a number of methods of assessing the performance of the network and determine which works best for our task.

Underwater target classification using wavelet packets and neural networks

In this paper, a new subband-based classification scheme is developed for classifying underwater mines and mine-like targets from the acoustic backscattered signals. The system consists of a feature extractor using wavelet packets in conjunction with linear predictive coding (LPC), a feature selection scheme, and a backpropagation neural-network classifier. The data set used for this study consists of the backscattered signals from six different objects: two mine-like targets and four nontargets for several aspect angles. Simulation results on ten different noisy realizations and for signal-to-noise ratio (SNR) of 12 dB are presented. The receiver operating characteristic (ROC) curve of the classifier generated based on these results demonstrated excellent classification performance of the system. The generalization ability of the trained network was demonstrated by computing the error and classification rate statistics on a large data set. A multiaspect fusion scheme was also adopted in order to further improve the classification performance.

Comparison of Algorithms for Classifying Swedish Landcover Using Landsat TM and ERS-1 SAR Data

Sixteen landcover classes in a representative Swedish environment were analyzed and classified using one Landsat TM scene and seven ERS-1 SAR.PRI images acquired during 1993. Spectral and backscattering signature separabilities are analyzed using the Jeffries–Matusita distance measure to determine which combinations of channels/images contained the most information. Maximum likelihood, sequential maximum a posteriori (SMAP, a Bayesian image segmentation algorithm), and back propagation neural network classification algorithms were applied and their performances evaluated. Results of the separability analyses indicated that the multitemporal SAR data contained more separable landcover information than did the multispectral TM data; the highest separabilities were achieved when the TM and SAR data were combined. Classification accuracy evaluation results indicate that the SMAP algorithm outperformed the maximum likelihood algorithm which, in turn, outperformed the neural network algorithm. The best KAPPA values, using combined data, were 0.495 for SMAP, 0.445 for maximum likelihood, and 0.432 for neural network. Corresponding overall accuracy values were 57.1%, 52.4%, and 51.2%, respectively. A comparison between lumped crop area statistics with areal sums calculated from the classified satellite data gave the highest correspondence where the SMAP algorithm was used, followed by the maximum likelihood and neural network algorithms. Based on our application, we can therefore confirm the value of a multisource optical/SAR approach for analyzing landcover and the improvements to classification achieved using the SMAP algorithm.

Identification of bacterial rep-PCR genomic fingerprints using a backpropagation neural network

A backpropagation neural network (BPN) was used to identify bacterial plant pathogens based on their genomic fingerprints. Genomic fingerprint data, comprised of complex DNA band patterns generated using BOX, enterobacterial repetitive intergenic consensus (ERIC) and repetitive extragenic palindromic (REP)-primers (rep-PCR), were used to train three independent BPNs. 10 Strains of the genus Xanthomonas, each with a characteristic host plant range, were identified correctly using the three trained BPNs. When tested with fingerprints of bacterial strains not present in the training sets, the rejection rate was 100%, using the three BPN classifiers combined. Thus, BPN protocols can be employed to generate a powerful computer-based system for the identification of pathogenic bacteria in the genus Xanthomonas.

Identification of bacterial rep-PCR genomic fingerprints using a backpropagation neural network

A backpropagation neural network (BPN) was used to identify bacterial plant pathogens based on their genomic fingerprints. Genomic fingerprint data, comprised of complex DNA band patterns generated using BOX, enterobacterial repetitive intergenic consensus (ERIC) and repetitive extragenic palindromic (REP)-primers (rep-PCR), were used to train three independent BPNs. 10 Strains of the genus Xanthomonas, each with a characteristic host plant range, were identified correctly using the three trained BPNs. When tested with fingerprints of bacterial strains not present in the training sets, the rejection rate was 100%, using the three BPN classifiers combined. Thus, BPN protocols can be employed to generate a powerful computer-based system for the identification of pathogenic bacteria in the genus Xanthomonas.

Automatic classification of block-shaped parts based on their 2D projections

This paper presents a classification scheme for 3D block-shaped parts. A part is block-shaped if the contours of its orthographic projections are all rectangles. A block-shaped part is classified based on its partitioned view-contours, which are the result of partitioning the contours of its orthographic projections by visible or invisible projected line segments. The regions and their adjacency in a partitioned view-contour are first converted to a graph, then to a reference tree, and finally to a vector form, with which a back-propagation neural network classifier can be trained and applied. The proposed back-propagation neural network classifier is in a cascaded structure and has advantages that each network can be limited to a small size and trained independently. Based on the classification results on their partitioned view-contours, parts are grouped into families that can be in one of the three levels of similarity. Extensive empirical tests have been performed; the pros and cons of the approach are also investigated.

Performance of a Backpropagation Neural Network in Diagnostic Rhyme Test Word Recognition

This paper investigates isolated speaker- dependent word recognition of Diagnostic Rhyme Test words using a backpropagation neural network classifier. The performance of K-nearest neighbors and closest-class-mean classifiers are compared for several signal- to-noise ratios. The test and training data consisted of 40 frames of weighted eighth- order cepstral coefficients extracted from each word utterance. The neural network classifier correctly classified more than 92% of 2,400 testing examples not contained in the training data for the noise-free case. This performance was better than that of the K-nearest neighbor classifier, which was greater than 83%, and that of the closest class mean classifier, which was greater than 85%.

Automated segmentation and classification of multispectral magnetic resonance images of brain using artificial neural networks.

We present a fully automated process for segmentation and classification of multispectral magnetic resonance (MR) images. This hybrid neural network method uses a Kohonen self-organizing neural network for segmentation and a multilayer backpropagation neural network for classification. To separate different tissue types, this process uses the standard T1-, T2-, and PD-weighted MR images acquired in clinical examinations. Volumetric measurements of brain structures, relative to intracranial volume, were calculated for an index transverse section in 14 normal subjects (median age 25 years; seven male, seven female). This index slice was at the level of the basal ganglia, included both genu and splenium of the corpus callosum, and generally, showed the putamen and lateral ventricle. An intraclass correlation of this automated segmentation and classification of tissues with the accepted standard of radiologist identification for the index slice in the 14 volunteers demonstrated coefficients (ri) of 0.91, 0.95, and 0.98 for white matter, gray matter, and ventricular cerebrospinal fluid (CSF), respectively. An analysis of variance for estimates of brain parenchyma volumes in five volunteers imaged five times each demonstrated high intrasubject reproducibility with a significance of at least p < 0.05 for white matter, gray matter, and white/gray partial volumes. The population variation, across 14 volunteers, demonstrated little deviation from the averages for gray and white matter, while partial volume classes exhibited a slightly higher degree of variability. This fully automated technique produces reliable and reproducible MR image segmentation and classification while eliminating intra- and interobserver variability.

Image feature selection by a genetic algorithm: application to classification of mass and normal breast tissue.

We investigated a new approach to feature selection, and demonstrated its application in the task of differentiating regions of interest (ROIs) on mammograms as either mass or normal tissue. The classifier included a genetic algorithm (GA) for image feature selection, and a linear discriminant classifier or a backpropagation neural network (BPN) for formulation of the classifier outputs. The GA-based feature selection was guided by higher probabilities of survival for fitter combinations of features, where the fitness measure was the area Az under the receiver operating characteristic (ROC) curve. We studied the effect of different GA parameters on classification accuracy, and compared the results to those obtained with stepwise feature selection. The data set used in this study consisted of 168 ROIs containing biopsy-proven masses and 504 ROIs containing normal tissue. From each ROI, a total of 587 features were extracted, of which 572 were texture features and 15 were morphological features. The GA was trained and tested with several different partitionings of the ROIs into training and testing sets. With the best combination of the GA parameters, the average test Az value using a linear discriminant classifier reached 0.90, as compared to 0.89 for stepwise feature selection. Test Az values with a BPN classifier and a more limited feature pool were 0.90 with GA-based feature selection, and 0.89 for stepwise feature selection. The use of a GA in tailoring classifiers with specific design characteristics was also discussed. This study indicates that a GA can provide versatility in the design of linear or nonlinear classifiers without a trade-off in the effectiveness of the selected features.

Classification using splines

In a J class classification problem with data of the form: ( y n , x n ), n = 1,…, N, where y n ϵ{1,…, J} and x n = ( x 1 n ,…, x Mn ), linear discriminant analysis produces estimated class boundaries which are linear in x 1,…, x M . In this paper, a method is developed which estimates the conditional class probabilities in a function space which is bigger than the linear function space. The decision rule based on those estimated conditional class probabilities can have very nonlinear class boundaries. The method projects the conditional class probabilities onto a space spanned by cubic splines, and, hence, is called classification using splines (CUS). This new method seems to achieve comparable and in some cases lower misclassification error rates than existing methods like CART or the back-propagation neural network classifier.