Committee Of Networks Research Articles

Multiple Neural Network Classification Scheme for Detection of Colonic Polyps in CT Colonography Data Sets

Rationale and Objectives A new classification system for colonic polyp detection, designed to increase sensitivity and reduce the number of false-positive findings with computed tomographic colonography, was developed and tested in this study. Materials and Methods The system involves classification by a committee of neural networks (NNs), each using largely distinct subsets of features selected from a general set. Back-propagation NNs trained with the Levenberg-Marquardt algorithm were used as primary classifiers (committee members). The set of features included region density, Gaussian and mean curvature and sphericity, lesion size, colon wall thickness, and the means and standard deviations of all of these values. Subsets of variables were initially selected because of their effectiveness according to training and test sample misclassification rates. The final decision for each case is based on the majority vote across the networks and reflects the weighted votes of all networks. The authors also introduce a smoothed cross-validation method designed to improve estimation of the true misclassification rates by reducing bias and variance. Results This committee method reduced the false-positive rate by 36%, a clinically meaningful reduction, and improved sensitivity by an average of 6.9% compared with decisions made by any single NN. The overall sensitivity and specificity were 82.9% and 95.3%, respectively, when sensitivity was estimated by means of smoothed cross-validation. Conclusion The proposed method of using multiple classifiers and majority voting is recommended for classification tasks with large sets of input features, particularly when selected feature subsets may not be equally effective and do not provide satisfactory true- and false-positive rates. This approach reduces variance in estimates of misclassification rates.

Scaled conjugate gradient and Bayesian training of neural networks for fault identification in cylinders

This paper compares Bayesian training of neural networks using hybrid Monte Carlo to scaled conjugate gradient method for fault identification in cylinders using vibration data. From the measured data pseudo-modal energies and modal properties are calculated and the coordinate pseudo-modal energy assurance criterion (COMEAC) and the coordinate modal assurance criterion (COMAC) are computed respectively. The pseudo-modal energies, modal properties, COMEAC and COMAC are used to train four neural networks. On average, the pseudo-modal-energy-networks are more accurate than the modal-property-networks. The weighted averages of the pseudo-modal-energy- and modal-property-networks form a committee of networks. The committee method gives lower mean squared errors and better classification of faults than the individual methods. The Bayesian training is found to be more accurate and computationally expensive than the scaled conjugate gradient method and to give confidence levels.

Probabilistic Fault Identification Using a Committee of Neural Networks and Vibration Data

Bayesian-formulated neural network architecture is implemented using a hybrid Monte Carlo method for probabilistic fault identification in a population of ten nominally identical cylindrical shells using vibration data. Each cylinder is divided into three substructures. Holes of 12 mm in diameter are introduced in each of the substructures. Vibration data are measured by impacting the cylinders at selected positions using a modal hammer and measuring the acceleration responses at a fixed position. Modal energies, defined as the integrals of the real and imaginary components of the frequency response function over 12-Hz frequency bandwidths, are extracted and transformed into the coordinate modal energy assurance criterion. This criterion and the identity of faults are used to train the frequency response function (FRF) neural network. Modal analysis is then employed to identify modal properties. Mode shapes are transformed into the coordinate modal assurance criterion. The natural frequencies and the coordinate modal assurance criterion, as well as the identities of faults, are utilized to train the modal-property neural network. The weighted average of the modal-property network and the FRF network form a committee of two networks. The committee approach is observed to give lower mean square errors and standard deviations (thus, a higher probability of giving the correct solution) than the individual methods. This approach gives accurate identities of damage and their respective confidence intervals while requiring affordable computational resources.

Smoothed Bagging with Kernel Bandwidth Selectors

Recently, a combined approach of bagging (bootstrap aggregating) and noise addition was proposed and shown to result in a significantly improved generalization performance. But, the level of noise introduced, a crucial factor, was determined by trial and error. The procedure is not only ad hoc but also time consuming since bagging involves training a committee of networks. Here we propose a principled procedure of computing the level of noise, which is also computationally less expensive. The idea comes from kernel density estimation (KDE), a non-parametric probability density estimation method where appropriate kernel functions such as Gaussian are imposed on data. The kernel bandwidth selector is a numerical method for finding the width of a kernel function (called bandwidth). The computed bandwidth can be used as the variance of added noise. The proposed approach makes the trial and error procedure unnecessary, and thus provides a much faster way of finding an appropriate level of noise. In addition, experimental results show that the proposed approach results in an improved performance over bagging, particularly for noisy data.

Hybrid fuzzy logic committee neural networks for recognition of swallow acceleration signals

Biological signals are complex and often require intelligent systems for recognition of characteristic signals. In order to improve the reliability of the recognition or automated diagnostic systems, hybrid fuzzy logic committee neural networks were developed and the system was used for recognition of swallow acceleration signals from artifacts. Two sets of fuzzy logic-committee networks (FCN) each consisting of seven member networks were developed, trained and evaluated. The FCN-I was used to recognize dysphagic swallow from artifacts, and the second committee FCN-II was used to recognize normal swallow from artifacts. Several networks were trained and the best seven were recruited into each committee. Acceleration signals from the throat were bandpass filtered, and several parameters were extracted and fed to the fuzzy logic block of either FCN-I or FCN-II. The fuzzified membership values were fed to the committee of neural networks which provided the signal classification. A majority opinion of the member networks was used to arrive at the final decision. Evaluation results revealed that FCN correctly identified 16 out of 16 artifacts and 31 out of 33 dysphagic swallows. In two cases, the decision was ambiguous due to the lack of a majority opinion. FCN-II correctly identified 24 out of 24 normal swallows, and 28 out of 29 artifacts. In one case, the decision was ambiguous due to the lack of a majority opinion. The present hybrid intelligent system consisting of fuzzy logic and committee networks provides a reliable tool for recognition and classification of acceleration signals due to swallowing.

MEASURING AND IMPROVING NEURAL NETWORK GENERALIZATION FOR MODEL UPDATING

This paper compares various techniques of measuring the generalization ability of a neural network used for model-updating purposes. An appropriate metric for measuring generalization ability is suggested, and it is used to investigate and compare various neural network architectures and training algorithms. The effect of noise on generalization ability is considered, and it is shown that the form of the noise does not appear important to the networks. This implies that the optimum training location may be obtained by considering a simple noise model such as Gaussian noise. Various radial basis function neurons and training algorithms are considered. Significant improvements to generalization ability are noted by merging the holdout and training data sets before training the second layer of the network, after the network architecture has been decided. The Gaussian radial basis function is rejected as the radial basis function of choice, due to uncertainty regarding an appropriate value for the spread constant. It is noted that several alternative radial basis functions without spread constants, such as the thin-plate spline, give excellent results. Finally, the use of jitter and committees to improve the generalization ability of networks is considered. It is found that jitter makes neither improvement nor degrades the results. It is also found that a committee of networks performs better than any single network. A good method of generating committee members is to split the available data evenly into multiple random holdout and training data sets.

Damage Identification Using Committee of Neural Networks

This paper presents a committee of neural networks technique, which employs frequency response functions (FRFs), modal properties (natural frequencies and mode shapes), and wavelet transform (WT) data simultaneously to identify damage in structures. The experimental demonstration of the method is obtained by studying the sensitivities of the FRFs, modal properties, and WT data to four types of faults in a cylindrical shell. The experimental results show that different faults affect data in a different manner. The proposed approach is tested on simulated data from a three-degree-of-freedom mass-spring-damper system. The results from the simulated study show that the performance of the approach is not influenced by the noise in the data. Finally, the method is used to identify damage in a population of ten steel seam-welded cylindrical shells. The proposed method is able to identify damage cases better than the three approaches used individually. The committee approach gives results that generally have a lower mean square error (MSE) than the average MSE of the individual methods. It is found that the effectiveness of the method is enhanced when experimentally measured data are used, which is in contrast to many existing methods. This is because the committee approach assumes that the errors given by the three approaches are uncorrelated, a situation that becomes more apparent when using measured data rather than simulated data.

2000-2010: The Bone and Joint Decade

On January 13, 2000, The Bone and Joint Decade was formally launched at the headquarters of the World Health Organization in Geneva, Switzerland. This comes on the heels of the November 30, 1999, endorsement of the initiative by the United Nations. The Bone and Joint Decade is a global campaign to improve the quality of life for people who have musculoskeletal conditions and to advance the understanding and treatment of these conditions through research, prevention, and education. The aims of the campaign are to raise awareness of the increasing societal impact of musculoskeletal injuries and disorders, to empower patients to participate in decisions about their care, to increase funding for prevention activities and research, and to promote cost-effective prevention and treatment. The Bone and Joint Decade has thus far been endorsed by fifteen nations, including the United Kingdom, Austria, Sweden, Brazil, Tanzania, and The Netherlands; by three states in the United States; and by more than 650 organizations worldwide. More than forty multidisciplinary National Action Network committees have been formed around the world to coordinate the activities of National Action Network member organizations. In his endorsement statement, United Nations Secretary-General Kofi Annan noted that The Bone and Joint Decade initiative appeals to governments worldwide to start taking action now and to work together with national and international organizations and health-care professionals in the prevention and treatment of musculoskeletal disorders2. Health-care services worldwide will face severe financial pressures in the next ten to twenty years because of an escalation in the numbers of people affected by musculoskeletal diseases. Bone and joint disorders account for more than one-half of all chronic conditions in people older than fifty years of age in developed countries and are the most common cause of severe, long-term pain and disability3. In developing nations, …

Bayesian neural networks for classification: how useful is the evidence framework?

This paper presents an empirical assessment of the Bayesian evidence framework for neural networks using four synthetic and four real-world classification problems. We focus on three issues; model selection, automatic relevance determination (ARD) and the use of committees. Model selection using the evidence criterion is only tenable if the number of training examples exceeds the number of network weights by a factor of five or ten. With this number of available examples, however, cross-validation is a viable alternative. The ARD feature selection scheme is only useful in networks with many hidden units and for data sets containing many irrelevant variables. ARD is also useful as a hard feature selection method. Results on applying the evidence framework to the real-world data sets showed that committees of Bayesian networks achieved classification accuracies similar to the best alternative methods. Importantly, this was achievable with a minimum of human intervention.

FAULT IDENTIFICATION USING FINITE ELEMENT MODELS AND NEURAL NETWORKS

When vibration data are used to identify faults in structures it is not completely clear whether to use either frequency response functions or modal parameters. This paper presents a committee of neural networks technique, which employs both frequency response functions and modal data simultaneously to identify faults in structures. The new approach is tested on simulated data from a cantilevered beam, which is substructured into five regions. It is observed that irrespective of the noise levels in the data, the committee of neural networks gives results that have lower mean-squares errors and standard deviations than the two existing methods. It is found that the new method is able to identify fault cases better than the two approaches used individually. It is established that for the problem analysed, giving equal weights to the frequency-response-based method and modal-properties-based method minimise the errors on identifying faults.

Colour classification by neural networks in graphic arts

This paper presents a hierarchical modular neural network for colour classification in graphic arts, capable of distinguishing among very similar colour classes. The network performs analysis in a rough to fine fashion, and is able to achieve a high average classification speed and a low classification error. In the rough stage of the analysis, clusters of highly overlapping colour classes are detected. Discrimination between such colour classes is performed in the next stage by using additional colour information from the surroundings of the pixel being classified. Committees of networks make decisions in the next stage. Outputs of members of the committees are adaptively fused through the BADD defuzzification strategy or the discrete Choquet fuzzy integral. The structure of the network is automatically established during the training process. Experimental investigations show the capability of the network to distinguish among very similar colour classes that can occur in multicoloured printed pictures. The classification accuracy obtained is sufficient for the network to be used for inspecting the quality of multicoloured prints.

Fast committee learning: Preliminary results

Fast committee learning can, to some extent, achieve the generalisation advantages of a committee of neural networks, without the need for independent learning of the committee members. This is achieved by selecting committee members from time-slices of the learning trajectory of one neural network.

Boosting and Other Ensemble Methods

We compare the performance of three types of neural network-based ensemble techniques to that of a single neural network. The ensemble algorithms are two versions of boosting and committees of neural networks trained independently. For each of the four algorithms, we experimentally determine the test and training error curves in an optical character recognition (OCR) problem as both a function of training set size and computational cost using three architectures. We show that a single machine is best for small training set size while for large training set size some version of boosting is best. However, for a given computational cost, boosting is always best. Furthermore, we show a surprising result for the original boosting algorithm: namely, that as the training set size increases, the training error decreases until it asymptotes to the test error rate. This has potential implications in the search for better training algorithms.