Committee Machine Research Articles

A cost-sensitive technique for positive-example learning supporting content-based product recommendations in B-to-C e-commerce

Existing supervised learning techniques are able to support product recommendations in business-to-consumer e-commerce but become ineffective in scenarios characterized by single-class learning, such as a training sample that consists of some examples pertaining to only one outcome class (positive or negative). To address such challenges, we develop a COst-sensitive Learning-based Positive Example Learning (COLPEL) technique, which constructs an automated classifier from a training sample comprised of positive examples and a much larger number of unlabeled examples. The proposed technique incorporates cost-proportionate rejection sampling to derive, from unlabeled examples, a subset that is likely to feature negative examples in the training sample. Our technique follows a committee machine approach and thereby constructs a set of classifiers that make joint product recommendations while mitigating the potential biases common to the use of a single classifier. We evaluate the proposed method with customers' book ratings collected from Amazon.com and include two prevalent techniques for benchmark purposes; namely, positive naïve Bayes and positive example-based learning. According to our results, the proposed COLPEL technique outperforms both benchmarks, as measured by accuracy and positive and negative F1 scores.

A committee machine approach to multiple response optimization

Multiple responses optimization problems have three phases including design of experiments, modeling and optimization. Artificial neural networks and genetic algorithm are applied for second and third phases. Committee machines include some experts such as some neural networks which operate together to get response. Current article applies a committee machine including four different artificial neural networks to model multiple responses optimization problems. Genetic algorithm is applied to calculate weights of committee machine and also it optimizes desirability function of all responses to get optimum point. Seven different cases in multiple responses optimization were modeled and analyzed. The results show the error of committee machine is near half of average error of artificial neural networks and global desirability of committee machine is the same as average global desirability of artificial neural networks. Key words: Committee machines, multiple responses optimization, genetic algorithm, neural networks.

Direct Parallel Perceptrons (DPPs): Fast Analytical Calculation of the Parallel Perceptrons Weights With Margin Control for Classification Tasks

Parallel perceptrons (PPs) are very simple and efficient committee machines (a single layer of perceptrons with threshold activation functions and binary outputs, and a majority voting decision scheme), which nevertheless behave as universal approximators. The parallel delta (P-Delta) rule is an effective training algorithm, which, following the ideas of statistical learning theory used by the support vector machine (SVM), raises its generalization ability by maximizing the difference between the perceptron activations for the training patterns and the activation threshold (which corresponds to the separating hyperplane). In this paper, we propose an analytical closed-form expression to calculate the PPs' weights for classification tasks. Our method, called Direct Parallel Perceptrons (DPPs), directly calculates (without iterations) the weights using the training patterns and their desired outputs, without any search or numeric function optimization. The calculated weights globally minimize an error function which simultaneously takes into account the training error and the classification margin. Given its analytical and noniterative nature, DPPs are computationally much more efficient than other related approaches (P-Delta and SVM), and its computational complexity is linear in the input dimensionality. Therefore, DPPs are very appealing, in terms of time complexity and memory consumption, and are very easy to use for high-dimensional classification tasks. On real benchmark datasets with two and multiple classes, DPPs are competitive with SVM and other approaches but they also allow online learning and, as opposed to most of them, have no tunable parameters.

Prediction of Compressional, Shear, and Stoneley Wave Velocities from Conventional Well Log Data Using a Committee Machine with Intelligent Systems

Measurement of compressional, shear, and Stoneley wave velocities, carried out by dipole sonic imager (DSI) logs, provides invaluable data in geophysical interpretation, geomechanical studies and hydrocarbon reservoir characterization. The presented study proposes an improved methodology for making a quantitative formulation between conventional well logs and sonic wave velocities. First, sonic wave velocities were predicted from conventional well logs using artificial neural network, fuzzy logic, and neuro-fuzzy algorithms. Subsequently, a committee machine with intelligent systems was constructed by virtue of hybrid genetic algorithm-pattern search technique while outputs of artificial neural network, fuzzy logic and neuro-fuzzy models were used as inputs of the committee machine. It is capable of improving the accuracy of final prediction through integrating the outputs of aforementioned intelligent systems. The hybrid genetic algorithm-pattern search tool, embodied in the structure of committee machine, assigns a weight factor to each individual intelligent system, indicating its involvement in overall prediction of DSI parameters. This methodology was implemented in Asmari formation, which is the major carbonate reservoir rock of Iranian oil field. A group of 1,640 data points was used to construct the intelligent model, and a group of 800 data points was employed to assess the reliability of the proposed model. The results showed that the committee machine with intelligent systems performed more effectively compared with individual intelligent systems performing alone.

Hybrid models in agent-based environmental decision support

Abstract Providing informational support in decision making is one of the priority directions of research in the sphere of public health management. Modern approaches suggest wide usage of intelligent data mining methods and Web-services, but just a few enable to study a complex system from an interdisciplinary point of view. In this paper an agent-based decision support system (ADSS), which embodies the principles of the interdisciplinary approach and facilitates multi-focal view and examination of the “Environment–Public health” system, is introduced. The detailed design of the system with the emphasis on the roles, scenarios and its implementation is presented. The data mining procedures used for data preparation, modeling and simulation, which include statistics, methods of artificial intelligence and hybrid models in form of cascade committee machines, are described. Then, the advantages of the proposed hybrid models against “singular” modeling methods are demonstrated. Finally, a case study for a selected region is presented and its results are discussed.

The Application of Committee Machine Model in Power Load Forecasting for the Western Region of Saudi Arabia

The Application of Committee Machine Model in Power Load Forecasting for the Western Region of Saudi Arabia

Using a Committee Machine With Artificial Neural Networks To Predict PVT Properties of Iran Crude Oil

Summary Reservoir-fluid properties are very important in material-balance calculations, well testing, and reserves estimates. Ideally, those data should be obtained experimentally. Sometimes, the results obtained from experimental tests are not reliable or accessible. In this study, we predict the pressure/volume/temperature (PVT) properties by a new artificial-neural-network (ANN) model using component mole percent, solution gas/oil ratio (GOR) (Rs), bubblepoint pressure (Pb), reservoir pressure, API oil gravity, and temperature as input data. The employed ANN model is from the committee machine type. The designed model processes its inputs using two parallel multilayer perceptron (MLP) networks, and then recombines their results. The results obtained show that the committee-machine model is a dependable network for prediction of PVT properties in reservoirs among the other ANNs and empirical correlations.

Computational capabilities of multilayer committee machines

We obtained an analytical expression for the computational complexity of many layered committee machines with a finite number of hidden layers (L < ∞) using the generalization complexity measure introduced by Franco et al (2006) IEEE Trans. Neural Netw. 17 578. Although our result is valid in the large-size limit and for an overlap synaptic matrix that is ultrametric, it provides a useful tool for inferring the appropriate architecture a network must have to reproduce an arbitrary realizable Boolean function.

A new approach to improve neural networks' algorithm in permeability prediction of petroleum reservoirs using supervised committee machine neural network (SCMNN)

Reservoir permeability is a critical parameter for the evaluation of hydrocarbon reservoirs. There are a lot of well log data related with this parameter. In this study, permeability is predicted using them and a supervised committee machine neural network (SCMNN) which is combined of 30 estimators. Statistically speaking, the results of simple neural network are not proper, for permeability data have a large range and are multi-populations. Therefore, all of data were divided in two low and high permeability populations using statistical study. Each estimator of SCMNN was combined of two simple networks to predict permeability in both low and high classes and one gating network, considered as a classifier, classified data to these two classes. Thus, each low and/or high input data would predict in related network. This SCMNN was used to predict permeability on the data of one of petroleum reservoirs of south-west of Iran. 220 samples of this reservoir were available that 80% of them were used as training data and 20% of them were used as validation and testing data. The overall fitting between predicted permeability versus measured ones was qualified through R 2 (R = correlation coefficient) to be 97.72% which is considered appropriate. Whereas, R 2 in the simple network in the best stat was 84.14%. The high power and efficiency of SCMNN are indicated by lower bias and better R 2 in results.

A committee machine with two‐layer expert nets

PurposeThe purpose of this paper is to present a committee machine (CM) with two‐layer expert nets to overcome the lack of approximating ability of CM with single‐layer expert nets.Design/methodology/approachA frequently used structure of CM, with a fuzzy c‐means clustering algorithm as splitting and combining unit and some single‐layer linear neural nets as expert modules, was applied to short‐term climate prediction. Considering the complexity of the climate conditions, use was made of two‐layer back propagation (BP) neural nets instead of single‐layer linear nets to test the effect of the model. Experiments were performed on both synthetic and realistic climatic data.FindingsPrediction accuracy is raised when the BP nets were used and as the number of hidden neurons increased at some stages. It implies that improving the approximating ability of individual expert module of a CM is beneficial.Research limitations/implicationsThe optimal learning rate, the optimal cluster numbers and the maximal number of iteration were not well treated.Practical implicationsThe paper is a useful alternative worth consideration for the complicated prediction problems.Originality/valueA CM with two‐layer expert nets are presented. Comparisons are made between CMs with simple and complex expert nets.

Estimation of NMR log parameters from conventional well log data using a committee machine with intelligent systems: A case study from the Iranian part of the South Pars gas field, Persian Gulf Basin

Nuclear Magnetic Resonance (NMR) log provides useful information for petrophysical study of the hydrocarbon bearing intervals. Free fluid porosity (effective porosity), rock permeability and bound fluid volume (BFV) could be obtained by processing and interpretation of NMR data. The present study proposes an improved strategy to make a quantitative correlation between the NMR log parameters and conventional well logs by integration of different intelligent systems using the concept of committee machine. The proposed committee machine with intelligent systems (CMIS) combines the results of Fuzzy Logic (FL), Neuro-Fuzzy (NF) and Neural Network (NN) algorithms for overall estimation of the NMR log parameters from conventional well log data. It assigns a weight factor to each of the individual intelligent algorithms showing its contribution in overall prediction. The weight factors are derived in two ways: simple averaging and weighted averaging. In the weighted averaging method a genetic algorithm (GA) was employed to obtain the optimal contribution of each algorithm in construction of the CMIS. The proposed methodology was applied to the South Pars gas field, Persian Gulf Basin. The petrophysical logs from two wells were used for constructing the intelligent models and a third well from the field was used to evaluate the reliability of the developed models. The results indicate the higher performance of the GA optimized model over the individual intelligent systems performing alone.

Statistical mechanics of an error correcting code using monotonic and nonmonotonic treelike multilayer perceptrons

An error correcting code using a treelike multilayer perceptron is proposed. An original message s0 is encoded into a codeword y0 using a treelike committee machine (committee tree) or a treelike parity machine (parity tree). Based on these architectures, several schemes featuring monotonic or nonmonotonic units are introduced. The codeword y0 is then transmitted via a binary asymmetric channel where it is corrupted by noise. The analytical performance of these schemes is investigated using the replica method of statistical mechanics. Under some specific conditions, some of the proposed schemes are shown to saturate the Shannon bound at the infinite codeword length limit. The influence of the monotonicity of the units on the performance is also discussed.

Learning with incomplete information in the committee machine

We study the problem of learning with incomplete information in a student-teacher setup for the committee machine. The learning algorithm combines unsupervised Hebbian learning of a series of associations with a delayed reinforcement step, in which the set of previously learnt associations is partly and indiscriminately unlearnt, to an extent that depends on the success rate of the student on these previously learnt associations. The relevant learning parameter lambda represents the strength of Hebbian learning. A coarse-grained analysis of the system yields a set of differential equations for overlaps of student and teacher weight vectors, whose solutions provide a complete description of the learning behavior. It reveals complicated dynamics showing that perfect generalization can be obtained if the learning parameter exceeds a threshold lambda ( c ), and if the initial value of the overlap between student and teacher weights is non-zero. In case of convergence, the generalization error exhibits a power law decay as a function of the number of examples used in training, with an exponent that depends on the parameter lambda. An investigation of the system flow in a subspace with broken permutation symmetry between hidden units reveals a bifurcation point lambda* above which perfect generalization does not depend on initial conditions. Finally, we demonstrate that cases of a complexity mismatch between student and teacher are optimally resolved in the sense that an over-complex student can emulate a less complex teacher rule, while an under-complex student reaches a state which realizes the minimal generalization error compatible with the complexity mismatch.

Binary Tissue Classification on Wound Images With Neural Networks and Bayesian Classifiers

A pressure ulcer is a clinical pathology of localized damage to the skin and underlying tissue caused by pressure, shear, or friction. Diagnosis, treatment, and care of pressure ulcers are costly for health services. Accurate wound evaluation is a critical task for optimizing the efficacy of treatment and care. Clinicians usually evaluate each pressure ulcer by visual inspection of the damaged tissues, which is an imprecise manner of assessing the wound state. Current computer vision approaches do not offer a global solution to this particular problem. In this paper, a hybrid approach based on neural networks and Bayesian classifiers is used in the design of a computational system for automatic tissue identification in wound images. A mean shift procedure and a region-growing strategy are implemented for effective region segmentation. Color and texture features are extracted from these segmented regions. A set of k multilayer perceptrons is trained with inputs consisting of color and texture patterns, and outputs consisting of categorical tissue classes which are determined by clinical experts. This training procedure is driven by a k-fold cross-validation method. Finally, a Bayesian committee machine is formed by training a Bayesian classifier to combine the classifications of the k neural networks. Specific heuristics based on the wound topology are designed to significantly improve the results of the classification. We obtain high efficiency rates from a binary cascade approach for tissue identification. Results are compared with other similar machine-learning approaches, including multiclass Bayesian committee machine classifiers and support vector machines. The different techniques analyzed in this paper show high global classification accuracy rates. Our binary cascade approach gives high global performance rates (average sensitivity =78.7% , specificity =94.7% , and accuracy =91.5% ) and shows the highest average sensitivity score ( =86.3%) when detecting necrotic tissue in the wound.

Combining MLPs and RBFNNs to Detect Signals With Unknown Parameters

The detection of Gaussian signals with an unknown correlation coefficient rhos is considered. Solutions based on neural networks (NNs) are studied, and a strategy for designing committee machines in a composite hypothesis test is proposed. A single multilayer perceptron (MLP) has been trained with rhos uniformly varying in [0, 1]. Considering the decision boundaries for rhos = 0 and rhos = 1 and how an MLP approximates them, a detection scheme composed of two MLPs has been proposed. One of them MLP1 has been trained with rhos uniformly varying in [0, 0.5], and the other one MLP2 has been trained with rhos uniformly varying in [0.5, 1]. For making a decision, the higher output is compared to a threshold for each false-alarm probability (P FA). This strategy simplifies the task of finding a compromise solution between the computational cost and the approximation error and outperforms the single-MLP detector. When MLP1 is substituted with a radial basis function NN (RBFNN), a new combination strategy of the outputs is required. We propose separately thresholding the outputs and applying them to an or logic function. The performance of this detector is slightly better than the two-MLP one, and the computational cost is significantly reduced.

A hybrid learning approach to tissue recognition in wound images

PurposePressure ulcer is a clinical pathology of localized damage to the skin and underlying tissue caused by pressure, shear, and friction. Diagnosis, treatment and care of pressure ulcers involve high costs for sanitary systems. Accurate wound evaluation is a critical task to optimize the efficacy of treatments and health‐care. Clinicians evaluate the pressure ulcers by visual inspection of the damaged tissues, which is an imprecise manner of assessing the wound state. Current computer vision approaches do not offer a global solution to this particular problem. The purpose of this paper is to use a hybrid learning approach based on neural and Bayesian networks to design a computational system to automatic tissue identification in wound images.Design/methodology/approachA mean shift procedure and a region‐growing strategy are implemented for effective region segmentation. Color and texture features are extracted from these segmented regions. A set of k multi‐layer perceptrons is trained with inputs consisting of color and texture patterns, and outputs consisting of categorical tissue classes determined by clinical experts. This training procedure is driven by a k‐fold cross‐validation method. Finally, a Bayesian committee machine is formed by training a Bayesian network to combine the classifications of the k neural networks (NNs).FindingsThe authors outcomes show high efficiency rates from a two‐stage cascade approach to tissue identification. Giving a non‐homogeneous distribution of pattern classes, this hybrid approach has shown an additional advantage of increasing the classification efficiency when classifying patterns with relative low frequencies.Practical implicationsThe methodology and results presented in this paper could have important implications to the field of clinical pressure ulcer evaluation and diagnosis.Originality/valueThe novelty associated with this work is the use of a hybrid approach consisting of NNs and Bayesian classifiers which are combined to increase the performance of a pattern recognition task applied to the real clinical problem of tissue detection under non‐controlled illumination conditions.

Committee machines for facial-gender recognition

This paper presents a new approach for building a committee machine (LVQCM) that is based on learning vector quantization (LVQ) neural networks. The proposed committee machine was then applied to solve the problem of facial gender recognition. Design

A two‐stage comittee machine of neural networks

In solving pattern recognition problems, many ensemble methods have been proposed to replace a single classifier by a classifier committee. These methods can be divided roughly into two categories: serial and parallel approaches. In the serial approach, component classifiers are created by focusing on different parts of the training set in different learning phases. In contrast, without paying special attention to any part of the dataset, the parallel approach generates classifiers independently. By integrating these two techniques and by using a neural network approach for the base classifier, this work proposes a design method for a two‐stage committee machine. In the first stage of the approach the entire dataset is used to train an averaging ensemble. Based on the classification results of this first stage, hard‐to‐classify samples are selected and sent to the second stage. To improve the classification accuracy for these samples, a computationally more intensive bagging ensemble is employed in the second stage. These two neural network ensembles work in series whereas the component neural networks in each of the ensembles are trained in parallel. Experimental results demonstrate the accuracy and robustness of the proposed approach.

Multiple Model Soft Sensor Based on Affinity Propagation, Gaussian Process and Bayesian Committee Machine

Presented is a multiple model soft sensing method based on Affinity Propagation (AP), Gaussian process (GP) and Bayesian committee machine (BCM). AP clustering arithmetic is used to cluster training samples according to their operating points. Then, the sub-models are estimated by Gaussian Process Regression (GPR). Finally, in order to get a global probabilistic prediction, Bayesian committee machine is used to combine the outputsof the sub-estimators. The proposed method has been applied to predict the light naphtha end point in hydrocracker fractionators. Practical applications indicate that it is useful for the online prediction of quality monitoring in chemical processes.

A committee neural network for prediction of normalized oil content from well log data: An example from South Pars Gas Field, Persian Gulf

Normalized oil content (NOC) is an important geochemical factor for identifying potential pay zones in hydrocarbon source rocks. The present study proposes an optimal and improved model to make a quantitative and qualitative correlation between NOC and well log responses by integration of neural network training algorithms and the committee machine concept. This committee machine with training algorithms (CMTA) combines Levenberg–Marquardt (LM), Bayesian regularization (BR), gradient descent (GD), one step secant (OSS), and resilient back-propagation (RP) algorithms. Each of these algorithms has a weight factor showing its contribution in overall prediction. The optimal combination of the weights is derived by a genetic algorithm. The method is illustrated using a case study. For this purpose, 231 data composed of well log data and measured NOC from three wells of South Pars Gas Field were clustered into 194 modeling dataset and 37 testing samples for evaluating reliability of the models. The results of this study show that the CMTA provides more reliable and acceptable results than each of the individual neural networks differing in training algorithms. Also CMTA can accurately identify production pay zones (PPZs) from well logs.