Committee Machine Research Articles

Emotion Classification through Nonlinear EEG Analysis Using Machine Learning Methods

Background: Emotion recognition, as a subset of affective computing, has received considerable attention in recent years. Emotions are key to human-computer interactions. Electroencephalogram (EEG) is considered a valuable physiological source of information for classifying emotions. However, it has complex and chaotic behavior. Methods: In this study, an attempt is made to extract important nonlinear features from EEGs with the aim of emotion recognition. We also take advantage of machine learning methods such as evolutionary feature selection methods and committee machines to enhance the classification performance. Classification performed concerning both arousal and valence factors. Results: Results suggest that the proposed method is successful and comparable to the previous works. A recognition rate equal to 90% achieved, and the most significant features reported. We apply the final classification scheme to 2 different databases including our recorded EEGs and a benchmark dataset to evaluate the suggested approach. Conclusion: Our findings approve of the effectiveness of using nonlinear features and a combination of classifiers. Results are also discussed from different points of view to understand brain dynamics better while emotion changes. This study reveals useful insights about emotion classification and brain-behavior related to emotion elicitation.

On the impact of outlier filtering on the electricity price forecasting accuracy

Increasing the accuracy of short-term electricity price forecasting allows day-ahead power market participants to obtain a positive economic effect by bidding close to the equilibrium price. However the electricity price time-series is generally infested with extreme values due to high price volatility. This paper discusses the impact of outlier filtering on forecasting accuracy based on a recently introduced seasonal component autoregressive model. We consider such methods of outlier detection (with a priori defined cut-off parameter) as threshold, standard deviation, percentage, recursive, and moving filter on prices. It is shown that such data pre-processing often leads to the forecasting accuracy gain while the error decrease (relative to the approach without filtering) in a number of cases may reach 1.8–1.9% of the average weekly price (in absolute values). For an a priori defined cut-off parameter, the simple threshold and standard deviation filter on prices outperform other considered methods, and yield to the accuracy gain in 63% and 67% of cases, correspondingly. At the same time, in case of the out-of-sample filter parameter grid-optimization all of the methods demonstrate comparable prediction power (equal to the marginal performance). But, practically speaking, such optimization is time-consuming and cannot be carried out on unavailable future data. As an competitive alternative, we propose a combined filter on prices based on a committee machine which uses the results of individual non-optimized algorithms and is not time-consuming, but gives accuracy comparable to the best one obtained for each of the studied electricity markets and leads to forecast gain in 63% of the considered cases.

Committee machine that votes for similarity between materials.

A method has been developed to measure the similarity between materials, focusing on specific physical properties. The information obtained can be utilized to understand the underlying mechanisms and support the prediction of the physical properties of materials. The method consists of three steps: variable evaluation based on nonlinear regression, regression-based clustering, and similarity measurement with a committee machine constructed from the clustering results. Three data sets of well characterized crystalline materials represented by critical atomic predicting variables are used as test beds. Herein, the focus is on the formation energy, lattice parameter and Curie temperature of the examined materials. Based on the information obtained on the similarities between the materials, a hierarchical clustering technique is applied to learn the cluster structures of the materials that facilitate interpretation of the mechanism, and an improvement in the regression models is introduced to predict the physical properties of the materials. The experiments show that rational and meaningful group structures can be obtained and that the prediction accuracy of the materials' physical properties can be significantly increased, confirming the rationality of the proposed similarity measure.

Neural Classifiers with Limited Connectivity and Recurrent Readouts.

For many neural network models in which neurons are trained to classify inputs like perceptrons, the number of inputs that can be classified is limited by the connectivity of each neuron, even when the total number of neurons is very large. This poses the problem of how the biological brain can take advantage of its huge number of neurons given that the connectivity is sparse. One solution is to combine multiple perceptrons together, as in committee machines. The number of classifiable random patterns would then grow linearly with the number of perceptrons, even when each perceptron has limited connectivity. However, the problem is moved to the downstream readout neurons, which would need a number of connections as large as the number of perceptrons. Here we propose a different approach in which the readout is implemented by connecting multiple perceptrons in a recurrent attractor neural network. We prove analytically that the number of classifiable random patterns can grow unboundedly with the number of perceptrons, even when the connectivity of each perceptron remains finite. Most importantly, both the recurrent connectivity and the connectivity of downstream readouts also remain finite. Our study shows that feedforward neural classifiers with numerous long-range afferent connections can be replaced by recurrent networks with sparse long-range connectivity without sacrificing the classification performance. Our strategy could be used to design more general scalable network architectures with limited connectivity, which resemble more closely the brain neural circuits that are dominated by recurrent connectivity.SIGNIFICANCE STATEMENT The mammalian brain has a huge number of neurons, but the connectivity is rather sparse. This observation seems to contrast with the theoretical studies showing that for many neural network models the performance scales with the number of connections per neuron and not with the total number of neurons. To solve this dilemma, we propose a model in which a recurrent network reads out multiple neural classifiers. Its performance scales with the total number of neurons even when each neuron of the network has limited connectivity. Our study reveals an important role of recurrent connections in neural systems like the hippocampus, in which the computational limitations due to sparse long-range feedforward connectivity might be compensated by local recurrent connections.

Rigorous prognostication of permeability of heterogeneous carbonate oil reservoirs: Smart modeling and correlation development

Permeability estimation has a major role in mapping quality of the reservoir, reservoir engineering calculation, reserve estimation, numerical reservoir simulation and planning for the drilling operations. In carbonate formations, it is of great challenge to predict permeability by reason of natural heterogeneity, nonuniformity of rock, complexity and nonlinearity of parameters. Various approaches have been developed for measuring/predicting this parameter, which are associated with high expenditures, time consuming processes and low accuracy. In this study, comprehensive efforts have been made to the development of radial basis function neural network (RBF-ANN), multilayer perceptron neural network (MLP-ANN), least square support vector machine (LSSVM), adaptive neuro-fuzzy inference system (ANFIS), genetic programming (GP), and committee machine intelligent system (CMIS). For this purpose, a widespread databank of 701 core permeability datapoints as a function of well log data was adopted from the open literature for heterogonous formations. Moreover, several optimization techniques like genetic algorithm (GA), particle swarm optimization (PSO), and levenberg marquardt (LM) were employed to enhance the prediction capability of the proposed tools in this study. For assessing the models efficiency, several tools like crossplot and error distribution diagram were applied in association with statistical calculation. As a result, the CMIS model is identified as the most accurate model with the highest determination coefficient (R2 near to unity) and the lowest root mean square error (RMSE near to zero). As a result of GP mathematical strategy, a new user-friendly empirically-derived correlation was developed for rapid and accurate estimation of reservoir permeability. The outcome of outlier detection shows the validity of dataset used for modeling, and the effective porosity is perceived to be the most affecting parameter on the permeability estimation in terms of sensitivity analysis. The main novelty of this modeling study was the proposal of CMIS and GP-based empirically-derived models for the first time in literature. To this end, the outcome of this study can be of great value for reservoir engineers dealing with simulation and characterization of the heterogonous carbonate reservoirs.

A Surrogate Modeling Technique for Electromagnetic Scattering Analysis of 3-D Objects With Varying Shape

A surrogate modeling technique for electromagnetic scattering analysis of 3-D objects with varying shape is presented by means of Gaussian process regression (GPR) and Bayesian committee machine (BCM). The technique based on GPR and BCM constructs surrogate models of 3-D objects with varying shape to reduce the expensive computational resource consumption of the electromagnetic scattering analysis. Based on the resample method, several subsets of training data are selected from the total training data set, and several subsurrogate models can be constructed by GPR with these subsets. The predicted values from each subsurrogate model for a test input can be fused by BCM to obtain the final predicted output with a high accuracy. For a series of test inputs on the varying geometric dimensions of objects, the uncertainty analysis of 3-D objects with varying shape will be achieved efficiently through a statistical analysis. Numerical results demonstrate the effectiveness of the proposed method.

Modelling asphaltene precipitation titration data: A committee of machines and a group method of data handling

AbstractAsphaltene precipitation causes different problems in the oil industry. In this study, a large data bank was used to model asphaltene precipitation titration data as a function of temperature, type of solvent, and solvent to crude oil dilution ratio. Three multilayer perceptron (MLP) neural networks and three radial basis function (RBF) neural networks were developed to estimate asphaltene precipitation data. The MLP models were optimized with scaled conjugate gradient (SCG), Levenberg‐Marquardt (LM), and Bayesian regularization (BR) algorithms and the RBF models were optimized with the particle swarm optimization (PSO) algorithm, imperialist competitive algorithm (ICA), and genetic algorithm (GA). All of the proposed models show an acceptable degree of accuracy and have an average absolute percent relative error (AAPRE) less than 4 %. Afterwards, three of the best models including MLP‐LM, MLP‐BR, and RBF‐GA were combined and a committee machine intelligent system (CMIS) was designed, which offers a higher accuracy compared to the other intelligent models. Aside from these intelligent models, the group method of data handling (GMDH) was used to develop an explicit and simple expression for estimating asphaltene precipitation. The proposed CMIS and GMDH models were compared to models developed based on scaling theory and the results show that the proposed models in this study outperform pre‐existing models. The validity and accuracy of the CMIS and GMDH models were proven by statistical and graphical techniques. Finally, a sensitivity analysis suggested that the solvent to crude oil dilution ratio has the largest effect on asphaltene precipitation.

Computational Deep Intelligence Vision Sensing for Nutrient Content Estimation in Agricultural Automation

This paper presents a novel computational intelligence vision sensing approach to estimate nutrient content in wheat leaves by analyzing color features of the leaves images captured on field with various lighting conditions. We propose the development of deep sparse extreme learning machines (DSELM) fusion and genetic algorithm (GA) to normalize plant images as well as to reduce color variability due to a variation of sunlight intensities. We also apply the DSELM in image segmentation to differentiate wheat leaves from a complex background. In this paper, four moments of color distribution of the leaves images (mean, variance, skewness, and kurtosis) are extracted and utilized as predictors in the nutrient estimation. We combine a number of DSELMs with committee machine and optimize them using the GA to estimate nitrogen content in wheat leaves. The results have shown the superiority of the proposed method in the term of quality and processing speed in all steps, i.e., color normalization, image segmentation, and nutrient prediction, as compared with other existing methods. Note to Practitioners —In order to support precision farming technology and agricultural automation, it is very essential to estimate the nutrient content in plants to prevent over fertilizing that will harm environment. Furthermore, the existing method to determine nutrient content in leaves is destructive as well as time consuming and requires special expertise to operate expensive devices. A number of methods have been developed to estimate nutrient content in leaves nondestructively based on the color features of the leaves. Most of these methods, however, are conducted in a controlled environment with artificial lighting. Such methods are not practical and need various equipment. We propose a low-cost, simple, and accurate technique to estimate nitrogen content in wheat leaves by analyzing RGB color of the leaves images. In this paper, we found that the developed DSELMs fusion has enabled better performance in normalizing images and is faster than other neural network types, i.e., backpropagation-based multilayer perceptron and original extreme learning machine. In image segmentation step, the established DSELM shows good performance to recognize and distinguish wheat leaves from other undesired background images. Furthermore, the developed weighted DSELMs have demonstrated enhanced ability in estimating nutrient content.

Applying the committee machine method to forecast responses of oil prices to changes in oil reserves

Applying the committee machine method to forecast responses of oil prices to changes in oil reserves

Building a Globally Optimized Computational Intelligent Image Processing Algorithm for On-Site Inference of Nitrogen in Plants

Estimating nutrient content in plants is a crucial task in the application of precision farming. This work will be more challenging if it is conducted nondestructively based on plant images captured in the field due to the variation of lighting conditions. This paper proposes a computational intelligence image processing to analyze nitrogen status in wheat plants. We developed an ensemble of deep learning multilayer perceptron-using committee machines for color normalization and image segmentation. This paper also focuses on building a genetic-algorithm-based global optimization to fine tune the color normalization and nitrogen estimation results. We discovered that the proposed method can successfully normalize plant images by reducing color variabilities compared to other color normalization techniques. Furthermore, this algorithm is able to enhance the nitrogen estimation results compared to other non-global optimization methods as well as the most renowned SPAD meter based nitrogen measurement.

Ensemble committee-based data intelligent approach for generating soil moisture forecasts with multivariate hydro-meteorological predictors

Soil moisture (SM) is a key component of the global energy cycle that regulates all domains of the natural environmental and the agricultural system. In this research, the challenge is to develop a low-cost data-intelligent SM forecasting model using climate dynamics (i.e., the climate indices, atmospheric and hydro-meteorological parameters) as the model inputs. A newly designed, multi-model ensemble committee machine learning approach based on the artificial neural network (ANN-CoM) is developed to forecast monthly upper layer (∼0.2 m from the surface) and the lower layer (∼0.2–1.5 m deep) SM at four agricultural sites in Australia’s Murray-Darling Basin. ANN-CoM model is validated with respect to non-tuned second-order Volterra, M5 model tree, random forest, and an extreme learning machine (ELM) models. To construct the ANN-CoM model, the input variables comprised of the hydro-meteorological data from the Australian Water Availability Project, large-scale climate indices and atmospheric parameters derived from the Interim ERA European Centre for Medium-Range Weather Forecasting ECMWF reanalysis fields leads to a total of 60 potential predictors used for SM forecasting. To reduce the model input data dimensionality for accurate forecasts, the Neighborhood Component Analysis (NCA) based feature selection algorithm for regression purposes (fsrnca) is applied to determine the relative feature weights related to the targeted variable. The optimal predictor variables are then screened with an ELM model as the fitness function of the fsrnca algorithm to identify the set of most pertinent model variables. Extensive performance evaluation using statistical score metrics with visual and diagnostic plots show that the ensemble committee based, ANN-CoM model is able to effectively capture the nonlinear dynamics involved in the modeling of monthly upper and lower layer SM levels. Therefore, the ANN-CoM multi-model ensemble-based approach can be considered to be a superior SM forecasting tool, portraying as an amicable, integrated (or ensemble) machine learning stratagem that can be explored for soil moisture modeling and applications in agriculture and other hydro-meteorological phenomena.

Application of the Committee Machine Method to Forecast the Movement of Exchange Rates and Oil Prices

Application of the Committee Machine Method to Forecast the Movement of Exchange Rates and Oil Prices

Application of the Committee Machine Method to Forecast the Movement of Exchange Rates and Oil Prices

Application of the Committee Machine Method to Forecast the Movement of Exchange Rates and Oil Prices

Combining of intelligent models through committee machine for estimation of wax deposition

Deposition of the wax is one of the thorny issues in the petroleum industry, invoking costly problems during the transportation and production of crude oil. Owing to its devastating impacts on oil companies' economy, it is essential to develop a simple and robust strategy for the quantitative estimation of wax deposition. In this paper, support vector regression (SVR) is first proposed to estimate the amount of wax deposition. Subsequently, an artificial neural network (ANN) is developed for wax deposition prediction. Eventually, a sophisticated committee machine (CM) is constructed for combining the results of the SVR and ANN models. Optimal contribution of each model in final prediction of the wax deposit is determined through genetic algorithm in CM. Statistical error analysis shows that the CM model performs better than the individual models performing alone.

Accurate prediction of solubility of gases within H2-selective nanocomposite membranes using committee machine intelligent system

In-depth knowledge about the gas sorption within hydrogen (H2) selective nanocomposite membranes at various conditions is crucial, particularly in petrochemical and separation processes. Hence, various artificial intelligence (AI) methods such as multilayer perceptron artificial neural network (MLP-ANN), adaptive neuro-fuzzy inference system (ANFIS), the adaptive neuro-fuzzy inference system optimized by genetic algorithm (GA-ANFIS), Genetic Programming (GP) and Committee Machine Intelligent System (CMIS) were applied to predict the sorption of gases in H2-selective nanocomposite membranes consist of porous nanoparticles as the dispersed phase and polymer matrix as continuous phase. The momentous purpose of this paper was to estimate the sorption of C3H8, H2, CH4 and CO2 within H2-selective nanocomposite membranes considering the effect of nanoparticles loading, critical temperature (gas type characteristics) and upstream pressure. Obtained data were categorized into two parts including training and testing data set. The CMIS method showed more precise results rather than other intelligent models. Having developed different intelligent approaches rely on algorithms, a powerful successor for labor-intensive experimental processes of solubility was revealed. The prediction results and experimental data were significantly consistent in approach with a correlation coefficient (R2) of 0.9999, 0.9987, 0.9998, 0.9995, and 0.9997 for CMIS, GP, GA-ANFIS, ANFIS and ANN models respectively.

Fault diagnosis of rotating machinery based on multiple probabilistic classifiers

Abstract Intelligent fault diagnosis of rotating machinery is vital for industries to improve fault prediction performance and reduce the maintenance cost. The new fault diagnostic framework is proposed which consists of three stages: (1) signal processing and feature extraction, (2) fault diagnosis by combining the classification results through a probabilistic ensemble method, and (3) parameter optimization and performance evaluation. In the first stage, ensemble empirical mode decomposition (EEMD) decomposes the acquired signal into a suite of intrinsic mode functions (IMF) which encounters redundant components and large data problems. To eliminate the redundant IMF and select fault feature from residual IMF, correlation coefficient (CC) and singular value decomposition (SVD) method are applied respectively. In the second stage, to improve the performance of fault diagnosis based on single classifier and increase the number of detectable fault, a new probabilistic committee machine (PCM) method is proposed, in which multiple pairwise-coupled sparse Bayesian extreme learning machines (PCSBELM) are individually trained using air ration, ignition pattern and engine sound signal. In addition, each classifier is assigned with an optimal weight in accordance with their reliability and accuracy so that a reliable and widely-covered fault diagnostic result can be obtained from the weighted combination of the members. To verify the effectiveness of the proposed fault diagnostic framework, it is applied to automotive engine fault detection. The evaluation results show the proposed framework is superior to the existing single classifier in terms of both single- and simultaneous-faults in automotive engine.

On the importance of the long-term seasonal component in day-ahead electricity price forecasting with NARX neural networks

Daily and weekly seasonalities are always taken into account in day-ahead electricity price forecasting, but the long-term seasonal component has long been believed to add unnecessary complexity, and hence, most studies have ignored it. The recent introduction of the Seasonal Component AutoRegressive (SCAR) modeling framework has changed this viewpoint. However, this framework is based on linear models estimated using ordinary least squares. This paper shows that considering non-linear autoregressive (NARX) neural network-type models with the same inputs as the corresponding SCAR-type models can lead to yet better performances. While individual Seasonal Component Artificial Neural Network (SCANN) models are generally worse than the corresponding SCAR-type structures, we provide empirical evidence that committee machines of SCANN networks can outperform the latter significantly.

Modeling Music Emotion Judgments Using Machine Learning Methods.

Emotion judgments and five channels of physiological data were obtained from 60 participants listening to 60 music excerpts. Various machine learning (ML) methods were used to model the emotion judgments inclusive of neural networks, linear regression, and random forests. Input for models of perceived emotion consisted of audio features extracted from the music recordings. Input for models of felt emotion consisted of physiological features extracted from the physiological recordings. Models were trained and interpreted with consideration of the classic debate in music emotion between cognitivists and emotivists. Our models supported a hybrid position wherein emotion judgments were influenced by a combination of perceived and felt emotions. In comparing the different ML approaches that were used for modeling, we conclude that neural networks were optimal, yielding models that were flexible as well as interpretable. Inspection of a committee machine, encompassing an ensemble of networks, revealed that arousal judgments were predominantly influenced by felt emotion, whereas valence judgments were predominantly influenced by perceived emotion.

Erratum to: “Combining Rules Using Local Statistics and Uncertainty Estimates for Improved Ensemble Segmentation”

Segmentation using an ensemble of classifiers (or committee machine) combines multiple classifiers’ results to increase the performance when compared to single classifiers. In this paper, we propose new concepts for combining rules. They are based (1) on uncertainties of the individual classifiers, (2) on combining the result of existing combining rules, (3) on combining local class probabilities with the existing segmentation probabilities at each individual segmentation, and (4) on using uncertainty-based weights for the weighted majority rule. The results show that the proposed local-statistics-aware combining rules can reduce the effect of noise in the individual segmentation result and consequently improve the performance of the final (combined) segmentation. Also, combining existing combining rules and using the proposed uncertainty- based weights can further improve the performance.

Efficient estimation of hydrolyzed polyacrylamide (HPAM) solution viscosity for enhanced oil recovery process by polymer flooding

Polymers applications have been progressively increased in sciences and engineering including chemistry, pharmacology science, and chemical and petroleum engineering due to their attractive properties. Amongst the all types of polymers, partially Hydrolyzed Polyacrylamide (HPAM) is one of the widely used polymers especially in chemistry, and chemical and petroleum engineering. Capability of solution viscosity increment of HPAM is the key parameter in its successful applications; thus, the viscosity of HPAM solution must be determined in any study. Experimental measurement of HPAM solution viscosity is time-consuming and can be expensive for elevated conditions of temperatures and pressures, which is not desirable for engineering computations. In this communication, Multilayer Perceptron neural network (MLP), Least Squares Support Vector Machine approach optimized with Coupled Simulated Annealing (CSA-LSSVM), Radial Basis Function neural network optimized with Genetic Algorithm (GA-RBF), Adaptive Neuro Fuzzy Inference System coupled with Conjugate Hybrid Particle Swarm Optimization (CHPSO-ANFIS) approach, and Committee Machine Intelligent System (CMIS) were used to model the viscosity of HPAM solutions. Then, the accuracy and reliability of the developed models in this study were investigated through graphical and statistical analyses, trend prediction capability, outlier detection, and sensitivity analysis. As a result, it has been found that the MLP and CMIS models give the most reliable results with determination coefficients (R2) more than 0.98 and Average Absolute Relative Deviations (AARD) less than 4.0%. Finally, the suggested models in this study can be applied for efficient estimation of aqueous solutions of HPAM polymer in simulation of polymer flooding into oil reservoirs.