Ensemble Of Multilayer Perceptrons Research Articles

Predicting the perception of performed dynamics in music audio with ensemble learning.

By varying the dynamics in a musical performance, the musician can convey structure and different expressions. Spectral properties of most musical instruments change in a complex way with the performed dynamics, but dedicated audio features for modeling the parameter are lacking. In this study, feature extraction methods were developed to capture relevant attributes related to spectral characteristics and spectral fluctuations, the latter through a sectional spectral flux. Previously, ground truths ratings of performed dynamics had been collected by asking listeners to rate how soft/loud the musicians played in a set of audio files. The ratings, averaged over subjects, were used to train three different machine learning models, using the audio features developed for the study as input. The highest result was produced from an ensemble of multilayer perceptrons with an R2 of 0.84. This result seems to be close to the upper bound, given the estimated uncertainty of the ground truth data. The result is well above that of individual human listeners of the previous listening experiment, and on par with the performance achieved from the average rating of six listeners. Features were analyzed with a factorial design, which highlighted the importance of source separation in the feature extraction.

Accelerator-Based Human Activity Recognition Using Voting Technique with NBTree and MLP Classifiers

In evolution and ubiquitous computing systems, accelerometer-based human activity recognition has huge potential in a large number of application domains. Accelerometer-based human activity recognition aims to identify physical activities performed by human using accelerometer; a sensor device attached to the body and returns an actual valued estimate of acceleration along the x-, y- and z-axes from which the sensor location can be estimated. In this study, an accelerator-based activity recognition model using voting technique was proposed. Two machine learning classifiers, Naive Bayes Tree (NBTree) and Multilayer Perceptron (MLP), were used as ensemble classifiers in the voting technique. To evaluate the proposed voting technique, the performance of selected individual classifiers and existing voting technique was first examined, followed by the experiment to determine the performance of the proposed model. All of the experiments were performed using a standard dataset called Wireless Sensor Data Mining involving six physical human activities; jogging, walking, walking towards upstairs, walking towards downstairs, sitting and stand still. Results showed that the proposed voting technique with NBTree and MLP ensemble classifiers outperformed other individual classifiers and another previously suggested voting technique for accelerometer-based human activity recognition.

Introduction: Special issue on computational intelligence methods for big data and information analytics

Incremental learning has been investigated by many researchers. However, only few works have considered the situation where class imbalance occurs. In this paper, class imbalanced incremental learning was investigated and an ensemble-based method, named Selective Further Learning (SFL) was proposed. In SFL, a hybrid ensemble of Naive Bayes (NB) and Multilayer Perceptrons (MLPs) were employed. For the ensemble of MLPs, parts of the MLPs were selected to learning from the new data set. Negative Correlation Learning (NCL) with Dynamic Sampling (DyS) for handling class imbalance was used as the basic training method. Besides, as an additive model, Naive Bayes was employed as an individual of the ensemble to learn the data sets incrementally. A group of weights (with the number of the classes as the length) are updated for every individual of the ensemble to indicate the confidence of the individual learning about the classes. The ensemble combines all of the individuals by weighted average according to the weights. Experiments on 3 synthetic data sets and 10 real world data sets showed that SFL was able to handle class imbalance incremental learning and outperform a recently related approach.

Selective further learning of hybrid ensemble for class imbalanced increment learning

Incremental learning has been investigated by many researchers. However, only few works have considered the situation where class imbalance occurs. In this paper, class imbalanced incremental learning was investigated and an ensemble-based method, named Selective Further Learning (SFL) was proposed. In SFL, a hybrid ensemble of Naive Bayes (NB) and Multilayer Perceptrons (MLPs) were employed. For the ensemble of MLPs, parts of the MLPs were selected to learning from the new data set. Negative Correlation Learning (NCL) with Dynamic Sampling (DyS) for handling class imbalance was used as the basic training method. Besides, as an additive model, Naive Bayes was employed as an individual of the ensemble to learn the data sets incrementally. A group of weights (with the number of the classes as the length) are updated for every individual of the ensemble to indicate the 'confidence' of the individual learning about the classes. The ensemble combines all of the individuals by weighted average according to the weights. Experiments on 3 synthetic data sets and 10 real world data sets showed that SFL was able to handle class imbalance incremental learning and outperform a recently related approach.

On species distribution modelling, spatial scales and environmental flow assessment with Multi–Layer Perceptron Ensembles: A case study on the redfin barbel (Barbus haasi; Mertens, 1925)

Inconsistent performance of Species Distribution Models (SDMs), which may depend on several factors such as the initial conditions or the applied modelling technique, is one of the greatest challenges in ecological modelling. To overcome this problem, ensemble modelling combines the forecasts of several individual models. A commonly applied ensemble modelling technique is the Multi–Layer Perceptron (MLP) Ensemble, which was envisaged in the 1990s. However, despite its potential for ecological modelling, it has received little attention in the development of SDMs for freshwater fish. Although this approach originally included all the developed MLPs, Genetic Algorithms (GA) now allow selection of the optimal subset of MLPs and thus substantial improvement of model performance. In this study, MLP Ensembles were used to develop SDMs for the redfin barbel ( Barbus haasi ; Mertens, 1925) at two different spatial scales: the micro–scale and the meso–scale. Finally, the potential of the MLP Ensembles for environmental flow (e–flow) assessment was tested by linking model results to hydraulic simulation. MLP Ensembles with a candidate selection based on GA outperformed the optimal single MLP or the ensemble of the whole set of MLPs. The micro–scale model complemented previous studies, showing high suitability of relatively deep areas with coarse substrate and corroborating the need for cover and the rheophilic nature of the redfin barbel. The meso–scale model highlighted the advantages of using cross–scale variables, since elevation (a macro–scale variable) was selected in the optimal model. Although the meso–scale model also demonstrated that redfin barbel selects deep areas, it partially contradicted the micro–scale model because velocity had a clearer positive effect on habitat suitability and redfin barbel showed a preference for fine substrate in the meso–scale model. Although the meso–scale model suggested an overall higher habitat suitability of the test site, this did not result in a notable higher minimum environmental flow. Our results demonstrate that MLP Ensembles are a promising tool in the development of SDMs for freshwater fish species and proficient in e–flow assessment.

ADMET evaluation in drug discovery: 15. Accurate prediction of rat oral acute toxicity using relevance vector machine and consensus modeling.

Background Determination of acute toxicity, expressed as median lethal dose (LD50), is one of the most important steps in drug discovery pipeline. Because in vivo assays for oral acute toxicity in mammals are time-consuming and costly, there is thus an urgent need to develop in silico prediction models of oral acute toxicity. ResultsIn this study, based on a comprehensive data set containing 7314 diverse chemicals with rat oral LD50 values, relevance vector machine (RVM) technique was employed to build the regression models for the prediction of oral acute toxicity in rate, which were compared with those built using other six machine learning approaches, including k-nearest-neighbor regression, random forest (RF), support vector machine, local approximate Gaussian process, multilayer perceptron ensemble, and eXtreme gradient boosting. A subset of the original molecular descriptors and structural fingerprints (PubChem or SubFP) was chosen by the Chi squared statistics. The prediction capabilities of individual QSAR models, measured by qext2 for the test set containing 2376 molecules, ranged from 0.572 to 0.659.ConclusionConsidering the overall prediction accuracy for the test set, RVM with Laplacian kernel and RF were recommended to build in silico models with better predictivity for rat oral acute toxicity. By combining the predictions from individual models, four consensus models were developed, yielding better prediction capabilities for the test set (qext2 = 0.669–0.689). Finally, some essential descriptors and substructures relevant to oral acute toxicity were identified and analyzed, and they may be served as property or substructure alerts to avoid toxicity. We believe that the best consensus model with high prediction accuracy can be used as a reliable virtual screening tool to filter out compounds with high rat oral acute toxicity. Graphical abstractWorkflow of combinatorial QSAR modelling to predict rat oral acute toxicity

Shifts in the suitable habitat available for brown trout (Salmo trutta L.) under short-term climate change scenarios

The impact of climate change on the habitat suitability for large brown trout (Salmo trutta L.) was studied in a segment of the Cabriel River (Iberian Peninsula). The future flow and water temperature patterns were simulated at a daily time step with M5 models' trees (NSE of 0.78 and 0.97 respectively) for two short-term scenarios (2011–2040) under the representative concentration pathways (RCP 4.5 and 8.5). An ensemble of five strongly regularized machine learning techniques (generalized additive models, multilayer perceptron ensembles, random forests, support vector machines and fuzzy rule base systems) was used to model the microhabitat suitability (depth, velocity and substrate) during summertime and to evaluate several flows simulated with River2D©. The simulated flow rate and water temperature were combined with the microhabitat assessment to infer bivariate habitat duration curves (BHDCs) under historical conditions and climate change scenarios using either the weighted usable area (WUA) or the Boolean-based suitable area (SA). The forecasts for both scenarios jointly predicted a significant reduction in the flow rate and an increase in water temperature (mean rate of change of ca. −25% and +4% respectively). The five techniques converged on the modelled suitability and habitat preferences; large brown trout selected relatively high flow velocity, large depth and coarse substrate. However, the model developed with support vector machines presented a significantly trimmed output range (max.: 0.38), and thus its predictions were banned from the WUA-based analyses. The BHDCs based on the WUA and the SA broadly matched, indicating an increase in the number of days with less suitable habitat available (WUA and SA) and/or with higher water temperature (trout will endure impoverished environmental conditions ca. 82% of the days). Finally, our results suggested the potential extirpation of the species from the study site during short time spans.

Can multilayer perceptron ensembles model the ecological niche of freshwater fish species?

The potential of Multilayer Perceptron (MLP) Ensembles to explore the ecology of freshwater fish species was tested by applying the technique to redfin barbel (Barbus haasi Mertens, 1925), an endemic and montane species that inhabits the North-East quadrant of the Iberian Peninsula. Two different MLP Ensembles were developed. The physical habitat model considered only abiotic variables, whereas the biotic model also included the density of the accompanying fish species and several invertebrate predictors. The results showed that MLP Ensembles may outperform single MLPs. Moreover, active selection of MLP candidates to create an optimal subset of MLPs can further improve model performance. The physical habitat model confirmed the redfin barbel preference for middle-to-upper river segments whereas the importance of depth confirms that redfin barbel prefers pool-type habitats. Although the biotic model showed higher uncertainty, it suggested that redfin barbel, European eel and the considered cyprinid species have similar habitat requirements. Due to its high predictive performance and its ability to deal with model uncertainty, the MLP Ensemble is a promising tool for ecological modelling or habitat suitability prediction in environmental flow assessment.

Regional scale crop mapping using multi-temporal satellite imagery

Abstract. One of the problems in dealing with optical images for large territories (more than 10,000 sq. km) is the presence of clouds and shadows that result in having missing values in data sets. In this paper, a new approach to classification of multi-temporal optical satellite imagery with missing data due to clouds and shadows is proposed. First, self-organizing Kohonen maps (SOMs) are used to restore missing pixel values in a time series of satellite imagery. SOMs are trained for each spectral band separately using nonmissing values. Missing values are restored through a special procedure that substitutes input sample's missing components with neuron's weight coefficients. After missing data restoration, a supervised classification is performed for multi-temporal satellite images. An ensemble of neural networks, in particular multilayer perceptrons (MLPs), is proposed. Ensembling of neural networks is done by the technique of average committee, i.e. to calculate the average class probability over classifiers and select the class with the highest average posterior probability for the given input sample. The proposed approach is applied for regional scale crop classification using multi temporal Landsat-8 images for the JECAM test site in Ukraine in 2013. It is shown that ensemble of MLPs provides better performance than a single neural network in terms of overall classification accuracy, kappa coefficient, and producer's and user's accuracies for separate classes. The overall accuracy more than 85% is achieved. The obtained classification map is also validated through estimated crop areas and comparison to official statistics.

Hybrid Modelling of Multilayer Perceptron Ensembles for Predicting the Response of Bolted Lap Joints

Hybrid Modelling of Multilayer Perceptron Ensembles for Predicting the Response of Bolted Lap Joints

Multi-objective optimization of the HVAC (heating, ventilation, and air conditioning) system performance

A data-driven approach to optimize the total energy consumption of the HVAC (heating, ventilation, and air conditioning) system in a typical office facility is presented. A multi-layer perceptron ensemble is selected to build the total energy model integrating three indoor air quality models, the facility temperature model, the facility relative humidity model, and the facility CO2 concentration model. To balance the energy consumption and the indoor air quality, a quad-objective optimization problem is constructed. The problem is solved with a modified particle swarm optimization algorithm producing control settings of supply air temperature and static pressure of the air handling unit. By assigning different weights to the objectives to the model, the generated control settings optimize HVAC system with the trade-off between the energy consumption and the facility thermal comfort. Significant energy savings can be obtained even with air quality constraint.

Negative correlation learning for customer churn prediction: a comparison study.

Recently, telecommunication companies have been paying more attention toward the problem of identification of customer churn behavior. In business, it is well known for service providers that attracting new customers is much more expensive than retaining existing ones. Therefore, adopting accurate models that are able to predict customer churn can effectively help in customer retention campaigns and maximizing the profit. In this paper we will utilize an ensemble of Multilayer perceptrons (MLP) whose training is obtained using negative correlation learning (NCL) for predicting customer churn in a telecommunication company. Experiments results confirm that NCL based MLP ensemble can achieve better generalization performance (high churn rate) compared with ensemble of MLP without NCL (flat ensemble) and other common data mining techniques used for churn analysis.

Ensemble of Multilayer Perceptrons for Change Detection in Remotely Sensed Images

In this letter, a change detection technique using a multiple classifier system is proposed. Here, different architectures of multilayer perceptron (MLP) are used as base classifiers. An ensemble of different MLPs is utilized to increase the robustness of the system. This also avoids the problem of choosing an optimum architecture for MLP. First, the support values for each of the unlabeled patterns are estimated using different MLPs (trained with the labeled patterns). Then, each of the unlabeled patterns is assigned to a specific class by fusing the outcome of the base classifiers using different combination rules. Experiments are carried out on multitemporal and multispectral images. Results show that the proposed ensemble technique has an edge over individual base classifiers for change detection in remotely sensed images.

Breast cancer diagnosis from biopsy images with highly reliable random subspace classifier ensembles

Accurate and reliable classification of microscopic biopsy images is an important issue in computer assisted breast cancer diagnosis. In this paper, a new cascade Random Subspace ensembles scheme with reject options is proposed for microscopic biopsy image classification. The classification system is built as a serial fusion of two different Random Subspace classifier ensembles with rejection options to enhance the classification reliability. The first ensemble consists of a set of Support Vector Machine classifiers that converts the original $$K$$ -class classification problem into a number of $$K$$ 2-class problems. The second ensemble consists of a Multi-Layer Perceptron ensemble, that focuses on the rejected samples from the first ensemble. For both of the ensembles, the reject option is implemented by relating the consensus degree from majority voting to a confidence measure, and abstaining to classify ambiguous samples if the consensus degree is lower than some threshold. We also investigated the effectiveness of a feature description approach by combining Local Binary Pattern (LBP) texture analysis, statistics derived using the Gray Level Co-occurrence Matrix (GLCM) and the Curvelet Transform. While the LBP analysis efficiently describes local texture properties and the GLCM reflects global texture statistics, the Curvelet Transform is particularly appropriate for the representation of piece-wise smooth images with rich edge information. The combined feature description thus provides a comprehensive biopsy image characterization by taking advantages of their complementary strengths. Using a benchmark microscopic biopsy image dataset, obtained from the Israel Institute of Technology, a high classification accuracy of $$99.25 \%$$ was obtained (with a rejection rate of $$1.94 \%$$ ) using the proposed system.

Fuzzy artmap and neural network approach to online processing of inputs with missing values

An ensemble based approach for dealing with missing data, without predicting or imputing the missing values is proposed. This technique is suitable for online operations of of neural networks and as a result, is used for online condition monitoring. The proposed technique is tested in both classification and regression problems. An ensemble of Fuzzy-ARTMAPs is used for classification whereas an ensemble of multi-layer perceptrons is used for the regression problem. Results obtained using this ensemble-based technique are compared to those obtained using a combination of auto-associative neural networks and genetic algorithms and findings show that this method can perform up to 9% better in regression problems. Another advantage of the proposed technique is that it eliminates the need for finding the best estimate of the data, and hence, saves time.