Ensemble Performance Research Articles

Moving to Communicate, Moving to Interact

Skilled ensemble musicians coordinate with high precision, even when improvising or interpreting loosely defined notation. Successful coordination is supported primarily through shared attention to the musical output; however, musicians also interact visually, particularly when the musical timing is irregular. This study investigated the performance conditions that encourage visual signaling and interaction between ensemble members. Piano and clarinet duos rehearsed a new piece as their body motion was recorded. Analyses of head movement showed that performers communicated gesturally following held notes. Gesture patterns became more consistent as duos rehearsed, though consistency dropped again during a final performance given under no-visual-contact conditions. Movements were smoother and interperformer coordination was stronger during irregularly timed passages than elsewhere in the piece, suggesting heightened visual interaction. Performers moved more after rehearsing than before, and more when they could see each other than when visual contact was occluded. Periods of temporal instability and increased familiarity with the music and co-performer seem to encourage visual interaction, while specific communicative gestures are integrated into performance routines through rehearsal. We propose that visual interaction may support successful ensemble performance by affirming coordination throughout periods of temporal instability and serving as a social motivator to promote creative risk-taking.

End-to-End Ensemble Learning by Exploiting the Correlation Between Individuals and Weights.

Ensemble learning performs better than a single classifier in most tasks due to the diversity among multiple classifiers. However, the enhancement of the diversity is at the expense of reducing the accuracies of individual classifiers in general and, thus, how to balance the diversity and accuracies is crucial for improving the ensemble performance. In this paper, we propose a new ensemble method which exploits the correlation between individual classifiers and their corresponding weights by constructing a joint optimization model to achieve the tradeoff between the diversity and the accuracy. Specifically, the proposed framework can be modeled as a shallow network and efficiently trained by the end-to-end manner. In the proposed ensemble method, not only can a high total classification performance be achieved by the weighted classifiers but also the individual classifier can be updated based on the error of the optimized weighted classifiers ensemble. Furthermore, the sparsity constraint is imposed on the weight to enforce that partial individual classifiers are selected for final classification. Finally, the experimental results on the UCI datasets demonstrate that the proposed method effectively improves the performance of classification compared with relevant existing ensemble methods.

A gradient boosting approach with diversity promoting measures for the ensemble of surrogates in engineering

Ensemble of surrogates has shown to be effective in the modeling of a variety of engineering problems, from aerospace to automotive and oil and gas industries, among many others. The ensemble performance though can be negatively affected by the lack of diversity (correlated errors) among the members of the ensemble, i.e., similar errors throughout the input space. During the last decade, ensemble research efforts in engineering have mostly focused on the so-called ensemble integration (combining prediction) phase with, for example, linear weighted averages of surrogate predictions based on performance (error) measures of both local and global natures. On the ensemble generation side (members of the ensemble), the emphasis has been on promoting diversity through the so-called heterogeneous ensembles, that is the use of different learners (surrogates), such as, linear regression, radial basis functions, kriging, support vector regression, and neural networks. Because of the lack of control on diversity in heterogeneous ensembles, potentially correlated error among the learners remains a critical issue. This paper presents an alternative approach based on homogeneous ensembles (single learning algorithm) that promotes diversity both implicitly and explicitly, by randomly sampling the training data and deterministically changing the data supplied to each surrogate model in the ensemble generation process. Specifically, it includes decision trees as learners, subsampling of the training set, and training on random selections of the input features, i.e., random subspace. The ensemble is sequentially constructed as an additive model where at each stage a simple surrogate is fit to the negative gradient of the loss function at the training data of the latest ensemble (gradient boosting). In addition, as a by-product of the modeling process, the proposed approach provides importance measures to preliminary rank input features. Statistically significant prediction improvements over heterogeneous ensembles were observed in a variety of well-known analytical test functions and industrial case studies in the areas of petroleum engineering-net present value of oil production in a mature reservoir and structural analysis-FSAE brake pedal von Mises stress and buckling load factor.

Particle-swarm optimization of ensemble neural networks with negative correlation learning for forecasting short-term wind speed of wind farms in western China

In many countries, wind power is being developed as a primary source of renewable energy. However, it is difficult to describe and forecast wind speed features, which are stochastic and intermittently complex. Moreover, it is important both to obtain precise forecasts of wind speed for wind power generation and to determine their contribution to regional economies. In this paper, we propose a novel approach for wind speed forecasting. The proposed model is a hybrid that uses the wavelet analysis technique (WAT) for denoising, a negative correlation learning neural network (NCL-NN) ensemble, and an ensemble structure optimized using particle-swarm optimization (PSO). We name the approach WAT-NCL-PSO. We define a novel fitness function to optimize the performance of the NCL-NN ensemble. Then, we rebuild the new NCL-NN ensemble using the contribution rates (CRs) by applying the PSO algorithm. We compiled wind speed datasets from six wind power generator sites in western China and used them to test the performance of the proposed model. Further, we analyzed the model’s performance in terms of its robustness and time complexity. Finally, to illustrate the effectiveness of the proposed approach, we compared its performance with that of the back-propagation neural network (BPNN), support vector machine (SVM), bagging, AdaBoost, random forest (RF), long short-term memory (LSTM), seasonal autoregressive integrated moving average (SARIMA), SVM with ensemble empirical mode decomposition (EEMD-SVM), and NCL-NN with wavelet denoising (WAT-NCL) models. The simulation results demonstrate that the performance of the WAT-NCL-PSO model is superior to other methods in terms of forecasting accuracy for short-term wind speeds.

On the ergodic control of ensembles

Across smart-grid and smart-city application domains, there are many problems where an ensemble of agents is to be controlled such that both the aggregate behaviour and individual-level perception of the system’s performance are acceptable. In many applications, traditional PI control is used to regulate aggregate ensemble performance. Our principal contribution in this note is to demonstrate that PI control may not be always suitable for this purpose, and in some situations may lead to a loss of ergodicity for closed-loop systems. Building on this observation, a theoretical framework is proposed to both analyse and design control systems for the regulation of large scale ensembles of agents with a probabilistic intent. Examples are given to illustrate our results.

An Examination of Differential Item Functioning in a Rubric to Assess Solo Music Performance

The purpose of this study was to examine differential item functioning (DIF) in a rubric used to assess middle-school solo and ensemble performances. This study was guided by the following research questions: (a) does measurement equivalence for all items exist when used to measure subgroups of students based on their musical instrument? (b) what patterns of differential item functioning effects exist for items when used to measure subgroups of students based on their musical instrument? and (c) what size of differential item functioning effects exists for items when used to measure subgroups of students based on their musical instrument? In total, 17 adjudicators evaluated 138 middle-school instrumental students (ages 11–13) in the context of a live, formal solo and ensemble performance assessment. Using the Many Facets Rasch Partial Credit measurement model, measurement equivalence for all items did not exist when used to measure subgroups of students based on their musical instrument ([Formula: see text]2 (252) = 634.00, p < .01). Of the 252 total pairwise interactions examined between items and instruments, 57 (22.62%) significant interaction terms were identified. Overall, 26 (10.3%) significant interaction terms demonstrated a moderate to large effect (|DIF| >/= .63 logits), nine (3.6%) significant interaction terms demonstrated a slight to moderate effect (|DIF| = (.43, .63 logits)), and 22 (8.7%) significant interaction terms demonstrated a negligible effect (|DIF| < .43 logits). Implications for the fairness of music performance assessments and improved assessment protocols are discussed.

Boosting the Face Recognition Performance of Ensemble Based LDA for Pose, Non-uniform Illuminations, and Low-Resolution Images

Face recognition systems have several potential applications, such as security and biometric access control. Ongoing research is focused to develop a robust face recognition algorithm that can mimic the human vision system. Face pose, non-uniform illuminations, and low-resolution are main factors that influence the performance of face recognition algorithms. This paper proposes a novel method to handle the aforementioned aspects. Proposed face recognition algorithm initially uses 68 points to locate a face in the input image and later partially uses the PCA to extract mean image. Meanwhile, the AdaBoost and the LDA are used to extract face features. In final stage, classic nearest centre classifier is used for face classification. Proposed method outperforms recent state-of-the-art face recognition algorithms by producing high recognition rate and yields much lower error rate for a very challenging situation, such as when only frontal (00) face sample is available in gallery and seven poses (00, ±300, ±350, and ±450) as a probe on the LFW and the CMU Multi-PIE databases.

Perception of synchronization in singing ensembles.

Recent investigations analysing synchronization in singing ensembles have shown that the precision of synchronization during singing duo performances is better in the presence of visual contact between the singers than without. Research has also shown that synchronisation improves with practice across rehearsals in a newly formed singing quintet. However, whether listeners’ perceptions of synchronization reflect the different patterns of synchronization that were observed during ensemble performance with and without visual contact and across rehearsals has not yet been investigated. This study aims to analyse the perception of the synchronization associated with altered visual contact and amount of rehearsal during singing duo and quintet performances respectively, for listeners with different levels of musical expertise. A set of fifty-eight singing recordings selected from duo and quintet ensemble performances, collected from previous investigations of interpersonal synchrony, was presented to 33 listeners, including non-experts (university students with little or no music training), performers in the group (singers who performed the pieces used for the study), and other musicians (advanced music students). Participants were required to listen to each trial and judge the level of “togetherness” on a sliding scale from zero to 100. Results show that listeners, irrespective of their musical training and performance experience, perceived differences in the synchronization in the duo tokens depending on the presence/absence of visual contact between singers; on the other hand, the smaller asynchrony patterns measured across rehearsals in the singing quintet recordings were not perceived. This study contributes to our understanding of perceptions of synchronization by individuals with different levels of musical expertise, and underscores the perceptual salience of synchronization, regardless of musical experience.

Ensemble of optimal trees, random forest and random projection ensemble classification

The predictive performance of a random forest ensemble is highly associated with the strength of individual trees and their diversity. Ensemble of a small number of accurate and diverse trees, if prediction accuracy is not compromised, will also reduce computational burden. We investigate the idea of integrating trees that are accurate and diverse. For this purpose, we utilize out-of-bag observations as a validation sample from the training bootstrap samples, to choose the best trees based on their individual performance and then assess these trees for diversity using the Brier score on an independent validation sample. Starting from the first best tree, a tree is selected for the final ensemble if its addition to the forest reduces error of the trees that have already been added. Our approach does not use an implicit dimension reduction for each tree as random project ensemble classification. A total of 35 bench mark problems on classification and regression are used to assess the performance of the proposed method and compare it with random forest, random projection ensemble, node harvest, support vector machine, kNN and classification and regression tree. We compute unexplained variances or classification error rates for all the methods on the corresponding data sets. Our experiments reveal that the size of the ensemble is reduced significantly and better results are obtained in most of the cases. Results of a simulation study are also given where four tree style scenarios are considered to generate data sets with several structures.

Performance evaluation of deep neural ensembles toward malaria parasite detection in thin-blood smear images.

BackgroundMalaria is a life-threatening disease caused by Plasmodium parasites that infect the red blood cells (RBCs). Manual identification and counting of parasitized cells in microscopic thick/thin-film blood examination remains the common, but burdensome method for disease diagnosis. Its diagnostic accuracy is adversely impacted by inter/intra-observer variability, particularly in large-scale screening under resource-constrained settings.IntroductionState-of-the-art computer-aided diagnostic tools based on data-driven deep learning algorithms like convolutional neural network (CNN) has become the architecture of choice for image recognition tasks. However, CNNs suffer from high variance and may overfit due to their sensitivity to training data fluctuations.ObjectiveThe primary aim of this study is to reduce model variance, improve robustness and generalization through constructing model ensembles toward detecting parasitized cells in thin-blood smear images.MethodsWe evaluate the performance of custom and pretrained CNNs and construct an optimal model ensemble toward the challenge of classifying parasitized and normal cells in thin-blood smear images. Cross-validation studies are performed at the patient level to ensure preventing data leakage into the validation and reduce generalization errors. The models are evaluated in terms of the following performance metrics: (a) Accuracy; (b) Area under the receiver operating characteristic (ROC) curve (AUC); (c) Mean squared error (MSE); (d) Precision; (e) F-score; and (f) Matthews Correlation Coefficient (MCC).ResultsIt is observed that the ensemble model constructed with VGG-19 and SqueezeNet outperformed the state-of-the-art in several performance metrics toward classifying the parasitized and uninfected cells to aid in improved disease screening.ConclusionsEnsemble learning reduces the model variance by optimally combining the predictions of multiple models and decreases the sensitivity to the specifics of training data and selection of training algorithms. The performance of the model ensemble simulates real-world conditions with reduced variance, overfitting and leads to improved generalization.

Deep Network Ensembles for Aerial Scene Classification

It is well known in the machine learning community that the ensembles of neural networks outperform the respective individual networks; hence recent work on aerial scene classification has been trending toward various network fusion strategies. However, training multiple deep networks can be computationally expensive. “Snapshot ensembling” has recently appeared as an alternative and claims to provide the performance of network ensembles at the cost of a single network's training. In this letter, we present the results of a comparative study on deep network ensembles in the context of aerial scene classification, where homogeneous, heterogeneous, and snapshot ensembling strategies are explored with both DenseNet and Inception networks; contrary to the existing work, we employ model fusion at the last convolutional layer's level. The explored approaches are validated with the two largest and most challenging data sets available (AID and NWPU-RESISC45), and state-of-the-art results are achieved on both.

The psychometric evaluation of a wind band performance rubric using the Multifaceted Rasch Partial Credit Measurement Model

The purpose of this study was to develop a valid and reliable rubric to be used for the evaluation of large ensemble wind band performances. The guiding questions for this study were: (a) what are the psychometric qualities (i.e., reliability and validity) of the scale developed to assess wind band ensemble performance at the high school level? (b) how do the items fit the model and vary in difficulty? (c) how does the structure of the rating scale vary across individual items? and (d) how can the rating scale be transferred into an informative rubric? The primary data analysis tool used in this study was the Multifaceted Rasch Partial Credit Measurement Model. Music content experts ( N = 20) were solicited to evaluate 40 wind band performances, each evaluator listening to four. A 4-point Likert-type rating scale (e.g., Strongly Agree, Agree, Disagree, and Strongly Disagree) was used to evaluate each recorded performance. Results indicated good model data fit and resulted in a final rubric containing 24 items ranging from two to four performance categories. Implications for classroom teaching and consequential validity are discussed.

A Projection of Extreme Precipitation Based on a Selection of CMIP5 GCMs over North Korea

The numerous choices between climate change scenarios makes decision-making difficult for the assessment of climate change impacts. Previous studies have used climate models to compare performance in terms of simulating observed climates or preserving model variability among scenarios. In this study, the Katsavounidis-Kuo-Zhang algorithm was applied to select representative climate change scenarios (RCCS) that preserve the variability among all climate change scenarios (CCS). The performance of multi-model ensemble of RCCS was evaluated for reference and future climates. It was found that RCCS was well suited for observations and multi model ensemble of all CCS. Using the RCCS under RCP (Representative Concentration Pathway) 8.5, the future extreme precipitation was projected. As a result, the magnitude and frequency of extreme precipitation increased towards the farther future. Especially, extreme precipitation (daily maximum precipitation of 20-year return-period) during 2070-2099, was projected to occur once every 8.3-year. The RCCS employed in this study is able to successfully represent the performance of all CCS, therefore, this approach can give opportunities managing water resources efficiently for assessment of climate change impacts.

Ensemble Performance of Biometric Authentication Systems Based on Secret Key Generation

We study the ensemble performance of biometric authentication systems, based on secret key generation, which work as follows. In the enrollment stage, an individual provides a biometric signal that is mapped into a secret key and a helper message, the former being prepared to become available to the system at a later time (for authentication), and the latter is stored in a public database. When an authorized user requests authentication, claiming his/her identity as one of the subscribers, he/she has to provide a biometric signal again, and then the system, which retrieves also the helper message of the claimed subscriber, produces an estimate of the secret key, that is finally compared with the secret key of the claimed user. In case of a match, the authentication request is approved, otherwise, it is rejected. Referring to an ensemble of systems based on Slepian–Wolf binning, we provide a detailed analysis of the false–reject (FR) and false–accept (FA) probabilities, for a wide class of stochastic decoders. We also derive converse bounds. The converse bound of the FA probability matches the direct theorem, whereas the one for the FR probability is tight for some ranges of rates. Finally, we outline derivations of the secrecy leakage (for the typical code in the ensemble) and the privacy leakage.

Approximation of Ensemble Boundary Using Spectral Coefficients.

A spectral analysis of a Boolean function is proposed for approximating the decision boundary of an ensemble of classifiers, and an intuitive explanation of computing Walsh coefficients for the functional approximation is provided. It is shown that the difference between the first- and third-order coefficient approximations is a good indicator of optimal base classifier complexity. When combining neural networks, the experimental results on a variety of artificial and real two-class problems demonstrate under what circumstances ensemble performance can be improved. For tuned base classifiers, the first-order coefficients provide performance similar to the majority vote. However, for weak/fast base classifiers, higher order coefficient approximation may give better performance. It is also shown that higher order coefficient approximation is superior to the Adaboost logarithmic weighting rule when boosting weak decision tree base classifiers.

A hybrid transfer learning algorithm incorporating TrSVM with GASEN

Traditional machine learning is generally committed to obtaining classifiers which are well-performed over unlabeled test data. This usually relies on two critical assumptions: firstly, sufficient labeled training data are available; secondly, training and testing data are drawn from the same distribution and the same feature space. Unfortunately, in most cases, the actual situation is difficult to meet the above conditions. Transfer learning scheme is naturally proposed to alleviate this problem. In order to get robust classifiers with relatively lower computational costs, we incorporate the rationale of Support Vector Machine (SVM) into transfer learning scheme and propose a novel SVM-based transfer learning model, abbreviated as TrSVM. In this method, support vector sets are extracted to represent the source domain. New training datasets are respectively constructed by combining each support vector set and target labeled dataset. On the basis of these training datasets, a number of new base classifiers can be acquired. Since performance of a classifiers ensemble is generally superior to that of individual classifiers, ensemble selection is utilized in our work. A hybrid transfer learning algorithm, integrating the Genetic Algorithm based Selective Ensemble (GASEN) with TrSVM, is proposed, and abbreviated as TrGASVM, naturally. GASEN is a genetic algorithm-based heuristic algorithm for solving combinatorial optimization problems. It can not only enhance the generalization ability of an ensemble, but also alleviate the local minimum problem of greedy ensemble pruning methods. Since TrGASVM is under frame of TrSVM and GASEN, it inevitably inherits the advantages of both algorithms. The reasonable incorporation of TrSVM with GASEN endows TrGASVM with favorable transfer learning capability, with its effectiveness being demonstrated by the experimental results on three real-world text classification datasets.

Cast harmony and performance through uncertainty: learning to achieve dance at a distance

ABSTRACTNarrating the process of directing a performance almost entirely at a distance of 1600 miles, “Dance at a Distance” describes my ongoing interactions with the performers throughout the process. Specifically, I focus on how the development of harmony among the cast helped to mitigate the uncertainty and the significant communicative challenges of our circumstance. I use this experience to uncover a more intimate understanding of the larger process of creating and teaching ensemble performance, with implications beyond the scope of this particular production.

Building the COllaborative Carbon Column Observing Network (COCCON): long-term stability and ensemble performance of the EM27/SUN Fourier transform spectrometer

Abstract. In a 3.5-year long study, the long-term performance of a mobile, solar absorption Bruker EM27/SUN spectrometer, used for greenhouse gas observations, is checked with respect to a co-located reference Bruker IFS 125HR spectrometer, which is part of the Total Carbon Column Observing Network (TCCON). We find that the EM27/SUN is stable on timescales of several years; the drift per year between the EM27/SUN and the official TCCON product is 0.02 ppmv for XCO2 and 0.9 ppbv for XCH4, which is within the 1σ precision of the comparison, 0.6 ppmv for XCO2 and 4.3 ppbv for XCH4. The bias between the two data sets is 3.9 ppmv for XCO2 and 13.0 ppbv for XCH4. In order to avoid sensitivity-dependent artifacts, the EM27/SUN is also compared to a truncated IFS 125HR data set derived from full-resolution TCCON interferograms. The drift is 0.02 ppmv for XCO2 and 0.2 ppbv for XCH4 per year, with 1σ precisions of 0.4 ppmv for XCO2 and 1.4 ppbv for XCH4, respectively. The bias between the two data sets is 0.6 ppmv for XCO2 and 0.5 ppbv for XCH4. With the presented long-term stability, the EM27/SUN qualifies as an useful supplement to the existing TCCON network in remote areas. To achieve consistent performance, such an extension requires careful testing of any spectrometers involved by application of common quality assurance measures. One major aim of the COllaborative Carbon Column Observing Network (COCCON) infrastructure is to provide these services to all EM27/SUN operators. In the framework of COCCON development, the performance of an ensemble of 30 EM27/SUN spectrometers was tested and found to be very uniform, enhanced by the centralized inspection performed at the Karlsruhe Institute of Technology prior to deployment. Taking into account measured instrumental line shape parameters for each spectrometer, the resulting average bias across the ensemble with respect to the reference EM27/SUN used in the long-term study in XCO2 is 0.20 ppmv, while it is 0.8 ppbv for XCH4. The average standard deviation of the ensemble is 0.13 ppmv for XCO2 and 0.6 ppbv for XCH4. In addition to the robust metric based on absolute differences, we calculate the standard deviation among the empirical calibration factors. The resulting 2σ uncertainty is 0.6 ppmv for XCO2 and 2.2 ppbv for XCH4. As indicated by the executed long-term study on one device presented here, the remaining empirical calibration factor deduced for each individual instrument can be assumed constant over time. Therefore the application of these empirical factors is expected to further improve the EM27/SUN network conformity beyond the scatter among the empirical calibration factors reported above.

Improving undersampling-based ensemble with rotation forest for imbalanced problem

As one of the most challenging and attractive issues in pattern recognition and machine learning, the imbalanced problem has attracted increasing attention. For two-class data, imbalanced data are characterized by the size of one class (majority class) being much larger than that of the other class (minority class), which makes the constructed models focus more on the majority class and ignore or even misclassify the examples of the minority class. The undersampling-based ensemble, which learns individual classifiers from undersampled balanced data, is an effective method to cope with the class-imbalance data. The problem in this method is that the size of the dataset to train each classifier is notably small; thus, how to generate individual classifiers with high performance from the limited data is a key to the success of the method. In this paper, rotation forest (an ensemble method) is used to improve the performance of the undersampling-based ensemble on the imbalanced problem because rotation forest has higher performance than other ensemble methods such as bagging, boosting, and random forest, particularly for small-sized data. In addition, rotation forest is more sensitive to the sampling technique than some robust methods including SVM and neural networks; thus, it is easier to create individual classifiers with diversity using rotation forest. Two versions of the improved undersampling-based ensemble methods are implemented: 1) undersampling subsets from the majority class and learning each classifier using the rotation forest on the data obtained by combing each subset with the minority class and 2) similarly to the first method, with the exception of removing the majority class examples that are correctly classified with high confidence after learning each classifier for further consideration. The experimental results show that the proposed methods show significantly better performance on measures of recall, g-mean, f-measure, and AUC than other state-of-the-art methods on 30 datasets with various data distributions and different imbalance ratios.

Comparative analysis of whole-genome sequencing pipelines to minimize false negative findings

Comprehensive and accurate detection of variants from whole-genome sequencing (WGS) is a strong prerequisite for translational genomic medicine; however, low concordance between analytic pipelines is an outstanding challenge. We processed a European and an African WGS samples with 70 analytic pipelines comprising the combination of 7 short-read aligners and 10 variant calling algorithms (VCAs), and observed remarkable differences in the number of variants called by different pipelines (max/min ratio: 1.3~3.4). The similarity between variant call sets was more closely determined by VCAs rather than by short-read aligners. Remarkably, reported minor allele frequency had a substantial effect on concordance between pipelines (concordance rate ratio: 0.11~0.92; Wald tests, P < 0.001), entailing more discordant results for rare and novel variants. We compared the performance of analytic pipelines and pipeline ensembles using gold-standard variant call sets and the catalog of variants from the 1000 Genomes Project. Notably, a single pipeline using BWA-MEM and GATK-HaplotypeCaller performed comparable to the pipeline ensembles for ‘callable’ regions (~97%) of the human reference genome. While a single pipeline is capable of analyzing common variants in most genomic regions, our findings demonstrated the limitations and challenges in analyzing rare or novel variants, especially for non-European genomes.