Ensemble Accuracy Research Articles

Boosting Speech Recognition Performance: A Robust and Accurate Ensemble Method based on HMMs

Boosting Speech Recognition Performance: A Robust and Accurate Ensemble Method based on HMMs

Decoding time-resolved neural representations of orientation ensemble perception.

The visual system can compute summary statistics of several visual elements at a glance. Numerous studies have shown that an ensemble of different visual features can be perceived over 50-200 ms; however, the time point at which the visual system forms an accurate ensemble representation associated with an individual's perception remains unclear. This is mainly because most previous studies have not fully addressed time-resolved neural representations that occur during ensemble perception, particularly lacking quantification of the representational strength of ensembles and their correlation with behavior. Here, we conducted orientation ensemble discrimination tasks and electroencephalogram (EEG) recordings to decode orientation representations over time while human observers discriminated an average of multiple orientations. We modeled EEG signals as a linear sum of hypothetical orientation channel responses and inverted this model to quantify the representational strength of orientation ensemble. Our analysis using this inverted encoding model revealed stronger representations of the average orientation over 400-700 ms. We also correlated the orientation representation estimated from EEG signals with the perceived average orientation reported in the ensemble discrimination task with adjustment methods. We found that the estimated orientation at approximately 600-700 ms significantly correlated with the individual differences in perceived average orientation. These results suggest that although ensembles can be quickly and roughly computed, the visual system may gradually compute an orientation ensemble over several hundred milliseconds to achieve a more accurate ensemble representation.

Stacked ensemble deep learning for pancreas cancer classification using extreme gradient boosting.

Ensemble learning aims to improve prediction performance by combining several models or forecasts. However, how much and which ensemble learning techniques are useful in deep learning-based pipelines for pancreas computed tomography (CT) image classification is a challenge. Ensemble approaches are the most advanced solution to many machine learning problems. These techniques entail training multiple models and combining their predictions to improve the predictive performance of a single model. This article introduces the idea of Stacked Ensemble Deep Learning (SEDL), a pipeline for classifying pancreas CT medical images. The weak learners are Inception V3, VGG16, and ResNet34, and we employed a stacking ensemble. By combining the first-level predictions, an input train set for XGBoost, the ensemble model at the second level of prediction, is created. Extreme Gradient Boosting (XGBoost), employed as a strong learner, will make the final classification. Our findings showed that SEDL performed better, with a 98.8% ensemble accuracy, after some adjustments to the hyperparameters. The Cancer Imaging Archive (TCIA) public access dataset consists of 80 pancreas CT scans with a resolution of 512 * 512 pixels, from 53 male and 27 female subjects. A sample of two hundred and twenty-two images was used for training and testing data. We concluded that implementing the SEDL technique is an effective way to strengthen the robustness and increase the performance of the pipeline for classifying pancreas CT medical images. Interestingly, grouping like-minded or talented learners does not make a difference.

Seasonal forecasts of the rainy season onset over Africa: Preliminary results from the FOCUS-Africa project

Precipitation seasonality is the main factor controlling vegetation phenology in many tropical and subtropical regions. Anticipating the rain onset is of paramount importance for field preparation and seeding. This is of particular importance in various African countries that rely on agriculture as a main source of food, subsistence and income. In such countries, skilful and accurate onset forecasts could also inform early warning and early actions, such as aids logistics planning, for food security. Here, we assess the skill of the seasonal forecast data provided by the Copernicus Climate Change Service in predicting the rain onset over Africa. The skill, i.e. the accuracy of the seasonal forecasts simulation ensemble compared to the climatology, is computed in a probabilistic fashion by accounting for the frequencies of normal, early and late onsets predicted by the forecast system. We compute the skill using the hindcasts (forecast simulations conducted for the past) starting at the beginning of each month in the period 1993–2016. We detect the onset timing of the rainy season using a non-parametric method that accounts for double seasonality and is suitable for the specific time-window of the seasonal forecast simulations. We find positive skills in some key African agricultural regions some months in advance. Overall, the multi-model ensemble outperforms any individual model ensemble. We provide targeted recommendations to develop a useful climate service for the agricultural sector in Africa.

WEASEL 2.0: a random dilated dictionary transform for fast, accurate and memory constrained time series classification

A time series is a sequence of sequentially ordered real values in time. Time series classification (TSC) is the task of assigning a time series to one of a set of predefined classes, usually based on a model learned from examples. Dictionary-based methods for TSC rely on counting the frequency of certain patterns in time series and are important components of the currently most accurate TSC ensembles. One of the early dictionary-based methods was WEASEL, which at its time achieved SotA results while also being very fast. However, it is outperformed both in terms of speed and accuracy by other methods. Furthermore, its design leads to an unpredictably large memory footprint, making it inapplicable for many applications. In this paper, we present WEASEL 2.0, a complete overhaul of WEASEL based on two recent advancements in TSC: Dilation and ensembling of randomized hyper-parameter settings. These two techniques allow WEASEL 2.0 to work with a fixed-size memory footprint while at the same time improving accuracy. Compared to 15 other SotA methods on the UCR benchmark set, WEASEL 2.0 is significantly more accurate than other dictionary methods and not significantly worse than the currently best methods. Actually, it achieves the highest median accuracy over all data sets, and it performs best in 5 out of 12 problem classes. We thus believe that WEASEL 2.0 is a viable alternative for current TSC and also a potentially interesting input for future ensembles.

Completeness out of incompleteness: Inferences from regularities in imperfect information ensembles.

Handling imperfect information problems is fundamental to perception, learning, and decision-making. Ensemble perception may partially overcome imperfect information by providing global clues. However, if not all cluster elements are readily accessible, the observations required for computing statistics are incomplete. In this case, these elements' internal correlations (i.e., regularity) could serve as clues to elucidate the missing pieces. We thus investigated spatial regularity's role in ensemble perception under imperfect information situations created using partially occluded stimuli. In two experiments, we manipulated circle size (Experiment 1) and line orientation (Experiment 2) to linearly vary with its location; spatial regularity thus supplied clues for inferring information of the invisible parts. Participants estimated the mean of the targeted feature of the entire cluster, including visible and invisible parts. We observed robust biases toward the overall cluster in the estimations, implying the invisible parts were considered during ensemble perception. We proposed this effect could be understood as assessing evidence from visible parts to construct the missing parts. Experiment 3 employed a periodicity regularity to deter participants from using specific strategies, and consistent results were found. We then developed a generative model, the Regularity-Based Model, to simulate the inference process, which better captured the pattern of human outcomes than the comparative model. These findings indicate the visual system could use high-level structural information to infer scenes with incomplete information, thus producing more accurate ensemble representations. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

A Pointwise-Optimal Ensemble of Surrogate Models

AbstractThe ensemble of surrogate models is commonly used to replace computationally expensive simulations due to their superior prediction accuracy and robustness compared to individual models. This paper proposes a new pointwise ensemble of surrogate models, namely, a pointwise-optimal ensemble of surrogate models (POEMs). To address the limitations of the cross-validation (CV) error in evaluating the performance of regression surrogate models, this paper introduces the compensated cross-validation error, which is more reliable in selecting better individual surrogate models and improving the accuracy of surrogate model ensembles. To overcome the limitations of CV error in calculating pointwise weight factors, this paper designs and solves an optimization problem at training points to obtain corresponding pointwise weight factors. Additionally, this paper proposes two weight calculation methods to be applied in the interpolation and extrapolation regions, respectively, to reduce the instability of ensembles caused by extrapolation. Thirty test functions are employed to investigate the appropriate hyperparameters of POEM and the Friedman test is used to verify the rationality of the α value. The thirty test functions are also used to examine the performance of POEM and compare it with state-of-the-art ensemble surrogate models. Furthermore, POEM is applied to a large-aperture mirror holder optimization case to verify its superiority. The results demonstrate that POEM presents better accuracy and robustness than individual surrogates and other compared ensembles of surrogate models.

Adaptive neural network ensemble using prediction frequency

AbstractNeural network (NN) ensembles can reduce large prediction variance of NN and improve prediction accuracy. For highly non-linear problems with insufficient data set, the prediction accuracy of NN models becomes unstable, resulting in a decrease in the accuracy of ensembles. Therefore, this study proposes a prediction frequency-based ensemble that identifies core prediction values, which are core prediction members to be used in the ensemble and are expected to be concentrated near the true response. The prediction frequency-based ensemble classifies core prediction values ​​supported by multiple NN models ​​by conducting statistical analysis with a frequency distribution, which is a collection of prediction values ​​obtained from various NN models for a given prediction point. The prediction frequency-based ensemble searches for a range of prediction values that contains prediction values above a certain frequency, and thus the predictive performance can be improved by excluding prediction values with low accuracy ​​and coping with the uncertainty of the most frequent value. An adaptive sampling strategy that sequentially adds samples based on the core prediction variance calculated as the variance of the core prediction values is proposed to improve the predictive performance of the prediction frequency-based ensemble efficiently. Results of various case studies show that the prediction accuracy of the prediction frequency-based ensemble is higher than that of Kriging and other existing ensemble methods. In addition, the proposed adaptive sampling strategy effectively improves the predictive performance of the prediction frequency-based ensemble compared with the previously developed space-filling and prediction variance-based strategies.

IDDANet: An Input-Driven Dynamic Adaptive Network ensemble method for edge intelligence

Dynamic pruning and conditional convolution are crucial for reducing the computational complexity of convolutional neural networks, enabling their deployment on resource-limited devices. However, existing methods only focus on one aspect in isolation, ignoring the collaborative potential of devices and the similarity between input samples in edge cluster scenarios, making it difficult to eliminate redundancy in a given network structure. In order to reduce computation costs while enhancing model representation, we propose an input-driven dynamic adaptive network ensemble method, namely IDDANet. Our method is a two-stage model synthesis strategy that first dynamically prunes the network by using sample complexity and similarity to obtain a lightweight model and a set of input-dependent combination coefficients. We then combine the convolution parameters of multiple models to synthesize a more accurate ensemble model. As demonstrated through extensive experiments on different datasets, our method can significantly reduce computation complexity while improving model representation ability compared to state-of-the-art methods. For example, our method can reduce 58.5% of the FLOPs of ResNet-18, while only having a 0.1% Top-1 accuracy drop on CIFAR-10.

Ensemble determination by NMR data deconvolution.

Nuclear magnetic resonance (NMR)is the spectroscopic technique of choice for determining molecular conformations in solution at atomic resolution. As solution NMR spectra are rich in structural and dynamic information, the way in whichthe data should be acquired and handled to deliver accurate ensembles is not trivial. This Review provides a guide to the NMRexperiment selection and parametrization process, the generation of viable theoretical conformer pools and the deconvolution of time-averaged NMR data into a conformer ensemble that accurately represents a flexible molecule in solution. In addition to reviewing the key elements of solution ensemble determination of flexible mid-sized molecules, the feasibility and pitfalls of data deconvolution are discussed with a comparison of the performance of representative algorithms.

Quantitative Paramagnetic NMR-Based Analysis of Protein Orientational Dynamics on Membranes: Dissecting the KRas4B-Membrane Interactions.

Peripheral membrane proteins can adopt distinct orientations on the surfaces of lipid bilayers that are often short-lived and challenging to characterize by conventional experimental methods. Here we describe a robust approach for mapping protein orientational landscapes through quantitative interpretation of paramagnetic relaxation enhancement (PRE) data arising from membrane mimetics with spin-labeled lipids. Theoretical analysis, followed by experimental verification, reveals insights into the distinct properties of the PRE observables that are generally distorted in the case of stably membrane-anchored proteins. To suppress the artifacts, we demonstrate that undistorted Γ2 values can be obtained via transient membrane anchoring, based on which a computational framework is established for deriving accurate orientational ensembles obeying Boltzmann statistics. Application of the approach to KRas4B, a classical peripheral membrane protein whose orientations are critical for its functions and drug design, reveals four distinct orientational states that are close but not identical to those reported previously. Similar orientations are also found for a truncated KRas4B without the hypervariable region (HVR) that can sample a broader range of orientations, suggesting a confinement role of the HVR geometrically prohibiting severe tilting. Comparison of the KRas4B Γ2 rates measured using nanodiscs containing different types of anionic lipids reveals identical Γ2 patterns for the G-domain but different ones for the HVR, indicating only the latter is able to selectively interact with anionic lipids.

Cell-type classification of cancer single-cell RNA-seq data using the Subsemble ensemble-based machine learning classifier

Background The advent of single-cell RNA sequencing (scRNA-seq) has provided a high-resolution overview of the cellular heterogeneity of different tissue types. Manual cell type annotation of gene expression datasets remains a useful but time-intensive task. Ensemble machine learning methods leverage the predictive power of multiple classifiers and can be applied to classify high-dimensional gene expression data. Here, we present a novel application of the Subsemble supervised ensemble machine learning classifier used to classify novel cells with known cell type labels using gene expression data. Methods First, we tested the classification performance of different pre-processing steps used to normalize and upsample the training dataset for the Subsemble using a colorectal cancer dataset. Second, we conducted a cross-validated performance benchmark of the Subsemble classifier compared to nine other cell type classification methods across five metrics tested, using an acute myeloid leukemia dataset. Third, we conducted a comparative performance benchmark of the Subsemble classifier using a patient-based leave-one-out cross-validation scheme. Rank normalized scores were calculated for each classifier to aggregate performance across multiple metrics. Results The Subsemble classifier performed best when trained on a dataset that was log-transformed then upsampled to generate balanced class distributions. The Subsemble classifier was consistently the top-ranked classifier across five classification performance metrics compared to the nine other baseline classifiers and showed an improvement in performance as the training dataset increased. When tested using the patient-based leave-one-out cross-validation scheme, the Subsemble was the top-ranked classifier based on rank normalized scores. Conclusions Our proof-of-concept study showed that the Subsemble classifier can be used to accurately predict known cell type labels from single-cell gene expression data. The top-ranked classification performance of the Subsemble across two validation datasets, two cross-validation schemes, and five performance metrics motivates future development of accurate ensemble classifiers of scRNA-seq datasets.

Optimal model selection for k-nearest neighbours ensemble via sub-bagging and sub-sampling with feature weighting

This paper proposes two novel approaches based on feature weighting and model selection for building more accurate kNN ensembles. The first approach identifies the nearest observations using a feature weighting scheme concerning the response variable via support vectors. A randomly selected subset of features is used for the feature weighting and model construction. After building a sufficiently large number of base models on bootstrap samples, a subset of the models is selected based on out-of-bag prediction error for the final ensemble. The second approach builds base learners build on random subsamples instead of bootstrap samples with a random subset of features. The method uses feature weighting while building the models. The remaining observations from each sample are used to assess the corresponding base learner and select a subset of the models for the final ensemble. The suggested ensemble methods are assessed on 12 benchmark datasets against other classical methods, including kNN-based models. The analyses reveal that the proposed methods are often better than the others.

Model ensembles of ecosystem services fill global certainty and capacity gaps

Sustaining ecosystem services (ES) critical to human well-being is hindered by many practitioners lacking access to ES models ("the capacity gap") or knowledge of the accuracy of available models ("the certainty gap"), especially in the world's poorer regions. We developed ensembles of multiple models at an unprecedented global scale for five ES of high policy relevance. Ensembles were 2 to 14% more accurate than individual models. Ensemble accuracy was not correlated with proxies for research capacity, indicating that accuracy is distributed equitably across the globe and that countries less able to research ES suffer no accuracy penalty. By making these ES ensembles and associated accuracy estimates freely available, we provide globally consistent ES information that can support policy and decision-making in regions with low data availability or low capacity for implementing complex ES models. Thus, we hope to reduce the capacity and certainty gaps impeding local- to global-scale movement toward ES sustainability.

Data-Driven Prediction of Janus/Core-Shell Morphology in Polymer Particles: A Machine-Learning Approach.

The majority of research on Janus particles prepared by solvent evaporation-induced phase separation technique uses models based on interfacial tension or free energy to predict Janus/core-shell morphology. Data-driven predictions, in contrast, utilize multiple samples to identify patterns and outliers. Using machine-learning algorithms and explainable artificial intelligence (XAI) analysis, we developed a model based on a 200-instance data set to predict particle morphology. As model features, simplified molecular input line entry system syntax identifies explanatory variables, including cohesive energy density, molar volume, the Flory-Huggins interaction parameter of polymers, and the solvent solubility parameter. Our most accurate ensemble classifiers predict morphology with an accuracy of 90%. In addition, we employ innovative XAI tools to interpret system behavior, suggesting phase-separated morphology to be most affected by solvent solubility, polymer cohesive energy difference, and blend composition. While polymers with cohesive energy densities above a certain threshold favor the core-shell structure, systems with weak intermolecular interactions favor the Janus structure. The correlation between molar volume and morphology suggests that increasing the size of polymer repeating units favors Janus particles. Additionally, the Janus structure is preferred when the Flory-Huggins interaction parameter exceeds 0.4. XAI analysis introduces feature values that generate the thermodynamically low driving force of phase separation, resulting in kinetically stable morphologies as opposed to thermodynamically stable ones. The Shapley plots of this study also reveal novel methods for creating Janus or core-shell particles based on solvent evaporation-induced phase separation by selecting feature values that strongly favor a given morphology.

Ensemble Methods in Customer Churn Prediction: A Comparative Analysis of the State-of-the-Art

In the past several single classifiers, homogeneous and heterogeneous ensembles have been proposed to detect the customers who are most likely to churn. Despite the popularity and accuracy of heterogeneous ensembles in various domains, customer churn prediction models have not yet been picked up. Moreover, there are other developments in the performance evaluation and model comparison level that have not been introduced in a systematic way. Therefore, the aim of this study is to perform a large scale benchmark study in customer churn prediction implementing these novel methods. To do so, we benchmark 33 classifiers, including 6 single classifiers, 14 homogeneous, and 13 heterogeneous ensembles across 11 datasets. Our findings indicate that heterogeneous ensembles are consistently ranked higher than homogeneous ensembles and single classifiers. It is observed that a heterogeneous ensemble with simulated annealing classifier selection is ranked the highest in terms of AUC and expected maximum profits. For accuracy, F1 measure and top-decile lift, a heterogenous ensemble optimized by non-negative binomial likelihood, and a stacked heterogeneous ensemble are, respectively, the top ranked classifiers. Our study contributes to the literature by being the first to include such an extensive set of classifiers, performance metrics, and statistical tests in a benchmark study of customer churn.

Ensemble rainfall–runoff and inundation simulations using 100 and 1000 member rainfalls by 4D LETKF on the Kumagawa River flooding 2020

This paper presents the 1000 ensemble flood simulations using ensemble rainfalls simulated by 4D LETKF. The number of ensemble rainfall members is large as 1000 compared to the operational rainfall products of two-digit numbers to avoid sampling errors in the three-dimensional meteorological simulation based on chaotic theory. Using the large data set, 1000 ensemble rainfall–runoff for dam catchments and high-resolution inundation simulations of large area are carried out focusing on the Kumagawa river catchment. Herewith, the comparisons were carried out with 21-member ensemble rainfalls of an operational forecast by Japan Meteorological Agency and 100-member 4D-LETKF ensemble rainfalls simulated independent of 1000-member 4D-LETKF. At the same time, the accuracy of selective 100-member ensembles out of 1000 members is investigated. As a result, although many previous research works show a large number of ensemble simulations are necessary for three-dimensional meteorological field, the number could be reduced in the catchment-average rainfall–runoff and 2.5-dimensional inundation simulations given that the rainfall prediction has a certain level of accuracy since improving the discharge prediction accuracy with lower dimension is sometimes possible by adjusting the horizontally/vertically integrated model parameters determined by topography and soil characteristics in advance against the observed rainfall. Also, the 1000 ensembles could be classified into several patterns in horizontally accumulated 2D rainfall field. Likewise, the flood flow moves toward the low elevation area and river; thus, the resultant 2.5-dimensional flood field does not show much variety as three-dimensional meteorological simulation. The paper summarizes these studies.

Obtention of accurate Abeta-Zn(II) ensembles from molecular modelling and Bayesian/maximum entropy reweighing.

Amyloid fibrils are stable forms of misfolded proteins associated with numerous neurodegenerative diseases. Among these, Alzheimer's disease may be the most prevalent, with over 55 million dementia cases reported by the World Health Organisation in 2021. The molecular origin of Alzheimer's is linked to amyloid fibril formation by misfolded Aβ-peptide (Aβ). These fibrils form aggregates that are stabilised by the presence of Zn(II) cations. Many possible Aβ-Zn(II) structures have been proposed experimentally (see Figure 1) where Zn(II) adopts different coordinations. To obtain a representative and accurate ensemble of Aβ-Zn(II) systems is challenging as standard Zn(II) models employed by classical force fields, i.e., nonbonded models, tend to fail. These models heavily favour octahedral coordination although it has been reported that tetrahedral coordination is favoured in the protein environment. In this work, we will employ bonded models to accurately capture Aβ-Zn(II) coordination and extract precise conformational dynamics of the systems. Additionally, we use Bayesian/Maximum Entropy reweighing to obtain a more accurate ensemble description of these systems.

A Bayesian inference approach to determining structural ensembles using cryo-EM and molecular dynamics.

Advances in cryo-EM have led to an enormous increase in the number of high-resolution structures solved and deposited within the Protein Data Bank. However, the way protein models are represented has not kept pace as static single structure models continue to be widely accepted as accurate depictions of proteins despite their underlying conformational heterogeneity and dynamics. While advances in image processing allows for the construction of multiple distinct conformational states directly from raw 2D images, functionally important highly dynamic regions of proteins, including highly-dynamic side chains, are often averaged out resulting in low resolution and an inability to accurately determine local structures. Here we present ‘EMMIVox’, a Bayesian inference approach to determining structural ensembles of biological entities by combining cryo-EM data with molecular dynamics (MD) simulations. EMMIVox, which is able to automatically detect and downweigh noisy experimental data, calculates accurate structural ensembles of proteins and protein complexes including any lipids, small-molecules and ordered water present in experimental maps. Using EMMIVox, we generated single structure models for a variety of high-resolution cryo-EM maps (1.9 Å - 3.5 Å), including membrane proteins and ligand-protein complexes. EMMIVox models outperformed deposited structures in terms of stereochemical descriptors (MolProbity and Clashcore) as well as metrics that quantify how well the model fits the density map (map-model cross correlation and EMRinger). Finally, we determined the structural ensembles of the type 1a tau filament (1.9 Å) and the SPP1 bacteriophage (4 Å) in detail, demonstrating the accuracy of our structural ensemble compared to standard MD simulations. Inference of structural ensembles and their representative populations will have wide ranging applications in structural biology and drug discovery. EMMIVox is available in the Integrative Structural and Dynamical Biology module of the open-source, freely-available PLUMED library (www.plumed.org).

Computing the Ensemble Spread From Deterministic Weather Predictions Using Conditional Generative Adversarial Networks

AbstractEnsemble prediction systems are an invaluable tool for weather forecasting. Practically, ensemble predictions are obtained by running several perturbations of the deterministic control forecast. However, ensemble prediction is associated with a high computational cost and often involves statistical post‐processing steps to improve its quality. Here we propose to use deep‐learning‐based algorithms to learn the statistical properties of an ensemble prediction system, the ensemble spread, given only the deterministic control forecast. Thus, once trained, the costly ensemble prediction system will not be needed anymore to obtain future ensemble forecasts, and the statistical properties of the ensemble can be derived from a single deterministic forecast. We adapt the classical pix2pix architecture to a three‐dimensional model and train them against several years of operational (ensemble) weather forecasts for the 500 hPa geopotential height. The results demonstrate that the trained models indeed allow obtaining a highly accurate ensemble spread from the control forecast only.