Diverse Ensemble Research Articles

HDX-MS guided ensemble reweighting approach reveals cryptic drug binding sites in the cytoplasmic heme binding protein, PhuS.

Hydrogen-deuterium exchange mass spectrometry (HDX-MS) is a valuable approach to probe protein structure and dynamics. However, interpretation of HDX-MS data is often qualitative and typically limited to peptide-level resolution. Ongoing efforts aim to integrate experimental HDX-MS data with molecular dynamics (MD) approaches, to provide a high-resolution interpretation of the HDX-MS data. One such approach is an HDX-MS based maximum-entropy reweighting approach (HDXer) to reweight computationally generated ensembles to HDX-MS data. Here, HDXer is used in conjunction with enhanced MD (eMD) to characterize the structural dynamics of PhuS. PhuS is a cytoplasmic heme binding protein from P. aeruginosa which shuttles exogenous heme to Heme Oxygenase (HemO) for degradation and is essential for virulence. Through the implementation of a workflow that integrates eMD, HDXer, and dimensional reduction, we had previously modeled the native state ensemble of apo-PhuS revealing a locally diverse apo-PhuS ensemble characterized by large rearrangements of helices surrounding the heme binding. Here, the representative structures from the modeled native ensemble were used as starting points for computer aided drug design (CADD). Implementing the Site Identification by Ligand Competitive Saturation (SILCS) CADD method approximately 1.3 million compounds were screened for each representative structure as well as the published apo PhuS crystal structure, yielding a library of 50 chemically representative candidate compounds. The candidate compounds were then screened using a fluorescence quenching assay. From these assays, several hits were confirmed to bind apo PhuS at nanomolar binding affinities. Most significantly, none of the compounds were derived from CADD using the published apo PhuS crystal structure, demonstrating the utility of HDX-MS guiding ensemble reweighting in revealing cryptic drug binding site in CADD approaches.

Idpconformergenerator: A flexible software suite for sampling the conformational space of disordered protein states.

In the efforts to identify structure-function relationships of protein machinery, computational methods have developed rapidly as tools to assist the scientific community in both structural prediction as well as functional relationships in context of biological pathways. Although solving high resolution structural information has been clear for stable folded macromolecules, similar structure-function insight is more difficult to obtain for highly dynamic systems such as intrinsically disordered proteins (IDPs) which must be described as structural ensembles. Here, we present IDPConformerGenerator, a flexible, modular open-source software platform for generating and analyzing large diverse ensembles of disordered protein states. IDPConformerGenerator builds conformers that obey geometric, steric, and other physical restraints on the input sequence by sampling backbone phi (φ), psi (ψ), and omega (ω) torsion angles of relevant sequence fragments from loops and secondary structure elements extracted from folded protein structures in the RCSB Protein Data Bank. All-atom sidechains can be added using robust Monte Carlo algorithms with expanded rotamer libraries. IDPConformerGenerator has many user-configurable options enabling variable fractional sampling of secondary structures, supports Bayesian models for assessing the agreement of IDP ensembles for consistency with experimental data, and introduces a machine learning approach to transform between internal and Cartesian coordinates with reduced error. IDPConformerGenerator will facilitate the characterization of disordered proteins to provide structural insights into these states that have key biological functions. Ultimately, a better structural understanding of disordered protein ensembles can open up avenues for future research on pharmaceuticals targeting diseases mediated by IDPs.

Improving the Results in Credit Scoring by Increasing Diversity in Ensembles of Classifiers

Improving the Results in Credit Scoring by Increasing Diversity in Ensembles of Classifiers

Combining Deep Convolutional Neural Networks With Stochastic Ensemble Weight Optimization for Facial Expression Recognition in the Wild

Although recent emotion recognition methods (based on facial expression cues) achieve excellent performance in controlled scenarios, the recognition of emotion in the wild remains a challenging problem because of occlusion, large head poses, illumination variations, etc. Recent advances in deep learning show that combining an ensemble of deep learning models can considerably outperform the approach of using only a single deep learning model for challenging recognition problems. This paper presents a novel ensemble deep learning method, “deep convolutional neural network (DCNN) ensemble classifier”, for improved facial expression recognition (FER) in the wild. Our proposed DCNN ensemble classifier is novel in terms of the following aspects: (1) the process of finding ensemble weights for combining DCNN decision outputs is formulated as a stochastic optimization problem (via simulated annealing) in which the energy to be minimized represents the generalized (test) classification error of the DCNN ensemble and (2) for the creation of DCNN ensemble members, we propose the combined use of different types of face representations and bagging (T. G. Dietterich, 2000), which is quite useful in increasing the diversity of the DCNN ensemble. Extensive and comparative experiments on three wild FER datasets, namely FER2013, SFEW2.0, and RAF-DB, show that the proposed DCNN ensemble classifier achieves competitive FER performances when compared with other recently developed methods—76.69%, 58.68%, and 87.13% of FER accuracy under the FER2013, SFEW2.0, and RAF-DB evaluation protocols, respectively.

Diverse Instrument Symphonic Ensembles as a Strategy for Sustainable Innovation

Diverse Instrument Symphonic Ensembles as a Strategy for Sustainable Innovation

Multi-Method Stacked Feature Selection Approach based IDS for IoT Networks

In an era marked by the proliferation of interconnected devices, the Internet of Things (IoT) has emerged as a revolutionary technological paradigm. IoT networks encompass a vast array of devices, from smart appliances to industrial sensors, revolutionizing industries and everyday life. However, this ubiquitous connectivity has ushered in a new frontier of security challenges, necessitating the deployment of robust Intrusion Detection Systems (IDS). This paper presents a pioneering Multi-Method Stacked Feature Selection (M2SFS) approach-based IDS tailored explicitly for IoT networks. By orchestrating a diverse ensemble of feature selection techniques, the M2SFS framework adeptly curates the feature set derived from the CICIDS 2017 dataset. This comprehensive selection strategy optimally reduces dimensionality, mitigating the resource constraints inherent to IoT ecosystems. The proposed IDS, rooted in the M2SFS approach, demonstrates exceptional accuracy and efficiency in detecting anomalous network behaviors. By harnessing the collective strength of stacked feature selection, this IDS capitalizes on the synergistic potential of multiple feature selection methods. The result is an IDS uniquely adapted to the intricacies of IoT networks, effectively fortifying their security posture. Through rigorous experimentation using the CICIDS 2017 dataset, the proposed approach affirms the superiority of the M2SFS-based IDS over conventional methods. This study underscores the pressing need for IDS in IoT environments and highlights the promise of our innovative approach in safeguarding IoT networks against emerging threats.

Predictive uncertainty estimation for out-of-distribution detection in digital pathology.

Machine learning model deployment in clinical practice demands real-time risk assessment to identify situations in which the model is uncertain. Once deployed, models should be accurate for classes seen during training while providing informative estimates of uncertainty to flag abnormalities and unseen classes for further analysis. Although recent developments in uncertainty estimation have resulted in an increasing number of methods, a rigorous empirical evaluation of their performance on large-scale digital pathology datasets is lacking. This work provides a benchmark for evaluating prevalent methods on multiple datasets by comparing the uncertainty estimates on both in-distribution and realistic near and far out-of-distribution (OOD) data on a whole-slide level. To this end, we aggregate uncertainty values from patch-based classifiers to whole-slide level uncertainty scores. We show that results found in classical computer vision benchmarks do not always translate to the medical imaging setting. Specifically, we demonstrate that deep ensembles perform best at detecting far-OOD data but can be outperformed on a more challenging near-OOD detection task by multi-head ensembles trained for optimal ensemble diversity. Furthermore, we demonstrate the harmful impact OOD data can have on the performance of deployed machine learning models. Overall, we show that uncertainty estimates can be used to discriminate in-distribution from OOD data with high AUC scores. Still, model deployment might require careful tuning based on prior knowledge of prospective OOD data.

Sparse projection infinite selection ensemble for imbalanced classification

Imbalanced datasets pose frequent and challenging problems to many real-world applications. Classification models are often biased towards the majority class when learning from class-imbalanced data. Typical imbalanced learning (IL) approaches, e.g., SMOTE, AdaCost, and Cascade, often suffer from poor performance in complex tasks where class overlapping or a high imbalance ratio occurs. In this paper, we systematically investigate the IL problem and propose a novel framework named sparse projection infinite selection ensemble (SPISE). SPISE iteratively resamples balanced subsets and combines the classifiers trained on these subsets for imbalanced classification. The diversity of classifier ensembles and the similarity between the subsets and the whole dataset are considered in this process. Specifically, we present a graph-based approach named infinite subset selection to adaptively sample diverse and similar subsets. Additionally, a random sparse projection is combined with feature selection at the beginning of each iteration to augment the training features and enhance the diversity of the generated subsets. SPISE can be easily adapted to most existing classifiers (e.g., support vector machine and random forest) to boost their performance for IL. Quantitative experiments on 26 imbalanced benchmark datasets substantiate the effectiveness and superiority of the proposed model compared with other popular approaches.

Ensemble Generation for Linear and Cyclic Peptides Using a Reservoir Replica Exchange Molecular Dynamics Implementation in GROMACS.

The profile of shapes presented by a cyclic peptide modulates its therapeutic efficacy and is represented by the ensemble of its sampled conformations. Although some algorithms excel at creating a diverse ensemble of cyclic peptide conformations, they seldom address the entropic contribution of flexible conformations and often have significant practical difficulty producing an ensemble with converged and reliable thermodynamic properties. In this study, an accelerated molecular dynamics (MD) method, namely, reservoir replica exchange MD (R-REMD or Res-REMD), was implemented in GROMACS ver. 4.6.7 and benchmarked on two small cyclic peptide model systems: a cyclized furin cleavage site of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) spike (cyclo-(CGPRRARSG)) and oxytocin (disulfide-bonded CYIQNCPLG). Additionally, we also benchmarked Res-REMD on alanine dipeptide and Trpzip2 to demonstrate its validity and efficiency over REMD. For Trpzip2, Res-REMD coupled with an umbrella-sampling-derived reservoir generated similar folded fractions as regular REMD but on a much faster time scale. For cyclic peptides, Res-REMD appeared to be marginally faster than REMD in ensemble generation. Finally, Res-REMD was more effective in sampling rare events such as trans to cis peptide bond isomerization. We provide a GitHub page with the modified GROMACS source code for running Res-REMD at https://github.com/PlotkinLab/Reservoir-REMD.

Optimum Bayesian thresholds for rebalanced classification problems using class-switching ensembles

Asymmetric label switching is an effective and principled method for creating a diverse ensemble of learners for imbalanced classification problems. This technique can be combined with other rebalancing mechanisms, such as those based on cost policies or class proportion modifications. In this study, and under the Bayesian theory framework, we specify the optimal decision thresholds for the combination of these mechanisms. In addition, we propose using a gating network to aggregate the learners contributions as an additional mechanism to improve the overall performance of the system.

Characterization of ensemble generation strategies: Application to three illustrative examples of Mediterranean high-impact weather

Predictability of high-impact weather constitutes a major challenge. In this study, we contribute to this relevant topic by analyzing the characteristics of multiple ensemble prediction systems aimed at sampling initial and/or model uncertainties for three exemplary heavy precipitation systems that occurred over the western Mediterranean. In particular, initial condition perturbations are generated by (i) directly downscaling the ensemble members from a global ensemble and (ii) creating perturbations by sampling a wide range of scales using an adaptation of the breeding methodology. Model error is sampled by applying stochastic perturbations to physical parameterizations and to microphysics parameters. A positive impact in terms of ensemble diversity is obtained when initial condition perturbations across a broad range of scales are applied, especially at low levels and for cases in which local factors are more relevant. Regarding stochastic model perturbations, they display very localized perturbations with low amplitude, which are insufficient to capture extreme scenarios, except when local factors play a dominant role. However, when combined with initial condition perturbations, they generally increase diversity and perturbation amplitude over areas characterized by deep moist convection. This outcome exhibits the positive impact of sampling both uncertainty sources, even in the extremely nonlinear regime that defines convective-scale phenomena.

OLP++: An online local classifier for high dimensional data

Ensemble diversity is an important characteristic of Multiple Classifier Systems (MCS), which aim at improving the overall performance of a classification system by combining the response of several models. While diversity may be introduced through various manipulations at the data level and the model level, some MCSs incorporate local information in order to increase it and/or take advantage of it, based on the idea that the different classifiers in the ensemble may have expertise in distinct areas of the feature space.11Code available at: https://github.com/marianaasouza/olp_plusplus.Following a similar reasoning, we introduced in a previous work an ensemble method which produces in test time a few experts in the local region where each given query sample is located. These local experts, which are generated with slightly differing views of the target area, are then used to label the corresponding unknown instance. While the framework was shown to perform well especially over imbalanced problems, the locality definition in the method is based on the nearest neighbors rule and Euclidian distance, as is the case of various local-based ensembles, which may suffer from the effects of the curse of dimensionality over high dimensional problems. Thus, in this work, we propose a local ensemble method in which we leverage the data partitions given by decision trees for locality definition. More specifically, the partitions defined at different levels of the decision path that a given query instance traverses in the tree(s) are used as the regions over which the local experts are produced. By using different node levels from the path, each classifier in the local pool has a moderately distinct view of the target region without resorting to a dissimilarity metric, which might be susceptible to high dimensional spaces. Experimental results over 39 high dimensional problems showed that the proposed approach was significantly superior to our previous, distance-based framework in balanced accuracy rate. Compared to other six local-based ensemble methods, including dynamic selection and weighting schemes, the proposed method achieved competitive results, outperforming the random forest baseline and two state-of-the-art dynamic ensemble selection techniques.

Artificial intelligence for load forecasting: A stacking learning approach based on ensemble diversity regularization

State-of-art artificial intelligence (AI) has made great breakthroughs in various industries. Ensemble learning mixed with various predictors provides a considerable solution for electric load forecasting in power system. In our paper, the generalization error of ensemble learning is statistically decomposed to exhibit the significance of base-learner diversity. A diversity regularized Stacking learning approach is proposed to solve the electric load forecasting issue. In our model, the input features are comprehensively selected by various tree-based embedded methods to understand the feature contribution. The robust candidate base-learners are extracted from sub-model pool depending on diversity regularization besides the individual learning capability. Mutual information theory and hierarchical clustering quantitatively assess the dissimilarity degree among base-leaners by exploiting error distribution. The Stacking ensemble framework is utilized to avoid the over-fitting occurrence by employing leave-one-out data splitting procedure for raw dataset block. At last, various cases from different time horizons or geographical scopes are deployed to verify the validity of the model. The case shows that the diversity regularized Stacking learning has better prediction performance compared with the traditional ensemble model or single model. Load forecasting results become more accurate and stable when elaborately selecting base-learners portfolio.

A molecular evolution algorithm for ligand design in DOCK.

As a complement to virtual screening, de novo design of small molecules is an alternative approach for identifying potential drug candidates. Here, we present a new 3D genetic algorithm to evolve molecules through breeding, mutation, fitness pressure, and selection. The method, termed DOCK_GA, builds upon and leverages powerful sampling, scoring, and searching routines previously implemented into DOCK6. Three primary experiments were used during development: Single-molecule evolution evaluated three selection methods (elitism, tournament, and roulette), in four clinically relevant systems, in terms of mutation type and crossover success, chemical properties, ensemble diversity, and fitness convergence, among others. Large scale benchmarking assessed performance across 651 different protein-ligand systems. Ensemble-based evolution demonstrated using multiple inhibitors simultaneously to seed growth in a SARS-CoV-2 target. Key takeaways include: (1) The algorithm is robust as demonstrated by the successful evolution of molecules across a large diverse dataset. (2) Users have flexibility with regards to parent input, selection method, fitness function, and molecular descriptors. (3) The program is straightforward to run and only requires a single executable and input file at run-time. (4) The elitism selection method yields more tightly clustered molecules in terms of 2D/3D similarity, with more favorable fitness, followed by tournament and roulette.

Prediction and uncertainty quantification of the diffuse sound absorption of finite absorbers

This contribution presents a numerical approach to predict diffuse sound absorption. Conventionally, this is done by considering an infinite absorber coupled to an acoustic halfspace. However, the diffuse absorption coefficient for a finite absorber can be quite different from the corresponding infinite-size value due to what is referred to in literature as the size or edge effect. A finite size correction has been developed previously, but it is only applicable to homogeneous absorbers, neglects boundary conditions, and is based on a computationally costly numerical integration. This contribution presents an alternative approach which overcomes these drawbacks. In this approach, a deterministic model of the absorber, e.g., constructed with the finite element or modal transfer matrix method, is coupled to a diffuse model of the room through the diffuse field reciprocity relation. The theoretical uncertainty on this diffuse sound absorption that is inherent in the diffuse field assumption can also be quantified, or more precisely, the variance of the sound absorption that can be theoretically expected across a diverse ensemble of rooms of the same volume. It is demonstrated that the diffuse absorption coefficient corresponds to the average of the absorption coefficient across an ensemble of rooms, implying that the former is of practical use even at low frequencies. The methodology is experimentally validated for porous and membrane absorbers.

IDPConformerGenerator: A Flexible Software Suite for Sampling the Conformational Space of Disordered Protein States.

The power of structural information for informing biological mechanisms is clear for stable folded macromolecules, but similar structure–function insight is more difficult to obtain for highly dynamic systems such as intrinsically disordered proteins (IDPs) which must be described as structural ensembles. Here, we present IDPConformerGenerator, a flexible, modular open-source software platform for generating large and diverse ensembles of disordered protein states that builds conformers that obey geometric, steric, and other physical restraints on the input sequence. IDPConformerGenerator samples backbone phi (φ), psi (ψ), and omega (ω) torsion angles of relevant sequence fragments from loops and secondary structure elements extracted from folded protein structures in the RCSB Protein Data Bank and builds side chains from robust Monte Carlo algorithms using expanded rotamer libraries. IDPConformerGenerator has many user-defined options enabling variable fractional sampling of secondary structures, supports Bayesian models for assessing the agreement of IDP ensembles for consistency with experimental data, and introduces a machine learning approach to transform between internal and Cartesian coordinates with reduced error. IDPConformerGenerator will facilitate the characterization of disordered proteins to ultimately provide structural insights into these states that have key biological functions.

Automated Hashtag Hierarchy Generation Using Community Detection and the Shannon Diversity Index, with Applications to Twitter and Parler

Developing semantic hierarchies from user-created hashtags in social media can provide useful organizational structure to large volumes of data. However, construction of these hierarchies is difficult using established ontologies (e.g. WordNet [C. Fellbaum (ed.), WordNet: An Electronic Lexical Database (MIT Press, Cambridge, MA, 1998)]) due to the differences in the semantic and pragmatic use of words versus hashtags in social media. While alternative construction methods based on hashtag frequency are relatively straightforward, these methods can be susceptible to the dynamic nature of social media, such as hashtags with brief surges in popularity. We drew inspiration from the ecologically based Shannon Diversity Index (SDI) [J. L. Wilhm, Use of biomass units in Shannon’s formula, Ecology 49(1) (1968) 153–156] to create a more representative and resilient method of semantic hierarchy construction that relies upon network-based community detection and a novel, entropy-based ensemble diversity index (EDI) score. The EDI quantifies the contextual diversity of each hashtag, resulting in thousands of semantically related groups of hashtags organized along a general-to-specific spectrum. Through an application of EDI to social media data (Twitter and Parler) and a comparison of our results to prior approaches, we demonstrate our method’s ability to create semantically consistent hierarchies that can be flexibly applied and adapted to a range of use cases.

AlphaFold, Artificial Intelligence (AI), and Allostery.

AlphaFold has burst into our lives. A powerful algorithm that underscores the strength of biological sequence data and artificial intelligence (AI). AlphaFold has appended projects and research directions. The database it has been creating promises an untold number of applications with vast potential impacts that are still difficult to surmise. AI approaches can revolutionize personalized treatments and usher in better-informed clinical trials. They promise to make giant leaps toward reshaping and revamping drug discovery strategies, selecting and prioritizing combinations of drug targets. Here, we briefly overview AI in structural biology, including in molecular dynamics simulations and prediction of microbiota–human protein–protein interactions. We highlight the advancements accomplished by the deep-learning-powered AlphaFold in protein structure prediction and their powerful impact on the life sciences. At the same time, AlphaFold does not resolve the decades-long protein folding challenge, nor does it identify the folding pathways. The models that AlphaFold provides do not capture conformational mechanisms like frustration and allostery, which are rooted in ensembles, and controlled by their dynamic distributions. Allostery and signaling are properties of populations. AlphaFold also does not generate ensembles of intrinsically disordered proteins and regions, instead describing them by their low structural probabilities. Since AlphaFold generates single ranked structures, rather than conformational ensembles, it cannot elucidate the mechanisms of allosteric activating driver hotspot mutations nor of allosteric drug resistance. However, by capturing key features, deep learning techniques can use the single predicted conformation as the basis for generating a diverse ensemble.

Prediction of spontaneous imbibition in porous media using deep and ensemble learning techniques

Spontaneous imbibition (SI), which is a process of displacing a nonwetting fluid by a wetting fluid in porous media, is of critical importance to hydrocarbon recovery from fractured reservoirs. In the present study, we utilize deep and ensemble learning techniques to predict SI recovery in porous media under different boundary conditions including All-Faces-Open (AFO), One-End-Open (OEO), Two-Ends-Open (TEO), and Two-Ends-Closed (TEC). An extensive experimental dataset reported in literature representing a multiplicity of non-wetting fluid recovery-time curves was used in our analysis. The prepared dataset was used to learn diverse ensemble and deep learning algorithms of Random Forest (RF), Gradient Boosting Machine (GBM), Extreme Gradient Boosting (XGBoost), Light Gradient Boosting Machine (LightGBM), Voting Regressor (VR), Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), and Gated Recurrent Unit (GRU). The training procedure provided us with robust models linking the SI recovery to the absolute permeability (k), porosity (ϕ), characteristic length (Lc), interfacial tension (σ), wetting-phase viscosity (μw), non-wetting-phase viscosity (μnw), and imbibition time (t). To evaluate and validate the models’ prediction, we used two well-established approaches: (i) 10-fold cross-validation and (ii) predicting the SI behavior of a set of unseen data excluded from the model training. Our results illustrate an excellent performance of deep and ensemble learning techniques for prediction of SI with the test RMSE values of 4.642, 4.088, 4.524, 3.933, 3.875, 3.975, 4.513, and 4.807 percent for RF, GBM, XGBoost, LightGBM, VR, CNN, LSTM, and GRU models, respectively. The models have significant benefits in terms of accuracy and generality. Furthermore, they alleviate the sophistications associated with tuning the traditional correlation functions. The findings of this study can pave the road toward a more comprehensive characterization of fluid flow in porous materials which is important to a wide range of environmental and energy-related challenges such as contaminant transport, soil remediation, and enhanced oil recovery.

The Bacillaceae-1 RNA motif comprises two distinct classes

Non-coding RNAs are key regulatory players in bacteria. Many computationally predicted non-coding RNAs, however, lack functional associations. An example is the Bacillaceae-1 RNA motif, whose Rfam model consists of two hairpin loops. We find the motif conserved in nine of 13 non-pathogenic strains of the genus Bacillus but only in one pathogenic strain. To elucidate functional characteristics, we studied 118 hits of the Rfam model in 11 Bacillus spp. and found two distinct classes based on the ensemble diversity of their RNA secondary structure and the genomic context concerning the ribosomal RNA (rRNA) cluster. Forty hits are associated with the rRNA cluster, of which all 19 hits upstream flanking of 16S rRNA have a reverse complementary structure of low structural diversity. Fifty-two hits have large ensemble diversity, of which 38 are located between two coding genes. For eight hits in Bacillus subtilis, we investigated public expression data under various conditions and observed either the forward or the reverse complementary motif expressed. Five hits are associated with the rRNA cluster. Four of them are located upstream of the 16S rRNA and are not transcriptionally active, but instead, their reverse complements with low structural diversity are expressed together with the rRNA cluster. The three other hits are located between two coding genes in non-conserved genomic loci. Two of them are independently expressed from their surrounding genes and are structurally diverse. In summary, we found that Bacillaceae-1 RNA motifs upstream flanking of ribosomal RNA clusters tend to have one stable structure with the reverse complementary motif expressed in B. subtilis. In contrast, a subgroup of intergenic motifs has the thermodynamic potential for structural switches.