Optimal Ensemble Research Articles

Accelerated ensemble optimization using momentum methods

We investigate the use of various momentum methods in combination with an ensemble approximation of gradients, for accelerated optimization. Although momentum gradient descent methods are popular in machine learning, it is unclear how they perform when applied to time-consuming dynamic problems such as production optimization for petroleum reservoir management. Four different momentum methods are extensively tested on a reservoir test case in one deterministic and one robust setting. The numerical experiments show that momentum strategies yield, on average, a higher net present value with fewer simulations needed.

Efficient Optimization: Unveiling the Application of Ensemble Learning Combined with the CMA-ES Algorithm in Hydraulic Fracturing Design

Optimizing fracturing parameters is crucial for enhancing production and reducing costs in oil and gas exploration and development. Effectively integrating geological and engineering parameters for the automated optimization of fracturing design continues to pose challenges. This study utilizes the cluster-based local outlier factor method for anomaly detection and removal from the dataset, significantly enhancing data quality. By integrating diverse models, including tree-based models and neural networks, an ensemble model for production prediction was developed. This approach successfully addresses the limitations of relying on a single model and achieves high-precision production forecasting. Furthermore, a Covariance Matrix Adaptation Evolution Strategy (CMA-ES)-based framework was established to comprehensively optimize the design parameters of fracturing projects. Optimization practices for two selected wells resulted in a 168.54% increase in production and identified the optimal design parameter configuration for all cases studied. The results of this study demonstrate the feasibility and effectiveness of the proposed ensemble prediction model and optimization framework in practical applications. Data-driven optimization strategies are expected to play a larger role in future oil and gas development, driving technological innovation and advancement in the field.

A highly accurate and robust prediction framework for drilling rate of penetration based on machine learning ensemble algorithm

The rate of penetration (ROP) is a key indicator of drilling efficiency. Many researchers have explored the application of machine learning in ROP prediction. However, few studies have focused on the robustness of the constructed models, and developing a ROP prediction model that can achieve both high accuracy and strong robustness remains a challenge. This paper introduces a novel machine learning approach to constructing a ROP prediction model through ensemble learning algorithms. The model is based on field data from oilfields, incorporating 16 input parameters that influence ROP. First, the feasibility of the collected dataset is verified using correlation analysis. Then, ROP prediction models are developed based on various machine learning algorithms, including Decision Tree Regression (DTR), Random Forest (RF), eXtreme Gradient Boosting (XGB), Light Gradient Boosting Machine (LGBM), Support Vector Regression (SVR), and Backpropagation Neural Network (BPNN). By comparing the performance of these models under noise levels of 0%, 1.7%, and 5.1%, RF, LGBM, XGB, and SVR are selected as base learners. These base learners are then combined to construct multiple ensemble models, and the performance of the optimal ensemble model is evaluated under varying noise levels. The results show that the prediction error of the optimal model remains within 10%, and R2 is greater than 0.96. Finally, the Shapley Additive Explanations (SHAP) method is applied to perform interpretability analysis on the optimal ROP prediction model, examining the impact of different input factors on the model's predictive performance. Compared to single models and other ensemble models, the proposed ensemble model not only achieves higher accuracy but also demonstrates strong robustness and generalization capability.

Ensemble and optimization algorithm in support vector machines for classification of wheat genotypes.

This study aimed to classifying wheat genotypes using support vector machines (SVMs) improved with ensemble algorithms and optimization techniques. Utilizing data from 302 wheat genotypes and 14 morphological attributes to evaluate six SVM kernels: linear, radial basis function (RBF), sigmoid, and polynomial degrees 1-3. Various optimization methods, including grid search, random search, genetic algorithms, differential evolution, and particle swarm optimization, were used. The radial basis function kernel achieves the highest accuracy at 93.2%, and the weighted accuracy ensemble further improves it to 94.9%. This study shows the effectiveness of these methods in agricultural research and crop improvement. Notably, optimization-based SVM classification, particularly with particle swarm optimization, saw a significant 1.7% accuracy gain in the test set, reaching 94.9% accuracy. These findings underscore the efficacy of RBF kernels and optimization techniques in improving wheat genotype classification accuracy and highlight the potential of SVMs in agricultural research and crop improvement endeavors.

Evaluating fine tuned deep learning models for real-time earthquake damage assessment with drone-based images

Earthquakes pose a significant threat to life and property worldwide. Rapid and accurate assessment of earthquake damage is crucial for effective disaster response efforts. This study investigates the feasibility of employing deep learning models for damage detection using drone imagery. We explore the adaptation of models like VGG16 for object detection through transfer learning and compare their performance to established object detection architectures like YOLOv8 (You Only Look Once) and Detectron2. Our evaluation, based on various metrics including mAP, mAP50, and recall, demonstrates the superior performance of YOLOv8 in detecting damaged buildings within drone imagery, particularly for cases with moderate bounding box overlap. This finding suggests its potential suitability for real-world applications due to the balance between accuracy and efficiency. Furthermore, to enhance real-world feasibility, we explore two strategies for enabling the simultaneous operation of multiple deep learning models for video processing: frame splitting and threading. In addition, we optimize model size and computational complexity to facilitate real-time processing on resource-constrained platforms, such as drones. This work contributes to the field of earthquake damage detection by (1) demonstrating the effectiveness of deep learning models, including adapted architectures, for damage detection from drone imagery, (2) highlighting the importance of evaluation metrics like mAP50 for tasks with moderate bounding box overlap requirements, and (3) proposing methods for ensemble model processing and model optimization to enhance real-world feasibility. The potential for real-time damage assessment using drone-based deep learning models offers significant advantages for disaster response by enabling rapid information gathering to support resource allocation, rescue efforts, and recovery operations in the aftermath of earthquakes.

Optimal Twirling Depth for Classical Shadows in the Presence of Noise.

The classical shadows protocol is an efficient strategy for estimating properties of an unknown state ρ using a small number of state copies and measurements. In its original form, it involves twirling the state with unitaries from some ensemble and measuring the twirled state in a fixed basis. It was recently shown that for computing local properties, optimal sample complexity (copies of the state required) is remarkably achieved for unitaries drawn from shallow depth circuits composed of local entangling gates, as opposed to purely local (zero depth) or global twirling (infinite depth) ensembles. Here, we consider the sample complexity as a function of the depth of the circuit, in the presence of noise. We find that this noise has important implications for determining the optimal twirling ensemble. Under fairly general conditions, we (i)show that any single-site noise can be accounted for using a depolarizing noise channel with an appropriate damping parameter f, (ii)compute thresholds f_{th} at which optimal twirling reduces to local twirling for Pauli operators, (iii)nth order Renyi entropies (n≥2), and (iv)provide a meaningful upper bound t_{max} on the optimal circuit depth for any finite noise strength f, which applies to observables and entanglement entropy measurements. These thresholds strongly constrain the search for optimal strategies to implement shadow tomography and are easily tailored to the experimental system at hand.

An ensemble optimizer with a stacking ensemble surrogate model for identification of groundwater contamination source

The application of the simulation-optimization method for groundwater contamination source identification (GCSI) encounters two main challenges: the substantial time cost of calling the simulation model, and the limitations on the accuracy of identification results due to the complexity, nonlinearity, and ill-posed nature of the inverse problem. To address these issues, we have innovatively developed an inversion framework based on ensemble learning strategies. This framework comprises a stacking ensemble model (SEM), which integrates three distinct machine learning models (Extremely Randomized Trees, Adaptive Boosting, and Bidirectional Gated Recurrent Unit), and an ensemble optimizer (E-GKSEEFO), which combines two newly proposed swarm intelligence optimizers (Genghis Khan Shark Optimizer and Electric Eel Foraging Optimizer). Specifically, the SEM serves as a surrogate model for the groundwater numerical simulation model. Compared to the original simulation model, it significantly reduces time cost while maintaining accuracy. The E-GKSEEFO, functioning as the search strategy for the optimization model, greatly enhances the accuracy of the optimization results. We have verified the performance of the SEM-E-GKSEEFO ensemble inversion framework through two hypothetical scenarios derived from an actual coal gangue pile. The results are as follows. (1) The SEM exhibits improved fitting performance compared to single machine learning models when dealing with high-dimensional nonlinear data from GCSI. (2) The E-GKSEEFO achieves significantly higher accuracy in the identification results of GCSI than individual optimizers. These findings affirm the effectiveness and superiority of the proposed SEM-E-GKSEEFO ensemble inversion framework.

Early Events in G-quadruplex Folding Captured by Time-Resolved Small-Angle X-Ray Scattering.

Time-resolved small-angle X-ray experiments (TR-SAXS) are reported here that capture and quantify a previously unknown rapid collapse of the unfolded oligonucleotide as an early step in G4 folding of hybrid 1 and hybrid 2 telomeric G-quadruplex structures. The rapid collapse, initiated by a pH jump, is characterized by an exponential decrease in the radius of gyration from 20.6 to 12.6 Å. The collapse is monophasic and is complete in less than 600 ms. Additional hand-mixing pH-jump kinetic studies show that slower kinetic steps follow the collapse. The folded and unfolded states at equilibrium were further characterized by SAXS studies and other biophysical tools, to show that G4 unfolding was complete at alkaline pH, but not in LiCl solution as is often claimed. The SAXS Ensemble Optimization Method (EOM) analysis reveals models of the unfolded state as a dynamic ensemble of flexible oligonucleotide chains with a variety of transient hairpin structures. These results suggest a G4 folding pathway in which a rapid collapse, analogous to molten globule formation seen in proteins, is followed by a confined conformational search within the collapsed particle to form the native contacts ultimately found in the stable folded form.

Teaching old docks new tricks with machine learning enhanced ensemble docking

We here introduce Ensemble Optimizer (EnOpt), a machine-learning tool to improve the accuracy and interpretability of ensemble virtual screening (VS). Ensemble VS is an established method for predicting protein/small-molecule (ligand) binding. Unlike traditional VS, which focuses on a single protein conformation, ensemble VS better accounts for protein flexibility by predicting binding to multiple protein conformations. Each compound is thus associated with a spectrum of scores (one score per protein conformation) rather than a single score. To effectively rank and prioritize the molecules for further evaluation (including experimental testing), researchers must select which protein conformations to consider and how best to map each compound’s spectrum of scores to a single value, decisions that are system-specific. EnOpt uses machine learning to address these challenges. We perform benchmark VS to show that for many systems, EnOpt ranking distinguishes active compounds from inactive or decoy molecules more effectively than traditional ensemble VS methods. To encourage broad adoption, we release EnOpt free of charge under the terms of the MIT license.

A Reciprocity Between Tree Ensemble Optimization and Multilinear Optimization

A Reciprocity Between Tree Ensemble Optimization and Multilinear Optimization

Retraction Note: effective heart disease prediction using improved particle swarm optimization algorithm and ensemble classification technique

Retraction Note: effective heart disease prediction using improved particle swarm optimization algorithm and ensemble classification technique

Risk prediction based on oversampling technology and ensemble model optimized by tree-structured parzed estimator

High accuracy prediction of urban flood risk is conducive to avoid potential losses, however, it's negatively affected by unbalanced data. Furthermore, ensemble model has been demonstrated to have the ability to improve to prediction accuracy. Nevertheless, the performance of ensemble model is influenced by basic model and ensemble rules, and determining the best ensemble model remains an open issue. To improve the accuracy of flood risk prediction, an approach covering data optimization and ensemble modeling was presented to optimize unbalanced flood data and the selection of various ensemble models based on efficiency and performance. A practical application in Zhengzhou City shows that Borderline-SMOTE2 is the most applicable for optimizing the flood risk data among the state-of-the-art oversampling algorithm utilized, because of the excellent entropy value. The effect of unbalanced data on the performance of the basic models was pervasive according to changes of the common indicators. The optimal ensemble model for flood risk prediction is composed of K-Nearest Neighbor, Decision Tree, Gaussian Naive Bayes and Extreme Gradient Boosting under Stacking rule in the current study. The results of this study supply the valuable reference for the flood prediction and mitigation.

Exploring The Efficiency of Metaheuristics in Optimal Hyperparameter Tuning for Ensemble Models on Varied Data Modalities

Effective disease detection systems play an important role in healthcare by supporting diagnosis and treatment. This study provides a comparison of hyperparameter tuning methods for disease detection systems using four health datasets; kidney disease, diabetes detection, heart disease and breast cancer detection. The main objective of this research is to prepare datasets by normalizing the input and testing machine learning models such as Naive Bayes Support Vector Machine (SVM), Logistic Regression and k Nearest Neighbor (kNN). to identify effective models for each data set. After implementing the models, we apply three hyperparameter tuning techniques: Grid search, random search, and particle ensemble optimization (PSO). These methods are used to tune the model parameters. Improve overall performance metrics. The evaluation focuses on accuracy measurements to compare model performance before and after hyperparameter tuning. The results of this study illustrate how different tuning techniques can improve the performance of disease detection systems across a range of healthcare datasets. By conducting testing and analysis, we determine the appropriate tuning method for each data set, yielding valuable insights, to develop an accurate and effective disease detection system .These discoveries serve to advance the field of healthcare analytics and machine learning to deliver outcomes for patients and healthcare services.

Deciphering Optimal Radar Ensemble for Advancing Sleep Posture Prediction through Multiview Convolutional Neural Network (MVCNN) Approach Using Spatial Radio Echo Map (SREM).

Assessing sleep posture, a critical component in sleep tests, is crucial for understanding an individual's sleep quality and identifying potential sleep disorders. However, monitoring sleep posture has traditionally posed significant challenges due to factors such as low light conditions and obstructions like blankets. The use of radar technolsogy could be a potential solution. The objective of this study is to identify the optimal quantity and placement of radar sensors to achieve accurate sleep posture estimation. We invited 70 participants to assume nine different sleep postures under blankets of varying thicknesses. This was conducted in a setting equipped with a baseline of eight radars-three positioned at the headboard and five along the side. We proposed a novel technique for generating radar maps, Spatial Radio Echo Map (SREM), designed specifically for data fusion across multiple radars. Sleep posture estimation was conducted using a Multiview Convolutional Neural Network (MVCNN), which serves as the overarching framework for the comparative evaluation of various deep feature extractors, including ResNet-50, EfficientNet-50, DenseNet-121, PHResNet-50, Attention-50, and Swin Transformer. Among these, DenseNet-121 achieved the highest accuracy, scoring 0.534 and 0.804 for nine-class coarse- and four-class fine-grained classification, respectively. This led to further analysis on the optimal ensemble of radars. For the radars positioned at the head, a single left-located radar proved both essential and sufficient, achieving an accuracy of 0.809. When only one central head radar was used, omitting the central side radar and retaining only the three upper-body radars resulted in accuracies of 0.779 and 0.753, respectively. This study established the foundation for determining the optimal sensor configuration in this application, while also exploring the trade-offs between accuracy and the use of fewer sensors.

Application of hybridized ensemble learning and equilibrium optimization in estimating damping ratios of municipal solid waste

The dynamic analysis of municipal solid waste (MSW) is essential for optimizing landfills and advancing sustainable development goals. Assessing damping ratio (D), a critical dynamic parameter, under laboratory conditions is costly and time-consuming, requiring specialized equipment and expertise. To streamline this process, this research leveraged several novel ensemble machine learning models integrated with the equilibrium optimizer algorithm (EOA) for the predictive analysis of damping characteristics. Data were gathered from 153 cyclic triaxial experiments on MSW, which examined the age, shear strain, weight, frequency, and percentage of plastic content. Analysis of a correlation heatmap indicated a significant dependence of D on shear strain within the collected MSW data. Subsequently, five advanced machine learning methods—adaptive boosting (AdaBoost), gradient boosting regression tree (GBRT), extreme gradient boosting (XGBoost), random forest (RF), and cubist regression—were employed to model D in landfill structures. Among these, the GBRT-EOA model demonstrated superior performance, with a coefficient of determination (R2) of 0.898, root mean square error of 1.659, mean absolute error of 1.194, mean absolute percentage error of 0.095, and an a20-index of 0.891 for the test data. A Shapley additive explanation analysis was conducted to validate these models further, revealing the relative contributions of each studied variable to the predicted D-MSW. This holistic approach not only enhances the understanding of MSW dynamics but also aids in the efficient design and management of landfill systems.

A New Post-Processing Method for Improving Track and Rainfall Ensemble Forecasts for Typhoons over Eastern China

This paper proposes a new post-processing method for model data in order to improve typhoon track and rainfall forecasts. The model data used in the article include low-resolution ensemble forecasts and high-resolution forecasts. The entire improvement method contains the following three steps. The first step is to correct the typhoon track forecast: three ensemble member optimization methods are applied to the low-resolution ensemble forecasts, and then the best optimization method is selected with the principle of the smallest average distance error. The results of rainfall forecasts show that the corrected rainfall forecast performs better than the original forecasts. The second step is to derive the high-resolution probability rainfall forecast: the neighborhood method is applied to the deterministic high-resolution rainfall forecast. The last step is to correct the typhoon rainfall forecast: the low- and high-resolution forecasts are blended using the probability-matching method with two different schemes. The results show that the forecasts of the two schemes perform better than the original forecast under all rainfall thresholds and all forecast lead times. In terms of bias score, a rain forecast from one scheme corrects the rainfall deviation from observation better for light and moderate rainfall, whereas a rain forecast from another scheme corrects the rainfall deviation better for heavy and torrential rainfall. The better performance of corrected rain forecasts in the case of Typhoon Lekima and Rumbia over eastern China is demonstrated.

Ensemble docking based virtual screening of SARS-CoV-2 main protease inhibitors.

During the first years of COVID-19 pandemic, X-ray structures of the coronavirus drug targets were acquired at an unprecedented rate, giving hundreds of PDB depositions in less than a year. The main protease (Mpro) of severe acute respiratory syndrome-related coronavirus 2 (SARS-CoV-2) is the primary validated target of direct-acting antivirals. The selection of the optimal ensemble of structures of Mpro for the docking-driven virtual screening campaign was thus non-trivial and required a systematic and automated approach. Here we report a semi-automated active site RMSD based procedure of ensemble selection from the SARS-CoV-2 Mpro crystallographic data and virtual screening of its inhibitors. The procedure was compared with other approaches to ensemble selection and validated with the help of hand-picked and peer-reviewed activity-annotated libraries. Prospective virtual screening of non-covalent Mpro inhibitors resulted in a new chemotype of thienopyrimidinone derivatives with experimentally confirmed enzyme inhibition.

Research on User Default Prediction Algorithm Based on Adjusted Homogenous and Heterogeneous Ensemble Learning

In the field of risk assessment, the traditional econometric models are generally used to assess credit risk. And with the introduction of the “dual-carbon” goals to promote the development of a low-carbon economy, the scale of green credit in China has rapidly expanded. But with the advent of the big data era, due to the poor interpretability of a traditional single machine learning model, it is difficult to capture nonlinear relationships, and there are shortcomings in prediction accuracy and robustness. This paper selects the adjusted ensemble learning model based on the homogeneous and heterogeneous factors for user default prediction, which can efficiently process large quantities of high-dimensional data. This article adjusts each model to adapt to the task and innovatively compares various models. In this paper, the missing value filling method, feature selection, and ensemble model are studied and discussed, and the optimal ensemble model is obtained. When comparing the predictions of single models and ensemble models, the accuracy, sensitivity, specificity, F1-Score, Kappa, and MCC of Categorical Features Gradient Boosting (CatBoost) and Random undersampling Boosting (RUSBoost) all reach 100%. The experimental results prove that the algorithm based on adjusted homogeneous and heterogeneous ensemble learning can predict the user default efficiently and accurately. This paper also provides some references for establishing a risk assessment index system.

Non-Gaussian Ensemble Optimization

Non-Gaussian Ensemble Optimization

Probabilistic assessment of rockburst risk in TBM-excavated tunnels with multi-source data fusion

In recent years, tunnel boring machines (TBMs) have become extensively utilized in underground engineering projects. However, as a prevalent type of dynamic geological hazard, rockburst poses a serious threat to the safety of personnel, equipment, and property. To ensure the safe advancement of TBMs, we integrate the data of microseismic monitoring, TBM excavation, and surrounding rock and establish multiple probabilistic assessment models of rockburst risk using probabilistic ensemble learning and quantum particle swarm optimization. In order to identify the most optimal model for geological engineers, a comprehensive evaluation system is devised based on metrics like accuracy, Cohen’s kappa, and F1-score. This system provides a thorough assessment of the models’ global and local generalization performance. The results indicate that the classification and regression tree-extremely randomized trees (CART-ERT) hybrid model achieves the highest score, demonstrating superior generalization performance with an accuracy of 93.06% and a Cohen’s kappa of 0.9074. Moreover, the F1-scores for no rockburst, low rockburst, moderate rockburst, and strong rockburst are 0.9412, 0.9444, 0.9189, and 0.9189, respectively. Based on the operational framework of the CART-ERT hybrid model, a user-friendly graphical user interface (GUI) system is developed. This significantly enhances the practicality and deployability of the model. Through application in a TBM diversion tunnel project in Xinjiang, the on-site early-warning accuracy of rockburst with the GUI system reaches 90%. Notably, the GUI system maintains high early-warning sensitivity and reliability for high-intensity rockburst such as moderate and strong ones. Lastly, to enhance the model’s interpretability, a variable importance measure (VIM) analysis on 14 features extracted from three types of heterogeneous data is conducted to assess their contributions to the early warning of rockburst. We discover that the cumulative energy change rate of microseismic events exhibits the highest importance score, indicating its crucial role in rockburst early warning.