Ensemble Regression Research Articles

A Novel DWT-ERT-based Fault Location for Distribution Network

Background: A new DWT-ERT-based fault location method is suggested in the IEEE test feeder. Objective: The fault location approach in the distribution network has been proposed in this pa-per that utilizes the discrete wavelet transform (DWT) and ensemble regression tree (ERT). Methods: The fault location methodology has been validated by simulations conducted on an IEEE 13 bus node test feeder. Results: The results show that the suggested solution has low compute burden and memory re-quirements, and is unaffected by system and fault situations. Conclusion: In this study, the fault location approach for the distribution system employing DWT and ERT has been proposed.

A Building Heat Load Prediction Method Driven by a Multi-Component Fusion LSTM Ridge Regression Ensemble Model

Under the background of “double carbon”, building carbon emission reduction is urgent, and improving energy efficiency through short-term building heat load forecasting is an efficient means of building carbon emission reduction. Aiming at the characteristics of the decomposed short-term building heat load data, such as complex trend changes, significant seasonal changes, and randomness, a single-step short-term building heat load prediction method driven by the multi-component fusion LSTM Ridge Regression Ensemble Model (ST-LSTM-RR) is designed and implemented. First, the trend and seasonal components of the heat load are decomposed by the STL seasonal decomposition algorithm, which are fused into the original data to construct three diversified datasets; second, three basic models, namely, the trend LSTM, the seasonal LSTM, and the original LSTM, are trained; and then, the ridge regression model is trained to fuse the predicted values of the three basic models to obtain the final predicted values. Finally, the method of this paper is applied to the heat load prediction of eight groups in a large mountain hotel park, and the root mean square error (RMSE) and mean absolute error (MAE) are used as the evaluation indexes. The experimental results show that the average RMSE and average MAE of the prediction results of the proposed method in this paper are minimized on the eight groups.

Block-wise imputation EM algorithm in multi-source scenario: ADNI case

Alzheimer’s disease is the most common form of dementia and the early detection is essential to prevent its proliferation. Real data available has been of paramount importance in order to achieve progress in the automatic detection despite presenting two major challenges: Multi-source observations containing Magnetic resonance (MRI), Positron emission tomography (PET) and Cerebrospinal fluid data (CSF); and also missing values within all these sources. Most machine learning techniques perform this predictive task by using a single data modality. Nevertheless, the integration of all these sources of evidence could possibly bring a higher performance at different stages of disease progression. The Expectation Maximization (EM) algorithm has been successfully employed to handle missing values, but it is not designed for typical Machine Learning scenarios where an imputation model is created over training data and subsequently applied on a testing set. In this work, we propose EMreg-KNN, a novel supervised and multi-source imputation algorithm. Based on the EM algorithm, EMreg-KNN builds a regression ensemble model for the imputation of future data thus allowing the further utilization of any vector-based Machine Learning method to automatically assess the Alzheimer’s disease diagnosis. Using the ADNI database, the proposed method achieves significant improvements on F1, AUC and Accuracy measures over classical imputation methods for this database using four classification algorithms. Considering these classifiers in four different classification scenarios, our algorithm is experimentally superior in terms of the F measure, in nearly 82% of the cases under evaluation.

Understanding heterogeneity, comorbidity, and variability in depression: Idiographic models and depression outcomes

This study uses time-intensive, item-level assessment to examine individual depressive and co-occurring symptom dynamics. Participants experiencing moderate-severe depression (N = 31) completed ecological momentary assessment (EMA) four times per day for 20 days (total observations = 2480). We estimated idiographic networks using MDD, anxiety, and ED items. ED items were most frequently included in individual networks relative to depression and anxiety items. We built ridge and logistic regression ensembles to explore how idiographic network centrality metrics performed at predicting between-subject depression outcomes (PHQ-9 change score and clinical deterioration, respectively) at 6-months follow-up. For predicting PHQ-9 change score, R2 ranged between 0.13 and 0.28. Models predicting clinical deterioration ranged from no better than chance to 80 % accuracy. This pilot study shows how co-occurring anxiety and ED symptoms may contribute to the maintenance of depressive symptoms. Future work should assess the predictive utility of psychological networks to develop understanding of how idiographic models may inform clinical decisions.

Optimal weight support vector regression ensemble with cluster-based subsampling for electricity price forecasting

Accurate prediction of short-term electricity price is the key to obtain economic benefit and also an important index of power system planning and management. Support vector regression (SVR) based ensemble works have gained remarkable achievements in terms of high accuracy and steady performance, but they are highly dependent on data representativeness and have a high computational complexity O (k * N 3) of data samples and parameter selection. To further improve the data representativeness and reduce its computational complexity, this paper develops a new approach to forecast electricity price via optimal weighted ensemble. In the model, the cluster-based subsampling algorithm is proposed to categorize the inputs being seasonally decomposed into several groups, and representative data are drawn from each group in a certain proportion to ensure that each subset trained with SVR has the same representativeness and features. Moreover, the optimal weighted combination method is presented to assign weights to the sub-SVRs to obtain the optimal support vector regression ensemble model (OWSSVRE). The experimental results show that the improved support vector regression ensemble model with the same features and representativeness of the subset has better performance in electricity price forecasting. As a result, it is suitable to support decision making in the energy and other sectors.

CatNorth: An Improved Gaia DR3 Quasar Candidate Catalog with Pan-STARRS1 and CatWISE

A complete and pure sample of quasars with accurate redshifts is crucial for quasar studies and cosmology. In this paper, we present CatNorth, an improved Gaia Data Release 3 (Gaia DR3) quasar candidate catalog with more than 1.5 million sources in the 3π sky built with data from Gaia, Pan-STARRS1, and CatWISE2020. The XGBoost algorithm is used to reclassify the original Gaia DR3 quasar candidates as stars, galaxies, and quasars. To construct training/validation data sets for the classification, we carefully built two different master stellar samples in addition to the spectroscopic galaxy and quasar samples. An ensemble classification model is obtained by averaging two XGBoost classifiers trained with different master stellar samples. Using a probability threshold of p QSO_mean > 0.95 in our ensemble classification model and an additional cut on the logarithmic probability density of zero proper motion, we retrieved 1,545,514 reliable quasar candidates from the parent Gaia DR3 quasar candidate catalog. We provide photometric redshifts for all candidates with an ensemble regression model. For a subset of 89,100 candidates, accurate spectroscopic redshifts are estimated with the convolutional neural network from the Gaia BP/RP spectra. The CatNorth catalog has a high purity of ∼90%, while maintaining high completeness, which is an ideal sample to understand the quasar population and its statistical properties. The CatNorth catalog is used as the main source of input catalog for the Large Sky Area Multi-Object Fiber Spectroscopic Telescope phase III quasar survey, which is expected to build a highly complete sample of bright quasars with i < 19.5.

The Identification of Acetic Acid-Ethanol Mixture Using Gas Sensor Array and Ensemble Regression

Identification of acetic acid-ethanol mixtures using a commercial gas sensor array equipped with ensemble regression has been carried out. The gas sensor analysis was simple, rapid, and fast since it did not require any sample preparation. A quantitative analysis of the acetic acid-ethanol mixture was carried out to determine the sensitivity and selectivity of the sensor in distinguishing the concentration of the acetic acid and ethanol mixture. This study focuses on the coefficient of determination of 80% of the calibration data set and recovery of 20% of the testing data set. The models showed excellent performance,specifically, the Bagging and Random Forest r2 for the ethanol calibration data reached 0.91 and 0.94, respectively. The corresponding ethanol test recoveries were 99.95% and 97.84%, indicating the robustness of the model in accurately predicting ethanol concentration. Acetic acid test recoveries were 100.56% and 101.38% with r2 of 0.89 and 0.93 for Bagging and Random Forest regression, respectively. Hence, the commercial gas sensor array equipped with ensemble regression can be applied to the quantification of the acetic acid – ethanol mixture and demonstrate opportunities for the practical use of this gas sensor array in analyzing real samples, i.e. human breath or environmental monitoring samples.

Design optimization of solar collectors with hybrid nanofluids: An integrated ansys and machine learning study

The current study discussed the integration between two computational approaches to evaluate the hydrothermal properties, such as pressure drop (ΔP), energy efficiency (ηeng), and absorbed energy (Qabs) of solar collectors using hybrid nanofluids. The physical problem was solved through a 3D model using Ansys 2021R1. After that, the three outputs were predicted through MATLAB R2022b using Optimize a Boosted Regression Ensemble by implementing Least-Squares Boosting (LSBoost) with Bayesian optimization (BO). Various working fluids were tested: distilled water (DW), Graphene oxide (GO-DW) nanofluids, and hybrid graphene oxide/silicon dioxide (GO/SiO2-DW) nanofluids in the mixing ratio (50:50), in the concentration range 0.01-1 vol%. The simulations covered a range of Reynolds numbers (300 ≤ Re ≤ 1500) and inlet temperatures (30 °C, 40 °C, 50 °C, and 60 °C). The results indicated that, the maximum variation compared to the first and second validations for mass flow rates and different nanofluid concentrations was between 3.96-4.71% and 4.32–6.45%. At an inlet temperature of 30 °C, GO-DW-1% exhibited the highest ΔP, ηeng, and Qabs, while GO-SiO2-DW-1% closely followed with slightly lower energy values. The maximum errors between computational fluid dynamics and machine learning predictions for (ΔP), (ηeng), and (Qabs) ranged from 2.40% to 6.32%, depending on the type of nanofluids (single or hybrid) and the volume concentration. To conclude, the current approach showed a promising result in the photothermal devices.

Assessment and regression of carbon emissions from the building and construction sector in China: A provincial study using machine learning

The building and construction sector is a major contributor to carbon emissions in China. Hence, it is crucial to explore the characteristics and trends of building carbon emissions to achieve the carbon peak and neutrality. While previous studies have made efforts to analyze the influencing factors through different approaches, developing an effective and intelligent regression model based on machine learning algorithms remains challenging in predicting the carbon emission trend. This study analyzed carbon emissions and per capita indicators of the building and construction sector in 30 provincial regions in China from 2005 to 2021. While embodied and operational carbon emissions contribute equally to the total emissions, the results showed a significant spatial-temporal correlation. Considering the emissions as target features, nine alternative machine learning regression models were developed using eight identified explanatory features incorporating scale, economic, technological, and classification factors. Based on performance metrics encompassing root mean squared error, coefficient of determination, and mean absolute percentage error, the stacking ensemble regression model was identified to have superior performance. This model was further employed to conduct a sensitivity analysis of explanatory features on carbon emissions. The results indicated that urbanization rate and population were the most sensitive factors, with varying effects on different target features. These findings can be used to predict carbon emission trends and promote carbon reduction policies in the building industry.

A Slicing‐Free Perspective to Sufficient Dimension Reduction: Selective Review and Recent Developments

SummarySince the pioneering work of sliced inverse regression, sufficient dimension reduction has been growing into a mature field in statistics and it has broad applications to regression diagnostics, data visualisation, image processing and machine learning. In this paper, we provide a review of several popular inverse regression methods, including sliced inverse regression (SIR) method and principal hessian directions (PHD) method. In addition, we adopt a conditional characteristic function approach and develop a new class of slicing‐free methods, which are parallel to the classical SIR and PHD, and are named weighted inverse regression ensemble (WIRE) and weighted PHD (WPHD), respectively. Relationship with recently developed martingale difference divergence matrix is also revealed. Numerical studies and a real data example show that the proposed slicing‐free alternatives have superior performance than SIR and PHD.

Explainable prediction of surface roughness in multi-jet polishing based on ensemble regression and differential evolution method

Surface roughness is a critical parameter for quantifying the surface quality of a workpiece. Accurate surface roughness prediction can significantly influence the overall quality of the final components by reducing costs and enhancing productivity. Although various prediction methods have been explored in previous research, less attention has been given to explainable prediction methods for surface roughness. This paper proposes a surface roughness prediction model based on the Differential Evolution (DE) algorithm with ensemble learning. It aims to elucidate the intrinsic correlation between processing parameters and surface roughness values in Multi-Jet Polishing (MJP) using explainable analysis methods. The proposed Ensemble Regression with Differential Evolution (ERDE) algorithm comprises two main modules: data processing and analytics, and ensemble model prediction. These modules investigate the surface roughness of MJP from the data and model levels, respectively. To validate the effectiveness of ERDE, MJP experiments were conducted on three-dimensional printed components of 316L stainless steel. The experimental data indicated that ERDE outperforms other existing algorithms, reducing the mean absolute percentage error by approximately 42 %.

Machine Learning-Assisted Determination of C6H14 Mole Fraction From Molecular Emissions of Laser-Induced Hexane-Air Plasmas.

Laser-induced plasmas of materials containing hydrocarbons present strong carbon molecular emission features. Using these emissions to build models relating changes in spectral features to a physical parameter of the system, such as hydrocarbon content, can be difficult because of the dynamic complexity of the spectral features and temperature disequilibrium between molecular species. This study presents machine learning models trained to quantify the mole fraction of hexane in hexane-air plasmas from CN Violet and C2 Swan spectral features. Ensemble regression methods provide the most accurate predictions with root mean squared error on the order 10-2. Artificial neural network regressions produce predictions with superlative sensitivity, exhibiting detection limits as low as 0.008. These foundational models can be further refined with more advanced data to quantify the presence of carbon species in complex plasma environments, such as high-speed reacting flows.

Monitoring Disease Severity of Mild Cognitive Impairment from Single-Channel EEG Data Using Regression Analysis.

A deviation in the soundness of cognitive health is known as mild cognitive impairment (MCI), and it is important to monitor it early to prevent complicated diseases such as dementia, Alzheimer's disease (AD), and Parkinson's disease (PD). Traditionally, MCI severity is monitored with manual scoring using the Montreal Cognitive Assessment (MoCA). In this study, we propose a new MCI severity monitoring algorithm with regression analysis of extracted features of single-channel electro-encephalography (EEG) data by automatically generating severity scores equivalent to MoCA scores. We evaluated both multi-trial and single-trail analysis for the algorithm development. For multi-trial analysis, 590 features were extracted from the prominent event-related potential (ERP) points and corresponding time domain characteristics, and we utilized the lasso regression technique to select the best feature set. The 13 best features were used in the classical regression techniques: multivariate regression (MR), ensemble regression (ER), support vector regression (SVR), and ridge regression (RR). The best results were observed for ER with an RMSE of 1.6 and residual analysis. In single-trial analysis, we extracted a time-frequency plot image from each trial and fed it as an input to the constructed convolutional deep neural network (CNN). This deep CNN model resulted an RMSE of 2.76. To our knowledge, this is the first attempt to generate automated scores for MCI severity equivalent to MoCA from single-channel EEG data with multi-trial and single data.

FY‐4A/AGRI Infrared Brightness Temperature Estimation of Precipitation Based on Multi‐Model Ensemble Learning

AbstractSatellite infrared detectors cannot penetrate clouds, especially precipitating clouds. Improving precipitation estimation accuracy based on infrared brightness temperature has always been important but challenging. In this paper, based on the infrared brightness temperature of the Advanced Geosynchronous Radiation Imager (AGRI) onboard China's Feng‐Yun 4A satellite, we develop and evaluate a new precipitation estimation method. First, using static data, physical characteristics of clouds, cloud image texture features, temporal motion features, and AGRI infrared channel brightness temperature, we construct features for a machine learning model. Then, we develop precipitation estimation methods. Precipitation is estimated in two steps: classification and regression. We employ a random forest classification model to identify whether there is precipitation in a given field of view. If there is precipitation, a multi‐model ensemble regression learning method is used to estimate the areas with this precipitation. The ensemble learning method uses convex optimization to integrate prediction results based on the optimization of hyperparameters of five basic models (i.e., those of random forest, XGBoost, LightGBM, decision tree, and extra tree models). Furthermore, two regression stacking ensemble models—the Least Absolute Shrinkage and Selection Operator (herein referred to as Stacking1‐LASSO) and K‐nearest neighbor (herein referred to as Stacking2‐KNN)—are used to predict the results of the aforementioned basic models. The results of basic models are used as inputs of these two stacking models. Finally, based on the Integrated Multi‐satellitE Retrievals for GPM (IMERG) precipitation product and rain gauge precipitation data, we conduct precipitation estimation experiments and evaluate our methods. The results show that ensemble learning models have greater accuracy in estimating precipitation than the basic models. When using IMERG precipitation as the target precipitation, ensemble learning models can estimate the central area of heavy precipitation during typhoons Ampil and Maria. The ensemble learning estimation effect is better than that of Stacking2‐KNN. Moreover, when rain gauge data is used as the target precipitation, ensemble learning can also estimate the center of heavy precipitation and with good consistency with recorded satellite brightness temperature data.

CSER: a gene regulatory network construction method based on causal strength and ensemble regression.

Gene regulatory networks (GRNs) reveal the intricate interactions between and among genes, and understanding these interactions is essential for revealing the molecular mechanisms of cancer. However, existing algorithms for constructing GRNs may confuse regulatory relationships and complicate the determination of network directionality. We propose a new method to construct GRNs based on causal strength and ensemble regression (CSER) to overcome these issues. CSER uses conditional mutual inclusive information to quantify the causal associations between genes, eliminating indirect regulation and marginal genes. It considers linear and nonlinear features and uses ensemble regression to infer the direction and interaction (activation or regression) from regulatory to target genes. Compared to traditional algorithms, CSER can construct directed networks and infer the type of regulation, thus demonstrating higher accuracy on simulated datasets. Here, using real gene expression data, we applied CSER to construct a colorectal cancer GRN and successfully identified several key regulatory genes closely related to colorectal cancer (CRC), including ADAMDEC1, CLDN8, and GNA11. Importantly, by integrating immune cell and microbial data, we revealed the complex interactions between the CRC gene regulatory network and the tumor microenvironment, providing additional new biomarkers and therapeutic targets for the early diagnosis and prognosis of CRC.

Ekstrapolacija vertikalne brzine vetra korišćenjem statističkih pristupa

The wind power industry has experienced a significant increase and popularity in recent times, and the latest statistics indicate that this sector is still thriving. However, one of the essential steps in developing wind energy projects is finding suitable sites for wind farms, which involves understanding the nature of wind speed, wind direction, terrain, and environmental impacts. To predict the wind energy production over the expected lifespan of a wind farm, vertical wind speed extrapolation to the hub height of the wind turbine is necessary. Therefore, this study presents a comprehensive evaluation of seven statistical approaches for vertical wind speed extrapolation, including Generalized Linear Models (GLM), Linear Regression (LR), Support Vector Machines (SVM), Generalized Additive Models (GAM), Gaussian Process Regression (GPR), Regression Tree (RT), and Ensemble Regression (ER). The accuracy of these methods is assessed using performance metrics such as Mean Squared Error (MSE), Root Mean Squared Error (RMSE), Normalized RMSE (NRMSE), Normalized MSE (NMSE), Mean Bias Error (MBE), Mean Absolute Error (MAE), Mean Percentage Error (MPE), Mean Absolute Percentage Error (MAPE), Symmetric Mean Absolute Percentage Error (SMAPE), and R-squared (R2). The study concludes that, on average, GLM performs the best out of all seven statistical methods.

Ensemble Regression Model-Based Missing Not at Random Type Missing Data Imputation on the Internet of Medical Things

Ensemble Regression Model-Based Missing Not at Random Type Missing Data Imputation on the Internet of Medical Things

An optimization-based stacked ensemble regression approach to predict the compressive strength of self-compacting concrete

An optimization-based stacked ensemble regression approach to predict the compressive strength of self-compacting concrete

Enhanced desalination with polyamide thin-film membranes using ensemble ML chemometric methods and SHAP analysis.

Addressing global freshwater scarcity requires innovative technological solutions, among which desalination through thin-film composite polyamide membranes stands out. The performance of these membranes plays a vital role in desalination, necessitating advanced predictive modeling for optimization. This study harnesses machine learning (ML) algorithms, including support vector machine (SVM), neural networks (NN), linear regression (LR), and multivariate linear regression (MLR), alongside their ensemble techniques to predict and enhance average water flux (AWF) and average salt rejection (ASR) essential metrics of desalination efficiency. To ensure model interpretability and feature importance analysis, SHapley Additive exPlanations (SHAP) were employed, providing both global and local insights into feature contributions. Initially, the individual models were validated, with NN demonstrating superior performance for both AWF and ASR, achieving the lowest mean absolute error (MAE = 0.001) and root mean squared error (RMSE = 0.0111) for AWF and an MAE = 0.0107 and RMSE = 0.0982 for ASR. The accuracy of predictions improved significantly with ensemble models, as evidenced by the near-perfect Nash-Sutcliffe efficiency (NSE) values. Specifically, the NN ensemble (NN-E) and Linear Regression ensemble (LR-E) reached an MAE and RMSE of 0.001 and 0.0111, respectively, for AWF. For ASR, NN-E reduced the MAE to 0.0013 and the RMSE to 0.0089, while LR-E maintained competitive performance with an MAE of 0.0133 and an RMSE of 0.0936. SHAP analysis revealed that features such as MDP and TMC were critical drivers of performance, with MDP showing the most significant positive impact on ASR. These findings demonstrate the dominance of ensemble methods over individual algorithms in predicting key desalination parameters. The enhanced precision in estimating AWF and ASR offered by these neuro-intelligent ensembles, combined with the interpretability provided by SHAP analysis, can lead to significant environmental and operational improvements in membrane performance, optimizing resource usage and minimizing ecological impacts. This study paves the way for integrating intelligent ML ensembles and SHAP-based interpretability into the practical field of membrane technology, marking a step forward toward sustainable and efficient desalination processes.

Wi-Fi Fingerprint for Indoor Keyless Entry Systems with Ensemble Learning Regression-Classification Model

Keyless entry systems have made human life more comfortable because the possibility of someone leaving a key is non-existent. However, there is a research opportunity to use Wi-Fi fingerprint localization in an indoor keyless entry system. The use of Wi-Fi as a solution is low cost because of the already hdeployed Wi-Fi access points (AP) around buildings. This study uses ensemble learning to utilize Wi-Fi fingerprints for keyless entry systems in indoor environments. The first step of this research is to obtain the UJIIndoorLoc Data Set, an open-source Wi-Fi fingerprint dataset. The next step is to design and implement a keyless entry system based on Wi-Fi Fingerprint. We compared four ensemble learning methods: Random Forest, AdaBoost, gradient boosting, and XGBoost. We use several metrics to compare the four methods, namely r2, the area under the receiver operating curve (AUROC), and the equal error rate (EER). The test results show that for localization based on ensemble regression, XGBoost has the best performance compared to the other three ensemble methods for both longitude and latitude predictions. The r2values are 0.94 and 0.98, respectively. For authentication based on ensemble classification, the gradient-boosting model performance increases as the number of weak learners increases. The optimum number of weak learners is 60. With AUROC = 0.99, gradient boosting outperforms random forest, AdaBoost, and XGBoost on authentication. Finally, our authentication method has EER = 0.01, which outperforms other state-of-the-art keyless entry systems. This research focuses on local building map datasets for future work.