Ensemble Of Regression Models Research Articles

Learning sparse feature representation for blind quality assessment of night-time images

Capturing Night-Time Images (NTIs) with high-quality is quite challenging for consumer photography and several practical applications. Thus, addressing the quality assessment of night-time images is urgently needed. Since there is no available reference image for such images, Night-Time image Quality Assessment (NTQA) should be done blindly. Although Blind natural Image Quality Assessment (BIQA) has attracted a great deal of attention for a long time, very little work has been done in the field of NTQA. Due to the capturing conditions, NTIs suffer from various complex authentic distortions that make it a challenging field of research. Therefore, previous BIQA methods, do not provide sufficient correlation with subjective scores in the case of NTIs and special methods of NTQA should be developed. In this paper we conduct an unsupervised feature learning method for blind quality assessment of night-time images. The features are the sparse representation over the data-adaptive dictionaries learned on the image exposure and gradient magnitude maps. Having these features, an ensemble regression model trained using least squares gradient boosting scheme predicts high correlated objective scores on the standard datasets.

Variable importance analysis with interpretable machine learning for fair risk prediction.

Machine learning (ML) methods are increasingly used to assess variable importance, but such black box models lack stability when limited in sample sizes, and do not formally indicate non-important factors. The Shapley variable importance cloud (ShapleyVIC) addresses these limitations by assessing variable importance from an ensemble of regression models, which enhances robustness while maintaining interpretability, and estimates uncertainty of overall importance to formally test its significance. In a clinical study, ShapleyVIC reasonably identified important variables when the random forest and XGBoost failed to, and generally reproduced the findings from smaller subsamples (n = 2500 and 500) when statistical power of the logistic regression became attenuated. Moreover, ShapleyVIC reasonably estimated non-significant importance of race to justify its exclusion from the final prediction model, as opposed to the race-dependent model from the conventional stepwise model building. Hence, ShapleyVIC is robust and interpretable for variable importance assessment, with potential contribution to fairer clinical risk prediction.

A Building Heat Load Prediction Method Driven by a Multi-Component Fusion LSTM Ridge Regression Ensemble Model

Under the background of “double carbon”, building carbon emission reduction is urgent, and improving energy efficiency through short-term building heat load forecasting is an efficient means of building carbon emission reduction. Aiming at the characteristics of the decomposed short-term building heat load data, such as complex trend changes, significant seasonal changes, and randomness, a single-step short-term building heat load prediction method driven by the multi-component fusion LSTM Ridge Regression Ensemble Model (ST-LSTM-RR) is designed and implemented. First, the trend and seasonal components of the heat load are decomposed by the STL seasonal decomposition algorithm, which are fused into the original data to construct three diversified datasets; second, three basic models, namely, the trend LSTM, the seasonal LSTM, and the original LSTM, are trained; and then, the ridge regression model is trained to fuse the predicted values of the three basic models to obtain the final predicted values. Finally, the method of this paper is applied to the heat load prediction of eight groups in a large mountain hotel park, and the root mean square error (RMSE) and mean absolute error (MAE) are used as the evaluation indexes. The experimental results show that the average RMSE and average MAE of the prediction results of the proposed method in this paper are minimized on the eight groups.

Block-wise imputation EM algorithm in multi-source scenario: ADNI case

Alzheimer’s disease is the most common form of dementia and the early detection is essential to prevent its proliferation. Real data available has been of paramount importance in order to achieve progress in the automatic detection despite presenting two major challenges: Multi-source observations containing Magnetic resonance (MRI), Positron emission tomography (PET) and Cerebrospinal fluid data (CSF); and also missing values within all these sources. Most machine learning techniques perform this predictive task by using a single data modality. Nevertheless, the integration of all these sources of evidence could possibly bring a higher performance at different stages of disease progression. The Expectation Maximization (EM) algorithm has been successfully employed to handle missing values, but it is not designed for typical Machine Learning scenarios where an imputation model is created over training data and subsequently applied on a testing set. In this work, we propose EMreg-KNN, a novel supervised and multi-source imputation algorithm. Based on the EM algorithm, EMreg-KNN builds a regression ensemble model for the imputation of future data thus allowing the further utilization of any vector-based Machine Learning method to automatically assess the Alzheimer’s disease diagnosis. Using the ADNI database, the proposed method achieves significant improvements on F1, AUC and Accuracy measures over classical imputation methods for this database using four classification algorithms. Considering these classifiers in four different classification scenarios, our algorithm is experimentally superior in terms of the F measure, in nearly 82% of the cases under evaluation.

Optimal weight support vector regression ensemble with cluster-based subsampling for electricity price forecasting

Accurate prediction of short-term electricity price is the key to obtain economic benefit and also an important index of power system planning and management. Support vector regression (SVR) based ensemble works have gained remarkable achievements in terms of high accuracy and steady performance, but they are highly dependent on data representativeness and have a high computational complexity O (k * N 3) of data samples and parameter selection. To further improve the data representativeness and reduce its computational complexity, this paper develops a new approach to forecast electricity price via optimal weighted ensemble. In the model, the cluster-based subsampling algorithm is proposed to categorize the inputs being seasonally decomposed into several groups, and representative data are drawn from each group in a certain proportion to ensure that each subset trained with SVR has the same representativeness and features. Moreover, the optimal weighted combination method is presented to assign weights to the sub-SVRs to obtain the optimal support vector regression ensemble model (OWSSVRE). The experimental results show that the improved support vector regression ensemble model with the same features and representativeness of the subset has better performance in electricity price forecasting. As a result, it is suitable to support decision making in the energy and other sectors.

CatNorth: An Improved Gaia DR3 Quasar Candidate Catalog with Pan-STARRS1 and CatWISE

A complete and pure sample of quasars with accurate redshifts is crucial for quasar studies and cosmology. In this paper, we present CatNorth, an improved Gaia Data Release 3 (Gaia DR3) quasar candidate catalog with more than 1.5 million sources in the 3π sky built with data from Gaia, Pan-STARRS1, and CatWISE2020. The XGBoost algorithm is used to reclassify the original Gaia DR3 quasar candidates as stars, galaxies, and quasars. To construct training/validation data sets for the classification, we carefully built two different master stellar samples in addition to the spectroscopic galaxy and quasar samples. An ensemble classification model is obtained by averaging two XGBoost classifiers trained with different master stellar samples. Using a probability threshold of p QSO_mean > 0.95 in our ensemble classification model and an additional cut on the logarithmic probability density of zero proper motion, we retrieved 1,545,514 reliable quasar candidates from the parent Gaia DR3 quasar candidate catalog. We provide photometric redshifts for all candidates with an ensemble regression model. For a subset of 89,100 candidates, accurate spectroscopic redshifts are estimated with the convolutional neural network from the Gaia BP/RP spectra. The CatNorth catalog has a high purity of ∼90%, while maintaining high completeness, which is an ideal sample to understand the quasar population and its statistical properties. The CatNorth catalog is used as the main source of input catalog for the Large Sky Area Multi-Object Fiber Spectroscopic Telescope phase III quasar survey, which is expected to build a highly complete sample of bright quasars with i < 19.5.

Estimating Body Mass Index from Facial Images

Abstract: A person's health status may have a significant impact on many aspects of their life, from mental health to lifespan to financial security. The health of a person can be calculated by a value which is called Body Mass Index (BMI), it uses both the height and weight of a person. Numerous variables, including physical health, mental health, and popularity, have been linked to BMI. With the increasing number of people being obese, self-diagnostic solutions for healthy weight monitoring are grabbing significant attention. Calculating BMI using the statistical formula requires precise measurements of the height and weight of a person and is time-consuming. The main objective of this project is to predict the BMI of a person by giving the image as input. While developing Fitness apps, we can use this system to detect the BMI of a person daily and suggest suitable exercises. The developed system can also be used to find whether a person is suffering from malnutrition and some other diseases that can be detected using BMI. The models used in our project are FaceNet, Ridge Linear Regression, Random Forest Regression, Support Vector Regression, and ensemble of regression models.

Assessment and regression of carbon emissions from the building and construction sector in China: A provincial study using machine learning

The building and construction sector is a major contributor to carbon emissions in China. Hence, it is crucial to explore the characteristics and trends of building carbon emissions to achieve the carbon peak and neutrality. While previous studies have made efforts to analyze the influencing factors through different approaches, developing an effective and intelligent regression model based on machine learning algorithms remains challenging in predicting the carbon emission trend. This study analyzed carbon emissions and per capita indicators of the building and construction sector in 30 provincial regions in China from 2005 to 2021. While embodied and operational carbon emissions contribute equally to the total emissions, the results showed a significant spatial-temporal correlation. Considering the emissions as target features, nine alternative machine learning regression models were developed using eight identified explanatory features incorporating scale, economic, technological, and classification factors. Based on performance metrics encompassing root mean squared error, coefficient of determination, and mean absolute percentage error, the stacking ensemble regression model was identified to have superior performance. This model was further employed to conduct a sensitivity analysis of explanatory features on carbon emissions. The results indicated that urbanization rate and population were the most sensitive factors, with varying effects on different target features. These findings can be used to predict carbon emission trends and promote carbon reduction policies in the building industry.

Stacked Ensemble Regression Model for Prediction of Furan

Furan tests provide a non-intrusive and cost-effective method of estimating the degradation of paper insulation, which is critical for ensuring the reliability of power grids. However, conducting routine furan tests can be expensive and challenging, highlighting the need for alternative methods, such as machine learning algorithms, to predict furan concentrations. To establish the generalizability and robustness of the furan prediction model, this study investigates two distinct datasets from different geographical locations, Utility A and Utility B. Three scenarios are proposed: in the first scenario, a round-robin cross-validation method was used, with 75% of the data for training and the remaining 25% for testing. The second scenario involved training the model entirely on Utility A and testing it on Utility B. In the third scenario, the datasets were merged, and round-robin cross-validation was applied, similar to the first scenario. The findings reveal the effectiveness of machine learning algorithms in predicting furan concentrations, and particularly the stacked generalized ensemble method, offering a non-intrusive and cost-effective alternative to traditional testing methods. The results could significantly impact the maintenance strategies of power and distribution transformers, particularly in regions where furan testing facilities are not readily available.

Multi-spatial-scale land/use land cover influences on seasonally dominant water quality along Middle Ganga Basin.

Studying spatiotemporal water quality characteristics and their correlation with land use/land cover (LULC) patterns is essential for discerning the origins of various pollution sources and for informing strategic land use planning, which, in turn, requires a comprehensive analysis of spatiotemporal water quality data to comprehend how surface water quality evolves across different time and space dimensions. In this study, we compared catchment, riparian, and reach scale models to assess the effect of LULC on WQ. Using various multivariate techniques, a 14-year dataset of 20 WQ variables from 20 monitoring stations (67,200 observations) is studied along the Middle Ganga Basin (MGB). Based on the similarity and dissimilarity of WQPs, the K-means clustering algorithm classified the 20 monitoring stations into four clusters. Seasonally, the three PCs chosen explained 75.69% and 75% of the variance in the data. With PCs > 0.70, the variables EC, pH, Temp, TDS, NO2 + NO3, P-Tot, BOD, COD, and DO have been identified as dominant pollution sources. The applied RDA analysis revealed that LULC has a moderate to strong contribution to WQPs during the wet season but not during the dry season. Furthermore, dense vegetation is critical for keeping water clean, whereas agriculture, barren land, and built-up area degrade WQ. Besides that, the findings suggest that the relationship between WQPs and LULC differs at different scales. The stacked ensemble regression (SER) model is applied to understand the model's predictive power across different clusters and scales. Overall, the results indicate that the riparian scale is more predictive than the watershed and reach scales.

Estimating Sea Surface Salinity in the East China Sea Using Satellite Remote Sensing and Machine Learning

AbstractSea surface salinity (SSS) is a master variable in oceanography and important to understand marine biogeochemical and physical processes. In the East China Sea (ECS), a random forest based regression ensemble model (RF) was developed to estimate the SSS with a spatial resolution of ∼1 km based on a large synchronous data set of in situ SSS observations, MODIS‐derived remote sensing reflectance (Rrs) and sea surface temperature (SST). The model showed the best performance when the Rrs(412), Rrs(488), Rrs(555), Rrs(667), SST and Julian day (JD) were used as inputs, with a root mean square error (RMSE) of 0.84, mean absolute error (MAE) of 0.31 and coefficient of determination (R2) of 0.81 for model training (N = 4,504), and a RMSE of 0.77, MAE of 0.30 and R2 of 0.86 for the model test (N = 1,153). The accuracy of the SSS model was examined using an independent data set during the period of 2020–2022 with a RMSE of 0.66 and MAE of 0.39 (N = 2,151). The interannual and seasonal signal of modeled SSS of the ECS, showed that important drivers of variability are the Changjiang discharge and the East‐Asian monsoon. Applications of the model to other Chinese marginal seas (Yellow and Bohai seas) showed good agreement in distribution patterns when compared with the estimated SSS from NASA Soil Moisture Active Passive. Once more empirical oceanographic data is made available, this robust model can be applied to other regions retraining the model with informed local data sets.

EmPDBA: protein-DNA binding affinity prediction by combining features from binding partners and interface learned with ensemble regression model.

Protein-deoxyribonucleic acid (DNA) interactions are important in a variety of biological processes. Accurately predicting protein-DNA binding affinity has been one of the most attractive and challenging issues in computational biology. However, the existing approaches still have much room for improvement. In this work, we propose an ensemble model for Protein-DNA Binding Affinity prediction (emPDBA), which combines six base models with one meta-model. The complexes are classified into four types based on the DNA structure (double-stranded or other forms) and the percentage of interface residues. For each type, emPDBA is trained with the sequence-based, structure-based and energy features from binding partners and complex structures. Through feature selection by the sequential forward selection method, it is found that there do exist considerable differences in the key factors contributing to intermolecular binding affinity. The complex classification is beneficial for the important feature extraction for binding affinity prediction. The performance comparison of our method with other peer ones on the independent testing dataset shows that emPDBA outperforms the state-of-the-art methods with the Pearson correlation coefficient of 0.53 and the mean absolute error of 1.11kcal/mol. The comprehensive results demonstrate that our method has a good performance for protein-DNA binding affinity prediction. Availability and implementation: The source code is available at https://github.com/ChunhuaLiLab/emPDBA/.

Detecting COVID-19 from Chest X-rays Using Convolutional Neural Network Ensembles

Starting in late 2019, the coronavirus SARS-CoV-2 began spreading around the world and causing disruption in both daily life and healthcare systems. The disease is estimated to have caused more than 6 million deaths worldwide [WHO]. The pandemic and the global reaction to it severely affected the world economy, causing a significant increase in global inflation rates, unemployment, and the cost of energy commodities. To stop the spread of the virus and dampen its global effect, it is imperative to detect infected patients early on. Convolutional neural networks (CNNs) can effectively diagnose a patient’s chest X-ray (CXR) to assess whether they have been infected. Previous medical image classification studies have shown exceptional accuracies, and the trained algorithms can be shared and deployed using a computer or a mobile device. CNN-based COVID-19 detection can be employed as a supplement to reverse transcription-polymerase chain reaction (RT-PCR). In this research work, 11 ensemble networks consisting of 6 CNN architectures and a classifier layer are evaluated on their ability to differentiate the CXRs of patients with COVID-19 from those of patients that have not been infected. The performance of ensemble models is then compared to the performance of individual CNN architectures. The best ensemble model COVID-19 detection accuracy was achieved using the logistic regression ensemble model, with an accuracy of 96.29%, which is 1.13% higher than the top-performing individual model. The highest F1-score was achieved by the standard vector classifier ensemble model, with a value of 88.6%, which was 2.06% better than the score achieved by the best-performing individual model. This work demonstrates that combining a set of top-performing COVID-19 detection models could lead to better results if the models are integrated together into an ensemble. The model can be deployed in overworked or remote health centers as an accurate and rapid supplement or back-up method for detecting COVID-19.

Analyzing the impact of COVID-19 on the electricity demand in Austin, TX using an ensemble-model based counterfactual and 400,000 smart meters

The COVID-19 pandemic caused lifestyle changes and has led to the new electricity demand patterns in the presence of non-pharmaceutical interventions such as work-from-home policy and lockdown. Quantifying the effect on electricity demand is critical for future electricity market planning yet challenging in the context of limited smart metered buildings, which leads to limited understanding of the temporal and spatial variations in building energy use. This study uses a large scale private smart meter electricity demand data from the City of Austin, combined with publicly available environmental data, and develops an ensemble regression model for long term daily electricity demand prediction. Using 15-min resolution data from over 400,000 smart meters from 2018 to 2020 aggregated by building type and zip code, our proposed model precisely formalizes the counterfactual universe in the without COVID-19 scenario. The model is used to understand building electricity demand changes during the pandemic and to identify relationships between such changes and socioeconomic patterns. Results indicate the increase in residential usage , demonstrating the spatial redistribution of energy consumption during the work-from-home period. Our experiments demonstrate the effectiveness of our proposed framework by assessing multiple socioeconomic impacts with the comparison between the counterfactual universe and observations.

ESTIMATING BODY MASS INDEX FROM FACIAL IMAGES

A person's health status may have a significant impact on many aspects of their life, from mental health to lifespan to financial security. The health of a person can be calculated by a value which is called Body Mass Index (BMI), it uses both the height and weight of a person. Numerous variables, including physical health, mental health, and popularity, have been linked to BMI. With the increasing number of people being obese, self-diagnostic solutions for healthy weight monitoring are grabbing significant attention. Calculating BMI using the statistical formula requires precise measurements of the height and weight of a person and is time-consuming. The main objective of this study is to predict the BMI of a person by giving the image as input. While developing Fitness apps, we can use this system to detect the BMI of a person daily and suggest suitable exercises. The developed system can also be used to find whether a person is suffering from malnutrition and some other diseases that can be detected using BMI. The models used in this study are FaceNet, Ridge Linear Regression, Random Forest Regression, Support Vector Regression, and ensemble of regression models

Development of artificial neural network models to predict the PAMPA effective permeability of new, orally administered drugs active against the coronavirus SARS-CoV-2

Responding to the pandemic caused by SARS-CoV-2, the scientific community intensified efforts to provide drugs effective against the virus. To strengthen these efforts, the “COVID Moonshot” project has been accepting public suggestions for computationally triaged, synthesized, and tested molecules. The project aimed to identify molecules of low molecular weight with activity against the virus, for oral treatment. The ability of a drug to cross the intestinal cell membranes and enter circulation decisively influences its bioavailability, and hence the need to optimize permeability in the early stages of drug discovery. In our present work, as a contribution to the ongoing scientific efforts, we employed artificial neural network algorithms to develop QSAR tools for modelling the PAMPA effective permeability (passive diffusion) of orally administered drugs. We identified a set of 61 features most relevant in explaining drug cell permeability and used them to develop a stacked regression ensemble model, subsequently used to predict the permeability of molecules included in datasets made available through the COVID Moonshot project. Our model was shown to be robust and may provide a promising framework for predicting the potential permeability of molecules not yet synthesized, thus guiding the process of drug design.

Best practice for upscaling soil organic carbon stocks in salt marshes

Calculating the amount of soil organic carbon (SOC) stored in coastal environments, including salt marshes, is needed to determine their role in mitigating the Climate Crisis. Several techniques exist to calculate the SOC content of a unit of land from the upscaling of soil cores. However, no comprehensive assessment has been made on the performance of commonly used SOC upscaling techniques until now. We measured the SOC content of soil cores gathered from two Scottish salt marshes. Two SOC values were used for upscaling; SOC content for a 1 m standardised depth (as recommended by the IPCC), and SOC content of the modern marsh deposit (identified in the stratigraphy as a transition from organic-rich (marsh) to mineral-rich (intertidal flat) soil. Twenty-two upscaling techniques were used (SOC content × area, interpolative, and regression-based extrapolative calculations). Leave-one-out cross-validation procedures and prediction interval widths were used to assess the accuracy of each technique. Digital Terrain Models and Normalized Difference Vegetation Indices were used as covariate surfaces in the regression models. We found that marsh-scale SOC stocks varied by as much as fifty-two times depending on which sampling depth and upscaling technique was used. The largest differences emerged when comparing SOC stocks upscaled from 1 m deep and modern marsh deposits. Using the IPCC recommended 1 m sampling depth inflated the SOC stocks of salt marshes, as intertidal flat environments were included in the calculation. Ensemble regression models from the weighted average of seven machine learning algorithm outputs produced the highest upscaling accuracies across marshes and sampling depths. Simple SOC content × area calculations produced marsh-scale SOC stocks that were comparable to stock values produced by more advanced ensemble regression models. However, regression models produced detailed maps of SOC distribution across a marsh, and the associated uncertainty in the SOC values. Our findings are broadly applicable for other environments where large-scale SOC stock assessments and uncertainty are needed.

VAERHNN: Voting-averaged ensemble regression and hybrid neural network to investigate potent leads against colorectal cancer

In recent years, artificial intelligence (AI) has flourished in drug discovery and sped up the process of drug research and development. Recently, it has been demonstrated that the inhibition of phosphoglycerate kinase 1 (PGK1) can effectively inhibit colorectal cancer (CRC). In this paper, we propose an AI-based drug repurposing protocol, VAERHNN, to investigate potent leads against CRC. VAERHNN can comprehensively integrate the information of the target and its inhibitors or agonists for drug repurposing. During the protocol, we built a voting-averaged ensemble regression (VAER) model based on ensemble learning algorithm for molecular activity prediction. The VAER model outperforms other single ensemble learning regression models. Moreover, we also assemble a hybrid neural network (HNN) consisting of multiple neural networks to predict the drug-target affinity. In HNN, we picked a combination of several top-performing neural networks for weighted average computation. Hereafter, through the evaluation of voting-averaged scores from molecular docking and AI models, we singled out flavin adenosine dinucleotide (FAD) as a potential lead. Ultimately, molecular dynamics simulations and in vitro scratch and transwell assays confirmed the stability of FAD binding to PGK1 and that FAD can significantly inhibit the migration and invasion of CRC cells in vitro. In conclusion, through VAERHNN, we identified FAD as a potential lead compound against CRC, providing a new idea for CRC treatment. The source codes and data of this study are available at https://github.com/gxCaesar/VAERHNN.

Machine learning prediction models for compressive strength of calcined sludge-cement composites

• Compressive strength of calcined sludge-cement composites is studied under multiple influencing factors. • The prediction models of compressive strength of calcined sludge-cement composites are established by machine learning. • The robustness of compressive strength to different factors is analyzed and compared. Replacing part of ordinary Portland cement with sludge can reduce the use of cement while recycling sludge and achieve low CO 2 emissions, which is an environment-friendly method for sludge treatment. However, comprehensive research on compressive strength of calcined sludge-cement composites has not formed due to numerous influencing factors. In this paper, experiments are designed under six factors to study the effects of various factors on the compressive strength of calcined sludge-cement composites, involving ball milling time, calcination temperature, sludge replacement rate, curing age, admixtures amount of calcium chloride (CaCl 2 ) and calcium sulfate (CaSO 4 ). At the same time, this paper uses machine learning to establish six different regression prediction models to predict the compressive strength, including Linear Regression (LR), Support Vector Machine Regression (SVR), Random Forest Regression (RF), Multi-layer Perceptron-Artificial Neural Network Regression (MLP-ANN), Ensemble Regression and Convolutional Neural Network Regression (CNN). According to the results, CNN and Ensemble Regression models provide the excellent prediction accuracy. By comparing the robustness, curing age has the greatest impact on compressive strength, while the influence of ball milling time and CaSO 4 is small, which is consistent with the experimental results.

A new hybrid ensemble approach for the prediction of effluent total nitrogen from a full-scale wastewater treatment plant using a combined trickling filter-activated sludge system.

In this study, different K-nearest neighbors (KNN), support vector regression (SVR), decision tree (DT), and random forest (RF) algorithms integrated with the Bayesian optimization algorithm (BOP) have been applied as novel hybrid modeling/optimization tools to predict the total nitrogen in treated wastewater of Southern Tehran Wastewater Treatment Plant (STWWTP). In order to enhance the outcomes of hybrid models, the chosen sub-models (the best and least correlated hybrid models) were used to generate voting average and stacked regression ensemble models. Throughout the preprocessing step, two alternative scenarios were used to handle missing values from the samples, including elimination versus estimation via linear interpolation. The results of this research demonstrated that ensemble models were better than individual hybrid models, although not all ensemble models were superior to single models. The results also revealed that the stacking regression ensemble model using KNN-BOP and SVR-BOP as sub-models was the most superior model among the developed models, with the coefficient of determination (R2) = 0.640, root mean squared error (RMSE) = 2.378, and mean absolute error (MAE) = 1.838 on the test data. The best hybrid ensemble model that can accurately predict the concentration of total nitrogen (TN) in the effluent can give people a heads-up about water pollution caused by eutrophication before it gets bad.