Bagging Approach Research Articles

IoT and Ensemble Long-Short-Term-Memory-Based Evapotranspiration Forecasting for Riyadh.

Evapotranspiration (ET) is the fundamental component of efficient water resource management. Accurate forecasting of ET is essential for efficient water utilization in agriculture. ET forecasting is a complex process due to the requirements of large meteorological variables. The recommended approach is based on the Internet of Things (IoT) and an ensemble-learning-based approach for meteorological data collection and ET forecasting with limited meteorological conditions. IoT is part of the recommended approach to collect real-time data on meteorological variables. The daily maximum temperature (T), mean humidity (Hm), and maximum wind speed (Ws) are used to forecast evapotranspiration (ET). Long short-term memory (LSTM) and ensemble LSTM with bagged and boosted approaches are implemented and evaluated for their accuracy in forecasting ET values using meteorological data from 2001 to 2023. The results demonstrate that the bagged LSTM approach accurately forecasts ET with limited meteorological conditions in Riyadh, Saudi Arabia, with the coefficient of determination (R2) of 0.94 compared to the boosted LSTM and off-the-shelf LSTM with R2 of 0.91 and 0.77, respectively. The bagged LSTM model is also more efficient with small values of root mean squared error (RMSE) and mean squared error (MSE) of 0.42 and 0.53 compared to the boosted LSTM and off-the-shelf LSTM models.

Modified boosting and bagging for building tilt rate prediction in tunnel construction

The building tilt rate (BTR) prediction problem is of great importance for safety management in metro tunnel construction. To improve the BTR prediction accuracy, it is pertinent to accurately identify and properly handle less-reliable data which refers to data with an inconsistent input-output relation. A modified boosting and bagging approach is thus proposed, i.e., BooBag. Specifically, modified bagging of BooBag can create multiple artificial labels for differentiating reliable data and less-reliable data while modified boosting can be used for pace management to avoid both underfitting and overfitting. Investigations of a practical BTR prediction case show that BooBag can accurately differentiate reliable data and less-reliable data, which leads to a smaller BTR prediction error, which is further validated via extensive independence, strategy, and resilience comparisons. BooBag can be used for safety management in tunnel construction as a pre-step to be combined with varied machine learning approaches for key safety indicator prediction, irregularity/anomaly detection, fault diagnosis, etc.

EnsMulHateCyb: Multilingual hate speech and cyberbully detection in online social media

Nowadays, users across the globe interact with one another for information exchange, communication, and association on various online social media. However, some individuals exploit these venues for malicious practices like hate speech and cyberbully. In this paper, we present an improved multilingual hate speech and cyberbully detection model using bagging-stacking based hybrid ensemble deep learning techniques. The proposed model utilizes Bi-directional Long Short-Term Memory (BiLSTM), Bi-directional Gated Recurrent Unit (Bi-GRU), Convolutional Neural Network (CNN), and Long Short-Term Memory (LSTM) techniques to enhance the overall performance. We first preprocess the multilingual data streams followed by adoption of Global vectors for word Representation (GloVe) embeddings to convert words to a vector representation in parallel enabling the data streams for binary classification task. In order to construct an architecture for the detection of hate speech and cyberbully, we introduce a heterogeneous fusion of multiple effective models in a unique approach such that CNN-LSTM utilizes a stacking approach with stochastic gradient descent to achieve optimal weights, whereas all the base learners used bagging ensemble approach with cross-validation to reach optimal weights. The final output layer of the proposed ensemble deep learning architecture is achieved using a super learner approach on base learners. To show the efficacy of the proposed model, we conduct the simulation on a total of nine real-world social media datasets in different languages and compared the results with other contemporary hate speech and cyberbully detection methods. The collected findings show that the proposed model outperforms other models on considered datasets and shows an improvement of at least 4.44% in F1 scores.

Diabetes prediction using Hybrid Bagging Classifier

Diabetes is one of those diseases where the misdiagnosis may cause the heart failure, paralysis, loss of life and more. Diagnosis of such chronic diseases is essential and should be done carefully as it is the most common disease. According to the recent statistics in India it is major cause of death. Hence early prediction of diabetes may decrease the cause of death and increases the survival rate. Several Machine Learning algorithms were applied for the early detection of diabetes. These existing methods face the problems such as time consuming, feature selection not giving good accuracy, specificity and no accurate results. To overcome this problem, the proposed methodology implemented enhanced Recursive Feature Elimination (RFE) for feature selection. Then it classifies the diabetes using Hybrid Bagging Classifier because hybrid approach tunes the parameters of bagging approach according to the dataset.

Advanced modeling based on machine learning for evaluation of drug nanoparticle preparation via green technology: Theoretical assessment of solubility variations

Variability of drug solubility in supercritical solvent was investigated theoretically in this study using multiple machine learning models. The method of supercritical preparation can be used to manufacture solid-dosage drugs with enhanced solubility which can increase drug bioavailability and efficacy. In this study, we applied bagging regression to three core models, namely SVM, KNN, and linear regression (LR) to predict the solubility and density of Hyoscine drug based on Temperature and Pressure parameters as inputs. The WOA optimization algorithm was used to tune hyperparameters, and the performance of these models was compared with r2, RMSE, MAE, and AARD% metrics. The results revealed that the BAG-KNN model outperformed the others in predicting solubility, with an r2 score of 0.989 and a low RMSE of 0.041. Also, the BAG-LR model had the highest AARD% value of 7.50974, indicating that it has higher average relative deviation from the actual density values. Overall, the bagging approach proved to be effective in improving the accuracy of the core models, and the WOA optimization algorithm was successful in fine-tuning their hyperparameters. These findings can have significant implications for the accurate prediction of chemical properties in various fields, such as drug discovery and material science.

An intelligent identification and classification system for malicious uniform resource locators (URLs).

Uniform Resource Locator (URL) is a unique identifier composed of protocol and domain name used to locate and retrieve a resource on the Internet. Like any Internet service, URLs (also called websites) are vulnerable to compromise by attackers to develop Malicious URLs that can exploit/devastate the user's information and resources. Malicious URLs are usually designed with the intention of promoting cyber-attacks such as spam, phishing, malware, and defacement. These websites usually require action on the user's side and can reach users across emails, text messages, pop-ups, or devious advertisements. They have a potential impact that can reach, in some cases, to compromise the machine or network of the user, especially those arriving by email. Therefore, developing systems to detect malicious URLs is of great interest nowadays.This paper proposes a high-performance machine learning-based detection system to identify Malicious URLs. The proposed system provides two layers of detection. Firstly, we identify the URLs as either benign or malware using a binary classifier. Secondly, we classify the URL classes based on their feature into five classes: benign, spam, phishing, malware, and defacement. Specifically, we report on four ensemble learning approaches, viz. the ensemble of bagging trees (En_Bag) approach, the ensemble of k-nearest neighbor (En_kNN) approach, and the ensemble of boosted decision trees (En_Bos) approach, and the ensemble of subspace discriminator (En_Dsc) approach. The developed approaches have been evaluated on an inclusive and contemporary dataset for uniform resource locators (ISCX-URL2016). ISCX-URL2016 provides a lightweight dataset for detecting and categorizing malicious URLs according to their attack type and lexical analysis. Conventional machine learning evaluation measurements are used to evaluate the detection accuracy, precision, recall, F Score, and detection time. Our experiential assessment indicates that the ensemble of bagging trees (En_Bag) approach provides better performance rates than other ensemble methods. Alternatively, the ensemble of the k-nearest neighbor (En_kNN) approach provides the highest inference speed. We also contrast our En_Bag model with state-of-the-art solutions and show its superiority in binary classification and multi-classification with accuracy rates of 99.3% and 97.92%, respectively.

Preoperative 18F-FDG PET/CT radiomics analysis for predicting HER2 expression and prognosis in gastric cancer.

We aimed to establish and validate 2 machine learning models using 18F-flurodeoxyglucose positron emission tomography/computed tomography (18F-FDG PET/CT) radiomic features to predict human epidermal growth factor receptor 2 (HER2) expression and prognosis in gastric cancer (GC) patients. We retrospectively enrolled 90 patients diagnosed with GC, including their clinical information and the 18F-FDG PET/CT images. Patients were allocated to a training cohort of 72 patients and an independent validation cohort (IVC) of 18 patients. There were 2,100 radiomic features extracted from the 18F-FDG PET/CT scans. A sequential combination of multivariate and univariate feature selection was applied, including sequential forward selection and a redundancy-based analysis. The justification of the model performance was conducted by cross-validation analysis on the training set and an independent validation analysis. The machine learning models were developed using a balanced bagging approach for HER2 expression prediction and prognosis prediction, which differentiated HER2 positive expression from negative expression in the IVC with an area under the receiver operating characteristic curve (AUC) of 0.72, sensitivity of 0.85, and specificity of 0.80. The IVC for prognosis prediction achieved an AUC of 0.75, sensitivity of 0.82, and specificity of 0.71. We also conducted a reasonable interpretation for the selected features in each classification task from multiple aspects, including normalized feature importance analysis and statistical correlation analysis with the clinical features that were defaulted to be effective. 18F-FDG PET/CT radiomics analysis with a machine learning model provides a quantitative, efficient, and objective mechanism for predicting HER2 expression and prognosis in GC patients.

Effect of graphene and MXene as 2D filler material on physico‐mechanical properties of hemp/E‐glass fibers reinforced hybrid composite: A comparative study

AbstractThe present study concentrates on the mechanical, morphological, water absorption and burning behavior of hemp/E‐glass fiber reinforced hybrid composites (FRHC). The surface adhesive characteristics of the acquired bi‐directional hemp fiber mats were enhanced by treating them with an alkali solution. Furthermore, the present work extends to identify the effects of two 2‐dimensional graphene and MXene nanoparticles as filler on the hardness, flexural and tensile behavior as well as water absorption and flammability characteristics of the FRHC. The glass fiber (G), hemp fiber (H), and hemp/glass (G/H) fiber reinforced epoxy LY556 laminates were fabricated by using the hand lay‐up method followed by the vacuum bagging approach. A 30% and 12% increase in the flexural and tensile strength of the H/G hybrid composite has been observed by adding 1% of graphene as an additional reinforcement compared with the neat FRHC. However, the use of MXene as additional reinforcements shows 45% and 10% enhancement in the flexural and tensile strength of the H/G hybrid composite. In addition, the water absorption capacity of MXene‐filled G/H hybrid composites is found to be almost half of that filled with graphene. Similarly, the flammability results reveal that the burning rate of graphene‐reinforced H/G hybrid composite is 17.44 mm/min. When comparing the mechanical, water absorption, and flammability characteristics of graphene and MXene‐reinforced G/H hybrid composites, graphene gives more intensifying results than MXene. The scanning electron microscopic studies are also incorporated to investigate the fractured surface topography of the FRHC.

Improving the performance of fermionic neural networks with the Slater exponential Ansatz

AbstractIn this work, we propose a technique for the use of fermionic neural networks (FermiNets) with the Slater exponential Ansatz for electron‐nuclear and electron–electron distances, which provides faster convergence of target ground‐state energies due to a better description of the interparticle interaction in the vicinities of the coalescence points, while the accuracies of results, obtained with original and input‐modified FermiNets are similar due to high expressiveness of underlying neural network. Our analysis of learning curves indicates on the possibility to obtain accurate energies with smaller batch sizes using arguments of the bagging approach. In order to obtain even more accurate results for the ground‐state energies, we propose an extrapolation scheme for estimating Monte Carlo integrals in the limit of an infinite number of points. Numerical tests for a set of molecules demonstrate a good agreement with the results of the original FermiNets approach (achieved with larger batch sizes than required by our approach) and with results of the coupled‐cluster singles and doubles with perturbative triples (CCSD(T)) method that are calculated in the complete basis set limit.

A Weighted Deep Ensemble for Indian Sign Language Recognition

This work concentrates on developing an Indian sign language (ISL) recognition system using a forearm-worn wearable device to assist hearing-impaired persons. A novel ensemble of convolution neural networks (CNN) is proposed for robust ISL recognition using multi-sensor data. The accuracy for classification of 50 ISL signs improved from 92.5% obtained using a single CNN to 94.2% with 10 ensemble members created using the bagging approach and soft-voting for decision aggregation. Then, the ensemble of CNNs was optimized using weighted voting, where the weights were determined using a differential evolution algorithm. This further improved the classification accuracy to 96.6% with 10 ensemble members.

Phishing URL Detection using LSTM Based Ensemble Learning Approaches

Increasing incidents of phishing attacks tempt a significant challenge for cybersecurity personals. Phishing is a deceitful venture with an intention to steal confidential information of an organization or an individual. Many works have been performed to build anti-phishing solutions over the years, but attackers are coming with new manoeuvres from time to time. Many of the existing techniques are experimented based on limited set of URLs and dependent on other software to collect domain related information of the URLs. In this paper, with an aim to build a more accurate and effective phishing attack detection system, we used the concept of ensemble learning using Long Short-Term Memory (LSTM) models. We proposed ensemble of LSTM models using bagging approach and stacking approach. For performing classification using LSTM method, no separate feature extraction is done. Ensemble models are built integrating the predictions of multiple LSTM models. Performances of proposed ensemble LSTM methods are compared with five different machine learning classification methods. To implement these machine learning algorithms, different URL based lexical features are extracted. Mutual Information based feature selection algorithm is used to select more relevant features to perform classifications. Both the bagging and the stacking approaches of ensemble learning using LSTM models outperform other machine learning techniques. The results are compared with other anti-phishing solutions implemented using deep learning methods. Our approaches have proved to be the more accurate one with a low false positive rate of less than 0.15% performed comparatively on a larger dataset.

Machine Learning-Based Day-Ahead Prediction of Price-Setting Scheduled Energy in the Korean Electricity Trading Mechanism

Power generation companies, which participate in the electricity trading mechanism, need to determine optimal capacity bidding strategy to maximize their operational profit in Korea. Since the Price-setting Power Generation Schedule determines the profit of power generators, it is important to predict Price-setting Scheduled Energy during the trading day before the bidding phase. We propose a methodology for predicting the Pricing-setting Scheduled Energy from the power exchange without optimizing it. Instead of simulating the planning process, machine learning algorithms are applied and compared in the process of predicting the Pricing-setting Scheduled Energy. The input variables consist of seasonal and price information including calendar, fuel cost, and system marginal price. Three categories of machine learning (ML) algorithms including single, bagging and boosting approaches are implemented and tested to compare their performances. The computational experiments show that ML algorithms with price variable are shown to be better in terms of the considered measures. In addition, boosting approach is more effective than single and bagging approaches.

SMS Spam Detection using Relevance Vector Machine

Spam in Short Message Service(SMS) is a serious issue that impacts mobile phone consumers all around the world. Many strategies have been applied using several deep learning and machine learning techniques to overcome these issues. The bagging approach is used in the study to combine four different algorithms, namely RVM, SVM, Naive Bayes, and KNN. Then the final prediction is calculated from the predictions obtained from each of these algorithms by using a majority-based voting approach. So, this paper offers research on the comparative analysis of various text classification algorithms for accurately detecting and classifying spam SMS messages. The dataset is first preprocessed and then vectorized using the TF-IDF method which gives more importance to the less frequent words rather than common words. The Relevance vector machine (RVM) implementation on the dataset, achieves the best performance on this dataset with an F1 score of 0.975175. According to the study's findings, the suggested RVM model may successfully categorize SMS spam messages and be applied in practical settings.

An effective ensemble method for missing data imputation

The presence of missing data in a dataset plays a vital role in the design of classification, clustering, or regression methods. An efficient missing data imputation can enhance the overall performance of a machine learning method. This paper ensembles k-nearest neighbour imputation, local least square imputation, miss forest imputation, and k-means clustering imputation using the bagging approach to handle missing values over a wide range of datasets. The method has been tested with eight different datasets in terms of root mean square error, median absolute percentage error, mean absolute percentage error, and standard deviation. Experimental results show that our method gives a low error rate compared to its closed competitors.

An Adaptive Technique for Underwater Image Enhancement with CNN and Ensemble Classifier

Image Restoration is a significant phase to process images for their enhancement. Underwater photographs are subject to quality issues such as blurry photos, poor contrast, uneven lighting, etc. Image processing is crucial in the processing of these degraded images. This research introduced an ensemble-based classifier based on the bagging approach to enhance UW images. The support vector machine and random forest classifiers serve as the ensemble classifier's main classifiers. Additionally, to complement the feature optimization technique, the proposed ensemble classifier leverages particle swarm optimization. The feature selection method for the classifier is improved by the feature optimization process. To validate results, underwater images are collected by the Kaggle repository. In this process, Extreme Learning (EL) and Convolution Neural Network (CNN) are compared with suggested algorithms. The simulation results indicate that the proposed algorithm outperformed the existing work.

Improvement of interfacial adhesion of CuO nanostructured carbon fiber reinforced polymer composites

AbstractHydrothermal technique was used to deposit copper‐oxide (CuO) nanostructures on woven carbon fibers (WCF) to produce a nanostructured interphase that increased the interfacial strength with an epoxy resin matrix. In order to create laminated hybrid composites, CuO‐modified WCF was reinforced with mixture of bisphenol‐A epoxy resin and dimethyl aniline hardener as matrix using the vacuum bagging approach. Enhanced mechanical qualities are the result of increased volume fraction of CuO nanostructures, where the void content is minimal. The mechanical characteristics such as impact strength and tensile strength of CuO‐coated WCF reinforced epoxy resin composite samples were assessed using drop‐down impact test and universal material testing system. The outcomes demonstrate a considerable improvement in impact energy absorption (74.8%), elastic modulus (52%) tensile strength (42%) and in‐plane shear strength (32%) for the CuO‐coated WCF reinforced epoxy resin composites. This work demonstrates that the development of CuO nanostructures can lessen composite delamination and enhance composite performance because of the increase in interfacial surface area between the matrix and the fiber by CuO nanostructures potentially expanding the spectrum of structural applications for these materials.

One size does not fit all: Variation in anatomical traits associated with emersion behavior in mudskippers (Gobiidae: Oxudercinae)

Life histories involving transitions between differing habitats (i.e., aquatic to terrestrial or marine to freshwater) require numerous anatomical, physiological, and behavioral changes. Often, the traits associated with these changes are thought to come in suites, but all traits thought to be associated with particular life histories may not be required. While some traits are found in all species with a particular habitat transition, a grab bag approach may apply to other traits in that any trait may be sufficient for successful habitat transitions. We examine patterns of morphological traits associated with prolonged emersion in mudskipper, an amphibious fishes clade, where prolonged emersion appears twice. We test the evolutionary history of multiple characteristics associated with cutaneous respiration. We find most traits thought to be key for prolonged emersion show no phylogenetic signal and no tight correlation with prolonged emersion. Such traits appear in species with prolonged emersion but also non-emerging species. Only capillary density, which, when increased, allows for increased oxygen absorption, shows strong phylogenetic signal and correlation with prolonged emersion. Further experimental, functional genomics, and observational studies are needed to fully understand the mechanisms associated with each of these traits. With respect to traits associated with other particular behaviors, a comparative framework can be helpful in identifying evolutionary correlates.

Sparse Canonical Correlation Analysis Postprocessing Algorithms for GCM Daily Rainfall Forecasts

Abstract We introduce two variants of canonical correlation analysis (CCA) for model output statistics of GCM forecasts of daily rainfall. These approaches link the coarse-gridded GCM forecasts with the reference field through a projection onto highly correlated basis vectors to address the recurrent errors in daily rainfall forecasts due to spatial bias and subgrid variability. The first model, namely, sparse CCA (SCCA), includes the sparsity feature into the ordinary CCA to provide a reduced number of canonical coefficients. The second model (B-SCCA) employs the bagging approach to reduce the variance in the predictions due to the sample variability in the derived canonical series. The models are tested using simulated data imposed with a strong spatial bias, and then using subseasonal rainfall forecasts provided by the NASA GMAO GEOS model under the SubX project, as well as gridded rainfall data (MSWEP product) for the region of South Korea. A linear regression model is chosen as the baseline postprocessing algorithm and ordinary CCA is also evaluated against the proposed models. As for the simulated data, the SCCA model confirms its ability to address spatial bias in forecast fields compared with the baseline model. For the actual forecasts, the leading improvements of SCCA and B-SCCA over the baseline model are for the S1 skill score, suggesting that these models offer a relative gain in reproducing the spatial gradient of the reference rainfall field, which is relevant in hydrological applications that require a sound representation of spatial variability. Our results also highlight the importance of prefiltering the input data before applying CCA in such settings.

Prediction of Autogenous Shrinkage of Concrete Incorporating Super Absorbent Polymer and Waste Materials through Individual and Ensemble Machine Learning Approaches.

The use of superabsorbent polymers, sometimes known as SAP, is a tremendously efficacious method for reducing the amount of autogenous shrinkage (AS) that occurs in high-performance concrete. This study utilizes support vector regression (SVR) as a standalone machine-learning algorithm (MLA) which is then ensemble with boosting and bagging approaches to reduce the bias and overfitting issues. In addition, these ensemble methods are optimized with twenty sub-models with varying the nth estimators to achieve a robust R2. Moreover, modified bagging as random forest regression (RFR) is also employed to predict the AS of concrete containing supplementary cementitious materials (SCMs) and SAP. The data for modeling of AS includes water to cement ratio (W/C), water to binder ratio (W/B), cement, silica fume, fly ash, slag, the filer, metakaolin, super absorbent polymer, superplasticizer, super absorbent polymer size, curing time, and super absorbent polymer water intake. Statistical and k-fold validation is used to verify the validation of the data using MAE and RMSE. Furthermore, SHAPLEY analysis is performed on the variables to show the influential parameters. The SVM with AdaBoost and modified bagging (RF) illustrates strong models by delivering R2 of approximately 0.95 and 0.98, respectively, as compared to individual SVR models. An enhancement of 67% and 63% in the RF model, while in the case of SVR with AdaBoost, it was 47% and 36%, in RMSE and MAE of both models, respectively, when compared with the standalone SVR model. Thus, the impact of a strong learner can upsurge the efficiency of the model.

Bagging-based ridge estimators for a linear regression model with non-normal and heteroscedastic errors

Regression analysis is used to predict a dependent variable using one or more independent variables. In the linear regression model, when the independent variables are highly correlated, it leads toward the problem of multicollinearity. Subsequently, the ordinary least squares estimates become inconsistent and may lead to wrong inferences. In such a situation, ridge regression is the most commonly adopted technique. In this paper, we propose some new bootstrap aggregation (bagging) based ridge estimators. The performance of the proposed estimators is evaluated by a simulation study in terms of minimum mean squared error. The simulation results indicate that in the presence of multicollinearity with non-normal or heteroscedastic errors, the bagging-based ridge estimators perform better than conventional ridge estimators. The estimation of biasing parameters using bagging approach promotes the performance of the conventional ridge estimators. Finally, the real-life example is used to demonstrate the application of proposed estimators.