AdaBoost Ensemble Learning Research Articles

Subway track foundation settlement deformation prediction based on the BiLSTM-AdaBoost model

The rapid economic expansion has spurred extensive construction near subway networks, impacting the stability of their track foundations. Consequently, it’s crucial to monitor and predict settlement in subway track foundations. However, the dynamic deformation patterns often exhibit nonlinearity and non-stationarity, posing challenges for traditional linear regression models. To tackle this, our study integrates the BiLSTM (bi-directional long short-term memory) network with the AdaBoost ensemble learning algorithm. Using settlement data from Shanghai metro monitoring points, the model is trained and evaluated employing R2 (coefficient of determination), MAE (mean absolute error), and RMSE (root mean square error). Results show that our proposed model displays superior predictive accuracy compared to the LSTM and the BiLSTM, with an average training set R2 of 0.99, test set R2 of 0.78, average MAE of 0.32 mm, and average RMSE of 0.4 mm. Consequently, for forecasting subway track foundation deformations, employing our network model ensures highly accurate predictive capabilities.

Quantification of play behaviour in calves using automated ultra-wideband location data and its association with age, weaning and health status

Play behaviour can act as an indicator of positive animal welfare. Previous attempts to predict play behaviour in farmed calves are limited because of the classification methods used, which lead to overestimation, and the short time periods that calves are observed. The study aimed to automatically classify and quantify play behaviour in farmed calves using location data from ultra-wide band sensors and to investigate factors associated with play behaviour. Location data were collected from 46 calves in three cohorts for a period of 18 weeks. Behavioural observations from video footage were merged with location data to obtain a total of 101.36 h of labelled data. An AdaBoost ensemble learning algorithm was implemented to classify play behaviour. To account for overestimation, generally seen in low-prevalence behaviours, an adjusted count technique was applied to the outputs of the classifier. Two generalized linear mixed models were fitted to investigate factors (e.g. age, health) associated with duration of play and number of play instances per day. Our algorithm identified play behaviour with > 94% accuracy when evaluated on the test set with no animals used for training, and 16% overestimation, which was computed based on the predicted number of samples of play versus the number of samples labelled as play on the test set. The instances and duration of play behaviour per day significantly decreased with age and sickness, whilst play behaviour significantly increased during and after weaning. The instances of play also significantly decreased as mean temperature increased. We suggest that the quantification method that we used could be used to detect and monitor other low prevalence behaviours (e.g. social grooming) from location data, including indicators of positive welfare.

Biomolecular Adsorption on Nanomaterials: Combining Molecular Simulations with Machine Learning.

Adsorption free energies of 32 small biomolecules (amino acids side chains, fragments of lipids, and sugar molecules) on 33 different nanomaterials, computed by the molecular dynamics - metadynamics methodology, have been analyzed using statistical machine learning approaches. Multiple unsupervised learning algorithms (principal component analysis, agglomerative clustering, and K-means) as well as supervised linear and nonlinear regression algorithms (linear regression, AdaBoost ensemble learning, artificial neural network) have been applied. As a result, a small set of biomolecules has been identified, knowledge of adsorption free energies of which to a specific nanomaterial can be used to predict, within the developed machine learning model, adsorption free energies of other biomolecules. Furthermore, the methodology of grouping of nanomaterials according to their interactions with biomolecules has been presented.

Feature Optimization-Based Machine Learning Approach for Czech Land Cover Classification Using Sentinel-2 Images

The Czech Republic is one of the countries along the Belt and Road Initiative, and classifying land cover in the Czech Republic helps to understand the distribution of its forest resources, laying the foundation for forestry cooperation between China and the Czech Republic. This study aims to develop a practical approach for land cover classification in the Czech Republic, with the goal of efficiently acquiring spatial distribution information regarding its forest resources. This approach is based on multi-level feature extraction and selection, integrated with advanced machine learning or deep learning models. To accomplish this goal, the study concentrated on two typical experimental regions in the Czech Republic and conducted a series of classification experiments, using Sentinel-2 and DEM data in 2018 as the main data sources. Initially, this study extracted various features, including spectral, vegetation, and terrain features, from the study area, then assessed and selected key features based on their importance. Additionally, this study also explored multi-level spatial contextual features to improve classification performance. The extracted features include texture and morphological features, as well as deep semantic information learned by utilizing a deep learning model, 3D CNN. Finally, an AdaBoost ensemble learning model with the random forest as the base classifier is designed to produce land cover classification maps, thus obtaining the spatial distribution of forest resources. The experimental results demonstrate that feature optimization significantly enhances the extraction of high-quality features of surface objects, thereby improving classification performance. Specifically, morphological and texture features can effectively enhance the discriminability between different features of surface objects, thereby improving classification accuracy. Utilizing deep learning networks enables more efficient extraction of deep feature information, further enhancing classification accuracy. Moreover, employing an ensemble learning model effectively boosts the accuracy of the original classification results from different individual classifiers. Ultimately, the classification accuracy of the two experimental areas reaches 92.84% and 93.83%, respectively. The user accuracies for forests are 92.24% and 93.14%, while the producer accuracies are 97.71% and 97.02%. This study applies the proposed approach for nationwide classification in the Czech Republic, resulting in an overall classification accuracy of 90.98%, with forest user accuracy at 91.97% and producer accuracy at 96.2%. The results in this study demonstrate the feasibility of combining feature optimization with the 3D Convolutional Neural Network (3DCNN) model for land cover classification. This study can serve as a reference for research methods in deep learning for land cover classification, utilizing optimized features.

Harvesting Route Detection and Crop Height Estimation Methods for Lodged Farmland Based on AdaBoost

Addressing the challenge of the current harvester route detection method’s reduced robustness within lodging-affected farmland environments and its limited perception of crop lodging, this paper proposes a harvesting operation image segmentation method based on SLIC superpixel segmentation and the AdaBoost ensemble learning algorithm. This segmentation enables two essential tasks. Firstly, the RANSAC algorithm is employed to extract the harvester’s operational route through straight-line fitting from the segmented image. Secondly, the method utilizes a 3D point cloud generated by binocular vision, combined with IMU information for attitude correction, to estimate the height of the harvested crop in front of the harvester. Experimental results demonstrate the effectiveness of this method in successfully segmenting the harvested and unharvested areas of the farmland. The average angle error for the detected harvesting route is approximately 1.97°, and the average error for crop height detection in the unharvested area is around 0.054 m. Moreover, the algorithm exhibits a total running time of approximately 437 ms. The innovation of this paper lies in its simultaneous implementation of two distinct perception tasks, leveraging the same image segmentation results. This approach offers a robust and effective solution for addressing both route detection and crop height estimation challenges within lodging-affected farmland during harvesting operations.

Generative Adversarial Networks With AdaBoost Ensemble Learning for Anomaly Detection in High-Speed Train Automatic Doors

Due to the scarcity of abnormal condition data in components of transportation systems, only normal condition data are typically used to train models for anomaly detection. One of the main challenges is the difficulty of properly representing the data distribution which is typically non-smooth, high-dimensional and on a manifold. This work develops an anomaly detection model based on an Auto-Encoder (AE) formed by the generator of a Generative Adversarial Network (GAN) and an auxiliary encoder to capture the sophisticated data structure. The reconstruction error of the AE is, then, used as anomaly score to detect anomalies. Additionally, an adaptive noise is added to the data to make easier the GAN optimization, an AdaBoost-based ensemble learning scheme is used to improve detection performance and a new approach for setting the hyperparameters of the AE-GAN model based on the derivation of a lower bound of the Jensen-Shannon divergence between generator and normal condition data distributions is developed. The method has been applied to synthetic and real data collected from automatic doors of high-speed trains.

Estimating the heavy metal contents in farmland soil from hyperspectral images based on Stacked AdaBoost ensemble learning

• Assessment of heavy metal pollution in soil base on hyperspectral image. • Estimating the soil heavy metal content by constructing ensemble learning model. • The estimation accuracy of heavy metal content in soil is improved. • Mapping the distribution trend of heavy metals in farmland soil. • The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. Heavy metal pollution poses a huge challenge to the soil environment. With the increasing pollution level, the traditional monitoring methods cannot quickly obtain information on large-area pollution. Therefore, a large-scale mapping method with high precision is urgently needed to effectively control heavy metal pollution. This study explored a method for mapping soil heavy metal concentrations through hyperspectral images. On this basis, a new Stacked AdaBoost ensemble learning algorithm was constructed to construct the inversion model of soil heavy metal contents. The characteristic spectral bands of heavy metals were extracted as model input variables using Pearson’s correlation coefficient and successive projections algorithm. With three sets of heavy metal content data, the prediction accuracy and mapping outcomes of various machine learning methods were compared. Furthermore, the potential sources of heavy metal pollution in the study area were analyzed based on the Moran’s index. The results showed that the Stacked AdaBoost model was relatively stable with higher accuracy than traditional machine learning models. For Cr, Cu, and As, the determination coefficients ( R 2 ) of the verification set were 0.66, 0.61, and 0.74, respectively. Afterward, the results of this model were used to map the heavy metal concentration over the study area. The mapping results suggested that the heavy metal conditions of soils in the Ganhetan area were caused by nature and human activities. The As pollution in agricultural soils was the most serious, with an exceedance rate of 38.66%. Industrial areas were potential sources of soil heavy metal pollution in the study area. In summary, the Stacked AdaBoost ensemble learning model provides detailed and reliable data for agricultural ecological protection and industrial pollution control, allowing the effective management of heavy metal pollution sources.

Comparison of Ensemble and Base Learner Algorithms for the Prediction of Machining Induced Residual Stresses in Turning of Aerospace Materials

Estimation of residual stresses is important to prevent the catastrophic failures of the components used in the aerospace industry. The objective of this work is to predict the machining induced residual stresses with bagging, boosting, and single-based machine learning models based on the design and cutting parameters used in turning of Inconel 718 and Ti6Al4V alloys. Experimentally measured residual stress data of these two materials was compiled from the literature including the surface material of the cutting tools, cooling conditions, rake angles as well as the cutting speed, feed, and width of cut to show the robustness of the models. These variables were also grouped with different combinations to clearly show the contribution and necessity of each element. Various predictive models in machine learning (AdaBoost, Random Forest, Artificial Neural Network, K-Neighbors Regressor, Linear Regressor) were then applied to estimate the residual stresses on the machined surfaces for the classified groups using the generated data. It was found that the AdaBoost algorithm was able to predict the machining induced residual stresses with the mean absolute errors of 18.1 MPa for IN718 alloy and 31.3 MPa for Ti6Al4V by taking into account all the variables while artificial neural network provides the lowest mean absolute errors for the Ti6Al4V alloy. On the other hand, linear regression model gives poor agreement with the experimental data. All the analyses showed that AdaBoost (boosting) ensemble learning, and artificial neural network models can be used for the prediction of the machining induced residual stresses with the small datasets of the IN718 and Ti6Al4V materials.

Kernel PLS with AdaBoost ensemble learning for particulate matters forecasting in subway environment

Excessive exposure to hazardous air pollutants remains the leading cause of serious cardio-respiratory disease among commuters. To monitor and improve the air quality in time, soft-sensing techniques in metro systems need to be introduced. Adaptive boosting (AdaBoost), as an ensemble learning method, has greater flexibility and adaptability. Compared with a single conventional model, it has a stronger capability to explain the complex features contained in air quality data. In this study, an improved scheme employs AdaBoost to predict key air quality variables, such as PM2.5, in subway environment. To eliminate the adverse effects of noise interference and the collinearity problem among data variables, kernel latent variables are embedded into the original AdaBoost. The proposed method combines kernel partial least squares with AdaBoost (KPLS-AdaBoost). Root mean square error (RMSE) was used to evaluate the prediction performance of KPLS-AdaBoost. The results demonstrate that KPLS-AdaBoost could achieve better modeling performance, providing a new way to realize real-time monitoring of hazardous air pollutants. More precisely, in terms of PM2.5 in the hall, the RMSE value was optimized by approximately 14.44–31.86% compared with that of AdaBoost, random forest, artificial neural network, PLS, KPLS, and PLS-AdaBoost. For PM2.5 at the platform, RMSE value can be decreased by 13.57–38.51%.

Design of Intrusion Detection System based on Cyborg intelligence for security of Cloud Network Traffic of Smart Cities

The Internet of things (IoT) is an important technology that is highly beneficial in establishing smart items, connections and cities. However, there are worries regarding security and privacy vulnerabilities in IoT in which some emerge from numerous sources, including cyberattacks, unsecured networks, data, connections or communication. This paper provides an ensemble intrusion strategy based on Cyborg Intelligence (machine learning and biological intelligence) framework to boost security of IoT enabled networks utilized for network traffic of smart cities. To do this, multiple algorithms such Random Forest, Bayesian network (BN), C5.0, CART and Artificial Neural Network were investigated to determine their usefulness in identifying threats and attacks-botnets in IoT networks based on cyborg intelligence using the KDDcup99 dataset. The results reveal that the AdaBoost ensemble learning based on Cyborg Intelligence Intrusion Detection framework facilitates dissimilar network characteristics with the capacity to swiftly identify different botnet assaults efficiently. The suggested framework has obtained good accuracy, detection rate and a decreased false positive rate in comparison to other standard methodologies. The conclusion of this study would be a valuable complement to the efforts toward protecting IoT-powered networks and the accomplishment of safer smart cities.

SCNTA

Real-time network inspection applications face a threat of vulnerability as high-speed networks continue to expand. For companies and ISPs, real-time traffic classification is an issue. The classifier monitor is made up of three modules: Capturing_of_Packets (CoP) and pre-processing, Reconciliation_of_Flow (RoF), and categorization of Machine Learning (ML). Based on parallel processing along with well-defined interfacing of data, the modules are framed, allowing each module to be modified and upgraded separately. The Reconciliation_of_Flow (RoF) mechanism becomes the output bottleneck in this pipeline. In this implementation, an optimal reconciliation process was used, resulting in an average delivery time of 0.62 seconds. In order to verify our method, we equated the results of the AdaBoost Ensemble Learning Algorithm (ABELA), Naive Bayes (NB), Decision Tree (DT), K-Nearest Neighbor (KNN), and Flexible Naive Bayes (FNB) in the classification module. The architectural design of the run time CSNTA categorization (flow-based) scheme is presented in this paper.

Bayesian Hyperparameter Optimization and Ensemble Learning for Machine Learning Models on Software Effort Estimation

In recent decades, various software effort estimation (SEE) algorithms have been suggested. Unfortunately, generating high-precision accuracy is still a major challenge in the context of SEE. The use of traditional techniques and parametric approaches is largely inaccurate because they produce biased and subjective accuracy. Meanwhile, none of the machine learning methods performed well. This study applies the AdaBoost ensemble learning method and random forest (RF), on the other hand the Bayesian optimization method is applied to determine the hyperparameters of this model. The PROMISE repository and the ISBSG dataset were used to build the SEE model. The developed model was comprehensively compared with four machine learning methods (classification and regression tree, k-nearest neighbor, multilayer perceptron, and support vector regression) under 3-fold cross validation (CV). It can be seen that the RF method based on AdaBoost ensemble learning and bayesian optimization outperforms this approach. In addition, the AdaBoost-based model assigns a feature importance rating, which makes it a promising tool in software effort prediction.

Recognition and Classification Method for Fiber Optical Vibration Signal Using AdaBoost Ensemble Learning

Recognition and Classification Method for Fiber Optical Vibration Signal Using AdaBoost Ensemble Learning

Automated Diagnosis of Childhood Pneumonia in Chest Radiographs Using Modified Densely Residual Bottleneck-Layer Features

Pneumonia is a severe infection that affects the lungs due to viral or bacterial infections such as the novel COVID-19 virus resulting in mild to critical health conditions. One way to diagnose pneumonia is to screen prospective patient’s lungs using either a Computed Tomography (CT) scan or chest X-ray. To help radiologists in processing a large amount of data especially during pandemics, and to overcome some limitations in deep learning approaches, this paper introduces a new approach that utilizes a few light-weighted densely connected bottleneck residual block features to extract rich spatial information. Then, shrinking data batches into a single vector using four efficient methods. Next, an adaptive weight setup is proposed utilizing Adaboost ensemble learning which adaptively sets weight for each classifier depending on the scores generated to achieve the highest true positive rates while maintaining low negative rates. The proposed method is evaluated using the Kaggle chest X-ray public dataset and attained an accuracy of 99.6% showing superiority to other deep networks-based pneumonia diagnosis methods.

Application of artificial intelligence ensemble learning model in early prediction of atrial fibrillation

BackgroundAtrial fibrillation is a paroxysmal heart disease without any obvious symptoms for most people during the onset. The electrocardiogram (ECG) at the time other than the onset of this disease is not significantly different from that of normal people, which makes it difficult to detect and diagnose. However, if atrial fibrillation is not detected and treated early, it tends to worsen the condition and increase the possibility of stroke. In this paper, P-wave morphology parameters and heart rate variability feature parameters were simultaneously extracted from the ECG. A total of 31 parameters were used as input variables to perform the modeling of artificial intelligence ensemble learning model.ResultsThis paper applied three artificial intelligence ensemble learning methods, namely Bagging ensemble learning method, AdaBoost ensemble learning method, and Stacking ensemble learning method. The prediction results of these three artificial intelligence ensemble learning methods were compared. As a result of the comparison, the Stacking ensemble learning method combined with various models finally obtained the best prediction effect with the accuracy of 92%, sensitivity of 88%, specificity of 96%, positive predictive value of 95.7%, negative predictive value of 88.9%, F1 score of 0.9231 and area under receiver operating characteristic curve value of 0.911.ConclusionIn feature extraction, this paper combined P-wave morphology parameters and heart rate variability parameters as input parameters for model training, and validated the value of the proposed parameters combination for the improvement of the model’s predicting effect. In the calculation of the P-wave morphology parameters, the hybrid Taguchi-genetic algorithm was used to obtain more accurate Gaussian function fitting parameters. The prediction model was trained using the Stacking ensemble learning method, so that the model accuracy had better results, which can further improve the early prediction of atrial fibrillation.

AdaBoost Ensemble Learning on top of Naive Bayes Algorithm to Discriminate Fake and Genuine News from Social Media

There is a continuous increase in social media usage and a huge interaction takes place between users. In this context, fake news circulation or flood becomes a real thread for social media users from various perspectives. Fake news is defined as presentation of misleading information as true news. In this view, fake news is fabricated news that aims to manipulate public opinion to obtain a benefit. For example, increasing readership for profiting through clickbaits is such an aim. Social media users are manipulated through attention grabbing headlines or web-links to increase number of visitors. Therefore, an automated fake news identification model can be used by social media users to filter inadvertent web-traffic. For this goal machine learning algorithms are used in the literature as a solution for fake news problem. In machine learning literature, advancing performance of the base models is crucial. Ensemble learning is one of the key solutions to enhance model efficiency. In this work, we first generated a set of baseline machine learning algorithms and we tested them in terms of their fake news identification ability. We then made use of ensemble learning strategy to further enhance obtained results. More precisely, we obtained Naïve Bayes Multinomial classifier as the best fake news predictor having 96.74 % accuracy. We then further improved this prediction ability to 98.2 % by applying an AdaBoost ensemble learning strategy.

Adaptive Abnormal Oil Temperature Diagnosis Method of Transformer Based on Concept Drift

The diagnosis of abnormal transformer oil temperature is of great significance to guarantee the normal operation of the transformer. Due to concept drift, the oil temperature abnormal diagnosis of the oil-immersed main power transformer is usually unstable via the classic data mining method. Thus, this paper proposes an adaptive abnormal oil temperature diagnosis method (AAOTD) of the transformer based on concept drift. First, the bagging ensemble learning method was used to predict the oil temperature. Then, abnormal diagnosis was performed based on the difference between the predicted oil temperature and the actual measured oil temperature. At the same time, based on the concept drift detection strategy and Adaboost ensemble learning methods, adaptive update of the base classifier in the abnormal diagnosis model was realized. Experiments validated that the algorithm proposed in this paper can significantly reduce the influence of concept drift and has higher oil temperature prediction accuracy. Furthermore, since this method only utilizes the existing power grid data resources to realize abnormal oil temperature diagnosis without extra monitoring equipment, it is an economic and efficient solution for practical scenarios in the electric power industry.

Different Approaches to Reducing Bias in Classification of Medical Data by Ensemble Learning Methods

In this study, different models were created to reduce bias by ensemble learning methods. Reducing the bias error will improve the classification performance. In order to increase the classification performance, the most appropriate ensemble learning method and ideal sample size were investigated. Bias values and learning performances of different ensemble learning methods were compared. AdaBoost ensemble learning method provided the lowest bias value with n: 250 sample size while Stacking ensemble learning method provided the lowest bias value with n: 500, n: 750, n: 1000, n: 2000, n: 4000, n: 6000, n: 8000, n: 10000, and n: 20000 sample sizes. When the learning performances were compared, AdaBoost ensemble learning method and RBF classifier achieved the best performance with n: 250 sample size (ACC = 0.956, AUC: 0.987). The AdaBoost ensemble learning method and REPTree classifier achieved the best performance with n: 20000 sample size (ACC = 0.990, AUC = 0.999). In conclusion, for reduction of bias, methods based on stacking displayed a higher performance compared to other methods.

IoTBoT-IDS: A novel statistical learning-enabled botnet detection framework for protecting networks of smart cities

The rapid proliferation of the Internet of Things (IoT) systems, has enabled transforming urban areas into smart cities. Smart cities’ paradigm has resulted in improved quality of life and better services to citizens, like smart healthcare, smart parking, smart transport, smart buildings, smart homes, and so on. One of the major challenges of IoT devices is the limited capacity of their battery because the devices consume a large amount of energy once they communicate with each other. Furthermore, the IoT-based smart systems would contain sensitive data about network systems, introducing serious privacy and security issues. IoT-based smart systems are highly exposed to botnet attacks. Examples of such attacks are Mirai and BASHLITE malware launched from compromised surveillance devices, which are common in smart cities, resulting in paralysis of Internet-based services through distributed denial of service (DDoS) attacks. Such DDoS attacks on IoT devices and their networks further threaten the emerging concept of sustainable smart cities. To discover such cyberattacks, this paper proposes a novel statistical learning-based botnet detection framework, called IoTBoT-IDS, which protects IoT-based smart networks against botnet attacks. IoTBoT-IDS captures the normal behavior of IoT networks by applying statistical learning-based techniques, using Beta Mixture Model (BMM) and a Correntropy model. Any deviation from the normal behavior is detected as an anomalous event. To evaluate IoTBoT-IDS, three benchmark datasets generated from realistic IoT networks were used. The evaluation results showed that IoTBoT-IDS effectively identifies various types of botnets with an average detection accuracy of 99.2%, which is higher by about 2–5% compared with compelling intrusion detection methods, namely AdaBoost ensemble learning, fuzzy c-means, and deep feed forward neural networks.

Automatic colonic polyp detection using integration of modified deep residual convolutional neural network and ensemble learning approaches

Background and ObjectiveThe increased incidence of colorectal cancer (CRC) and its mortality rate have attracted interest in the use of artificial intelligence (AI) based computer-aided diagnosis (CAD) tools to detect polyps at an early stage. Although these CAD tools have thus far achieved a good accuracy level to detect polyps, they still have room to improve further (e.g. sensitivity). Therefore, a new CAD tool is developed in this study to detect colonic polyps accurately. MethodsIn this paper, we propose a novel approach to distinguish colonic polyps by integrating several techniques, including a modified deep residual network, principal component analysis and AdaBoost ensemble learning. A powerful deep residual network architecture, ResNet-50, was investigated to reduce the computational time by altering its architecture. To keep the interference to a minimum, median filter, image thresholding, contrast enhancement, and normalisation techniques were exploited on the endoscopic images to train the classification model. Three publicly available datasets, i.e., Kvasir, ETIS-LaribPolypDB, and CVC-ClinicDB, were merged to train the model, which included images with and without polyps. ResultsThe proposed approach trained with a combination of three datasets achieved Matthews Correlation Coefficient (MCC) of 0.9819 with accuracy, sensitivity, precision, and specificity of 99.10%, 98.82%, 99.37%, and 99.38%, respectively. ConclusionsThese results show that our method could repeatedly classify endoscopic images automatically and could be used to effectively develop computer-aided diagnostic tools for early CRC detection.