LightGBM Classifier Research Articles

RAIDS: Robust autoencoder-based intrusion detection system model against adversarial attacks

Machine learning-based intrusion detection systems (IDS) are essential security functions in conventional and software-defined networks alike. Their success and the security of the networks they protect depend on the accuracy of their classification results. Adversarial attacks against machine learning, which seriously threaten any IDS, are still not countered effectively. In this study, we first develop a method that employs generative adversarial networks to produce adversarial attack data. Then, we propose RAIDS, a robust IDS model, designed to be resilient against adversarial attacks. In RAIDS, an autoencoder's reconstruction error is used as a prediction value for a classifier. Also, to prevent the attacker from guessing about the feature set, multiple feature sets are created and used to train baseline machine learning classifiers. A LightGBM classifier is then trained with the results produced by two autoencoders and an ensemble of baseline machine learning classifiers. The results show that the proposed robust model can increase overall accuracy by at least 13.2% and F1-score by more than 110% against adversarial attacks without the need for adversarial training.

Gradient Boosting Approach to Predict Energy-Saving Awareness of Households in Kitakyushu

This paper aims to develop a machine-learning model based on a gradient-boosting algorithm to predict the energy-saving awareness of households using a questionnaire survey and 11-month energy data collected from more than 200 smart houses in Kitakyushu, Japan. We utilize the LightGBM (light gradient boosting machine) classifier to perform feature selection for the prediction. By using this approach, we demonstrate that the key features are the standard deviations of electricity purchased between 8 a.m. and 9 a.m. and electricity consumed between 7 p.m. and 9 p.m. Next, by using k-means clustering we split the households based on the obtained features into three groups. Finally, by using statistical hypothesis testing, we prove that these three groups have statistically distinct levels of energy-saving awareness. This model enables us to detect eco-friendly households from their energy data, which may support energy policymaking.

POINT-WISE CLASSIFICATION OF HIGH-DENSITY UAV-LIDAR DATA USING GRADIENT BOOSTING MACHINES

Abstract. Point-wise classification of 3D point clouds is a challenging task in point cloud processing, whereas, in particular, its application to high-density point clouds needs special attention because a large number of point clouds affect computational efficiency negatively. Although deep learning based models have been gaining popularity in recent years and have reached state-of-the-art results in accuracy for point-wise classification, their requirements of the high number of training samples and computational resources make those models inefficient for high-density 3D point clouds. However, traditional machine learning classifiers require less training samples, so they are capable of reducing computational requirements, even considering the latest machine learning classifiers, particularly in ensemble learning of gradient boosting machines, the results can compete with deep learning models. In this study, we are studying the point-wise classification of high-density UAV LiDAR data and focusing on efficient feature extraction and a recent state-of-the-art gradient boosting machine learning classifier, LightGBM. Our proposed framework includes the following steps: at first, we are using point cloud sampling for creating sub-sampled point clouds, then we are calculating the features based on those scales implemented on GPU. Finally, we are using the LightGBM classifier for training and testing. For the evaluation of our framework, we used a publicly available benchmark dataset, Hessigheim 3D. According to the results, we achieved an overall accuracy of 87.59% and an average F1 score of 75.92%. Our framework has promising results and scores closer to deep learning models. However, more distinctive features are required to obtain more accurate results.

Surface and Underwater Acoustic Source Recognition Using Multi-Channel Joint Detection Method Based on Machine Learning

Sound source recognition is a very important application of passive sonar. How to distinguish between surface and underwater acoustic sources has always been a challenge. Due to the mixing of underwater target radiated noise and marine environmental noise, especially in shallow water environments where multipath effects exist, it is difficult to distinguish them. To solve the surface and underwater acoustic source recognition problem, this paper proposes a multi-channel joint detection method based on machine learning. First, the simulation data are generated using the normal model KRAKEN setting in the same environment as the SACLANT 1993 experiment, which uses a vertical linear array of 48 hydrophones. Secondly, the GBDT classifier and LightGBM classifier are trained separately, and then the model is evaluated using precision, recall, F1, and accuracy. Finally, four ML models (kNN, random subspace kNN, GBDT, and LightGBM) are used to analyze all 48 channels of hydrophone data. For each model, two kinds of feature extraction methods (module features, real and imaginary features) are applied. Generally, the results show that both GBDT and LightGBM models have better performance than both kNN and random subspace kNN ones. For both GBDT and LightGBM models, the results using module features have better performance than using real and imaginary features.

Evaluation of Tropical Cyclone Disaster Loss Using Machine Learning Algorithms with an eXplainable Artificial Intelligence Approach

In the context of global warming, tropical cyclones (TCs) have garnered significant attention as one of the most severe natural disasters in China, particularly in terms of assessing the disaster losses. This study aims to evaluate the TC disaster loss (TCDL) using machine learning (ML) algorithms and identify the impact of specific feature factors on the prediction of model with an eXplainable Artificial Intelligence (XAI) approach, SHapley Additive exPlanations (SHAP). The results show that LightGBM outperforms Random Forest (RF), Support Vector Machine (SVM), and Naive Bayes (NB) for estimating the TCDL grades, achieving the highest accuracy value of 0.86. According to the SHAP values, the three most important factors in the LightGBM classifier model are proportion of stations with rainfall exceeding 50 mm (ProRain), maximum wind speed (MaxWind), and maximum daily rainfall (MaxRain). Specifically, in the estimation of high TCDL grade, events characterized with MaxWind exceeding 30 m/s, MaxRain exceeding 200 mm, and ProRain exceeding 30% tend to exhibit a higher susceptibility to TC disaster due to positive SHAP values. This study offers a valuable tool for decision-makers to develop scientific strategies in the risk management of TC disaster.

A Multi-Indicator Fusion-Based Approach for Fault Feature Selection and Classification of Rolling Bearings

Concerning the problems of harrowing extraction and poor classification accuracy of fault features in rolling bearing vibration signals, a fault feature selection and classification method based on multi-indicator fusion is proposed. Firstly, the original signal is decomposed through the improved complementary ensemble local mean decomposition (CELMD) method into several physically meaningful product functions (PF) and single residual components; then, the three indicators of kurtosis, correlation coefficient, and K-L divergence are combined to extract the most suitable PF components for signal reconstruction. Ultimately, the reconstructed signal's multi-domain characteristics and entropy value features are retrieved and fed into the LightGBM classifier for classification in order to achieve an intelligent diagnosis of rolling bearing problems. The statistical results demonstrate that the proposed method can efficiently identify the functional PF components and has notable benefits in extracting features from diverse experimental data sets and detecting faults.

Automated carcinoma classification using efficient nuclei-based patch selection and deep learning techniques

Breast cancer can be successfully treated if diagnosed at its earliest, though it is considered as a fatal disease among women. The histopathology slide turned images are the gold standard for tumor diagnosis. However, the manual diagnosis is still tedious due to its structural complexity. With the advent of computer-aided diagnosis, time and computation intensive manual procedure can be managed with the development of an automated classification system. The feature extraction and classification are quite challenging as these images involve complex structures and overlapping nuclei. A novel nuclei-based patch extraction method is proposed for the extraction of non-overlapping nuclei patches obtained from the breast tumor dataset. An ensemble of pre-trained models is used to extract the discriminating features from the identified and augmented non-overlapping nuclei patches. The discriminative features are further fused using p-norm pooling technique and are classified using a LightGBM classifier with 10-fold cross-validation. The obtained results showed an increase in the overall performance in terms of accuracy, sensitivity, specificity, and precision. The proposed framework yielded an accuracy of 98.3% for binary class classification and 95.1% for multi-class classification on ICIAR 2018 dataset.

A Light Gradient-Boosting Machine algorithm with Tree-Structured Parzen Estimator for breast cancer diagnosis

Breast cancer is a common and potentially life-threatening disease. Early and accurate diagnosis of breast cancer is crucial for effective treatment and improved patient outcomes. This study proposed using the Light Gradient-Boosting Machine (LightGBM) algorithm, Borderline- Synthetic Minority Oversampling Technique (SMOTE), and the Tree-Structured Parzen Estimator (TPE) for hyperparameter tuning to enhance the effectiveness of the Machine Learning (ML) model for diagnosing breast cancer. A 10-fold cross-validated TPE optimized Borderline-SMOTE LightGBM classifier was modelled on the Wisconsin Diagnostic Breast Cancer (WDBC) Dataset and evaluated for its performance compared to a baseline LightGBM model. The TPE-optimized Borderline-SMOTE LightGBM model exhibited a significant improvement in performance over the baseline model, achieving an average accuracy of 99.12%, specificity of 100%, precision of 100%, recall of 97.62%, F1-score of 98.80%, and a Mathews Correlation Coefficient of 98.12%. Compared to previous studies, the TPE-optimized Borderline-SMOTE LightGBM model performed exceptionally well. The study demonstrates the effectiveness of using data augmentation and hyperparameter optimization techniques to improve the performance of ML models for breast cancer diagnosis, which has significant implications for the medical field where the accurate and efficient diagnosis of breast cancer is critical.

EnLEFD‐DM: Ensemble Learning based Ethereum Fraud Detection using CRISP‐DM framework

AbstractCryptocurrencies continue to captivate businesses and investors despite market fluctuations. The number of crypto users has risen rapidly in the last few years, and alarmingly, many appear to be unaware of the risks involved. These risks aren't confined to market hazards but include very sophisticated cybercrimes related to cryptocurrencies. As cryptocurrencies have become a breeding ground for a variety of cybercrimes, resulting in enormous financial losses, it hinders user adoption limiting the utility of blockchain technology. It has become crucial to spot such scams and devise intelligent techniques to make this technology safer for investors. This paper proposes a classification model based on the Cross Industry Standard Process for Data Mining (CRISP‐DM) framework to identify fraudulent transactions over the Ethereum blockchain. Its contribution is multi‐faceted; first, the available imbalanced Ethereum dataset has been balanced to enhance the accuracy of the classification model. Second, the correlation‐based feature selection technique has been applied to retain the best discriminating features. Thirdly, an effective machine learning‐based ensemble classification model has been adopted for the identification of fraudulent transactions over the Ethereum network. A comparative analysis of 10 machine learning techniques has been presented consisting of both individual and ensemble classifiers. Evaluated outcomes show that ensemble classifiers appear to yield better performance measures over individual classifiers, and among all, the LightGbm classifier outperformed with 99.2% accuracy. Further, extensive experiments indicate that the proposed method outperforms the state‐of‐the‐art method when applied to a similar dataset.

Development and benchmarking of machine learning models to classify patients suitable for outpatient lower extremity joint arthroplasty

Study objectivePerforming hip or knee arthroplasty as an outpatient surgery has been shown to be operationally and financially beneficial for selected patients. By applying machine learning models to predict patients suitable for outpatient arthroplasty, health care systems can better utilize resources efficiently. The goal of this study was to develop predictive models for identifying patients likely to be discharged same-day following hip or knee arthroplasty. DesignModel performance was assessed with 10-fold stratified cross-validation, evaluated over baseline determined by the proportion of eligible outpatient arthroplasty over sample size. The models used for classification were logistic regression, support vector classifier, balanced random forest, balanced bagging XGBoost classifier, and balanced bagging LightGBM classifier. SettingThe patient records were sampled from arthroplasty procedures at a single institution from October 2013 to November 2021. PatientsThe electronic intake records of 7322 knee and hip arthroplasty patients were sampled for the dataset. After data processing, 5523 records were kept for model training and validation. InterventionsNone. MeasurementsThe primary measures for the models were the F1-score, area under the receiver operating characteristic curve (ROCAUC), and area under the precision-recall curve. To measure feature importance, the SHapley Additive exPlanations value (SHAP) were reported from the model with the highest F1-score. ResultsThe best performing classifier (balanced random forest classifier) achieved an F1-score of 0.347: an improvement of 0.174 over baseline and 0.031 over logistic regression. The ROCAUC for this model was 0.734. Using SHAP, the top determinant features of the model included patient sex, surgical approach, surgery type, and body mass index. ConclusionsMachine learning models may utilize electronic health records to screen arthroplasty procedures for outpatient eligibility. Tree-based models demonstrated superior performance in this study.

Performance Comparison of Boosting Algorithms in Spices Classification Using Histogram of Oriented Gradient Feature Extraction

Spice classification is an important task in the food industry to ensure food safety and quality. This study focuses on the classification of spices using the HoG feature extraction method and boosting algorithms. The objective of this research is to compare the performance of four different models of boosting algorithms, namely Adaboost Classifier, Gradient Boosting Classifier, XGB Classifier, and Light GBM Classifier, in classifying spices. The evaluation metrics used in this research are Precision, Recall, F1-Score, F2-Score, Jaccard Score, and Accuracy. The results show that the XGB Classifier model achieved the best performance, with a precision of 0.811, recall of 0.809, and F1-score of 0.809, while the Adaboost Classifier model had the lowest performance, with a precision of 0.709, recall of 0.689, and F1-score of 0.682. Overall, the results indicate a fairly good success rate in classifying spices using the HoG feature extraction method and boosting algorithms. However, further evaluation is needed to improve the accuracy of the classification results, such as increasing the number of training data or considering the use of other feature extraction methods

Research on supply chain risk evaluation based on clustering method

This paper presents a supply chain risk evaluation and prediction model based on K-means clustering and LightGBM classifier. Supply chain risk evaluation and prediction is an important problem. With the development of economic globalization, market competition is not the competition between enterprises as in the past, but the competition between supply chains. The model takes many variables such as market environment factors, resource constraints, transportation network factors and management factors as inputs. The data we used is a supply chain data set of an equipment group, which covers many variables such as market environment factors, resource constraints, transportation network factors and management factors. After the supply chain risk evaluation based on K-means clustering, the model is classified by LightGBM to generate the prediction model, and the effectiveness of the model is verified on the real data set. The results of K-Means cluster evaluation show that supply chain risk can be divided into four categories: market environment problems, resource constraints, transportation network problems and management problems, among which market environment problems and resource constraints are the most frequent.

Shapley Values for Explaining the Black Box Nature of Machine Learning Model Clustering

Machine learning (ML) models are becoming increasingly complex. In fact, a sophisticated model (XGBoost boosting or deep learning) generally leads to more accurate predictions than a simple model (linear regression or decision tree). There is therefore a trade-off between the performance of a model and its interpretability: what a model gains in performance, it loses in interpretability (and vice versa), where interpretability is the ability for a human to understand the reasons for a model's decision. However, explaining the predictions made by machine learning models aims at computing and interpreting the importance of features. To achieve this, game theory has recently gained attention for better understanding the similarity between group members. In this paper, we use SHAP (SHapley Additive exPlanations), which is a method based on cooperative game theory, to analyze and evaluate the properties of each group. More importantly, we rely k-means PCA and Light gbm classifier to improve the data preparation before grouping the features into multiple clusters. The simulation results prove the importance of shapley value in creating an accurate and meaningful representation of each cluster.

Predicting Coronary Heart Disease Using an Improved LightGBM Model: Performance Analysis and Comparison

Coronary heart disease (CHD) is a dangerous condition that cannot be completely cured. Accurate detection of early coronary artery disease can assist physicians in treating patients. In this study, a prediction model called HY_OptGBM was proposed for predicting CHD by using the optimized LightGBM classifier. To optimize the LightGBM classifier, the hyperparameters of the LightGBM model were adjusted. In addition, its loss function was improved, and the model was trained using adjusted hyperparameters. In this study, the hyperparameters of the prediction model were optimized by applying the most advanced hyperparameter optimization framework (OPTUNA). The improved loss function is referred to as the focal loss (FL). In this study, a prediction model was evaluated by using CHD data from the Framingham Heart Institute. To evaluate the performance of the prediction model, various metrics, including precision, recall, F score, accuracy, MCC, sensitivity, specificity, and AUC, were used. The AUC value of the proposed model was 97.9%, which was better than that of other comparative models. The results demonstrate that the rate of early identification of CHD among the general population can be improved by utilizing the proposed method. This, in turn, could serve to mitigate the costs associated with the medical treatment of patients suffering from CHD.

A two-stage intrusion detection method based on light gradient boosting machine and autoencoder.

Intrusion detection systems can detect potential attacks and raise alerts on time. However, dimensionality curses and zero-day attacks pose challenges to intrusion detection systems. From a data perspective, the dimensionality curse leads to the low efficiency of intrusion detection systems. From the attack perspective, the increasing number of zero-day attacks overwhelms the intrusion detection system. To address these problems, this paper proposes a novel detection framework based on light gradient boosting machine (LightGBM) and autoencoder. The recursive feature elimination (RFE) method is first used for dimensionality reduction in this framework. Then a focal loss (FL) function is introduced into the LightGBM classifier to boost the learning of difficult samples. Finally, a two-stage prediction step with LightGBM and autoencoder is performed. In the first stage, pre-decision is conducted with LightGBM. In the second stage, a residual is used to make a secondary decision for samples with a normal class. The experiments were performed on the NSL-KDD and UNSWNB15 datasets, and compared with the classical method. It was found that the proposed method is superior to other methods and reduces the time overhead. In addition, the existing advanced methods were also compared in this study, and the results show that the proposed method is above 90% for accuracy, recall, and F1 score on both datasets. It is further concluded that our method is valid when compared with other advanced techniques.

Recognizing Pigmented Skin Lesion based on LightGBM Classifier and Deep Saliency Segmentation using Convolutional Neural Network

Recognizing Pigmented Skin Lesion based on LightGBM Classifier and Deep Saliency Segmentation using Convolutional Neural Network

A practical intrusion detection system based on denoising autoencoder and LightGBM classifier with improved detection performance

A practical intrusion detection system based on denoising autoencoder and LightGBM classifier with improved detection performance

Optimized LightGBM Power Fingerprint Identification Based on Entropy Features.

The huge amount of power fingerprint data often has the problem of unbalanced categories and is difficult to upload by the limited data transmission rate for IoT communications. An optimized LightGBM power fingerprint extraction and identification method based on entropy features is proposed. First, the voltage and current signals were extracted on the basis of the time-domain features and V-I trajectory features, and a 56-dimensional original feature set containing six entropy features was constructed. Then, the Boruta algorithm with a light gradient boosting machine (LightGBM) as the base learner was used for feature selection of the original feature set, and a 23-dimensional optimal feature subset containing five entropy features was determined. Finally, the Optuna algorithm was used to optimize the hyperparameters of the LightGBM classifier. The classification performance of the power fingerprint identification model on imbalanced datasets was further improved by improving the loss function of the LightGBM model. The experimental results prove that the method can effectively reduce the computational complexity of feature extraction and reduce the amount of power fingerprint data transmission. It meets the recognition accuracy and efficiency requirements of a massive power fingerprint identification system.

Improved Mayfly Optimization and LightGBM Classifier for Smart City Traffic Prediction

Objectives: This research work focuses on predicting traffic for the Smart City. Methods: Current research methods for traffic prediction are based on machine learning (ML) model. This article presents two contributions related to it. First, it provides feature engineering that includes feature extraction and a nature inspired optimization algorithm for selecting the best features. The mayfly optimization algorithm is improved by using the mode-based ranking method to select the best feature. Second, it uses the light-weight boosting method to train the datasets for better accuracy.Findings: The proposed Improved MayFly Optimization with LightGBM (IMFO-LGBM) is experimented with popular smart city datasets which is available in Kaggle website. IMFOLGBM shows an improvement in the prediction accuracy when compared with the baseline methods. It shows 2% of increase in the overall accuracy. Novelty:Nature inspired Mayfly optimization is improved and used to find the best feature for prediction. The selected features are then trained with the light weight boosting algorithm (i.e., Light Gradient Boosting Model). The hybrid of improved mayfly optimization and light GBM outperformed well. Keywords: IoT; Smartcity; Mayfly optimization; Machine learning and LightGBM

Classification of acute poisoning exposures with machine learning models derived from the National Poison Data System.

The primary aim of this pilot study was to develop a machine learning algorithm to predict and distinguish eight poisoning agents based on clinical symptoms. Data were used from the National Poison Data System from 2014 to 2018, for patients 0-89 years old with single-agent exposure to eight drugs or drug classes (acetaminophen, aspirin, benzodiazepines, bupropion, calcium channel blockers, diphenhydramine, lithium and sulfonylureas). Four classifier prediction models were applied to the data: logistic regression, LightGBM, XGBoost, and CatBoost. There were 201 031 cases used to develop and test the algorithms. Among the four models, accuracy ranged 77%-80%, with precision and F1 scores of 76%-80% and recall of 77%-78%. Overall specificity was 92% for all models. Accuracy was highest for identifying sulfonylureas, acetaminophen, benzodiazepines and diphenhydramine poisoning. F1 scores were highest for correctly classifying sulfonylureas, acetaminophen and benzodiazepine poisonings. Recall was highest for sulfonylureas, acetaminophen, and benzodiazepines, and lowest for bupropion. Specificity was >99% for models of sulfonylureas, calcium channel blockers, lithium and aspirin. For single-agent poisoning cases among the eight possible exposures, machine learning models based on clinical signs and symptoms moderately predicted the causal agent. CatBoost and LightGBM classifier models had the highest performance of those tested.