Ensemble Classification Method Research Articles

Latest Advancements in Credit Risk Assessment with Machine Learning and Deep Learning Techniques

Abstract A loan is vital for individuals and organizations to meet their goals. However, financial institutions face challenges like managing losses and missed opportunities in loan decisions. A key issue is the imbalanced datasets in credit risk assessment, hindering accurate predictions of defaulters. Previous research has utilized machine learning techniques, including single or multiple classifier systems, ensemble methods, and class-balancing approaches. This review summarizes various factors and machine learning methods for assessing credit risk, presented in a tabular format to provide valuable insights for researchers. It covers data complexity, minority class distribution, sampling techniques, feature selection, and meta-learning parameters. The goal is to help develop novel algorithms that outperform existing methods. Even a slight improvement in defaulter prediction rates could significantly influence society by saving millions for lenders.

Predictive Analysis System for National Ranking and Accreditation of HEIs

Predictive analytical models have been applied in context of the Indian higher educational institutions dataset. This study is aimed to compare the predictive accuracy level of various machine learning algorithms (multi-class classification prediction algorithms) to assess the grading system applicable to a college based on various parameters outlined by NAAC (National Assessment and Accreditation Council). NAAC is an apex body to assess and accredit HEIs of India. An ensemble classification method is proposed to best predict the academic grading of university-affiliated colleges of India for the NAAC accreditation system. This maiden attempt for using PA (Predictive Analysis) on such institutional credential data would be beneficial to colleges applying for NAAC accreditation to judge their grade prior to final assessment. Thus they would be able to identify their strengths and weaknesses and eventually work towards further improving upon their grade. Such a system would pave the way for designing such predictive systems to assess the institutional rankings for several other assessment systems.

Envelope rotation forest: A novel ensemble learning method for classification

Ensemble learning is an important field and research hotpot of machine learning. However, the existing ensemble learning methods construct subsets by directly sampling the same original sample set. Therefore, the subsets suffer from the low quality of original sample set, such as low separability, low diversity, high sensitivity to noisy samples. As a result, the improvement effect of existing ensemble methods is limited. To address these problems, a new ensemble learning method - envelope rotation forest was proposed. Firstly, envelope sample transformation mechanism (ESTM) is designed to transform the original sample set to multiple layers of envelope sample sets, thereby improving the quality of the sample subsets. The ESTM consists of multilayer iterative mean clustering (MIMC) and interlayer consistency mechanism (ICM). Secondly, the two-dimensional base classifier matrix generation and fusion mechanism (2D_BCMG&FM) is designed to enhance classification performance. The final classification results are obtained by fusing the two-dimensional classification labels. For verification, the proposed method is tested on 26 datasets and is compared with representative methods. It is observed that the accuracy is improved by up to 18.77%, which highlights the significance of considering envelope transformation on original samples for subsequent subset construction.Data and code can be found at: https://github.com/GRabase/ERF

Exploring the impact of opioids on serum chemistry: Insights from FT-Raman spectroscopy and biochemical analysis

Opioids are drugs whose use has increased dramatically over the last two decades. Opioids are a family of drugs that includes the illegal drug heroin, synthetic opioids such as fentanyl, and painkillers that are legally available on prescription. Unfortunately, too long use of these drugs can lead to addiction, which might cause disturbed physiological and mental effects. Regular opioid usage might cause common mental disorders such as depression, anxiety, and panic disorder. However, there is still no rapid detection method to describe the chemical changes caused by the Opioid use disorder. For this reason, this study used conventional FT-Raman spectroscopic techniques with multivariate analysis to detect chemical changes in serum following opiate administration. In the FT-Raman spectrum collected from opioid-addicted patients, significant shifts of peaks at 892 cm−1, 966 cm−1, 1286 cm−1, 1459 cm−1 and 2940 cm−1 were visible in comparison with the spectrum of serum collected from the control group. Furthermore, changes in the ratio of amides and lipids were observed in the non-control group, suggesting that opioids could cause structural changes in the compounds. Synchronous spectra show auto peaks at 2914 cm−1, 3415 cm−1 while asynchronous negative cross area between 1005 cm−1 and 1800 cm−1 and around 3000 cm−1 indicate higher presence of polysaccharides and amides in comparison with other compounds in serum collected from non-control group. Finally, the PCA model, utilizing three components that explain 94.89 % of the data variation within the 300–3700 cm⁻¹ range, demonstrates excellent performance. The ensemble classification method achieves accuracies of 97.96 % and 95.00 % on the training and test datasets, respectively. Furthermore, the method achieves AUC-ROC scores of 1.00 and 0.99 for the training and test sets, effectively distinguishing between control and opioid-addicted individuals.

Using ensemble learning method and binary decision tree algorithm for drought intensity level classification

This study applies machine learning methods such as Decision Tree (CART) and Random Forest to classify drought intensity based on meteorological data. The goal of the study was to evaluate the effectiveness of these methods for drought classification and their use in water resource management and agriculture. The methodology involved using two machine learning models that analyzed temperature and humidity indicators, as well as wind speed indicators. The models were trained and tested on real meteorological data to assess their accuracy and identify key factors affecting predictions. Results showed that the Random Forest model achieved the highest accuracy of 94.4% when analyzing temperature and humidity indicators, while the Decision Tree (CART) achieved an accuracy of 93.2%. When analyzing wind speed indicators, the models’ accuracies were 91.3% and 93.0%, respectively. Feature importance revealed that atmospheric pressure, temperature at 2 m, and wind speed are key factors influencing drought intensity. One of the study’s limitations was the insufficient amount of data for high drought levels (classes 4 and 5), indicating the need for further data collection. The innovation of this study lies in the integration of various meteorological parameters to build drought classification models, achieving high prediction accuracy. Unlike previous studies, our approach demonstrates that using a wide range of meteorological data can significantly improve drought classification accuracy. Significant findings include the necessity to expand the dataset and integrate additional climatic parameters to improve models and enhance their reliability.

Online welding status monitoring method of T-joint double-sided double arc welding based on multi-source information fusion

In modern manufacturing, double-sided double arc (DSDA) welding brings advantages in properties and efficiency for T-shaped joints. However, the double-side forming makes it difficult to detect the imperfections and ensure weld joint quality, which few studies have covered. Most existing methods deal with single-arc welding with limited sensing sources. To fill the gap, an online welding status monitoring method for DSDA welding is proposed based on the fusion of welding current, arc voltage, arc sound, and weld pool images. In pre-processing, the automatic weld pool region of interest (ROI) detection method based on the lightweight YOLO-L model and the waveform denoising algorithm are designed. In feature engineering, waveform signals are analyzed in both the time and frequency domain. A weld pool feature extractor based on a convolutional neural network (CNN) is proposed with good effectiveness and interpretability. The output feature combination is refined by Fisher-based selection and evaluation. In model building, an ensemble learning model based on three high-fit basic learners is proposed, with an accuracy of 98.538 %. The proposed model shows significant advantages over the single basic classifier and other ensemble methods. Experiments verify high precision and robustness, laying a foundation for accurate real-time monitoring of DSDA welding production.

Crop Recommendation System Using Machine Learning: A Comparative Study

Agriculture, as a fundamental aspect of human existence, faces challenges in crop selection, impacting resource allocation and productivity. This project addresses these challenges by proposing a stable system employing a soft voting classifier ensemble method. The ensemble comprises Naive Bayes, Support Vector Machine (SVM), Decision Tree, and Random Forest classifiers, offering personalized crop recommendations. Feasibility analysis encompasses technical, operational, economic, and scheduling aspects, ensuring practicality and efficacy. Development follows an incremental model, emphasizing continuous enhancement through feedback. Results indicate accuracies for individual classifiers (’Decision Tree’: 98.38%, ’Random Forest’: 98.90%, ’Naive Bayes’: 98.14%, ’SVM’: 98.50%), with an ensemble accuracy of 98.99%. Cross-validation confirms robustness. Evaluation metrics such as recall, precision, and F1 score demonstrate that the soft voting ensemble outperforms individual classifiers, highlighting its effectiveness in optimizing crop selection processes in agriculture and facilitating improved resource management and productivity.

Ensemble learning method for classification: Integrating data envelopment analysis with machine learning

In classification tasks with large sample sets, the use of a single classifier carries the risk of overfitting. To overcome this issue, an ensemble of classifier models has often been shown to outperform the use of a single “best” model. Given the rich variety of classifier models available, the selection of the high-efficiency classifiers for a given task dataset remains an urgent challenge. However, most of the previous classifier selection methods only focus on the measurement of classification output performance without considering the computational cost. This paper proposes a new ensemble learning method to improve the classification quality for big datasets by using data envelopment analysis. It contains the following two stages: classifier selection and classifier combination. In the first stage, the commonly used classifiers are evaluated on the basis of their performance on resource consumption and classification output performance using the range directional model (RDM); then, the most efficient classifiers are selected. In the second stage, the classifier confusion matrix is evaluated using the data envelopment analysis (DEA) cross-efficiency model. Then, the weight for the classifier combination is determined to ensure that classifiers with higher performance have greater weights based on the cross-efficiency values. Experimental results demonstrate the superiority of the cross-efficiency model over the BCC model and the benchmark voting method in model ensemble. Furthermore, our method has been shown to save more computational resources and yields better results than existing methods.

Improving Driving Style in Connected Vehicles via Predicting Road Surface, Traffic, and Driving Style

This paper investigates the application of ensemble learning in improving the accuracy and reliability of predictions in connected vehicle systems, focusing on driving style, road surface quality, and traffic conditions. Our study’s central methodology is the voting classifier ensemble method, which integrates predictions from multiple machine learning models to improve overall predictive performance. Specifically, the ensemble method combines insights from random forest, decision tree, and K-nearest neighbors models, leveraging their individual strengths while compensating for their weaknesses. This approach resulted in high accuracy rates of 94.67% for driving style, 99.10% for road surface, and 98.80% for traffic predictions, demonstrating the robustness of the ensemble technique. Additionally, our research emphasizes the importance of model explanation ability, employing the tree interpreter tool to provide detailed insights into how different features influence predictions. This paper proposes a model based on the algorithm GLOSA for sharing data between connected vehicles and the algorithm CTCRA for sending road information to navigation application users. Based on prediction results using ensemble learning and similarity in driving styles, road surface conditions, and traffic conditions, an ensemble learning approach is used. This not only contributes to the predictions’ transparency and trustworthiness but also highlights the practical implications of ensemble learning in improving real-time decision-making and vehicle safety in intelligent transportation systems. The findings underscore the significant potential of advanced ensemble methods for addressing complex challenges in vehicular data analysis.

A clustering-based adaptive undersampling ensemble method for highly unbalanced data classification

The class imbalance issue is prevalent in various practical classification tasks. A high unbalanced rate will significantly decrease the classification performance of unbalanced learning. However, existing methods for highly unbalanced data classification still face two key difficulties: (1) fairly learning key information, and (2) maintaining consistency. To address these difficulties, we propose a novel majority clustering-based adaptive undersampling enhanced ensemble classification method, which integrates undersampling and ensemble techniques. In the adaptive undersampling process, we first consider the spatial distribution of majority samples to ensure distribution consistency. We then consider an adaptive sampling rate and introduce a feedback mechanism to obtain more representative majority samples from each cluster. In the classifier ensemble process, multiple ensemble iterations are introduced to achieve fair attention to key information in different classes. Finally, six kinds of experiments are conducted on 17 real highly unbalanced datasets from multiple fields. Experimental results demonstrate that the proposed method outperforms existing methods in terms of effectiveness, robustness, and adaptability.

Evaluating the Performance of Voting Classifier in Multiclass Classification of Dry Bean Varieties

This study explores the application of a voting classifier, integrating Decision Tree, Logistic Regression, and Gaussian Naive Bayes models, for the multiclass classification of dry bean varieties. Utilizing a dataset comprising 13,611 images of dry bean grains, captured through a high-resolution computer vision system, we extracted 16 features to train and test the classifier. Through a rigorous 5-fold cross-validation process, we assessed the model's performance, focusing on accuracy, precision, recall, and F1-score metrics. The results demonstrated significant variability in the classifier's performance across different data subsets, with accuracy rates fluctuating between 31.23% and 96.73%. This variability highlights the classifier's potential under specific conditions while also indicating areas for improvement. The research contributes to the agricultural informatics field by showcasing the effectiveness and challenges of using ensemble learning methods for crop variety classification, a crucial task for enhancing agricultural productivity and food security. Recommendations for future research include exploring additional features to improve model generalization, extending the dataset for broader applicability, and comparing the voting classifier's performance with other ensemble methods or advanced machine learning models. This study underscores the importance of machine learning in advancing agricultural classification tasks, paving the way for more efficient and accurate crop sorting and grading processes.

A new method of fault diagnosis for aeroengines with dispersedly clumped gas path parameters

In an aircraft fleet with multiple aeroengines in service, the gas path parameter measurement values from engines of the same type are dispersed within a certain range due to manufacturing technology, assembly tolerances, operation conditions, and maintenance. This dispersion cannot be ignored and makes it impossible to apply unified health diagnosis criteria to all engines. To address this issue, a fault diagnosis solution for aeroengines with dispersedly clumped gas path parameters based on meta aero-engine is proposed. The concept of meta aero-engine is proposed to derive an engine group with dispersedly clumped parameters. To diagnose engines, a sphenophyllum classifier is proposed, which has a unified judgement criterion. A new weak classifier ensemble method based on divide-and-conquer is proposed to enhance the classification ability of the sphenophyllum classifier. The results of the basic classifier are partitioned and then analyzed in combination with the results of other classifiers in the classifier ensemble method. The effectiveness of the proposed method is verified through turbofan engine simulation dataset. The proposed method has strong interpretability for engine fleet with dispersedly clumped gas path parameters.

A deep ensemble learning method for cherry classification

AbstractIn many agricultural products, information technologies are utilized in classification processes at the desired quality. It is undesirable to mix different types of cherries, especially in export-type cherries. In this study on cherries, one of the important export products of Turkey, the classification of cherry species was carried out with ensemble learning methods. In this study, a new dataset consisting of 3570 images of seven different cherry species grown in Isparta region was created. The generated new dataset was trained with six different deep learning models with pre-learning on the original and incremental dataset. As a result of the training with incremental data, the best result was obtained from the DenseNet169 model with an accuracy of 99.57%. The two deep learning models with the best results were transferred to ensemble learning and a 100% accuracy rate was obtained with the Maximum Voting model.

Enhancing Cancer Classification Through Ensemble Machine Learning and Gene Selection Approaches

The vast dimensionality of gene expression data and the limited number of relevant genes necessitate the adoption of gene selection techniques. Furthermore, the choice of an efficient classifier plays a pivotal role in achieving accurate results. In this study, we employ the Minimum Redundancy Maximum Relevance (mRMR) method for gene selection, coupled with ensemble classifiers and individual classifiers like K-Nearest Neighbors (KNN) and Decision Trees (DT). A comparative analysis between two ensemble classifiers and two individual classifiers is conducted, revealing the superior performance of the ensemble classifiers. Our investigation utilizes four distinct cancer gene expression datasets to showcase the efficacy of employing ensemble classifiers and gene selection methods for cancer classification. The ensemble classifier (Bagging Classifier), in conjunction with the MRMR method selecting only the top 30 genes, achieved an impressive overall accuracy of 94% across all four employed datasets.

A ROBUST DETECTION OF CYBER INCIDENTS UTILIZING MACHINE LEARNING TECHNIQUES

A reliable Cyber Attack Detection Model (CADM) is a system that works as safeguard for the users of modern technological devices and assistant for the operators of networks. The research paper aims to develop a CADM for analyzing the network data patterns to classify cyber-attacks. CADM finds out attack wise detection accuracy using ensemble classification method. LASSO has been used to extract important features. It can work with large datasets, and it has more visualization capability. Gradient Boosting and Random Forest algorithms have been used for classification of network traffic data to build an ensemble method. Gradient Boosting algorithm trains weak learning models and select the best decision trees to deliver more improved prediction accuracy and Random Forest algorithm trains each tree in parallel manner. In this research work, Jive datasets such as NSL-KDD, KDD Cup 99, UNSWNB15, URL 2016 and CICIDS 2017 are also applied to check the efficiency of the proposed model.

A ROBUST DETECTION OF CYBER INCIDENTS UTILIZING MACHINE LEARNING TECHNIQUES

A reliable Cyber Attack Detection Model (CADM) is a system that works as safeguard for the users of modern technological devices and assistant for the operators of networks. The research paper aims to develop a CADM for analyzing the network data patterns to classify cyber-attacks. CADM finds out attack wise detection accuracy using ensemble classification method. LASSO has been used to extract important features. It can work with large datasets, and it has more visualization capability. Gradient Boosting and Random Forest algorithms have been used for classification of network traffic data to build an ensemble method. Gradient Boosting algorithm trains weak learning models and select the best decision trees to deliver more improved prediction accuracy and Random Forest algorithm trains each tree in parallel manner. In this research work, Jive datasets such as NSL-KDD, KDD Cup 99, UNSWNB15, URL 2016 and CICIDS 2017 are also applied to check the efficiency of the proposed model.

Machine Learning Based Precision Agriculture using Ensemble Classification with TPE Model

Many tasks are part of smart farming, including predicting crop yields, analysing soil fertility, making crop recommendations, managing water, and many more. In order to execute smart agricultural tasks, researchers are constantly creating several Machine Learning (ML) models. In this work, we integrate ML with the Internet of Things. Either the UCI dataset or the Kaggle dataset was used to gather the data. Effective data pretreatment approaches, such as the Imputation and Outlier (IO) methods, are necessary to manage the intricacies and guarantee proper analysis when dealing with data that exhibits irregular patterns or contains little changes that can have a substantial influence on analysis and decision making. The goal of this research is to provide a more meaningful dataset by investigating data preparation approaches that are particular to processing data. Following the completion of preprocessing, the data is classified using an average approach based on the Ensemble of Adaptive Neuro-Fuzzy Inference System (ANFIS), Random Neural Network (PNN), and Clustering-Based Decision Tree (CBDT) techniques. The next step in optimising the hyperparameter tuning of the proposed ensemble classifier is to employ a new Tree-Structured Parzen Estimator (TPE). Applying the suggested TPE based Ensemble classification method resulted in a 99.4 percent boost in accuracy

Ensemble Feature Selection Method for Single Pulse Classification

Affected by a large number of radio frequency interference signals, it has become an important task for astronomical data processing to quickly and accurately identify single pulse signals from massive observation data. Designing and extracting effective data features is the key issue for efficient identification of single pulse signals using machine learning. This paper proposes an ensemble feature selection method for single pulse signal classification. The method first mixed three types of features, including the parametric features, statistical features, and abstract features of single pulse signals, and then used five individual feature selection methods to select the corresponding optimal feature set, respectively. At last, the features selected by the five individual methods are mixed, and the greedy strategy was used to select the optimal ensemble feature set. The experimental results show that the ensemble feature set can improve F1-score by a value of 1.8% at most, and can obtain higher accuracy than the features selected by individual methods. Under the background of high-speed and large-scale sky survey, the ensemble feature selection method plays an important role in reducing the number of features, improving classification performance, and speeding up data processing.

A Learning Approach for The Identification of Network Intrusions Based on Ensemble XGBoost Classifier

The limitation of signature-based Intrusion Detection systems has given rise for the popularity of Machine learning (ML) approaches r for building such intrusion detection systems (IDSs). ML is a sub-filed of Artificial Intelligence that enables algorithms to learn from data and its applications have been widely accepted and used in many domains. To achieve a promising ML-based model that can identify attacks and intrusions in networks and the cyber space, different stages of machine learning approach like pre-processing, attribute selection, model building, hyper parameter tuning can be very important. CICIDS2017 intrusion dataset was used for all the experimentations. This study focuses on building cyber threat detection model based on the ensemble feature selection and classification method. Innovative approaches were used for the analysis and pre-processing of the dataset. Thereafter, XGboost algorithm was used for selecting relevant features from the default input attributes in each of the captures. Thereafter, the reduced features were employed in the identification of cyber intrusions. The average accuracy achieved in the 8 captures of the dataset is 98% while precision is 0.98. Also, recall is 0.98, f1-score is 0.98 while AUC ROC score is 0.99. The study concluded that XGBoost-based model was able to achieve promising results based on the proper dataset encoding, feature importance-based feature selection and tuning of the algorithm for intrusion identification.

An ensemble classification method based on machine learning models for malicious Uniform Resource Locators (URL).

Web applications are important for various online businesses and operations because of their platform stability and low operation cost. The increasing usage of Internet-of-Things (IoT) devices within a network has contributed to the rise of network intrusion issues due to malicious Uniform Resource Locators (URLs). Generally, malicious URLs are initiated to promote scams, attacks, and frauds which can lead to high-risk intrusion. Several methods have been developed to detect malicious URLs in previous works. There has been a good amount of work done to detect malicious URLs using various methods such as random forest, regression, LightGBM, and more as reported in the literature. However, most of the previous works focused on the binary classification of malicious URLs and are tested on limited URL datasets. Nevertheless, the detection of malicious URLs remains a challenging task that remains open to research. Hence, this work proposed a stacking-based ensemble classifier to perform multi-class classification of malicious URLs on larger URL datasets to justify the robustness of the proposed method. This study focuses on obtaining lexical features directly from the URL to identify malicious websites. Then, the proposed stacking-based ensemble classifier is developed by integrating Random Forest, XGBoost, LightGBM, and CatBoost. In addition, hyperparameter tuning was performed using the Randomized Search method to optimize the proposed classifier. The proposed stacking-based ensemble classifier aims to take advantage of the performance of each machine learning model and aggregate the output to improve prediction accuracy. The classification accuracies of the machine learning model when applied individually are 93.6%, 95.2%, 95.7% and 94.8% for random forest, XGBoost, LightGBM, and CatBoost respectively. The proposed stacking-based ensemble classifier has shown significant results in classifying four classes of malicious URLs (phishing, malware, defacement, and benign) with an average accuracy of 96.8% when benchmarked with previous works.