Decision Tree-based Classifiers Research Articles

Improving Credit Card Fraud Detection with Data Reduction Approaches

Detecting fraudulent activities in credit card transactions can be challenging due to issues like high dimensionality and class imbalance that are often present in the datasets. To address these challenges, data reduction techniques such as data sampling and feature selection have become essential. In this study, we compare four approaches for data reduction: using data sampling alone, employing feature selection alone, applying data sampling followed by feature selection, and using feature selection followed by data sampling. Additionally, we include results using all features. We build classification models using five Decision Tree-based classifiers and Logistic Regression, and evaluate their performance using two performance metrics: the Area Under the Receiver Operating Characteristic Curve (AUC) and the Area under the Precision–Recall Curve (AUPRC). In this work, we adopt ensemble supervised feature selection (SFS) techniques and Random Undersampling (RUS) for data reduction. The experimental results demonstrate that all four data reduction techniques have the potential to improve the performance of classifiers. These results are valuable since the classifiers available are dependent upon application domains, computing environments, and licensing agreements. However, these techniques can be applied independently of all these dependencies. We recommend utilizing the ensemble SFS followed by RUS (SFS–RUS) approach as the preferred data reduction method due to its ability to run feature selection and data sampling in parallel. Additionally, we find that XGBoost and CatBoost outperform other classifiers.

A lexicographic optimisation approach to promote more recent features on longitudinal decision-tree-based classifiers: applications to the English Longitudinal Study of Ageing

Supervised machine learning algorithms rarely cope directly with the temporal information inherent to longitudinal datasets, which have multiple measurements of the same feature across several time points and are often generated by large health studies. In this paper we report on experiments which adapt the feature-selection function of decision tree-based classifiers to consider the temporal information in longitudinal datasets, using a lexicographic optimisation approach. This approach gives higher priority to the usual objective of maximising the information gain ratio, and it favours the selection of features more recently measured as a lower priority objective. Hence, when selecting between features with equivalent information gain ratio, priority is given to more recent measurements of biomedical features in our datasets. To evaluate the proposed approach, we performed experiments with 20 longitudinal datasets created from a human ageing study. The results of these experiments show that, in addition to an improvement in predictive accuracy for random forests, the changed feature-selection function promotes models based on more recent information that is more directly related to the subject’s current biomedical situation and, thus, intuitively more interpretable and actionable.

RedTell: an AI tool for interpretable analysis of red blood cell morphology.

Introduction: Hematologists analyze microscopic images of red blood cells to study their morphology and functionality, detect disorders and search for drugs. However, accurate analysis of a large number of red blood cells needs automated computational approaches that rely on annotated datasets, expensive computational resources, and computer science expertise. We introduce RedTell, an AI tool for the interpretable analysis of red blood cell morphology comprising four single-cell modules: segmentation, feature extraction, assistance in data annotation, and classification. Methods: Cell segmentation is performed by a trained Mask R-CNN working robustly on a wide range of datasets requiring no or minimum fine-tuning. Over 130 features that are regularly used in research are extracted for every detected red blood cell. If required, users can train task-specific, highly accurate decision tree-based classifiers to categorize cells, requiring a minimal number of annotations and providing interpretable feature importance. Results: We demonstrate RedTell's applicability and power in three case studies. In the first case study we analyze the difference of the extracted features between the cells coming from patients suffering from different diseases, in the second study we use RedTell to analyze the control samples and use the extracted features to classify cells into echinocytes, discocytes and stomatocytes and finally in the last use case we distinguish sickle cells in sickle cell disease patients. Discussion: We believe that RedTell can accelerate and standardize red blood cell research and help gain new insights into mechanisms, diagnosis, and treatment of red blood cell associated disorders.

Urban land-use classification using machine learning classifiers: comparative evaluation and post-classification multi-feature fusion approach

ABSTRACT Accurate spatial-temporal mapping of urban land-use and land-cover (LULC) provides critical information for planning and management of urban environments. While several studies have investigated the significance of machine learning classifiers for urban land-use mapping, the determination of the optimal classifiers for the extraction of specific urban LULC classes in time and space is still a challenge especially for multitemporal and multisensor data sets. This study presents the results of urban LULC classification using decision tree-based classifiers comprising of gradient tree boosting (GTB), random forest (RF), in comparison with support vector machine (SVM) and multilayer perceptron neural networks (MLP-ANN). Using Landsat data from 1984 to 2020 at 5-year intervals for the Greater Gaborone Planning Area (GGPA) in Botswana, RF was the best classifier with overall average accuracy of 92.8%, MLP-ANN (91.2%), SVM (90.9%) and GTB (87.8%). To improve on the urban LULC mapping, the study presents a post-classification multiclass fusion of the best classifier results based on the principle of feature in-feature out (FEI-FEO) under mutual exclusivity boundary conditions. Through classifier ensemble, the FEI-FEO approach improved the overall LULC classification accuracy by more than 2% demonstrating the advantage of post-classification fusion in urban land-use mapping.

Improved Defect Detection in Photovoltaic Panels Through Deep Learning and Decision Tree-Based Classifiers

Improved Defect Detection in Photovoltaic Panels Through Deep Learning and Decision Tree-Based Classifiers

Machine Learning Prediction and Recommendation Framework to Support Introductory Programming Course

The new students struggle to understand the introductory programming courses, due to its intricate nature, which results in higher dropout and increased failure rates. Despite implementing productive methodologies, the instructor struggles to identify the students with distinctive levels of skills. The modern institutes are looking for technology-equipped practices to classify the students and prepare personalized consultation procedures for each class. This paper applies decision tree-based machine learning classifiers to develop a prediction model competent to forecast the outcome of the introductory programming students at an early stage of the semester. The model is then transformed into an adaptive consultation framework which generates three types of colored signals; red, yellow, and green which illustrates whether the student is performing low, average, or high respectively. This provides an opportunity for the instructor to set precautionary measures for low performing students and set complicated tasks that help the highly skilled students to improve their skills further. The experiments compare a set of decision tree-based classifiers and conclude J48 as an efficient model in classifying students in all classes with high accuracy, sensitivity, and F-measure. Even though the aim of the research is to focus on introductory programming courses, however, the framework is flexible and can be implemented in other courses.

Predicting defects in object-oriented software using cost-sensitive classification

In this software era, it is vital to produce reliable and good quality software. Early detection of defects aids in building accurate software with reduced cost and other resources. Researchers have a keen interest in producing machine learning models for effective and accurate software defect prediction in the early stages of software development. Object-oriented metrics of the software are used in developing these models. These models may result in biased predictions owing to the class imbalance problem existing in most of the software datasets. This paper provides an effective defect prediction framework for imbalanced data by employing cost-sensitive classifiers and stable performance measures like GMean, Balance, and AUC. Four decision tree-based classifiers with different cost ratios are investigated to predict defects in three Apache projects. The empirical results are statistically validated using the nonparametric Friedman test and Wilcoxon signed-rank test. The results state with 99% confidence that the predictive capability of J48, AdaBoostM1, Bagging, and RandomSubSpace improved after employing cost-sensitive learning for the four classifiers used in this study.

Feasibility of machine learning algorithms for classifying damaged offshore jacket structures using SCADA data

The best practise for structural damage detection currently relies on the installation of structural health monitoring systems for the collection of dedicated high frequency measurements. Switching to the employment of the wind turbine’s SCADA (Supervisory Control and Data Acquisition) signals and their commonly recorded low frequency statistics can lead to a reduction in the number of ad-hoc monitoring sensors and quantity of data required. In this paper, aero-hydro-servo-elastic simulations for a model of a turbine are used to assess its loads and any changes in the dynamics under healthy state and a damaged configuration case study. To prove the feasibility of the damage detection through low-resolution data, the statistics of the typically recorded signals from the SCADA and the structural monitoring systems are fed into a database for training and testing of classification algorithms. The ability of the machine learning models to generalise the classification for both stochasticity and uncertainties in the environmental conditions are tested. Decision tree-based classifiers showed the capability to capture the damage for the majority of the operating conditions considered. Though the setup of the traditional SCADA sensors had to be supplemented with an additional structural health monitoring sensor, the detection of the damage has been shown feasible by referring to low-frequency statistics only.

Detection of Disease-Specific Parent Cells Via Distinct Population of Nano-Vesicles by Machine Learning.

The diagnosis and prognosis of pathological conditions, such as age-related macular degeneration (AMD) and cancer still need improvement. AMD is primarily caused due to the dysfunction of retinal pigment epithelium (RPE), whereas endothelial cells (ECs) play one of the major roles in angiogenesis; an important process which occurs in malignant progression of cancer. Several reports suggested the augmented release of nano-vesicles under pathological conditions, including from RPE as well as cancer-associated ECs, which take part in various biological processes, including intercellular communication in disease progression. Importantly, these nano-vesicles are around 30-1000 nm and carry the fingerprint of their initiating parent cells (IPCs). Therefore, these nano-vesicles could be utilized as the diagnostic tool for AMD and cancer, respectively. However, the analysis of nano-vesicles for biomarker study is confounded by their extensive heterogeneous nature. To confront this challenge, we utilized artificial intelligence (AI) based machine learning (ML) algorithms such as support vector machine (SVM) and decision tree model on the dataset of nano-vesicles from RPE and ECs cell lines with low dimensionality. Overall, Gaussian SVM demonstrated the highest prediction accuracy of the IPCs of nano-vesicles, among all the chosen SVM classifiers. Additionally, the bagged tree showed the highest prediction among the chosen decision tree-based classifiers. Therefore, the overall bagged tree showed the best performance for the prediction of IPCs of nanovesicles, suggesting the applicability of AI-based prediction approach in diagnosis and prognosis of pathological conditions, including non-invasive liquid biopsy via various biofluids-derived nano-vesicles.

Empirical Performance Analysis of Decision Tree and Support Vector Machine based Classifiers on Biological Databases

The classification and prediction of medical diseases is a cutting edge research problem in the medical field. The experts of machine learning are continuously proposing new classification methods for the prediction of diseases. The discovery of classification rules from medical databases for classification and prediction of diseases is a challenging and non-trivial task. It is very significant to investigate the more promising and efficient classification approaches for the discovery of classification rules from the medical databases. This paper focuses on the problem of selection of more efficient, promising and suitable classifier for the prediction of specific diseases by performing empirical studies on bunch mark medical databases. The research work under the focus concentrates on the benchmark medical data sets i.e. arrhythmia, breast-cancer, diabetes, hepatitis, mammography, lymph, liver-disorders, sick, cardiotocography, heart-statlog, breast-w, and lung-cancer. The medical data sets are obtained from the open-source UCI machine learning repository. The research work will be investigating the performance of Decision Tree (i.e. AdaBoost.NC, C45-C, CART, and ID3-C) and Support Vector Machines. For experimentation, Knowledge Extraction based on Evolutionary Learning (KEEL), a data mining tool will be used. This research work provides the empirical performance analysis of decision tree-based classifiers and SVM on a specific dataset. Moreover, this article provides a comparative performance analysis of classification approaches in terms of statistics.

Comparative performance of regression tree and parametric classification of savannah woody cover on SPOT 6 NAOMI imagery

Earth observation data potentially can provide ecosystem state indicative data for use in nature conservation. Full realisation of this potential requires robust algorithms that can extract the required ecosystem attributes from imagery. Woody cover is an indicative ecosystem state variable in savannah wilderness areas, whose extraction poses a challenge in reproducing the characteristically discontinuous cover. In this work the performance of the more modern, nonparametric regression tree and Random Forest classifiers in mapping savannah woody cover is compared with the parametric, potentially highly accurate sub-pixel classification approach. Detailed field quantification of % woody cover in 1 ha plots yielded data for the training stage of the two decision tree-based nonparametric classifiers. SPOT 6 NAOMI imagery whose timing excluded grass from the spectral signature of non-senescent vegetation was used. In order to improve the capability to extract woody cover the 6 m resolution of the multispectral bands was enhanced to 1.5 m, using high pass filter pan-sharpening. For the decision tree-based classifiers multiresolution image segmentation was then employed as preliminary step in extracting the woody cover. Objects approximating the 1 ha field plots were delineated at the coordinates of the respective field sampling sites. The univariate decision tree training stage used 60% of the sampling sites. The woody cover classes 0–19%, 20–39%, 40–59% and 60–79% were the labels of recursive partitioning terminal tree leaves. For Random Forest, up to 500 decision trees (Ntree) were specified. For regression tree modelling the 10-fold cross validation approach using the 1-SE rule was used for optimal tree selection, and pruning using 20% of the field sampling sites was accomplished using the cost complexity pruning technique. The remaining 20% of the field sampling sites were reserved for accuracy assessment, for which an additional stratified random sample of coordinates verified using a concurrent high resolution QuickBird image was used. Spectral signatures approximating Gaussian distributions were generated using woody end member sites for sub-pixel classification. Random Forest produced the highest overall accuracy (95.7%), followed by regression tree modelling (90.6%), and sub-pixel classification (73.3%). The two nonparametric classifiers correctly modelled the landscape distribution, occasional continuous formations and discontinuities in the woody cover. Based on the results, nonparametric classifiers appeared more appropriate for mapping the discontinuous savannah woody cover.

An islanding detection methodology combining decision trees and Sandia frequency shift for inverter‐based distributed generations

Distributed generations (DGs) for grid-connected applications require an accurate and reliable islanding detection methodology (IDM) for secure system operation. This study presents an IDM for grid-connected inverter-based DGs. The proposed method is a combination of passive and active islanding detection techniques for aggregation of their advantages and elimination/minimisation of the drawbacks. In the proposed IDM, the passive method utilises critical system attributes extracted from local voltage measurements at target DG locations as well as employs decision tree-based classifiers for characterisation and detection of islanding events. The active method is based on Sandia frequency shift technique and is initiated only when the passive method is unable to differentiate islanding events from other system events. Thus, the power quality degradation introduced into the system by active islanding detection techniques can be minimised. Furthermore, a combination of active and passive techniques allows detection of islanding events under low power mismatch scenarios eliminating the disadvantage associated with the use of passive techniques alone. Detailed case study results demonstrate the effectiveness of the proposed method in detection of islanding events under various power mismatch scenarios, load quality factors and in the presence of single or multiple grid-connected inverter-based DG units.

Subject identification via ECG fiducial-based systems: Influence of the type of QT interval correction

Electrocardiography (ECG) has been recently proposed as biometric trait for identification purposes. Intra-individual variations of ECG might affect identification performance. These variations are mainly due to Heart Rate Variability (HRV). In particular, HRV causes changes in the QT intervals along the ECG waveforms. This work is aimed at analysing the influence of seven QT interval correction methods (based on population models) on the performance of ECG-fiducial-based identification systems. In addition, we have also considered the influence of training set size, classifier, classifier ensemble as well as the number of consecutive heartbeats in a majority voting scheme. The ECG signals used in this study were collected from thirty-nine subjects within the Physionet open access database. Public domain software was used for fiducial points detection. Results suggested that QT correction is indeed required to improve the performance. However, there is no clear choice among the seven explored approaches for QT correction (identification rate between 0.97 and 0.99). MultiLayer Perceptron and Support Vector Machine seemed to have better generalization capabilities, in terms of classification performance, with respect to Decision Tree-based classifiers. No such strong influence of the training-set size and the number of consecutive heartbeats has been observed on the majority voting scheme.

An enhanced fuzzy min-max neural network for pattern classification.

An enhanced fuzzy min-max (EFMM) network is proposed for pattern classification in this paper. The aim is to overcome a number of limitations of the original fuzzy min-max (FMM) network and improve its classification performance. The key contributions are three heuristic rules to enhance the learning algorithm of FMM. First, a new hyperbox expansion rule to eliminate the overlapping problem during the hyperbox expansion process is suggested. Second, the existing hyperbox overlap test rule is extended to discover other possible overlapping cases. Third, a new hyperbox contraction rule to resolve possible overlapping cases is provided. Efficacy of EFMM is evaluated using benchmark data sets and a real medical diagnosis task. The results are better than those from various FMM-based models, support vector machine-based, Bayesian-based, decision tree-based, fuzzy-based, and neural-based classifiers. The empirical findings show that the newly introduced rules are able to realize EFMM as a useful model for undertaking pattern classification problems.

Improvement of Adequate Use of Warfarin for the Elderly Using Decision Tree-based Approaches

Due to the narrow therapeutic range and high drug-to-drug interactions (DDIs), improving the adequate use of warfarin for the elderly is crucial in clinical practice. This study examines whether the effectiveness of using warfarin among elderly inpatients can be improved when machine learning techniques and data from the laboratory information system are incorporated. Having employed 288 validated clinical cases in the DDI group and 89 cases in the non-DDI group, we evaluate the prediction performance of seven classification techniques, with and without an Adaptive Boosting (AdaBoost) algorithm. Measures including accuracy, sensitivity, specificity and area under the curve are used to evaluate model performance. Decision tree-based classifiers outperform other investigated classifiers in all evaluation measures. The classifiers supplemented with AdaBoost can generally improve the performance. In addition, weight, congestive heart failure, and gender are among the top three critical variables affecting prediction accuracy for the non-DDI group, while age, ALT, and warfarin doses are the most influential factors for the DDI group. Medical decision support systems incorporating decision tree-based approaches improve predicting performance and thus may serve as a supplementary tool in clinical practice. Information from laboratory tests and inpatients' history should not be ignored because related variables are shown to be decisive in our prediction models, especially when the DDIs exist.

Identification of Water Stress in Citrus Leaves Using Sensing Technologies

Water stress is a serious concern in the citrus industry due to its effect on citrus quality and yield. A sensor system for early detection will allow rapid implementation of control measures and management decisions to reduce any adverse effects. Laser-induced breakdown spectroscopy (LIBS) presents a potentially suitable technique for early stress detection through elemental profile analysis of the citrus leaves. It is anticipated that the physiological change in plants due to stress will induce changes in the element profile. The major goal of this study was to evaluate the performance of laser-induced breakdown spectroscopy as a method of water stress detection for potential use in the citrus industry. In this work, two levels of water stress were applied to Cleopatra (Cleo) mandarin, Carrizo citrange, and Shekwasha seedlings under the controlled conditions of a greenhouse. Leaves collected from the healthy and stressed plants were analyzed using LIBS, as well as with a spectroradiometer (visible-near infrared spectroscopy) and a thermal camera (thermal infrared). Statistical classification of healthy and stressed samples revealed that the LIBS data could be classified with an overall accuracy of 80% using a Naïve-Bayes and bagged decision tree-based classifiers. These accuracies were lower than the classification accuracies acquired from visible-near infrared spectra. An accuracy of 93% and higher was achieved using a bagged decision tree with visible-near infrared spectral reflectance data.

Information Decay in Building Predictive Models Using Temporal Data

Most predictive data mining methods are based on the assumption that the historical data involved in building and verifying a model are the best estimator of what will happen in the future. However, the relevance of past to the future depends on the application domain in a specific timeframe. In this paper, we present a method of predicting the future from temporal data using information decay technique that estimate the relevance of the past to the future with the help of the knowledge of the information lifetime of the temporal data. We show how to use a decision tree data mining technique to build an information-decay-based predictive model by introducing an information decay method in impurity measure functions. The relevance of the concepts is proven by comparing two decision tree-based classifiers: one built using information decay over a specific timeframe and the other built setting aside the time factor.