Tree-based Classification Techniques Research Articles

Tree-Based Modeling Methods to Predict Nitrate Exceedances in the Ogallala Aquifer in Texas

The performance of four tree-based classification techniques—classification and regression trees (CART), multi-adaptive regression splines (MARS), random forests (RF) and gradient boosting trees (GBT) were compared against the commonly used logistic regression (LR) analysis to assess aquifer vulnerability in the Ogallala Aquifer of Texas. The results indicate that the tree-based models performed better than the logistic regression model, as they were able to locally refine nitrate exceedance probabilities. RF exhibited the best generalizable capabilities. The CART model did better in predicting non-exceedances. Nitrate exceedances were sensitive to well depths—an indicator of aquifer redox conditions, which, in turn, was controlled by alkalinity increases brought forth by the dissolution of calcium carbonate. The clay content of soils and soil organic matter, which serve as indicators of agriculture activities, were also noted to have significant influences on nitrate exceedances. Likely nitrogen releases from confined animal feedlot operations in the northeast portions of the study area also appeared to be locally important. Integrated soil, hydrogeological and geochemical datasets, in conjunction with tree-based methods, help elucidate processes controlling nitrate exceedances. Overall, tree-based models offer flexible, transparent approaches for mapping nitrate exceedances, identifying underlying mechanisms and prioritizing monitoring activities.

Pushing Visualization Effects into Pushed Schema Enumerated Tree-Based Support Constraints

Based on the datasets from UCI and Obstructive Sleep Apnea, a disparate methodology of uncovering the visualization effects into the pushed support constraints of schema enumerated tree-based classification techniques is proposed and presented in this paper. This is to actively ‘wipe out’ the redundant growing effects of decision trees through itemset generation when visualization techniques are applied using Principal Component Analysis (PCA) and/or Principal Component Variable Grouping (PCVG) algorithms. Enumeration specification is based on the schema enumerated tree (SET) drawn after sorting out the features and characteristics on each dataset applied. The linchpin is to streamline the pre-tree classification effects for post-tree classification by using visualization techniques, i.e. PCA and/or PCVG, which are applied during the SET development. The over-fitting effects done during the SET development by the pushed support constraints can be counter-corrected by fewer PCA and/or PCVG imposed during visualization processes. The under-fitting effects done by the imprecise ‘early stopping’ of the SET development can be counter-corrected by greater PCA and/or PCVG imposed during the post-tree classification techniques through pushed SET support constraint learning. Research outcome on all the investigated datasets showed that the prediction accuracies have been profoundly improved after applying visualization of PCA and/or PCVG algorithms into the pushed SET-based or SET-based support constraints.

Predictive maintenance using tree-based classification techniques: A case of railway switches

With growing service demands, rapid deterioration due to extensive usage, and limited maintenance due to budget cuts, the railway infrastructure is in a critical state and require continuous maintenance. The infrastructure managers have to come up with smart maintenance decisions in order to improve the assets’ condition, spend optimal cost and keep the network available. Currently, the infrastructure managers lack the tools and decision support models that could assist them in taking (un) planned maintenance decisions effectively and efficiently. Recently, many literature studies have proposed to employ the machine learning techniques to estimate the performance state of an asset, predict the maintenance need, possible failure modes, and such similar aspects in advance. Most of these studies have utilised additional data collection measures to record the assets’ behaviour. Though useful for experimentation, it is expensive and impractical to mount monitoring devices on multiple assets across the network. Therefore, the objective of this study is to develop predictive models that utilise existing data from a railway agency and yield interpretable results. We propose to leverage the tree-based classification techniques of machine learning in order to predict maintenance need, activity type and trigger’s status of railway switches. Using the data from an in-use business process, predictive models based on the decision tree, random forest, and gradient boosted trees are developed. Moreover, to facilitate in models interpretability, we provided a detail explanation of models’ predictions by features importance analysis and instance level details. Our solution approach of predictive models development and their results explanation have wider applicability and can be used for other asset types and different (maintenance) planning scenarios.

Analysis and Prediction of Diabetes Complication Disease using Data Mining Algorithm

Diabetes is one of the most dangerous chronic disease that could lead to others serious complicating diseases. In Indonesia, the most common diabetes microvascular complications diseases are retinopathy, nephropathy and neuropathy. In order to prevent these complications to manifest, data mining technique to extract knowledge of risk factor for each complication becomes crucial. The goal of this research is to construct a prediction model for three major diabetes complication diseases in Indonesia and find out the significant features correlated with it. In this research, the diabetes risk factor narrowed into seven features, which are Age, Gender, BMI, Family history of diabetes, Blood pressure, duration of diabetes suffers and Blood glucose level. Thus, Naive Bayes Tree and C4.5 decision tree-based classification techniques and k-means clustering techniques were used to analyze this dataset. After this analysis, we evaluated the performance of each technique and found the correlated feature and sub feature as a disease risk factor for them. Resulting the most influential risk factor for Retinopathy is a female patient that having a hypertension crisis. As for Nephropathy, the most prominent risk factor is the duration of diabetes more than 4 years. But for Neuropathy, it dominated for female patients, with BMI more than 25. As for family history of diabetes, there is no distinct significant correlation with these complication diseases. The overall accuracy of the proposed model is 68% so it, could be used to as an alternative method to help predict diabetes complication diseases at an early stage.

Effective Diagnosis of Alzheimer’s Disease using Modified Decision Tree Classifier

Abstract Alzheimer’s disease (AD) is described as a severe form of the neural disorder that collectively degenerate the essential cognitive activities of a human being (thinking, memory retention, etc.,) in particular among the elderly individuals and eventually results in death. In addition to the adverse ill-health effects on the patients, AD imposes paramount responsibility and burden on the caretakers too. Several genetic and pathological traits and non-invasive diagnostic strategies are being vigorously investigated and explored to discover the early onset of this debilitating disease. The prognosis of AD assumes importance, as the deterioration of health due to its progression may be either contained or controlled. Moreover, early and accurate detection of AD helps medical practitioners to prescribe case-specific medical treatment procedure. Among the popular machine learning algorithms, decision tree technique is widely used for classification/prediction, due to its accuracy and speed.This research article presents a novel decision tree-based classification technique, with optimum hyper parametertuning, that is ideally suitable for AD diagnosis, even at the early stages of development. The performance of this newly proposed Decision Tree Classifier with Hyper Parameters Tuning (DTC-HPT) is validated on the Open Access Imaging Studies Series (OASIS) dataset that contains patients’ data on the different stages of AD. The DTC-HPT is designed with the primary objective to classify the nature of brain abnormality using the most relevantand potentially significant data attributes/parameters. The efficiency of DTC-HPT on AD classification is measured as Accuracy, Precision, Recall, and F1-Score. The correctness of AD classification by DTC-HPTwith an average accuracy of 99.10% endorse that this classification technique can be used for AD detection on the AD clinical datasets.

A Survey of Various Tree Based Classification Techniques

A Survey of Various Tree Based Classification Techniques

RULEM: A novel heuristic rule learning approach for ordinal classification with monotonicity constraints

In many real world applications classification models are required to be in line with domain knowledge and to respect monotone relations between predictor variables and the target class, in order to be acceptable for implementation. This paper presents a novel heuristic approach, called RULEM, to induce monotone ordinal rule based classification models. The proposed approach can be applied in combination with any rule- or tree-based classification technique, since monotonicity is guaranteed in a post-processing step. RULEM checks whether a rule set or decision tree violates the imposed monotonicity constraints and existing violations are resolved by inducing a set of additional rules which enforce monotone classification. The approach is able to handle non-monotonic noise, and can be applied to both partially and totally monotone problems with an ordinal target variable. Two novel justifiability measures are introduced which are based on RULEM and allow to calculate the extent to which a classification model is in line with domain knowledge expressed in the form of monotonicity constraints. An extensive benchmarking experiment and subsequent statistical analysis of the results on 14 public data sets indicates that RULEM preserves the predictive power of a rule induction technique while guaranteeing monotone classification. On the other hand, the post-processed rule sets are found to be significantly larger which is due to the induction of additional rules. E.g., when combined with Ripper a median performance difference was observed in terms of PCC equal to zero and an average difference equal to −0.66%, with on average 5 rules added to the rule sets. The average and minimum justifiability of the original rule sets equal respectively 92.66% and 34.44% in terms of the RULEMF justifiability index, and 91.28% and 40.1% in terms of RULEMS, indicating the effective need for monotonizing the rule sets.

Impact of Bio-inspired metaheuristics in the data clustering problem

The goal of data mining is to extract the knowledge from data. It is also a form of knowledge discovery essential for solving problem in a specific domain. This paper presents a novel approach to data clustering and classification problem. Clustering analysis is distribution of data into groups of similar objects and Classification focuses the data on the class boundaries. This research explores three different bio-inspired metaheuristic algorithms in the clustering problem: Ant Colony Optimization (ACO), Genetic Algorithms (GAs) and Artificial Immune Systems (AIS). Data mining approaches are applied in the field of medical diagnosis recently. The major class of problem in medical science involves diagnosis of disease based upon various tests. The computerized diagnostic tools are helpful to predict the diagnosis accurately. Breast cancer is one the most dangerous cancer type in the world. Early detection can save a life and increase survivability of the patients. This of research work analysed the performance of GA, ACO and AIS with ID3 for solving data clustering and classification problem in an experiment with Breast Cancer Dataset data of UCI repository. An efficient ID3 Decision tree based classification techniques are used to measure the performance of the system with GA, ACO and AIS system. Proposed AIS system produces the best classification result than the ACO and GA based decision tree ID3 classifiers. Instead of K-means clustering, this research work combines the simplicity of K-means algorithm with the robustness of AGA-Miner. This proposed approach has potential applications in hospital for decision-making and analyze/ research such as predictive medicine.

A Dirichlet process model for classifying and forecasting epidemic curves.

BackgroundA forecast can be defined as an endeavor to quantitatively estimate a future event or probabilities assigned to a future occurrence. Forecasting stochastic processes such as epidemics is challenging since there are several biological, behavioral, and environmental factors that influence the number of cases observed at each point during an epidemic. However, accurate forecasts of epidemics would impact timely and effective implementation of public health interventions. In this study, we introduce a Dirichlet process (DP) model for classifying and forecasting influenza epidemic curves.MethodsThe DP model is a nonparametric Bayesian approach that enables the matching of current influenza activity to simulated and historical patterns, identifies epidemic curves different from those observed in the past and enables prediction of the expected epidemic peak time. The method was validated using simulated influenza epidemics from an individual-based model and the accuracy was compared to that of the tree-based classification technique, Random Forest (RF), which has been shown to achieve high accuracy in the early prediction of epidemic curves using a classification approach. We also applied the method to forecasting influenza outbreaks in the United States from 1997–2013 using influenza-like illness (ILI) data from the Centers for Disease Control and Prevention (CDC).ResultsWe made the following observations. First, the DP model performed as well as RF in identifying several of the simulated epidemics. Second, the DP model correctly forecasted the peak time several days in advance for most of the simulated epidemics. Third, the accuracy of identifying epidemics different from those already observed improved with additional data, as expected. Fourth, both methods correctly classified epidemics with higher reproduction numbers (R) with a higher accuracy compared to epidemics with lower R values. Lastly, in the classification of seasonal influenza epidemics based on ILI data from the CDC, the methods’ performance was comparable.ConclusionsAlthough RF requires less computational time compared to the DP model, the algorithm is fully supervised implying that epidemic curves different from those previously observed will always be misclassified. In contrast, the DP model can be unsupervised, semi-supervised or fully supervised. Since both methods have their relative merits, an approach that uses both RF and the DP model could be beneficial.

Pattern Trees for Fault-Proneness Detection in Object-Oriented Software

Problem statement: This study introduced an application of pattern tree based classification technique in the area of object-oriented software quality estimation. This application explored the fault prediction accuracy of pattern trees. Approach: Similarity measures and fuzzy aggregations employed in the pattern tree technique had been used to generate tree models for fault detection in software modules. Experiments had been performed on datasets namely, KC1 and KC3 obtained from NASA's metric data program. Pattern tree models were built using metrics from the object-oriented software datasets. Results: AND/OR, OWA and WA had been selected for pattern tree induction. Pattern tree models build using RMSE similarity measure produced higher accuracy as compared to other similarity measures. Conclusion: The proposed application succeeded in improving the quality of the object-oriented software in terms of prediction accuracy. Pattern trees models were found to be less structural complex as compared to fuzzy decision trees.

A comparison of nearest neighbour and tree-based methods of non-parametric discriminant analysis

Nearest neighbour discriminant analysis is compared under cross-validation with a tree-based classification technique, for discrimination tasks involving two classes and Cauchy-distributed error on the independent variables. It is shown that, for the tasks employed, as the number of variables is increased, performance of the nearest neighbour algorithm declines, whereas that of the tree-based technique improves. Above six or seven variables, the tree-based method shows superior discrimination power. The results are explained in terms of the nearest neighbour technique being liable to overfitting when large numbers of variables are used, in contrast to the tree-based technique, which incorporates protection against overfitting by branching on only a subset of the independent variables.