Unpruned Decision Tree Research Articles

MACHINE LEARNING TO CREATE DECISION TREE MODEL TO PREDICT OUTCOME OF ENTERPRENEURSHIP PSYCHOLOGICAL READINESS (EPR)

This study aims to create a decision tree model using machine learning to predict psychological readiness for entrepreneurship in college graduates. This research was conducted through several stages of research. In the early stages, a survey was conducted on 700 students from several universities in Riau aged between 17-25 years. The survey was conducted using the Entrepreneur Psychology Readiness (EPR) instrument. Furthermore, the survey data was validated and obtained 604 valid data to be used in forming machine learning models The urgency of this research is to find a number of decision rules from the best decision tree model to be used in building AI-based counseling applications in measuring entrepreneurial psychology readiness for college graduates. In this research, the decision tree model that is formed is divided into 2 models, namely: decision tree with pruning model and decision tree with unpruning. The pruning decision tree model produces 180 decision rules, while the unpruning model produces 121 decision rules. Good accuracy results are obtained in the pruned decision tree, which is above 99% in the use training set mode, and 82.87% in the percentage split mode. Meanwhile, the accuracy results on the unpruned decision tree are 90.18% with the use training set mode test, and 80.38% in the percentage split mode. The decision tree model with pruning technique has better performance than the unpruning decision tree model.

Selection of the Right Undergraduate Major by Students Using Supervised Learning Techniques

University education has become an integral and basic part of most people preparing for working life. However, placement of students into the appropriate university, college, or discipline is of paramount importance for university education to perform its role. In this study, various explainable machine learning approaches (Decision Tree [DT], Extra tree classifiers [ETC], Random forest [RF] classifiers, Gradient boosting classifiers [GBC], and Support Vector Machine [SVM]) were tested to predict students’ right undergraduate major (field of specialization) before admission at the undergraduate level based on the current job markets and experience. The DT classifier predicts the target class based on simple decision rules. ETC is an ensemble learning technique that builds prediction models by using unpruned decision trees. RF is also an ensemble technique that uses many individual DTs to solve complex problems. GBC classifiers and produce strong prediction models. SVM predicts the target class with a high margin, as compared to other classifiers. The imbalanced dataset includes secondary school marks, higher secondary school marks, experience, and salary to select specialization for students in undergraduate programs. The results showed that the performances of RF and GBC predict the student field of specialization (undergraduate major) before admission, as well as the fact that these measures are as good as DT and ETC. Statistical analysis (Spearman correlation) is also applied to evaluate the relationship between a student’s major and other input variables. The statistical results show that higher student marks in higher secondary (hsc_p), university degree (Degree_p), and entry test (etest_p) play an important role in the student’s area of specialization, and we can recommend study fields according to these features. Based on these results, RF and GBC can easily be integrated into intelligent recommender systems to suggest a good field of specialization to university students, according to the current job market. This study also demonstrates that marks in higher secondary and university and entry tests are useful criteria to suggest the right undergraduate major because these input features most accurately predict the student field of specialization.

A Case Study of Applying the Classification Task for Students' Performance Prediction

In this study the application of data mining in an academic database is presented, aiming the identification of the reasons of student dropouts by preventing failures in Programming Language class from Internet Systems course of Federal Institute of Mato Grosso do Sul (IFMS). The classification task is used, as well as decision tree technique and J4.8 algorithm, wich is run with three different options and, in each option pruned and unpruned decision trees are generated. The results show that the most realistic test is Cross-validation, with a success rate of 75.8% in classification, and that it is possible to prevent student failure in specific cases and as a consequence, to lower the number of student dropouts by failure.

Design and Application of Reduced Error Pruning Tree for Traffic Incident Detection

For aim applied to develop Intelligent Transportation System (ITS), a traffic incident detection method based on Reduced Error Pruning Tree (REPTree) algorithm of decision tree is presented. Different from unpruned decision tree, REPTree model is a fast decision tree learner which builds a decision tree using information gain as the splitting criterion, and prunes it using reduced-error pruning. The detection performance of the REPTree was compared to multi-layer feed forward neural networks (MLFNN) and Radical Basis Function neural networks (RBFNN) which yield superior incident detection performance in the previous studies. The experimental results indicate that REPTree model is competitive with MLFNN and RBFNN.

Use of decision tree and artificial neural network approaches to model presence/absence of Telestes muticellus in piedmont (North‐Western Italy)

AbstractIn Piedmont (Italy) the impact of human beings is causing some deep environmental changes in freshwaters and their inhabitants, so much so that we need to develop some practical tools for immediate use in providing accurate ecological assessments of the freshwater system and of the conditions of the species living there, one of which is Telestes muticellus, an endangered Cyprinidae found in the western Alps and the central Apennines in Italy. We aimed to help manage this species by assessing its presence using two types of data‐mining approaches—decision‐tree models and artificial neural networks. We built models using 10 environmental input variables to classify sites as positive or negative for the species. The unpruned decision tree models classified a high percentage of instances correctly and made accurate predictions, as did the post‐pruned tree models. The post‐pruned methods yielded simpler trees and therefore clearer models. Generally, the artificial neural networks (ANN) performed better than the decision tree models, except in the case of Cohen's k. We used the sensitivity analysis technique to understand which inputs are the most important ones for building the ANN model we obtained. Copyright © 2008 John Wiley & Sons, Ltd.

Class-switching neural network ensembles

This article investigates the properties of class-switching ensembles composed of neural networks and compares them to class-switching ensembles of decision trees and to standard ensemble learning methods, such as bagging and boosting. In a class-switching ensemble, each learner is constructed using a modified version of the training data. This modification consists in switching the class labels of a fraction of training examples that are selected at random from the original training set. Experiments on 20 benchmark classification problems, including real-world and synthetic data, show that class-switching ensembles composed of neural networks can obtain significant improvements in the generalization accuracy over single neural networks and bagging and boosting ensembles. Furthermore, it is possible to build medium-sized ensembles ( ≈ 200 networks) whose classification performance is comparable to larger class-switching ensembles ( ≈ 1000 learners) of unpruned decision trees.

Predicting nearly as well as the best pruning of a decision tree through dynamic programming scheme

Helmbold and Schapire gave an on-line prediction algorithm that, when given an unpruned decision tree, produces predictions not much worse than the predictions made by the best pruning of the given decision tree. In this paper, we give two new on-line algorithms. The first algorithm is based on the observation that finding the best pruning can be efficiently solved by a dynamic programming in the “batch” setting where all the data to be predicted are given in advance. This algorithm works well for a wide class of loss functions, whereas the one given by Helmbold and Schapire is only described for the absolute loss function. Moreover, the algorithm given in this paper is so simple and general that it could be applied to many other on-line optimization problems solved by dynamic programming. We also explore the second algorithm that is competitive not only with the best pruning but also with the best prediction values which are associated with nodes in the decision tree. In this setting, a greatly simplified algorithm is given for the absolute loss function. It can be easily generalized to the case where, instead of using decision trees, data are classified in some arbitrarily fixed manner.

Approximate Match of Rules Using Backpropagation Neural Networks

This paper presents a method for approximate match of first-order rules with unseen data. The method is useful especially in case of a multi-class problem or a noisy domain where unseen data are often not covered by the rules. Our method employs the Backpropagation Neural Network for the approximation. To build the network, we propose a technique for generating features from the rules to be used as inputs to the network. Our method has been evaluated on four domains of first-order learning problems. The experimental results show improvements of our method over the use of the original rules. We also applied our method to approximate match of propositional rules converted from an unpruned decision tree. In this case, our method can be thought of as soft-pruning of the decision tree. The results on multi-class learning domains in the UCI repository of machine learning databases show that our method performs better than standard C4.5's pruned and unpruned trees.