Decision Tree Induction Research Articles

Data Mining Generating Decision Trees to Alert System Against Death and Losses in Egg Production

Climatic changes and high temperatures have been affecting animal production and the well-being of laying birds, with heat stress and high mortality rates, generating economic losses. Legacy databases can contain information to help model thermal comfort at climatic extremes. They can enable decision trees to be created through the use of data mining to prevent mortality and production losses. Thus, the objective of this study is to seek to develop decision trees, for application as an alert system, for the incidence of caloric stress in the production of layers. We used a database of three aviaries located in the city of Bastos-SP, collected in 2013. The data were organized in Excel® spreadsheets, and processed with the Weka® software with the J48 (C4.5) algorithm for mining of the data. The technique allowed the construction of decision trees that in the chosen sheds were classified with respectively 99.73%, 99.61%, and 98.71% of correct answers and with Kappa indexes equal to 0.9958, 0.9907 and 0.9663, which indicate that the three classifiers built are excellent. Thus, the proposed system, with the decision trees built, can serve as a basis for the construction of an alert system to be applied to the three warehouses simultaneously.

Analyse Success Model of Split Time and Cut-Off Point Values of Physical Demands to Keep Category in Semi-Professional Football Players

The aim of this study was to analyse different success models and split time on cut-off point values on physical demands to keep category in semi-professional football players. An ad hoc observational controlled study was carried out with a total of ten (840 match data) outfield main players (25.2 ± 6.3 years, 1.79 ± 0.75 m, 74.9 ± 5.8 kg and 16.5 ± 6 years of football experience) and monitored using 15 Hz GPS devices. During 14 official matches from the Spanish division B in the 2016/2017 season, match data were coded considering the situational variable (score) and classified by match results (winning, losing or drawing). The results show significant differences between high-intensity attributes criteria that considered split time in velocity zones of 0–15 min (p = 0.043, ηp2 = 0.065, medium), 30–45 min (p = 0.010, ηp2 = 0.094, medium) and 60–75 min (p = 0.015, ηp2 = 0.086, medium), as well as sprint 60–75 min (p = 0.042, ηp2 = 0.066, medium) and 75–90 min (p = 0.002, ηp2 = 0.129, medium). Decision tree induction was applied to reduce the disparity range of data according to six 15-min intervals and to determine the cut-off point values for every parameter combination. It was possible to establish multivariate models for the main high-intensity actions criteria, allowing the establishment of all rules with their attributes and enabling the detection and visualisation of relationships and the pattern sets of variables for determining success.

Efficient Closure Operators for FCA-Based Classification

Knowledge discovery in databases (KDD) aims to exploit the large amounts of data collected every day in various fields of computing application. The idea is to extract hidden knowledge from a set of data. It gathers several tasks that constitute a process, such as: data selection, pre-processing, transformation, data mining, visualization, etc. Data mining techniques include supervised classification and unsupervised classification. Classification consists of predicting the class of new instances with a classifier built on learning data of labeled instances. Several approaches were proposed such as: the induction of decision trees, Bayes, nearest neighbor search, neural networks, support vector machines, and formal concept analysis. Learning formal concepts always refers to the mathematical structure of concept lattice. This article presents a state of the art on formal concept analysis classifier. The authors present different ways to calculate the closure operators from nominal data and also present new approach to build only a part of the lattice including the best concepts. This approach is based on Dagging (ensemble method) that generates an ensemble of classifiers, each one represents a formal concept, and combines them by a voting rule. Experimental results are given to prove the efficiency of the proposed method.

Windowing as a Sub-Sampling Method for Distributed Data Mining

Windowing is a sub-sampling method, originally proposed to cope with large datasets when inducing decision trees with the ID3 and C4.5 algorithms. The method exhibits a strong negative correlation between the accuracy of the learned models and the number of examples used to induce them, i.e., the higher the accuracy of the obtained model, the fewer examples used to induce it. This paper contributes to a better understanding of this behavior in order to promote windowing as a sub-sampling method for Distributed Data Mining. For this, the generalization of the behavior of windowing beyond decision trees is established, by corroborating the observed negative correlation when adopting inductive algorithms of different nature. Then, focusing on decision trees, the windows (samples) and the obtained models are analyzed in terms of Minimum Description Length (MDL), Area Under the ROC Curve (AUC), Kulllback–Leibler divergence, and the similitude metric Sim1; and compared to those obtained when using traditional methods: random, balanced, and stratified samplings. It is shown that the aggressive sampling performed by windowing, up to 3% of the original dataset, induces models that are significantly more accurate than those obtained from the traditional sampling methods, among which only the balanced sampling is comparable in terms of AUC. Although the considered informational properties did not correlate with the obtained accuracy, they provide clues about the behavior of windowing and suggest further experiments to enhance such understanding and the performance of the method, i.e., studying the evolution of the windows over time.

Treant: training evasion-aware decision trees

Despite its success and popularity, machine learning is now recognized as vulnerable to evasion attacks, i.e., carefully crafted perturbations of test inputs designed to force prediction errors. In this paper we focus on evasion attacks against decision tree ensembles, which are among the most successful predictive models for dealing with non-perceptual problems. Even though they are powerful and interpretable, decision tree ensembles have received only limited attention by the security and machine learning communities so far, leading to a sub-optimal state of the art for adversarial learning techniques. We thus propose Treant, a novel decision tree learning algorithm that, on the basis of a formal threat model, minimizes an evasion-aware loss function at each step of the tree construction. Treant is based on two key technical ingredients: robust splitting and attack invariance, which jointly guarantee the soundness of the learning process. Experimental results on publicly available datasets show that Treant is able to generate decision tree ensembles that are at the same time accurate and nearly insensitive to evasion attacks, outperforming state-of-the-art adversarial learning techniques.

Predict Students' Performance by Using J48 Algorithm

The critical issue to the academic community of higher education is to monitor the progress of students’ academic performance. We can use data mining techniques for this purpose. J48 algorithm is one of the famous classification algorithms present today to generate decision trees in data mining technique. The data set used in this study is taken from University of Computer Studies (Mandalay). Weka machine learning tool is applied to make classification. In this work, we tested result classification accuracy was computed. This J48 classification algorithm give accuracy with 78.2%.

A New Direct Node Data Splitting Technique in Decision Tree Induction

Learning knowledge from the example data and then applying that knowledge on new applications is the goal of machine learning. Split attribute technique is inseparable and an important means of decision tree construction technique. It is a well known fact and universally accepted truth that a difficult and larger data mining model tends to create less significant generalized performance results. Researchers are being continuously trying to find new and the best split attribute techniques during decision tree induction. In this paper a new direct split attribute technique for decision tree induction is proposed based on the mathematical A implies B tautology principle. The experimental results show that this rule is worthy and useful in many real world applications, particularly in medical field. The resulted association relationships are perfectly matching with the expected results.

Optimizing an eDNA protocol for estuarine environments: Balancing sensitivity, cost and time

Environmental DNA (eDNA) analysis has gained traction as a precise and cost-effective method for species and waterways management. To date, publications on eDNA protocol optimization have focused primarily on DNA yield. Therefore, it has not been possible to evaluate the cost and speed of specific components of the eDNA protocol, such as water filtration and DNA extraction method when designing or choosing an eDNA protocol. At the same time, these two parameters are essential for the experimental design of a project. Here we evaluate and rank 27 different eDNA protocols in the context of Chinook salmon (Oncorhynchus tshawytscha) eDNA detection in an estuarine environment. We present a comprehensive evaluation of multiple eDNA protocol parameters, balancing time, cost and DNA yield. We collected samples composed of 500 mL estuarine water from Deverton Slough (38°11'16.7"N 121°58'34.5"W) and 500 mL from tank water containing 1.3 juvenile Chinook Salmon per liter. Then, we compared extraction methods, filter types, use of inhibitor removal kit for DNA yield, processing time, and protocol cost. Lastly, we used an MCMC algorithm together with machine learning to understand the DNA yield of each step of the protocol as well as the interactions between those steps. Glass fiber filtration was to be the most resilient to high turbidites, filtering the samples in 2.32 ± 0.08 min instead of 14.16 ± 1.86 min and 6.72 ± 1.99 min for nitrocellulose and paper filter N1, respectively. The filtration DNA yield percentages for paper filter N1, glass fiber, and nitrocellulose were 0.00045 ± 0.00013, 0.00107 ± 0.00013, 0.00172 ± 0.00013. The DNA extraction yield percentage for QIagen, dipstick, NaOH, magnetic beads, and direct dipstick ranged from 0.047 ± 0.0388 to 0.475 ± 0.0357. For estuarine waters, which are challenging for eDNA studies due to high turbidity, variable salinity, and the presence of PCR inhibitors, we found that a protocol combining glass filters, magnetic beads, and an extra step for PCR inhibitor removal, is the method that best balances time, cost, and yield. In addition, we provide a generalized decision tree for determining the optimal eDNA protocol for other studies in aquatic systems. Our findings should be applicable to most aquatic environments and provide a clear guide for determining which eDNA protocol should be used under different study constraints.

Traffic Accidents Classification and Injury Severity Prediction

Traffic accidents are one of the most life-threatening dangers to human being. Deaths and injuries due to traffic accidents have a great impact on society. Traffic accidents information and data provided by public can be useful to classify these accidents according to their type and severity, and consequently try to build predictive model. Detecting and identifying injury severity in traffic accidents in real time is primordial for speeding post-accidents protocols as well as developing general road safety policies. In this project we are using Logistic Regression algorithm to classify accident data. The data to be analysed is collected from various sources, is both structured and unstructured and has several attributes. In this project we are going to detect and analyse data together to generate decision trees that give insights on previous accidents.

In the Realm of Hungry Ghosts: Multi-Level Theory for Supplier Participation on Digital Platforms

ABSTRACT Digital platforms have transformed various industries, with profound changes witnessed in settings characterized by repeated, low value, and novel transactions, such as ride sharing, household services, and food delivery. Platform providers need to understand the factor(s) that suppliers consider before choosing to participate on their platforms. We argue that multi-level theorizing is necessary to explain the patterns of decision criteria that constitute the complex, yet boundedly-rational decision of platform participation. We draw on multiple theoretical perspectives which include predictors from firm-level strategic behavior, firm-level digital predictors, institutional predictors, and platform level, competition and contextual (environmental) predictors to model restaurant’s decision to participate on food delivery platforms, which are an exemplar of platforms that digitize transactions of ubiquitous and episodic nature. The population of 95,735 restaurants, serving a total of 135 different cuisines, located in the 37 largest cities of India, forms the dataset for developing our multi-level theory. Our decision tree induction methodology, which employs high levels of pruning, empowers us to discover context-specific rules that serve as credible approximations of the partial ordering of decision flows tacitly used by restaurants in deciding to participate on platforms. We identify six combinations of predictors, that is, decision rules, which are situated in at least four theoretical perspectives, to succinctly explain supplier participation on digital platforms. Finally, we abduct away from these specific decision rules to develop generalizable theoretical propositions. The decision trees, context-specific rules, general forms of the rules, and generalized propositions together form our multi-level theory for supplier participation on digital platforms. Our findings aid platform providers identify suppliers who can be the focus of efforts to increase platform participation, and help suppliers identify the participation status of competitors. For policymakers, our findings imply that incentives at the ends of the pricing spectrum can increase supplier participation on digital platforms.

Meteorological Application Based on Big Data Streaming Computation Method

Big data analysis in the form of streaming computation has always been a problem to be solved at present, with relatively few research results and practical experience. The random forest method is currently the most extensively used classification algorithm. However, in the application scenario of streaming computation, real-time, volatility, and disorder features presented by data will lead to a gradual reduction in the accuracy of the algorithm. In this paper, the characteristics of the random forest algorithm are analyzed, and the idea of random forest pruning based on the accuracy of the decision tree is proposed. Meanwhile, to adapt to the changes in meteorological data, the concept of accuracy interval is combined to propose a method for the generation, verification, and supplementation of a new decision tree. Finally, a random forest that can be constantly updated with data is established to meet the requirements of the streaming big data environment for the algorithm. Actual meteorological data are used to verify the feasibility of the improved method. The results show that the new method has a higher classification accuracy in the real streaming big data scenario.

Application of Bayesian Decision Tree Algorithm in Breast Cancer Prediction

Aiming at the low classification accuracy of common decision tree C4.5 algorithm and the condition attribute problem of decision tree splitting point, a Bayesian decision tree algorithm combining ordinary decision tree C4.5 algorithm and naive Bayesian correlation algorithm is proposed in this paper. On this basis, the consistency test coefficient(kappa coefficient) is introduced to improve the correlation algorithm in the information gain rate in C4.5. Finally, the newly generated decision tree is pruned according to the most suitable splitting point. The improved algorithm proposed in this paper has a distinct improvement compared with the algorithm proposed in other literatures considering both time and accuracy.

Rule-Based Classification using Multi Soft Set Theory

One of the important data mining functionality is classification. Presently, different methods exist for implementing classification. Rule-based classification using decision tree induction method is a conventional and simple method for identifying an unknown class of a given object. This method has a set of demerits and to remove these demerits, we depend on a soft computing tool which is known as soft set theory. One branch of soft set theory is called - multi soft theory- and it has a wide range of applications in the area of classification. We made a certain alteration in the rule-based classification using decision tree induction method by applying multi soft set theory. These changes will simplify the difficulties of the rule-based classification using decision tree induction method. The first two sections of this research work discuss introduction and preliminaries. In the remaining sections, the authors describe the multi soft set theory and its applications in rule base classification. Lastly, the paper finishes with a new algorithm, which the research scholars implemented as software using python programming. The suggested work experts can use in data mining industry. It has massive use in the fields of business, agriculture, health, education and many more.

Spam Detection in Social Media Networking Sites using Ensemble Methodology with Cross Validation

Social media networking sites are more popular over Internet. The Internet users spend more amount of time on social media sites like Twitter, Facebook, Instagram and LinkedIn etc. The social media networking users share their ideas, opinions, information and make new friends. Social networking sites provide large amount of valuable information to the users. This large amount of information in social media attracts spammers to misuse information. These spammers create fake accounts and spread irrelevant information to the genuine users. The spam message information may be advertisements, malicious links to disturb the natural users. This spam data in social media is a very serious problem. Spam detection in social media networking sites is critical process. To extract spam messages in social media various spam detection methodologies are developed by researchers. In this paper we proposed an ensemble methodology for identification spam on Twitter social media network. In this methodology we used Decision tree induction algorithm, Naïve bayes algorithm and KNN algorithm to construct a model. As part of this approach, we compare the classification results of any two classification algorithms, if both classifiers predict the same result, then we finalize the class of tweet under investigation. If the predicted classes of both classification algorithms differ, then we use the prediction of third algorithm as the final class label of tweet. To measure the performance of our model we used precision, recall and F measure.

Hybrid classification algorithms based on instance filtering

Basic classification algorithms induce a single model from training data. The interpretation of a model is relatively easy, while basic algorithms have limitations in achieving high accuracy. An instance misclassified by a model may be correctly predicted by another. Hybrid classification is a concept that employs basic classification algorithms for model induction and for data preprocessing. Misclassification instances are usually considered to be noise, yet those still may carry useful information for identifying the class values of some other instances. This study proposes hybrid classification algorithms in which training instances are filtered to build three models for prediction. Each testing instance is classified by exactly one of them. The algorithms involved in the proposed hybrid classification algorithms are decision tree induction and naïve Bayesian classifier. The testing results on twenty data sets demonstrate that our hybrid classification algorithms can significantly outperform the basic ones as well as the hybrid algorithm proposed in a previous study. The hybrid classification algorithms based on instance filtering achieve relatively high accuracy and maintain the easy interpretation of learning results.

A machine learning based decision aid for lameness in dairy herds using farm-based records

Lameness is a common welfare issue in dairy farms and can negatively influence productivity and farm profitability. On-farm measurements are time consuming and continuous monitoring of lameness can be thus challenging. A predictive model that is suitable for routine field applications can be thus an efficient strategy to improve lameness status in dairy herds. We explored the use of a machine learning approach based on decision tree induction to detect and monitor the risk level of dairy herds for lameness based on 20 routinely pre-collected farm-based records related to herd management and housing, milk production performance, reproduction performance, longevity, and genetic merit. The risk of lameness was determined for 229 dairy herds based on herd prevalence. The results based on 10-fold repeated cross-validation suggest that there was little difference between a simple decision tree and more advanced machine learning algorithms such as random forests or boosting algorithms with an area under the receiver operating characteristic curve (AUC) of 0.73–0.75, but machine learning performed better than classical multivariate logistic regression (AUC of 0.28). Model sensitivity based on machine learning was highest at 0.58, whereas model specificity was highest at 0.89 among all models tested. Stacking the individual machine learning algorithms to a higher-order model slightly lifted model performance (AUC of 0.76) at a model sensitivity of 0.54 and a model specificity of 0.94 but at a loss in model interpretability. Additional information more closely related to lameness are required to further improve model performance. Nevertheless, decision tree induction was found to be useful in analysing small data sets that frequently occur in various livestock farming disciplines. Overall, the machine learning approach described in our study may be an appropriate and powerful decision support and monitoring tool, and can be implemented in a computerized information system to detect herds with potential deficiencies in lameness.

An Online Tree-Based Approach for Mining Non-Stationary High-Speed Data Streams

This paper presents a new learning algorithm for inducing decision trees from data streams. In these domains, large amounts of data are constantly arriving over time, possibly at high speed. The proposed algorithm uses a top-down induction method for building trees, splitting leaf nodes recursively, until none of them can be expanded. The new algorithm combines two split methods in the tree induction. The first method is able to guarantee, with statistical significance, that each split chosen would be the same as that chosen using infinite examples. By doing so, it aims at ensuring that the tree induced online is close to the optimal model. However, this split method often needs too many examples to make a decision about the best split, which delays the accuracy improvement of the online predictive learning model. Therefore, the second method is used to split nodes more quickly, speeding up the tree growth. The second split method is based on the observation that larger trees are able to store more information about the training examples and to represent more complex concepts. The first split method is also used to correct splits previously suggested by the second one, when it has sufficient evidence. Finally, an additional procedure rebuilds the tree model according to the suggestions made with an adequate level of statistical significance. The proposed algorithm is empirically compared with several well-known induction algorithms for learning decision trees from data streams. In the tests it is possible to observe that the proposed algorithm is more competitive in terms of accuracy and model size using various synthetic and real world datasets.

A branch & bound algorithm to determine optimal cross-splits for decision tree induction

State-of-the-art decision tree algorithms are top-down induction heuristics which greedily partition the attribute space by iteratively choosing the best split on an individual attribute. Despite their attractive performance in terms of runtime, simple examples, such as the XOR-Problem, point out that these heuristics often fail to find the best classification rules if there are strong interactions between two or more attributes from the given datasets. In this context, we present a branch and bound based decision tree algorithm to identify optimal bivariate axis-aligned splits according to a given impurity measure. In contrast to a univariate split that can be found in linear time, such an optimal cross-split has to consider every combination of values for every possible selection of pairs of attributes which leads to a combinatorial optimization problem that is quadratic in the number of values and attributes. To overcome this complexity, we use a branch and bound procedure, a well known technique from combinatorial optimization, to divide the solution space into several sets and to detect the optimal cross-splits in a short amount of time. These cross splits can either be used directly to construct quaternary decision trees or they can be used to select only the better one of the individual splits. In the latter case, the outcome is a binary decision tree with a certain sense of foresight for correlated attributes. We test both of these variants on various datasets of the UCI Machine Learning Repository and show that cross-splits can consistently produce smaller decision trees than state-of-the-art methods with comparable accuracy. In some cases, our algorithm produces considerably more accurate trees due to the ability of drawing more elaborate decisions than univariate induction algorithms.

Analysis of Dimensionality Reduction Techniques on Big Data

Due to digitization, a huge volume of data is being generated across several sectors such as healthcare, production, sales, IoT devices, Web, organizations. Machine learning algorithms are used to uncover patterns among the attributes of this data. Hence, they can be used to make predictions that can be used by medical practitioners and people at managerial level to make executive decisions. Not all the attributes in the datasets generated are important for training the machine learning algorithms. Some attributes might be irrelevant and some might not affect the outcome of the prediction. Ignoring or removing these irrelevant or less important attributes reduces the burden on machine learning algorithms. In this work two of the prominent dimensionality reduction techniques, Linear Discriminant Analysis (LDA) and Principal Component Analysis (PCA) are investigated on four popular Machine Learning (ML) algorithms, Decision Tree Induction, Support Vector Machine (SVM), Naive Bayes Classifier and Random Forest Classifier using publicly available Cardiotocography (CTG) dataset from University of California and Irvine Machine Learning Repository. The experimentation results prove that PCA outperforms LDA in all the measures. Also, the performance of the classifiers, Decision Tree, Random Forest examined is not affected much by using PCA and LDA.To further analyze the performance of PCA and LDA the eperimentation is carried out on Diabetic Retinopathy (DR) and Intrusion Detection System (IDS) datasets. Experimentation results prove that ML algorithms with PCA produce better results when dimensionality of the datasets is high. When dimensionality of datasets is low it is observed that the ML algorithms without dimensionality reduction yields better results.

Classification Models for Determining Types of Academic Risk and Predicting Dropout in University Students

Academic performance is a topic studied not only to identify those students who could drop out of their studies, but also to classify them according to the type of academic risk they could find themselves. An application has been implemented that uses academic information provided by the university and generates classification models from three different algorithms: artificial neural networks, ID3 and C4.5. The models created use a set of variables and criteria for their construction and can be used to classify student desertion and more specifically to predict their type of academic risk. The performance of these models was compared to define the one that provided the best results and that will serve to make the classification of students. Decision tree algorithms, C4.5 and ID3, presented better measurements with respect to the artificial neural network. The tree generated using the C4.5 algorithm presented the best performance metrics with correctness, accuracy, and sensitivity equal to 0.83, 0.87, and 0.90 respectively. As a result of the classification to determine student desertion it was concluded, according to the model generated using the C4.5 algorithm, that the ratio of credits approved by a student to the credits that he should have taken is the variable more significant. The classification, depending on the type of academic risk, generated a tree model indicating that the number of abandoned subjects is the most significant variable. The admission scan modality through which the student entered the university did not turn out to be significant, as it does not appear in the generated decision tree.