Fast Decision Tree Research Articles

Feature Analysis of Coronary Artery Heart Disease Data Sets

Abstract Data sets dealing with the same medical problems like Coronary artery disease (CAD) may show different results when applying the same machine learning technique. The classification accuracy results and the selected important features are based mainly on the efficiency of the medical diagnosis and analysis. The aim of this work is to apply an integration of the results of the machine learning analysis applied on different data sets targeting the CAD disease. This will avoid the missing, incorrect, and inconsistent data problems that may appear in the data collection. Fast decision tree and pruned C4.5 tree are applied where the resulted trees are extracted from different data sets and compared. Common features among these data sets are extracted and used in the later analysis for the same disease in any data set. The results show that the classification accuracy of the collected dataset is 78.06% higher than the average of the classification accuracy of all separate datasets which is 75.48%.

Decision tree structure based classification of EEG signals recorded during two dimensional cursor movement imagery

Input signals of an EEG based brain computer interface (BCI) system are naturally non-stationary, have poor signal to noise ratio, depend on physical or mental tasks and are contaminated with various artifacts such as external electromagnetic waves, electromyogram and electrooculogram. All these disadvantages have motivated researchers to substantially improve speed and accuracy of all components of the communication system between brain and a BCI output device. In this study, a fast and accurate decision tree structure based classification method was proposed for classifying EEG data to up/down/right/left computer cursor movement imagery EEG data. The data sets were acquired from three healthy human subjects in age group of between 24 and 29 years old in two sessions on different days. The proposed decision tree structure based method was successfully applied to the present data sets and achieved 55.92%, 57.90% and 82.24% classification accuracy rate on the test data of three subjects. The results indicated that the proposed method provided 12.25% improvement over the best results of the most closely related studies although the EEG signals were collected on two different sessions with about 1 week interval. The proposed method required only a training set of the subject and automatically generated specific DTS for each new subject by determining the most appropriate feature set and classifier for each node. Additionally, with further developments of feature extraction and/or classification algorithms, any existing node can be easily replaced with new one without breaking the whole DTS. This attribute makes the proposed method flexible.

Learning accurate very fast decision trees from uncertain data streams

Most existing works on data stream classification assume the streaming data is precise and definite. Such assumption, however, does not always hold in practice, since data uncertainty is ubiquitous in data stream applications due to imprecise measurement, missing values, privacy protection, etc. The goal of this paper is to learn accurate decision tree models from uncertain data streams for classification analysis. On the basis of very fast decision tree (VFDT) algorithms, we proposed an algorithm for constructing an uncertain VFDT tree with classifiers at tree leaves (uVFDTc). The uVFDTc algorithm can exploit uncertain information effectively and efficiently in both the learning and the classification phases. In the learning phase, it uses Hoeffding bound theory to learn from uncertain data streams and yield fast and reasonable decision trees. In the classification phase, at tree leaves it uses uncertain naive Bayes (UNB) classifiers to improve the classification performance. Experimental results on both synthetic and real-life datasets demonstrate the strong ability of uVFDTc to classify uncertain data streams. The use of UNB at tree leaves has improved the performance of uVFDTc, especially the any-time property, the benefit of exploiting uncertain information, and the robustness against uncertainty.

`Homo heuristicus` in the financial world: From risk management to managing uncertainty

What — if anything — can psychology and decision science contribute to risk management in financial institutions? The turmoils of recent economic crises undermine the assumptions of classical economic models and threaten to dethrone Homo oeconomicus, who aims to make decisions by weighing and integrating all available information. But rather than proposing to replace the rational actor model with some notion of biased, fundamentally flawed and irrational agents, we advocate the alternative notion of Homo heuristicus, who uses simple, but ecologically rational strategies to make sound and robust decisions. Based on the conceptual distinction between risky and uncertain environments this paper presents theoretical and empirical evidence that boundedly rational agents prefer simple heuristics over more flexible models. We provide examples of successful heuristics, explain when and why heuristics work well, and illustrate these insights with a fast and frugal decision tree that helps to identify fragile banks. We conclude that all members of the financial community will benefit from simpler and more transparent products and regulations.

Nonstationary signal pattern recognition using fast time-time filtering and decision tree

This paper proposes a fast Time-Time (TT) filtering transform for analysis and pattern recognition of nonstationary signals. A fast TTtransform algorithm is developed with different types of frequency scaling, band pass filtering and interpolation techniques to reduce the computational cost. The new time-time transform uses dyadic and selective scaling that facilitates the extraction of relevant features from time-varying signals for recognizing their patterns. The extracted features are then passed through a decision tree based classifier for the identification of the signal patterns. Various real world simultaneous power signal disturbances have been simulated to prove the efficiency of the technique. The simulation results show superior performance of the new TT-Transform while classifying overlapping disturbance patterns. Because of the new fast TT – transform algorithm and a relatively simpler classifier methodology, this technique can be used for real time localization, detection, and classification of various power quality events including other nonstationary signal time series belonging to speech, biomedical signals, etc.

A new fast discrete S-transform and decision tree for the classification and monitoring of power quality disturbance waveforms

SUMMARY In this paper, a new fast discrete S-transform (ST)-based time-frequency signal analyzer has been proposed for the detection, classification, and monitoring of power quality (PQ) disturbances varying in an electric power system. The proposed algorithm is based on the generalized Fourier algorithm that is used to obtain the time-localized spectral characteristics of the time-varying voltage and current signals belonging to PQ events. The fast ST algorithm is realized with different types of frequency scaling, band pass filtering, and interpolation techniques based on Heisenberg's uncertainty principle resulting in a reduced computation cost. In the conventional ST, the window width decreases at higher frequencies with a reduction in frequency resolution and conversely at low frequencies with wider windows. Therefore, the time-varying PQ disturbance signal is down sampled at low frequencies and cropped at high frequencies resulting in the evaluation of a fewer samples. From the time–frequency matrix output, important features are extracted and used with a binary decision tree for an accurate classification of single and simultaneous PQ events. Further, a unified approach is presented to track the time-varying PQ disturbance waveforms like voltage sag, swell, harmonics, and oscillatory transients and produce estimation of their amplitudes and phase angles. Copyright © 2013 John Wiley & Sons, Ltd.

Enhanced approach for ECG classification based on wavelet transform and decision tree models

The classification of the electrocardiogram (ECG) into different pathological disease categories is a complex pattern recognition task. In this paper, we propose a scheme to integrate Discrete Wavelet Transform (DWT), Principal Component Analysis (PCA), Fast Independent Component Analysis (FastICA) and Decision Tree (DT) for ECG beat classification. The PCA and FastICA are used to transform the extracted wavelet coefficients into uncorrelated and mutually independent new features. Several decision tree methods are utilized to discriminate between different ECG arrhythmia types. Our results suggest the high reliability and high classification accuracy of the C4.5 algorithm. The problem of imbalanced arrhythmia datasets is also investigated by comparing different C4.5 versions. Therefore, various testing configurations and performance measures such as precision, recall and AUC were considered. The discrimination ability of selected features and the extracted rules were demonstrated.

Proposal of Credit Card Fraudulent Use Detection by Online-type Decision Tree Construction and Verification of Generality

Global society has experienced a flood of various types of data, as well as a growing desire to discover and use this information effectively. Moreover, this data is changing in increasingly numerous and complex ways. In particular, for data that is generated intermittently, attention has been focused on data streams that use sensor network and stream mining technologies to discover useful information. In this paper, we focus on classification learning, which is an analytical method of stream mining. We are concerned with a type of decision tree learning called the Very Fast Decision Tree (VFDT) learner, which regards real data as a data stream. We analyze credit card transaction data as a data stream and detect fraudulent use. In recent years, credit card users have increased. However, this also consequently increases the damage caused by fraudulent use. Therefore, the detection of fraudulent use by data stream mining is required. However, some data, such as credit card transaction data, is extremely different from the rate of classes. Therefore, we propose and implement new statistical criteria to be used in a node construction algorithm that implements the VFDT. We also evaluate whether this method can be applied to imbalanced distribution data streams. Recent developments in information processing techniques have enabled us to accumulate largescale data. The need for discovering and utilizing useful information in this data is growing. Because of this, data mining, which is a technology used to collect data to discover useful information, has attracted considerable attention. However, with the spread of the Internet and the development of sensor techniques, the complexity of this data is constantly changing, and the increasing amounts of data must be handled on a real-time basis. New knowledge-streammining techniques are required to process such large-scale data that arrives intermittently and at different intervals as data stream flows. Stream mining uses various analytical methods; in particular, classification learning is gaining considerable attention. Many classification learning methods have been proposed among which the decision tree learning method is commonly used, because it is fast and the derived description of classifiers is easily interpreted. One of the data streams that supports the decision tree learning method is called the Very Fast Decision Tree (VFDT) [1]. As data arrives, this data stream grows gradually while the data is classified. Credit card transaction data is considered as the data stream. Therefore, it is possible to detect fraudulent use by classifying transaction data using the VFDT. However, among the various data types, there are some data, such as the credit card transaction data discussed in this study, whose characteristics are extremely different. When such data is used in a data stream, some problems can reduce the accuracy of the VFDT [2, 3]. In this study, we propose a node construction algorithm that is applicable to imbalanced distribution data streams. We also implement and evaluate criteria for constructing nodes. This paper is organized as follows. First, in Section 2, we explain the VFDT. In Section 3, we describe our proposed method, which consists of a VFDT construction from imbalanced distribution data streams. In Section 4, we verify the effectiveness of the proposed method by experiments. In Section 5, we describe and consider the experimental result. In the final section, we conclude and discuss our future works.

IMPROVING ADAPTABILITY OF DECISION TREE FOR MINING BIG DATA

Big data has become a popular research topic since the data explosion in the past decade. An efficient analytical methodology provides a way of discovering the potential value from big data. Sampling technique is unsuitable any more that the full data will tell the truths. To this end, the data mining algorithm shall be robust to imperfect data, which may lead to tree size explosion and detrimental accuracy problems. In this paper, we propose an incremental optimization mechanism to solve these problems. The mechanism is called Optimized Very Fast Decision Tree (OVFDT) that possesses an optimized node-splitting control mechanism. Accuracy, tree size and the learning time are the significant factors, which contribute to a much-improved algorithm in performance. Naturally a bigger tree size takes longer computation time. OVFDT is a pioneer model equipped with an incremental optimization mechanism that seeks for a balance between accuracy and tree size for data stream mining. OVFDT operates incrementally by a test-then-train approach. Two new methods of functional tree leaves are proposed to improve the accuracy that the tree model makes a prediction for a new data stream in the testing phase. The optimized node-splitting mechanism controls the tree model growth in the training phase. The experiment supports our claim that OVFDT is an optimal tree structure in both numeric and nominal datasets.

Detection and characterization of multiple power quality disturbances with a fast S-transform and decision tree based classifier

This paper proposes fast variants of the discrete S-transform (FDST) algorithm to accurately extract the time localized spectral characteristics of nonstationary signals. Novel frequency partitioning schemes along with band pass filtering are proposed to reduce the computational cost of S-transform significantly. A generalized window function is introduced to improve the energy concentration of the time-frequency (TF) distribution. An application of the proposed algorithms is extended for detection and classification of various nonstationary power quality (PQ) disturbances. The relevant features required for classification were extracted from the time-frequency distribution of the nonstationary power signal patterns. An automated decision tree (DT) construction algorithm was employed to select optimal set of features based on a specified optimality criterion for extraction of the decision rules. The set of decision rules thus obtained were used for identification of the PQ disturbance types. Various single as well as simultaneous power signal disturbances were considered in this paper to prove the efficiency of proposed classification scheme. A comparison of the classification accuracies with techniques proposed earlier, clearly demonstrates the improved performance. The major contributions of this manuscript are new FDST algorithms for fast and accurate time-frequency representation and an efficient classification algorithm for identifying PQ disturbances. The advantages of the classification algorithm are (i) accurate feature derivation from the TF distribution and optimum feature selection by the DT construction algorithm, (ii) robust performance at different signal-to-noise ratios, (iii) simple decision rules for classification, and (iv) recognition of simultaneous PQ events.

Stream Mining Dynamic Data by Using iOVFDT

Dynamic data is referring to data that are being produced continuously and their volume can potentially amount to infinity. They can be found in many daily applications such as e-commerce, security surveillance and activities monitoring. Such data call for a new generation of mining algorithms, called stream mining that is able to mine dynamic data without the need of archiving them first. This paper 1 studies the efficacy of a prominent stream mining method, called iOVFDT that stands for Incrementally Optimized Very Fast Decision Tree, under the environments of dynamic data. Six scenarios of dynamic data which have different characteristics are tested in the experiment. Each type of dynamic data represents a decision-making problem which demands an efficient classification mechanism such as decision tee to quickly and accurately classify a new case into a defined group. iOVFDT is compared with other popular stream mining algorithms, and it shows its superior performance.

Incremental Optimization Mechanism for Constructing a Decision Tree in Data Stream Mining

Imperfect data stream leads to tree size explosion and detrimental accuracy problems. Overfitting problem and the imbalanced class distribution reduce the performance of the original decision-tree algorithm for stream mining. In this paper, we propose an incremental optimization mechanism to solve these problems. The mechanism is called Optimized Very Fast Decision Tree (OVFDT) that possesses an optimized node-splitting control mechanism. Accuracy, tree size, and the learning time are the significant factors influencing the algorithm’s performance. Naturally a bigger tree size takes longer computation time. OVFDT is a pioneer model equipped with an incremental optimization mechanism that seeks for a balance between accuracy and tree size for data stream mining. It operates incrementally by a test-then-train approach. Three types of functional tree leaves improve the accuracy with which the tree model makes a prediction for a new data stream in the testing phase. The optimized node-splitting mechanism controls the tree model growth in the training phase. The experiment shows that OVFDT obtains an optimal tree structure in both numeric and nominal datasets.

Performance evaluation of incremental decision tree learning under noisy data streams

Big data has become a significant problem in software applications nowadays. Extracting classification model from such data requires an incremental learning process. The model should update when new data arrive, without re-scanning historical data. A single-pass algorithm suits continuously arrival data environment. However, one practical and important aspect that has gone relatively unstudied is noisy data streams. Such data are inevitable in real-world applications. This paper presents a new classification model with a single decision tree, so called incrementally Optimised Very Fast Decision Tree iOVFDT that embeds multi-objectives incremental optimisation and functional tree leaf. In the performance evaluation, noisy values were added into synthetic data. This evaluation investigated the performance under noisy data scenario. The result showed that iOVFDT outperforms the existing algorithms.

A Very Fast Decision Tree Algorithm for Real-Time Data Mining of Imperfect Data Streams in a Distributed Wireless Sensor Network

Wireless sensor networks (WSNs) are a rapidly emerging technology with a great potential in many ubiquitous applications. Although these sensors can be inexpensive, they are often relatively unreliable when deployed in harsh environments characterized by a vast amount of noisy and uncertain data, such as urban traffic control, earthquake zones, and battlefields. The data gathered by distributed sensors—which serve as the eyes and ears of the system—are delivered to a decision center or a gateway sensor node that interprets situational information from the data streams. Although many other machine learning techniques have been extensively studied, real-time data mining of high-speed and nonstationary data streams represents one of the most promising WSN solutions. This paper proposes a novel stream mining algorithm with a programmable mechanism for handling missing data. Experimental results from both synthetic and real-life data show that the new model is superior to standard algorithms.

Building fast decision trees from large training sets

Decision trees are commonly used in supervised classification. Currently, supervised classification problems with large training sets are very common, however many supervised classifiers cannot handle this amount of data. There are some decision tree induction algorithms that are capable to process large training sets, however almost all of them have memory restrictions because they need to keep in main memory the whole training set, or a big amount of it. Moreover, algorithms that do not have memory restrictions have to choose a subset of the training set, needing extra time for this selection; or they require to specify the values for some parameters that could be very difficult to determine by the user. In this paper, we present a new fast heuristic for building decision trees from large training sets, which overcomes some of the restrictions of the state of the art algorithms, using all the instances of the training set without storing all of them in main memory. Experimental results show that our algorithm is faster than the most recent algorithms for building decision trees from large training sets.

Classifying peer-to-peer applications using imbalanced concept-adapting very fast decision tree on IP data stream

Peer-to-Peer (P2P) applications generate streaming data in large volumes, where new communities of peers regularly attend and existing communities of peers regularly leave, requiring the classification techniques to consider concept drift, and update the model incrementally. Concept-adapting Very Fast Decision Tree (CVFDT) is one of the well-known streaming data mining techniques that can be applied to P2P traffic. However, we observe that P2P traffic data is class imbalanced, namely, only about 30 % of examples can be labeled as “P2P”, biasing the trained models (e.g. decision tree) towards the majority class (i.e. “NonP2P”). In this paper, we propose a new technique, the imbalanced CVFDT (iCVFDT), by integrating the CVFDT with an efficient resampling technique to address the issue of the class imbalanced data. The iCVFDT classification technique enjoys the advantages of CVFDT (such as stability), and at the same time, is not sensitive to imbalanced data. We captured the Internet traffic at a main gateway and prepared a real data stream with 3.5 million examples to which the iCVFDT classification technique was applied. The experimental results demonstrate a significant improvement in the performance of the iCVFDT compared to that of the CVFDT.

Learning very fast decision tree from uncertain data streams with positive and unlabeled samples

Most data stream classification algorithms need to supply input with a large amount of precisely labeled data. However, in many data stream applications, streaming data contains inherent uncertainty, and labeled samples are difficult to be collected, while abundant data are unlabeled. In this paper, we focus on classifying uncertain data streams with only positive and unlabeled samples available. Based on concept-adapting very fast decision tree (CVFDT) algorithm, we propose an algorithm namely puuCVFDT (CVFDT for positive and unlabeled uncertain data). Experimental results on both synthetic and real-life datasets demonstrate the strong ability and efficiency of puuCVFDT to handle concept drift with uncertainty under positive and unlabeled learning scenario. Even when 90% of the samples in the stream are unlabeled, the classification performance of the proposed algorithm is still compared to that of CVFDT, which is learned from fully labeled data without uncertainty.

Exploiting empirical variance for data stream classification

Classification, using the decision tree algorithm, is a widely studied problem in data streams. The challenge is when to split a decision node into multiple leaves. Concentration inequalities, that exploit variance information such as Bernstein’s and Bennett’s inequalities, are often substantially strict as compared with Hoeffding’s bound which disregards variance. Many machine learning algorithms for stream classification such as very fast decision tree (VFDT) learner, AdaBoost and support vector machines (SVMs), use the Hoeffding’s bound as a performance guarantee. In this paper, we propose a new algorithm based on the recently proposed empirical Bernstein’s bound to achieve a better probabilistic bound on the accuracy of the decision tree. Experimental results on four synthetic and two real world data sets demonstrate the performance gain of our proposed technique.

A data stream mining algorithm used in power grid disturbance identification

The disturbance identification is a classification of data stream problem, VFDT (Very Fast Decision Tree) which is the classical data stream classification algorithm can only deal with discrete attribute values. However, the power grid data gathered by WAMS platform are mostly continuously present. This algorithm is improved on the VFDT, by introducing the sampling theorem, to make the continuous attribute discretization, and classify the common power grid disturbances. At last, the experiments prove that the new algorithm is feasible.

Stream-based Biomedical Classification Algorithms for Analyzing Biosignals

Classification in biomedical applications is an important task that predicts or classifies an outcome based on a given set of input variables such as diagnostic tests or the symptoms of a patient. Traditionally the classification algorithms would have to digest a stationary set of historical data in order to train up a decision-tree model and the learned model could then be used for testing new samples. However, a new breed of classification called stream-based classification can handle continuous data streams, which are ever evolving, unbound, and unstructured, for instance--biosignal live feeds. These emerging algorithms can potentially be used for real-time classification over biosignal data streams like EEG and ECG, etc. This paper presents a pioneer effort that studies the feasibility of classification algorithms for analyzing biosignals in the forms of infinite data streams. First, a performance comparison is made between traditional and stream-based classification. The results show that accuracy declines intermittently for traditional classification due to the requirement of model re-learning as new data arrives. Second, we show by a simulation that biosignal data streams can be processed with a satisfactory level of performance in terms of accuracy, memory requirement, and speed, by using a collection of stream-mining algorithms called Optimized Very Fast Decision Trees. The algorithms can effectively serve as a corner-stone technology for real-time classification in future biomedical applications.