Multivariate Decision Tree Research Articles

The role of hand dominance in padel: performance profiles of professional players

Profiling methods allow coaches to determine players’ profiles for designing training programs as specific as possible to maximize performance. We aimed to define the distinctive performance profiles for professional padel players according to technical, spatial, effectiveness and hand dominance characteristics. A multivariate decision tree approach was used to identify and classify players’ profiles. Hand dominance comparison revealed a similar technical game pattern along the match (p = 0.330) but different effectiveness rates (p = 0.012). Left-handed scored more points using smashes (63.3 % vs. 40.7 %) but made more errors using the wall (37.7 % vs. 19.5 %); right-handed secured the ball with fewer errors (11.2 % vs. 8.2 %) and more continuity actions (84.9 % vs. 79.9 %). According to these findings, the hand dominance has an important role in players’ game style and effectiveness. These differences should be taken into account for designing optimal training programs in padel.

Predictive analytics of insurance claims using multivariate decision trees

Abstract Because of its many advantages, the use of decision trees has become an increasingly popular alternative predictive tool for building classification and regression models. Its origins date back for about five decades where the algorithm can be broadly described by repeatedly partitioning the regions of the explanatory variables and thereby creating a tree-based model for predicting the response. Innovations to the original methods, such as random forests and gradient boosting, have further improved the capabilities of using decision trees as a predictive model. In addition, the extension of using decision trees with multivariate response variables started to develop and it is the purpose of this paper to apply multivariate tree models to insurance claims data with correlated responses. This extension to multivariate response variables inherits several advantages of the univariate decision tree models such as distribution-free feature, ability to rank essential explanatory variables, and high predictive accuracy, to name a few. To illustrate the approach, we analyze a dataset drawn from the Wisconsin Local Government Property Insurance Fund (LGPIF)which offers multi-line insurance coverage of property, motor vehicle, and contractors’ equipments.With multivariate tree models, we are able to capture the inherent relationship among the response variables and we find that the marginal predictive model based on multivariate trees is an improvement in prediction accuracy from that based on simply the univariate trees.

A comparison of random forest based algorithms: random credal random forest versus oblique random forest

Random forest (RF) is an ensemble learning method, and it is considered a reference due to its excellent performance. Several improvements in RF have been published. A kind of improvement for the RF algorithm is based on the use of multivariate decision trees with local optimization process (oblique RF). Another type of improvement is to provide additional diversity for the univariate decision trees by means of the use of imprecise probabilities (random credal random forest, RCRF). The aim of this work is to compare experimentally these improvements of the RF algorithm. It is shown that the improvement in RF with the use of additional diversity and imprecise probabilities achieves better results than the use of RF with multivariate decision trees.

Predictive Analytics of Insurance Claims Using Multivariate Decision Trees

Because of its many advantages, the use of decision trees has become an increasingly popular alternative predictive tool for building classification and regression models. Its origins date back for about five decades where the algorithm can be broadly described by repeatedly partitioning the regions of the explanatory variables and thereby creating a tree-based model for predicting the response. Innovations to the original methods, such as random forests and gradient boosting, have further improved the capabilities of using decision trees as a predictive model. In addition, the extension of using decision trees with multivariate response variables started to develop and it is the purpose of this paper to apply multivariate tree models to insurance claims data with correlated responses. This extension to multivariate response variables inherits several advantages of the univariate decision tree models such as distribution-free feature, ability to rank essential explanatory variables, and high predictive accuracy, to name a few. To illustrate the approach, we analyze a dataset drawn from the Wisconsin Local Government Property Insurance Fund (LGPIF) which offers multi-line insurance coverage of property, motor vehicle, and contractors' equipments. With multivariate tree models, we are able to capture the inherent relationship among the response variables and we find that the marginal predictive model based on multivariate trees is an improvement in prediction accuracy from that based on simply the univariate trees.

Rapid screen of aflatoxin‐contaminated peanut oil using Fourier transform infrared spectroscopy combined with multivariate decision tree

SummaryContamination of peanut oil is a great concern in the industry. FTIR spectroscopy combined with chemometrics could develop a rapid and nondestructive method to screen aflatoxin‐contaminated peanut oil. Aflatoxin B1 (AFB1)‐ and aflatoxin (AFT)‐positive peanut oils were screened by mid‐IR (MIR) with classification models established by a novel multivariate decision tree (MDT) method. Two discriminant functions were developed in the fingerprint region based on absorbance ratio and moving window Fisher discrimination analysis methods for the two complex nodes in each MDT model. Window adjustment reduced the total different spectral data points in modelling to 23. The true‐positive and true‐negative rate of calibration and validation could both reach up to 100%. These results would render the possibility of efficient and economical design of a portable high‐speed multitasking MIR instrument for screening AFB1‐ and AFT‐contaminated peanut oil.

Efficient tree classifiers for large scale datasets

Abstract Classification plays a significant role in production activities and lives. In this era of big data, it is especially important to design efficient classifiers with high classification accuracy for large scale datasets. In this paper, we propose a randomly partitioned and a Principal Component Analysis (PCA)-partitioned multivariate decision tree classifiers, of which the training time is quite short and the classification accuracy is quite high. Approximately balanced trees are created in the form of a full binary tree based on two simple ways of generating multivariate combination weights and a median-based method to select the divide value having ensured the efficiency and effectiveness of the proposed algorithms. Extensive experiments conducted on a series of large datasets have demonstrated that the proposed methods are superior to other classifiers in most cases.

An ensemble of decision trees with random vector functional link networks for multi-class classification

Ensembles of decision trees and neural networks are popular choices for solving classification and regression problems. In this paper, a new ensemble of classifiers that consists of decision trees and random vector functional link network is proposed for multi-class classification. The random vector functional link network (RVFL) partitions the original training samples into K distinct subsets, where K is the number of classes in a data set, and a decision tree is induced for each subset. Both univariate and multivariate (oblique) decision trees are used with RVFL. The performance of the proposed method is evaluated on 65 multi-class UCI datasets. The results demonstrate that the classification accuracy of the proposed ensemble method is significantly better than other state-of-the-art classifiers for medium and large sized data sets.

Finding structure in data using multivariate tree boosting.

Technology and collaboration enable dramatic increases in the size of psychological and psychiatric data collections, but finding structure in these large data sets with many collected variables is challenging. Decision tree ensembles such as random forests (Strobl, Malley, & Tutz, 2009) are a useful tool for finding structure, but are difficult to interpret with multiple outcome variables which are often of interest in psychology. To find and interpret structure in data sets with multiple outcomes and many predictors (possibly exceeding the sample size), we introduce a multivariate extension to a decision tree ensemble method called gradient boosted regression trees (Friedman, 2001). Our extension, multivariate tree boosting, is a method for nonparametric regression that is useful for identifying important predictors, detecting predictors with nonlinear effects and interactions without specification of such effects, and for identifying predictors that cause 2 or more outcome variables to covary. We provide the R package "mvtboost" to estimate, tune, and interpret the resulting model, which extends the implementation of univariate boosting in the R package "gbm" (Ridgeway, 2015) to continuous, multivariate outcomes. To illustrate the approach, we analyze predictors of psychological well-being (Ryff & Keyes, 1995). Simulations verify that our approach identifies predictors with nonlinear effects and achieves high prediction accuracy, exceeding or matching the performance of (penalized) multivariate multiple regression and multivariate decision trees over a wide range of conditions. (PsycINFO Database Record

Predictive factors for malignancy in incidental pulmonary nodules detected in breast cancer patients at baseline CT.

Pulmonary nodules are commonly encountered at staging CTs in patients with extrathoracic malignancies, but their significance on a per-patient basis remains uncertain. We undertook a retrospective analysis of pulmonary nodules identified in patients with a diagnosis of breast cancer from 2010 - 2015, evaluating nodules present at a baseline CT (i.e. prevalent nodules). We reviewed 211 patients with 248 individual nodules. The rate of malignancy in prevalent nodules is low, approximately 13%. Variables associated with metastasis include pleural studding, hilar lymphadenopathy and the presence of extrapulmonary metastasis, as well as number of nodules, nodule size and nodule shape. Using a combination of these factors, we have developed an evidence-based multivariate decision tree to predict which nodules are malignant in these patients, which is 91% accurate and 100% sensitive for metastasis. We propose a simplified clinical prediction algorithm to guide radiologists and oncologists in managing patients with breast cancer and incidental pulmonary nodules. • Incidental pulmonary nodules are common on computed tomography in breast cancer patients. • Nodules present at baseline have a lower malignancy risk than incident nodules. • We present an evidence-based decision algorithm predicting which nodules are likely malignant. • This algorithm can help direct patient management.

Multivariate decision trees with monotonicity constraints

Classification with monotonicity constraint is a fundamental task in social analysis, management and decision making, where a monotonic function guarantees that objects with better feature values are not assigned with worse decisions. In practice, an object may be better than another on some attributes, while worse on others. These objects are considered to be incomparable. Incomparable object pairs limit the performance of a monotone classifier. In this work, we design an algorithm to combat this issue by constructing multivariate decision trees with monotonicity constraints (MMT). The classification model is naturally deemed as an oblique tree as it discovers partitions via oblique hyperplane in the input space. Our algorithm generates the projections of the objects which are used to split the data by improved splitting criteria with rank mutual information (RMI) or rank Gini impurity (RGI). Moreover, an improved algorithm with L1-regularization is also proposed to compute the optimal subsets of features in the process of constructing the trees, which leads to a more compact tree. Experimental results show that the proposed algorithm improves the classification performance in monotone classification tasks. The proposed algorithm is also effective even if data is contaminated by non-monotonic noisy samples.

Comparison of a massive and diverse collection of ensembles and other classifiers for oil spill detection in SAR satellite images

We present a comparison of the largest collection of classifiers considered until now in the literature, composed by 428 methods belonging to 41 very different families. This collection, much larger than the one in our previous work (Fernandez-Delgado et al. in J Mach Learn Res 15:3133–3181, 2014), includes 320 ensembles (varying the base and meta-classifiers), alongside with Support Vector Machines, Bayesian, Neural Networks, Discriminant Analysis, Multivariate Adaptive Regression Splines, Random Forests, Decision Trees and many others. The classifier comparison is developed on the detection of oil spills on Synthetic Aperture Radar (SAR) images taken from satellites. The SAR images have revealed very useful to surveillance maritime agencies for the detection of regular offshore operational discharges, which, despite is commonly accepted, is one of the biggest causes of hydrocarbon marine pollution, instead of tanker and oil platform catastrophes. After a segmentation of the SAR images to select oil spill candidates, classifiers use the features extracted from these candidates to discard frequent and expensives look-alikes (false positives), caused by natural phenomena. Testing experiments revealed that the RotationForest ensemble of MultilayerPerceptron base classifiers, applying Kernel PCA on the original data, achieves the best accuracy and Cohen \(\kappa\) (87.1 % and 71.0 %, respectively) with a low frequency of false positives (5.13 %).

An exploratory decision tree analysis to predict cardiovascular disease risk in African American women.

African American (AA) women are at greater risk for cardiovascular disease (CVD) compared to White women, which can be attributed to disparities in risk factors. The built environment may contribute to improving CVD risk factors by increasing physical activity (PA). This study used recursive partitioning, a multivariate decision tree risk classification approach, to determine which built environment characteristics contributed to the classification of AA women as having 4 or more CVD risk factors at optimal levels. Recursive partitioning has the ability to detect interactions and does not have sample size limitations to detect effects. The Classification and Regression Trees (CR&T) growing method was used to group participants as having 4 or more versus 3 or fewer risk factors at optimal levels. Risk factors were smoking, body mass index (BMI), PA, healthy diet, cholesterol, glucose, and blood pressure. Built environment predictors were presence and quality of neighborhood PA resources (PARs), walkability, traffic safety, and crime. Participants (N = 30, mean age of 54.1 ± 7.5) all had at least 1 risk factor at the optimal level, none had all 7, and 66.7% had 4 or more risk factors at optimal levels. The CR&T identified participants with few, low-quality neighborhood PARs and who were older than 55 as least likely to have 4 or more CVD risk factors at optimal levels. Being younger than 55 years old and having many, high-quality neighborhood PARs may predict lower risk for CVD in AA women. Results should be used in future studies with larger sample sizes to inform logistic regression models. (PsycINFO Database Record

Classifying component failures of a hybrid electric vehicle fleet based on load spectrum data

Component failures in hybrid electric vehicles (HEV) can cause high warranty costs for car manufacturers. Hence, in order to (1) predict whether a component of the hybrid power-train of a HEV is faulty, and (2) to identify loads related to component failures, we train several random forest variants on so-called load spectrum data, i.e., the state-of-the-art data employed for calculating the fatigue life of components in fatigue analysis. We propose a parameter tuning framework that enables the studied random forest models, formed by univariate and multivariate decision trees, respectively, to handle the class imbalance problem of our dataset and to select only a small number of relevant variables in order to improve classification performance and to identify failure-related variables. By achieving an average balanced accuracy value of 85.2 %, while reducing the number of variables used from 590 to 22 variables, our results for failures of the hybrid car battery (approx. 200 faulty, 7000 non-faulty vehicles) demonstrate that especially balanced random forests using univariate decision trees achieve promising classification results on load spectrum data. Moreover, the selected variables can be related to component failures of the hybrid power-train.

MP72-09 ADJUVANT CISPLATIN LEADS TO A LARGER DECLINE IN GFR THAN NEOADJUVANT CISPLATIN IN RADICAL CYSTECTOMY PATIENTS

RESULTS: No significant difference was noted in 5-year CSS between GC NAC (58%) and non-NAC cohorts (61%). The median followup was 19.6 months for GC NAC and 106.5 months for non-NAC. Patients with residual non-muscle-invasive disease (pTis, pTa, pT1) after GC NAC exhibit similar 5-year CSS relative to patients with no residual carcinoma (pT0), p1⁄40.99. NAC pathologic responders ( T1, n1⁄480) demonstrated improved 5-year CSS rates of 90.6% vs. 27.1% (p<0.01) and decreased nodal positivity rates of 0% vs. 41.3% (p<0.01) compared to NAC pathologic non-responders ( pT2). Clinical and pathologic outcomes were inferior in NAC pathologic non-responders compared to the entire RC only treated cohort (pT2-T4). A significantly lower pathologic non-response rate was seen in those able to tolerate sufficient dosing of NAC when patients were stratified by the JHH-DI (p1⁄40.049) which is congruent with NCCN guidelines. A multivariate decision tree model demonstrated age 60 years and clinical stage cT2 as significant of response to NAC (p<0.05). CONCLUSIONS: Pathologic non-responders fare worse than patients proceeding directly to RC alone, suggesting the need to identify clinical features predictive of poor response prior to the initiation of chemotherapy. To this effect, multiple predictive models incorporating clinical, histopathologic, and molecular features are currently being developed to identify patients, prior to the initiation of therapy, that are most likely to benefit or not benefit from GC NAC.

Sparse alternating decision tree

Alternating decision tree (ADTree) is a special decision tree representation that brings interpretability to boosting, a well-established ensemble algorithm. This has found success in wide applications. However, existing variants of ADTree are implementing univariate decision nodes where potential interactions between features are ignored. To date, there has been no multivariate ADTree. We propose a sparse version of multivariate ADTree such that it remains comprehensible. The proposed sparse ADTree is empirically tested on UCI datasets as well as spectral datasets from the University of Eastern Finland (UEF). We show that sparse ADTree is competitive against both univariate decision trees (original ADTree, C4.5, and CART) and multivariate decision trees (Fisher's decision tree and a single multivariate decision tree from oblique Random Forest). It achieves the best average rank in terms of prediction accuracy, second in terms of decision tree size and faster induction time than existing ADTree. In addition, it performs especially well on datasets with correlated features such as UEF spectral datasets. Thus, the proposed sparse ADTree extends the applicability of ADTree to a wider variety of applications.

Gemcitabine and cisplatin neoadjuvant chemotherapy for muscle-invasive urothelial carcinoma: Predicting response and assessing outcomes.

336 Background: To evaluate gemcitabine-cisplatin (GC) neoadjuvant chemotherapy (NAC) for pathologic response (pR) and cancer-specific outcomes following radical cystectomy (RC) for muscle-invasive bladder cancer (MIBC) and identify clinical parameters associated with pR. Methods: We studied 150 consecutive cases of MIBC that received GC NAC followed by open RC (2000-2013). A cohort of 121 patients treated by RC alone was used for comparison. Pathologic response and cancer-specific survival (CSS) were compared. We created the Johns Hopkins Hospital Dose Index (JHH-DI) to characterize chemotherapeutic dosing regimens and accurately assess sufficient neoadjuvant dosing regarding patient tolerance. Results: No significant difference was noted in 5-year CSS between GC NAC (58%) and non-NAC cohorts (61%). The median follow-up was 19.6 months (GC NAC) and 106.5 months (non-NAC). Patients with residual non-muscle-invasive disease after GC NAC exhibit similar 5-year CSS relative to patients with no residual carcinoma (p=0.99). NAC pR (≤pT1) demonstrated improved 5-year CSS rates (90.6% vs. 27.1%, p&lt;0.01) and decreased nodal positivity rates (0% vs. 41.3%, p&lt;0.01) compared to non-responders (≥pT2). Clinicopathologic outcomes were inferior in NAC pathologic non-responders (pNR) compared to the entire RC-only treated cohort. A lower pNR rate was seen in patients tolerating sufficient dosing of NAC as stratified by the JHH-DI (p=0.049), congruent with NCCN guidelines. A multivariate decision tree model demonstrated age ≤60 years and clinical stage cT2 as significant of NAC response (p&lt;0.05). Conclusions: Pathologic non-responders fare worse than patients proceeding directly to RC alone. Multiple predictive models incorporating clinical, histopathologic, and molecular features are currently being developed to identify patients most likely to benefit from GC NAC.

Risk factors and safe dosage of levodopa for wearing-off phenomenon in Chinese patients with Parkinson's disease.

The objective of this study was to investigate the risk factors of wearing-off phenomenon in Parkinson's disease (PD) and propose safe dosage of levodopa to reduce wearing-off development based on Chinese cohort. Patients with PD who had taken levodopa (L-dopa) for at least 1 month were recruited. Wearing-off was diagnosed based on validated Chinese version of a patient self-rated 9-question Wearing-Off Questionnaire (WOQ-9) and clinical definition. Eleven variables (gender, disease duration at L-dopa initiation, disease duration at assessment, age at onset, age at assessment, H-Y stage, UPDRS III, L-dopa daily total dosage and dosage adjusted to weight, duration of L-dopa treatment, initial drug recipe) were included in our analysis. Univariate analysis, multivariate logistic regression analysis and decision tree classification model(DTC) were used to detect risk factors of wearing-off. Receiver operating characteristic (ROC) curve and DTC were used to investigate cut-off value of L-dopa to best predict wearing-off. Two hundred and thirty-four patients were investigated in our study, among whom 111 developed wearing-off. Patients with wearing-off tended to receive higher L-dopa dosage and endure longer duration of L-dopa treatment. L-Dopa dosage as 281 mg/day and 4.2 mg/kg/day by ROC, as well as 269 mg/day and 3.2 mg/kg/day by DTC were cut-off values for wearing-off. L-Dopa dosage and duration of L-dopa treatment were related to increased wearing-off development. Cumulative L-dopa dosage and L-dopa daily dosage were better predictive of wearing-off. Inadequate evidence was present for delayed L-dopa initiation. L-Dopa daily dosage no more than 275 mg or 4.2 mg/kg was regarded as safe.

Interpreting and Estimating the Risk of Iron Failures

Abstract Metals and particulates accumulate in the distribution system and are mobilised by hydraulic events which can result in discolouration and exceedance of regulatory standards. Traditional decision tools for targeting preventive work are single parameter, based for example on proportion of unlined iron pipe or the number of customer contacts per district metering area (DMA). We show that this approach is too simplistic and propose a multivariate Decision Tree process, using the Random Under-Sampling ensemble method. The outputs gave a classification of High or Low risk per DMA. Initial findings demonstrate an 80% success rate in identifying high risk DMAs across the supply area for a UK water company.

Random Projection Random Discretization Ensembles—Ensembles of Linear Multivariate Decision Trees

In this paper, we present a novel ensemble method random projection random discretization ensembles (RPRDE) to create ensembles of linear multivariate decision trees by using a univariate decision tree algorithm. The present method combines the better computational complexity of a univariate decision tree algorithm with the better representational power of linear multivariate decision trees. We develop random discretization (RD) method that creates random discretized features from continuous features. Random projection (RP) is used to create new features that are linear combinations of original features. A new dataset is created by augmenting discretized features (created by using RD) with features created by using RP. Each decision tree of a RPRD ensemble is trained on one dataset from the pool of these datasets by using a univariate decision tree algorithm. As these multivariate decision trees (because of features created by RP) have more representational power than univariate decision trees, we expect accurate decision trees in the ensemble. Diverse training datasets ensure diverse decision trees in the ensemble. We study the performance of RPRDE against other popular ensemble techniques using C4.5 tree as the base classifier. RPRDE matches or outperforms other popular ensemble methods. Experiments results also suggest that the proposed method is quite robust to the class noise.

SMT: Sparse multivariate tree

AbstractA multivariate decision tree attempts to improve upon the single variable split in a traditional tree. With the increase in datasets with many features and a small number of labeled instances in a variety of domains (bioinformatics, text mining, etc.), a traditional tree‐based approach with a greedy variable selection at a node may omit important information. Therefore, the recursive partitioning idea of a simple decision tree combined with the intrinsic feature selection of L1 regularized logistic regression (LR) at each node is a natural choice for a multivariate tree model that is simple, but broadly applicable. This natural solution leads to the sparse multivariate tree (SMT) considered here. SMT can naturally handle non‐time‐series data and is extended to handle time‐series classification problems with the power of extracting interpretable temporal patterns (e.g., means, slopes, and deviations). Binary L1 regularized LR models are used here for binary classification problems. However, SMT may be extended to solve multiclass problems with multinomial LR models. The accuracy and computational efficiency of SMT is compared to a large number of competitors on time series and non‐time‐series data. © 2013 Wiley Periodicals, Inc. Statistical Analysis and Data Mining, 2013