Decision Tree Forest Research Articles

An artificial intelligence approach to early predict symptom-based exacerbations of COPD

ABSTRACTAcute exacerbations are one of the main causes that reduce health-related quality of life and lead to hospitalisations of patients of chronic obstructive pulmonary disease (COPD). Prediction of exacerbations could diminish those negative effects and reduce the high costs associated with COPD patients. In this study, 16 patients were telemonitored at home during six months. Respiratory sounds were recorded daily with an electronic sensor ad-hoc designed. In order to enable an automatic prediction of symptom-based exacerbations, recorded data were used to train and validate a decision tree forest classifier. The developed model was capable of predicting early acute exacerbations of COPD, as average, with a 4.4 days margin prior to onset. Thirty-two out of 41 exacerbations were detected early. A percentage of 75.8% (25 out of 33) of detected episodes were reported exacerbation and 87.5% (7 out of 8) were unreported events. The achieved results demonstrated that machine-learning techniques have significant potential to support the early detection of COPD exacerbations.

Anomaly detection for elderly home care

In this paper, we propose a model for detecting anomalies in elderly home care. Two scenarios are investigated in detecting anomalies: 1) the elderly person's vital signs and their surrounding environment; 2) the mobility patterns of the elderly. We evaluated our proposed model by employing the isolation forest which detects anomalies using an isolation approach on a random forest of decision trees. We compare isolation forest on unlabeled data with statistical methods on labelled data. Subsequently, to show the reliability of the isolation concept, we compare it with a distance measure concept. The experiment shows that isolation forest has higher detection accuracy and lower error prediction for two attributes in the first scenario: skin temperature and heart rate, whereas, in the second scenario, multi-covariance determinant has a slightly better accuracy compared to isolation forest (3.9% difference in accuracy) and has a small number of prediction errors compared to isolation forest.

Lost in the woods: The value of tree ensemble modelling for adult age-at-death estimation from skeletal degeneration

Accurate and precise age estimation is of paramount importance in forensic analysis of human remains. Despite several research efforts most commonly applied techniques provide poor age estimates. The low predictive power of macroscopic skeletal age estimation methods can be attributed to several factors such as the inappropriate statistical modelling, poorly sampled reference material, and an over reliance on an indicator-specific approach. The objective of this pilot study is to illustrate a general approach to age-at-death estimation from skeletal degeneration of major articulations of the appendicular skeleton, cervical and lumbar spine. A sample of 256 adult male individuals with known age-at-death was constructed from the identified skeletal collections held at the University of Coimbra. Articular degeneration of the appendicular skeleton was mapped by analysing marginal lipping, bone formation, porosity and eburnation of bone surfaces. Vertebral degeneration was analysed by scoring marginal lipping and porosity of body surfaces. To tackle the complexity of age-related skeletal degeneration, a machine learning approach is proposed around decision trees forests. Decision trees offer a flexible tool for age-at-death estimation as they can map regression functions without assuming a structural form for the predictive surface. Individual tree ensembles were created for each articulation and spinal segment, forming what is called here skeletal aging woods. To solve the issue of combining age prediction obtained from different aging woods, a final and larger forest is trained on the cross-validation predictions of the woods. From individual aging woods, older individuals tend to be accurately identified but age predictions for young individuals tend to be very broad and imprecise. However, prediction error from the final forest can be as low as 7.59 years, with the major errors more broadly concentrated on middle-aged individuals. Tools for visualizing age-related subspaces spanned by the proximity matrix of tree ensembles are also presented.

Building forests of local trees

Ensemble methods have shown to be more effective than monolithic classifiers, in particular when diversity holds among their components. How to enforce diversity in classifier ensembles has received much attention from machine learning researchers, yielding a variety of different techniques and algorithms. In this paper, a novel algorithm for ensemble classifiers is proposed, in which ensemble components are trained with focus on different regions of the sample space. In so doing, diversity is mainly a consequence of the intention to limit the scope of base classifiers. The algorithm proposed in this paper shares roots with several ensemble paradigms, in particular with random forests, as it generates forests of decision trees as well. As decision trees are trained with focus on specific subsets of the sample space, the resulting ensemble is in fact a forest of “local” trees. Comparative experimental results highlight that, on average, these ensembles perform better than other relevant kinds of ensemble classifiers, including random forests.

Implementation and Comparison of Machine Learning Algorithms for Recognition of Fingerspelling in Indian Sign Language

Communication is the biggest hurdle faced by the hearing and speech impaired in leading a normal life. In this context, Sign Language is the most prominent means of communication. Machine learning and Computer Vision is an integral part of today’s computing world. This research paper proposes a Machine Learning based system to recognize fingerspelling gestures present in Indian Sign Language. Edge Frequency technique is chosen for Feature Extraction. The system was implemented using Aforge.NET framework. A comparative analysis of the Machine Learning Algorithms consisting of Support Vector Machine (SVM), K- Nearest Neighbor (KNN), Adaptive Naive Bayes Classifier (ANBC), Decision Tree (DT) and Random Forests (RF) is performed to find out which algorithm is the most suitable to recognize ISL. Comparison is done based on validation accuracies and confusion matrices obtained. The accuracy for KNN was found to be 97.44% while SVM and ANBC have an accuracy of 96.15% and 82.05% respectively.

IoT/M2M wearable-based activity-calorie monitoring and analysis for elders.

With the growth of aging population, elder care service has become an important part of the service industry of Internet of Things. Activity monitoring is one of the most important services in the field of the elderly care service. In this paper, we proposed a wearable solution to provide an activity monitoring service on elders for caregivers. The system uses wireless signals to estimate calorie burned by the walking and localization. In addition, it also uses wireless motion sensors to recognize physical activity, such as drinking and restroom activity. Overall, the system can be divided into four parts: wearable device, gateway, cloud server, and caregiver's android application. The algorithms we proposed for drinking activity are Decision Tree (J48) and Random Forest (RF). While for restroom activity, we proposed supervised Reduced Error Pruning (REP) Tree and Variable Order Hidden Markov Model (VOHMM). We developed a prototype service Android app to provide a life log for the recording of the activity sequence which would be useful for the caregiver to monitor elder activity and its calorie consumption.

The application of discriminant analysis for mapping cereals and pasture using object-based features

ABSTRACTHigh mapping accuracies occur where crops differ spectrally (e.g.>90.0%; canola, corn, soybeans) and vice versa (e.g. <75.0%; cereals and pasture). Developing improved mapping methods has been an ongoing priority of Agriculture and Agri-Food Canada (AAFC) remote-sensing science. To this end, this study tests a data-driven object-based classification method using Discriminant Analysis (DA) method for mapping cereals and pasture from satellite data. In this approach, variables (number >400) derived from the image segmentation and object-based feature extraction of multi-date and multi-band optical (RapidEye) and microwave (RADARSAT-2) imagery were applied in a data-driven approach. We use in situ and satellite information collected over two study sites with different levels of heterogeneity (Winnipeg, Brandon) situated in the Canadian Prairies during the 2013 growing season to assess: (a) the type of DA model that most accurately classifies the cereals and pasture cover classes; and (b) how the classification accuracies obtained by the application of this DA model compare to those obtained from more traditional Maximum Likelihood (ML), Decision Tree (DT), and Random Forest (RF) classifications. We found that our DA-based approach was able to map cereals and pastures at our two study sites with the highest accuracies, but these accuracies did not improve significantly with the use of more complex DA model (including priori classification probabilities, more input principle components (PCs), the use of weights proportional to field area). Our results are encouraging for the wider application of the data-driven pre-processing of the inputs to the image classification by DA.

Detecting and tracking mesoscale precipitating objects using machine learning algorithms

ABSTRACTAccurate identification of precipitating clouds is a challenging task. In the present work, Support Vector Machines (SVMs), Decision Trees (DT), and Random Forests (RD) algorithms were applied to extract and track mesoscale convective precipitating clouds from a series of 22 Geostationary Operational Environmental Satellite-13 meteorological image sub-scenes over the continental territory of Colombia. This study’s aims are twofold: (i) to establish whether the use of five meteorological spectral channels, rather than a single infrared (IR) channel, improves rainfall objects detection and (ii) to evaluate the potential of machine learning algorithms to locate precipitation clouds. Results show that while the SVM algorithm provides more accurate classification of rainfall cloud objects than the traditional IR brightness temperature threshold method, such improvement is not statistically significant. Accuracy assessment was performed using STEP (shape (S), theme (T), edge (E), and position (P)) object-based similarity matrix method, taking as reference precipitation satellite images from the Tropical Rainfall Measuring Mission. Best thematic and geometric accuracies were obtained applying the SVM algorithm.

Performance Analysis of Classification Algorithms in Predicting Diabetes

In Data mining classification is one the most important technique. Today, we have data in abundant from numerous sources but in order to get meaningful information from it is very tedious task. Machine learning algorithms to train classifiers to decode the meaningful information from the data, this analysis approach has gained much popularity in recent years. This paper explores evaluation performance of Naive Bayes, Logistic Regression and Decision tree, Random forest using datasets (Pima Indian Diabetes data from UCI Repository). Naive Bayes algorithm is depending upon likelihood and probability; it is fast and stable to data changes. Logistic Regression, calculate the relationship of each feature and weights them based on their impact on result. Random forest algorithm is an ensemble algorithm, fits multiple trees with subset of data and averages tree result to improve performance and control over-fitting. Decision tree can be nicely visualized uses binary tree structure with each node making a decision depending upon the value of the feature. This paper concludes with a comparative evaluation of Naive Bayes, Logistic Regression, Decision tree and Random Forest in the context of Pima Indian Diabetes Dataset( take from UCI repository) in order to predict the diabetic patients. Keywords: Naive Bayes, Logistic Regression, Random Forest, Classification, Decision tree

Mining Data of Noisy Signal Patterns in Recognition of Gasoline Bio-Based Additives using Electronic Nose

Abstract The paper analyses the distorted data of an electronic nose in recognizing the gasoline bio-based additives. Different tools of data mining, such as the methods of data clustering, principal component analysis, wavelet transformation, support vector machine and random forest of decision trees are applied. A special stress is put on the robustness of signal processing systems to the noise distorting the registered sensor signals. A special denoising procedure based on application of discrete wavelet transformation has been proposed. This procedure enables to reduce the error rate of recognition in a significant way. The numerical results of experiments devoted to the recognition of different blends of gasoline have shown the superiority of support vector machine in a noisy environment of measurement.

Fault Diagnostics of a Gearbox with Acoustic Signals Using Wavelets and Decision Tree

Objectives: This study aims at devising a methodology for accurately predicting the different fault conditions of gears in a gearbox using acoustic signals. Statistical Analysis: The acoustic signals are captured for several artificially created fault conditions of different magnitude and the wavelet features are extricated from captured acoustic signals. Subsequently,prominent features are selected by utilizing J48 Decision tree which discerns the most dominant traits among the allocated data obtained from wavelet transform of the acoustic signals followed by Random Forest for the classification of features. Findings: Out of a total of eleven features extracted, six were selected through Decision Tree and Random forest was used for feature classification of acoustic signals using wavelet features. Several iterations were conducted on the wavelet features by varying different parameters and the maximum percentage accuracy was found to be 99.76%. The instances of misclassification of features were minimal in Random Forest and it proved to be an efficient and precise classifier. Hence, Random Forest proved to be an easy to use, fast and accurate classifier that could classify various kinds of wavelet features efficiently. Applications: The methodology can be used to provide accurate real time results about the condition of gear teeth.

Determination of Voting Tendencies in Turkey through Data Mining Algorithms

Political elections can be defined as systems that contain political tendencies and voters' perceptions and preferences. The outputs of those systems are formed by specific attributes of individuals such as age, gender, occupancy, educational status, socio-economic status, religious belief, etc. Those attributes can create a data set, which contains hidden information and undiscovered patterns that can be revealed by using data mining methods and techniques. The main purpose of this study is to define voting tendencies in politics by using some of data mining methods. According to that purpose, the survey results, which were prepared and applied before 2011 elections of Turkey by KONDA Research and Consultancy Company, were used as raw data set. After Preprocessing of data, models were generated via data mining algorithms, such as Gini, C4.5 Decision Tree, Naive Bayes and Random Forest. Because of increasing popularity and flexibility in analyzing process, R language and Rstudio environment were used.

Tree-based techniques to predict soil units

Quantitative soil–landscape models offer a method for conducting soil surveys that use statistical tools to predict natural patterns in the occurrence of particular map units across a landscape. The aim of the present study was to predict soil units in a watershed with wide variation in landscape conditions. The approach relied on a modelling of soil-forming factors in order to understand the variability of the landscape components in the region. Models were generated for landscape attributes related to pedogenesis, specifically elevation, slope, curvature, compound topographic index, Euclidean distance from stream networks, landforms map, clay minerals index, iron oxide index and normalised difference vegetation index, along with an existing geology map. The soil classification was adapted from the World Reference Base System for Soil Resources, and the predominant soil taxonomic orders observed were Ferrasols, Acrisols, Gleysols, Cambisols, Fluvisols and Regosols. The algorithms used to predict the soil units were based on decision tree (DT) and random forest (RF) methods. The criteria used to evaluate the models’ performance were statistical indices, coherence between predicted units and the legacy map, as well as accuracy checks based on control samples. The best performing model was found to be the RF algorithm, with resulting statistical indices considered excellent (overall=0.966, kappa=0.962). The accuracy of the map as determined by control points was 67.89%, with a kappa value of 61.39%.

Hybrid framework using data mining techniques for early detection and prevention of oral cancer

This paper presents the usage of classification and association data mining techniques for early detection and prevention of oral cancer. The indigenous dataset of 1,025 patients who visited a tertiary care centre during 2004 to 2009 was used for the research. Ten classification data mining models are designed using varied types of data mining techniques like regression analysis, classification trees and neural networks. Regression analysis models are linear regression model and logistic regression model; classification tree models are decision tree model, decision tree forest model and TreeBoost model and artificial neural networks are multilayer perceptron model, radial basis function model, group method of data handling model, cascade correlation model and probabilistic - general regression neural network model. Association rules are generated using apriori algorithm. The classification models and association rules are evaluated using various estimation parameters. Finally, a hybrid oral cancer management system is designed using the classification model and the association rules.

Prediction of water saturation in a tight gas sandstone reservoir by using four intelligent methods: a comparative study

Reservoir water saturation is an important property of tight gas reservoirs. Improper calculation of water saturation leads to remarkable errors in following studies for development and production from reservoir. There are conventional methods to determine water saturation, but these methods suffer from poor generalization and cannot be applicable for various conditions of reservoirs. These methods also depend on core measurements. On the other hand, well log data are usually accessible for all the wells and provide continuous information across the well. Customary techniques are not fully capable to prepare meaningful results for predicting petrophysical properties, especially in presence of small data sets. In this regard, soft computing approaches have been used here. In this research, Support Vector Machine, Multilayer Perceptron Neural Network, Decision Tree Forest and Tree Boost methods have been employed to predict water saturation of Mesaverde tight gas sandstones located in Uinta Basin. Tree Boost and Decision Tree Forest are powerful predictors which have been applied in many research fields. Multilayer Perceptron is the most common neural network, and Support Vector Machine has been used in many petrophysical and reservoir studies. In this research, by using a small data set, the ability of these methods in predicting water saturation has been studied. Based on the data from four wells, two data set patterns were designed to evaluate training and generalization capabilities of methods. In each pattern, different combinations of well data were used. Three error indexes including correlation coefficient, average absolute error and root-mean-square error were used to compare the methods results. Results show that Support Vector Machine models perform better than other models across data sets, but there are some exceptions exhibiting better performance of Multilayer Perceptron Neural Network and Decision Tree Forest models. Correlation coefficient values vary from 0.6 to 0.8 for support vector machine, which exhibits better performance in comparison with other methods.

Fault Diagnosis of Helical Gearbox through Vibration Signals using Wavelet Features, J48 Decision Tree and Random Forest Classifiers

Objective: Gearbox being the backbone of transmission system is designed and manufactured very carefully so that there is minimum compliance in the system. However, there are still faults and failures which usually occur in the system. The failure in helical gearbox is more prominent in bearings rather than gears which are the main components of the system. Gearbox is susceptible to failures because of reasons like misalignment, vibration and shocks. In this paper wavelet feature extraction is used along with random forest algorithm to diagnose faults in gearbox. The vibration signals were used for extracting wavelet features. Features were selected using J48 Decision Tree and were classified using random forest algorithm. A detailed study has been done to ensure that the optimum number of features was used and the factor was iterated so that maximum classification accuracy is obtained. The results are presented along with the conclusion. Method Analysis: The classification accuracy is obtained by 3 steps namely, feature extraction, feature selection and feature classification. By obtaining the Decision Tree the most important factors are selected to obtain maximum classification accuracy at minimum number of features to reduce calculations in real time application. The number of features and depth of data is iterated to obtain the maximum classification accuracy. Findings: Through this research random forest algorithm was tested for fault diagnosis of gearbox and a better classification accuracy was obtained. These results can be further used for fault diagnosis in industries for any gearbox related problems. Application/Improvements: An extensive investigation is done by a random forest algorithm which produced better forecasting than the other algorithms. Based on the overall study, random forest was found as the most preferred classification algorithm that achieved the best classification accuracy of 93.08% which is better than the other algorithms.

Exploration of noisy data in differential electronic nose

Purpose – The paper is concerned with exploration of sensor signals in differential electronic nose. It is a special type of nose, which applies double sensor matrices and exploits only their differential signals, which are used in recognition of patterns associated with them. The purpose of this paper is to study the application of differential nose in dynamic measurement of aroma of 11 brands of cigarettes. Design/methodology/approach – The most important task in pattern recognition using electronic nose is its resistance to the noise corrupting the measurement. The authors will analyze and compare the performance of the nose in the noisy environment by applying two classifier systems: the support vector machine (SVM) and random forest (RF) of decision trees. Findings – On the basis of numerical experiments the authors have found that application of SVM as the classifier in the electronic nose is more advantageous than RF, especially at high level of noise and small number of measuring sensors. Its application allowed to recognize 11 brands of cigarettes with the accuracy close to 100 percent. Practical implications – Thanks to application of two identical sensors working in a differential mode the authors avoid the baseline estimation and thus the solution is well suited for on-line dynamic measurements of the process. Originality/value – The paper has studied the advantages and limitations of the differential electronic nose following from the existence of the noise, corrupting the measurements. It has pointed an important role of the applied classifier system in getting the electronic nose of the highest quality.

Room-temperature and temperature-dependent QSRR modelling for predicting the nitrate radical reaction rate constants of organic chemicals using ensemble learning methods

Experimental determinations of the rate constants of the reaction of NO3 with a large number of organic chemicals are tedious, and time and resource intensive; and the development of computational methods has widely been advocated. In this study, we have developed room-temperature (298 K) and temperature-dependent quantitative structure–reactivity relationship (QSRR) models based on the ensemble learning approaches (decision tree forest (DTF) and decision treeboost (DTB)) for predicting the rate constant of the reaction of NO3 radicals with diverse organic chemicals, under OECD guidelines. Predictive powers of the developed models were established in terms of statistical coefficients. In the test phase, the QSRR models yielded a correlation (r2) of >0.94 between experimental and predicted rate constants. The applicability domains of the constructed models were determined. An attempt has been made to provide the mechanistic interpretation of the selected features for QSRR development. The proposed QSRR models outperformed the previous reports, and the temperature-dependent models offered a much wider applicability domain. This is the first report presenting a temperature-dependent QSRR model for predicting the nitrate radical reaction rate constant at different temperatures. The proposed models can be useful tools in predicting the reactivities of chemicals towards NO3 radicals in the atmosphere, hence, their persistence and exposure risk assessment.

Three-Tier Strategy for Screening High-Energy Molecules Using Structure–Property Relationship Modeling Approaches

Experimental determination of the explosive properties of chemicals is a tedious, time- and resource-intensive, and involved risk. In this study, we have established a three-tier structure–property relationship (SPR) modeling strategy for screening the chemicals for their explosive behavior. Accordingly, qualitative (ternary classification: nonexplosive, ideal and nonideal explosive), semiquantitative (binary classification: industrial and high explosive), and quantitative SPR models based on decision tree forest (DTF) and decision tree boost (DTB) algorithms were developed for discriminating the chemicals in different categories and predicting the detonation velocity (DV) of ideal and lower flammability limit (LFL) of nonideal explosives in accordance with the OECD guidelines. The statistical quality and external predictive power of the developed SPR models were evaluated through the internal and external validation procedures. In the test set, the qualitative and semiquantitative SPR models (DTF, DTB) r...

In silico prediction of the developmental toxicity of diverse organic chemicals in rodents for regulatory purposes.

The experimental determination of the developmental toxicity potential (LEL) of chemicals is not only tedious, time and resource intensive, but it also involves unethical tests on animals. In this study, we have established quantitative structure activity relationship (QSAR) models for predicting the developmental toxicity potential of chemicals in rodents following the OECD guidelines. Accordingly, decision tree forest (DTF) and decision tree boost (DTB) based local (L-QSAR), global (G-QSAR) and interspecies quantitative structure activity-activity relationship (ISC QSAAR) models were developed for estimating the LEL (lowest effective level) dose of chemicals for developmental toxicity in rats and rabbits. The structural features of chemicals responsible for developmental toxicity in rodents were extracted and used in QSAR/QSAAR analysis. The external predictive power of the developed models was evaluated through the internal and external validation procedures. In test data, the L-QSAR models (DTF, DTB) yielded R2 values of >0.846 (rat) and >0.906 (rabbit), whereas in G-QSAR, the correlation value was >0.870 between the measured and predicted endpoint values. In ISC QSAAR models, the R2 values in test data were 0.830 (DTF) and 0.927 (DTB), respectively. Values of various statistical validation coefficients derived from the test data (except rm2 in DTF based rat L-QSAR and ISC QSAAR models) were above their respective threshold limits, thus putting a high confidence in this analysis. The prediction quality of the developed QSAR/QSAAR models was also assessed using the mean absolute error (MAE) criteria and found good. The applicability domains of the constructed models were defined using the descriptor range, leverage, and standardization approaches. The results suggest that the developed QSAR/QSAAR models can reliably predict the developmental toxicity potential of structurally diverse chemicals in rodents, generating useful toxicity data for risk assessment in humans.