Regression Tree Induction Research Articles

Cytotoxic attributes of antiviral plants rich in polyphenolic and volatile compounds

The application of natural products for treatment and prevention of various diseases is becoming more important. Not only antibacterial natural preparations are being developed, but also antiviral natural products are investigated. There are two main problems in such investigations: (i) special chemical analysis and data analysis methods have to be used, (ii) not only antiviral effect has to be taken into account, but also cytotoxic and other unwanted effects. In this work, a combination of separation methods (capillary electrophoresis and gas chromatography), spectrophotometric methods, data mining/data analysis methods (hierarchical clusterization, multidimensional scaling and classification and regression tree induction) for clarifying chemical attributes in cytotoxic plant extracts was applied. The selected plants belonged to families: (i) Lamiaceae, (ii) Asteraceae and (iii) Fabaceae. Most cytotoxic were Agastache foeniculum Pursh. Kuntze. and Salvia officinalis L. extracts. Least cytotoxic were Perilla frutescens L. and Satureja Montana L. extracts. The tendency of cytotoxic extracts having abundant concentrations of medium and low electrophoretic mobility cations was clarified.

A fuzzy KNN-based model for significant wave height prediction in large lakes

Summary Some algorithms based on fuzzy set theory (FST) such as fuzzy inference system (FIS) and adaptive-network-based fuzzy inference system (ANFIS) have been successfully applied to significant wave height (SWH) prediction. In this paper, perhaps for the first time, the fuzzy K-nearest neighbor (FKNN) algorithm is utilized to develop a fuzzy wave height prediction model for large lakes, where the fetch length depends on the wind direction. As fetch length (or wind direction) can affect the wave height in lakes, this variable is also considered as one of the inputs of the prediction model. The results of the FKNN model are compared with those of some soft computing techniques such as Bayesian networks (BNs), regression tree induction (named M5P), and support vector regression (SVR). The developed FKNN model is used for SWH prediction in the western part of Lake Superior in North America. The results show that the FKNN and M5P model can outperform the other soft computing techniques.

Wave Height Prediction Using Artificial Immune Recognition Systems (AIRS) and Some Other Data Mining Techniques

Significant wave height (SWH) prediction is an important task in coastal engineering activities. Due to difficulties and complexities in predicting SWH using traditional numerical and empirical models, in the past decades, a high tendency to use soft computing approaches has been observed. In this paper, an artificial immune recognition system (AIRS), which is a recently developed data mining approach, is utilized for SWH prediction. The results of AIRS model are compared with those of artificial neural networks (ANN), regression tree induction (named M5P), Rough Set Theory, Bayesian networks and support vector regression models. The AIRS has been successfully employed in some engineering applications, but its efficiency and applicability for SWH prediction have not been investigated. In this paper, the AIRS model is utilized for the SWH prediction in the Lake Superior in North America. Comparing the results of this model and five other data mining techniques shows that the AIRS and ANN can outperform other models.

The Prediction of Moulding Sand Moisture Content Based on the Knowledge Acquired by Data Mining Techniques

Abstract The subject of the study is the improvement of the quality of moulding sand preparation. An exploration research performed on the data concerning moulding sand quality parameters was described. The aim of the research was to find relationships between various factors determining the properties of moulding sands and, based on the results obtained, build models predicting the sand moisture content with the induction of classification and regression trees. A two-match prediction approach was demonstrated and its effectiveness in evaluating the moulding sand moisture content was discussed. The knowledge in the form of rules acquired in this way can be used in the creation of knowledge bases for systems supporting decisions in the diagnostics of the moulding sand rebonding process. Formalized knowledge also facilitates further processing of the measurement data.

Optimal water and waste load allocation in reservoir–river systems: a case study

In this paper, a new methodology is developed for optimization of water and waste load allocation in reservoir–river systems considering the existing uncertainties in reservoir inflow, waste loads and water demands. A stochastic dynamic programming (SDP) model is used to optimize reservoir operation considering the inflow uncertainty, and another model called PSO-SA is developed and linked with the SDP model for optimizing water and waste load allocation in downstream river. In the PSO-SA model, a particle swarm optimization technique with a dynamic penalty function for handling the constraints is used to optimize water and waste load allocation policies. Also, a simulated annealing technique is utilized for determining the upper and lower bounds of constraints and objective function considering the existing uncertainties. As the proposed water and waste load allocation model has a considerable run-time, some powerful soft computing techniques, namely, Regression tree Induction (named M5P), fuzzy K-nearest neighbor, Bayesian network, support vector regression and an adaptive neuro-fuzzy inference system, are trained and validated using the results of the proposed methodology to develop real-time water and waste load allocation rules. To examine the efficiency and applicability of the methodology, it is applied to the Dez reservoir–river system in the south-western part of Iran.

An evolutionary algorithm for global induction of regression and model trees

Most tree-based algorithms are typical top-down approaches that search only for locally optimal decisions at each node and does not guarantee the globally optimal solution. In this paper, we would like to propose a new evolutionary algorithm for global induction of univariate regression trees and model trees that associate leaves with simple linear regression models. The general structure of our solution follows a typical framework of evolutionary algorithms with an unstructured population and a generational selection. We propose specialised genetic operators to mutate and cross-over individuals (trees), fitness function that base on the Bayesian information criterion and smoothing process that improves the prediction accuracy of the model tree. Performed experiments on 15 real-life datasets show that proposed solution can be significantly less complex with at least comparable performance to the classical top-down counterparts.

NUMERIC PREDICTION ON OWL KNOWLEDGE BASES THROUGH TERMINOLOGICAL REGRESSION TREES

In the context of semantic knowledge bases, among the possible problems that may be tackled by means of data-driven inductive strategies, one can consider those that require the prediction of the unknown values of existing numeric features or the definition of new features to be derived from the data model. These problems can be cast as regression problems so that suitable solutions can be devised based on those found for multi-relational databases. In this paper, a new framework for the induction of logical regression trees is presented. Differently from the classic logical regression trees and the recent fork of the terminological classification trees, the novel terminological regression trees aim at predicting continuous values, while tests at the tree nodes are expressed with Description Logic concepts. They are intended for multiple uses with knowledge bases expressed in the standard ontology languages for the Semantic Web. A top-down method for growing such trees is proposed as well as algorithms for making predictions with the trees and deriving rules. The system that implements these methods is experimentally evaluated on ontologies selected from popular repositories.

Self-Adaptive Induction of Regression Trees

A new algorithm for incremental construction of binary regression trees is presented. This algorithm, called SAIRT, adapts the induced model when facing data streams involving unknown dynamics, like gradual and abrupt function drift, changes in certain regions of the function, noise, and virtual drift. It also handles both symbolic and numeric attributes. The proposed algorithm can automatically adapt its internal parameters and model structure to obtain new patterns, depending on the current dynamics of the data stream. SAIRT can monitor the usefulness of nodes and can forget examples from selected regions, storing the remaining ones in local windows associated to the leaves of the tree. On these conditions, current regression methods need a careful configuration depending on the dynamics of the problem. Experimentation suggests that the proposed algorithm obtains better results than current algorithms when dealing with data streams that involve changes with different speeds, noise levels, sampling distribution of examples, and partial or complete changes of the underlying function.

Indicators of environmentally sound land use in the humid tropics: The potential roles of expert opinion, knowledge engineering and knowledge discovery

Despite abundant literature on indicators for sustainable resource management, practical tools to help local users to apply its general concepts at a local to regional level are scarce. This means that decisions over land evaluation and land use at a local level are often not based on the formal application of indicators or decision support systems for environmentally sound management but instead on the opinion of local expertise, for instance forest managers, cattle breeders, farmers and/or academics. This is particularly seen to be the case in the tropics where access to modern communication and information technologies is restricted. As the opinions of experts are often based on and influenced by personal experience, intuition, heuristics and bias, their evaluations and decision are often unclear to the non-expert working at a local level. In order to make their reasoning more comprehensible to the non-expert, the ecological condition of 176 plots in the tropical Southeast of Mexico were evaluated by experts on soil fertility, forest management, cattle breeding and agriculture. With the assistance of a knowledge engineer (one who converts expert knowledge and reasoning into a model), these expert opinions and reasoning were then translated into a formal computer model. As an alternative approach we applied a knowledge discovery technique, namely the induction of regression trees and automatically developed models using the expert evaluations as training data. Where knowledge engineering was tedious and time consuming, regression models could be rapidly generated. Moreover, the correspondence between regression trees and expert opinions was considerably higher than the correspondence between expert opinion and their own models. The regression trees used less explicative variables than the models generated by the experts. The minimisation of sampling effort due to variable space reduction means that the application of regression tree induction has a high potential for a rapid development of indicators for narrowly defined ecological assessments, needed for decision making on a local or regional scale.

Potential of multi‐objective models for risk‐based mapping of the resilience characteristics of soils: demonstration at a national level

AbstractPolicy makers rely on risk‐based maps to make informed decisions on soil protection. Producing the maps, however, can often be confounded by a lack of data or appropriate methods to extrapolate using pedotransfer functions. In this paper, we applied multi‐objective regression tree analysis to map the resistance and resilience characteristics of soils onto stress. The analysis used a machine learning technique of multiple regression tree induction that was applied to a data set on the resistance and resilience characteristics of a range of soils across Scotland. Data included both biological and physical perturbations. The response to biological stress was measured as changes in substrate mineralization over time following a transient (heat) or persistent (copper) stress. The response to physical stress was measured from the resistance and recovery of pore structure following either compaction or waterlogging. We first determined underlying relationships between soil properties and its resistance and resilience capacity. This showed that the explanatory power of such models with multiple dependent variables (multi‐objective models) for the simultaneous prediction of interdependent resilience and resistance variables was much better than a piecewise approach using multiple regression analysis. We then used GIS techniques coupled with an existing, extensive soil data set to up‐scale the results of the models with multiple dependent variables to a national level (Scotland). The resulting maps indicate areas with low, moderate and high resistance and resilience to a range of biological and physical perturbations applied to soil. More data would be required to validate the maps, but the modelling approach is shown to be extremely valuable for up‐scaling soil processes for national‐level mapping.

The feasibility of co-existence between conventional and genetically modified crops: Using machine learning to analyse the output of simulation models

Simulation models are a commonly used tool for the study of the co-existence of conventional and genetically modified (GM) crops. Among other things, they allow us to investigate the effects of using different crop varieties, cropping systems and farming practices on the levels of adventitious presence of GM material in conventional crops. We propose to use machine learning methods to analyse the output of simulation models to learn co-existence rules that directly link the above mentioned causes and effects. The outputs of the GENESYS model, designed to study the co-existence of conventional and GM oilseed rape crops, were analysed by using the machine learning methods of regression tree induction and relational decision tree induction. Co-existence and adventitious presence of GM material were studied in several contexts, including gene flow between pairs of fields, the interactions of this process with farming practices (cropping systems), and gene flow in the context of an entire field plan. Accurate models were learned, which also make use of the relational aspects of a field plan, using information on the neighboring fields of a field, and the farming practices applied in it. The use of relational decision tree induction to analyse the results of simulation models is a novel approach and holds the promise of learning more general co-existence rules by allowing us to vary the target field within a chosen field plan, as well as to consider completely different field plans at the same time.

Post-pruning in regression tree induction: An integrated approach

The regression tree (RT) induction process has two major phases: the growth phase and the pruning phase. The pruning phase aims to generalize the RT that was generated in the growth phase by generating a subtree that avoids over-fitting to the training data. Most post-pruning methods essentially address post-pruning as if it were a single objective problem (i.e., maximize validation accuracy), and address the issue of simplicity (in terms of the number of leaves) only in the case of a tie. However, it is well known that apart from accuracy there are other performance measures (e.g., stability, simplicity) that are important for evaluating DT quality. In this paper we present an integrated approach to post-pruning phase that simultaneously accommodates multiple performance measures that are important for evaluating RT quality, and obtains the optimal subtree based on user provided preference and value function information.

Instance-Based Regression by Partitioning Feature Projections

A new instance-based learning method is presented for regression problems with high-dimensional data. As an instance-based approach, the conventional method, KNN, is very popular for classification. Although KNN performs well on classification tasks, it does not perform as well on regression problems. We have developed a new instance-based method, called Regression by Partitioning Feature Projections (RPFP) which is designed to meet the requirement for a lazy method that achieves high levels of accuracy on regression problems. RPFP gives better performance than well-known eager approaches found in machine learning and statistics such as MARS, rule-based regression, and regression tree induction systems. The most important property of RPFP is that it is a projection-based approach that can handle interactions. We show that it outperforms existing eager or lazy approaches on many domains when there are many missing values in the training data.

Using regression trees to identify the habitat preference of the sea cucumber ( Holothuria leucospilota) on Rarotonga, Cook Islands

In the Pacific Islands, invertebrates including sea cucumbers are among the most valuable and vulnerable inshore fisheries resources. As human activities continue to force substantial impacts on coral reef ecosystems, the management of inshore fisheries has become an increasingly important priority. Knowledge of the distribution, biology and habitat requirements of a species can significantly enhance conservation efforts. The sea cucumber ( Holothuria leucospilota) forms an important part of the traditional subsistence fishery on Rarotonga, Cook Islands, yet little is known of this species’ present spatial distribution and abundance around the island. We apply two machine learning approaches and a classical statistical approach to predict the number of sea cucumber individuals from site characteristics. The machine learning methods used are induction of regression trees and instance-based learning. These are compared to the classical statistical approach of linear regression. The most accurate predictions are obtained using instance-based learning, while the most understandable descriptions are obtained using regression tree induction.

Predicting Chemical Parameters of River Water Quality from Bioindicator Data

We address the problem of inferring chemical parameters of river water quality from biological ones. This task is important for enabling selective chemical monitoring of river water quality. We apply machine learning, in particular regression tree induction, to biological and chemical data on the water quality of Slovenian rivers. Regression trees are constructed that predict values of chemical parameters from data on the presence of bioindicator taxa at the species and family levels.

A Comparison of Logistic Regression to Decision Tree Induction in the Diagnosis of Carpal Tunnel Syndrome

This paper aims to compare and contrast two types of model (logistic regression and decision tree induction) for the diagnosis of carpal tunnel syndrome using four ordered classification categories. Initially, we present the classification performance results based on more than two covariates (multivariate case). Our results suggest that there is no significant difference between the two methods. Further to this investigation, we present a detailed comparison of the structure of bivariate versions of the models. The first surprising result of this analysis is that the classification accuracy of the bivariate models is slightly higher than that of the multivariate ones. In addition, the bivariate models lend themselves to graphical analysis, where the corresponding decision regions can easily be represented in the two-dimensional covariate space. This analysis reveals important structural differences between the two models.

Data mining applications to dermatological diseases, colorectal cancer, diabetes, cardiology and medical audit systems

Diagnosis of Dermatological Diseases This project aims to develop tools that would provide doctors with the ability to discover the typical characteristics of certain inflammatory dermatological diseases. It is hoped that this will aid doctors in accurately diagnosing the dermatological disorder, especially between two skin diseases with very similar characteristics such as psoriasis and atopic dermatitis. Prognostic Models for Colorectal Cancer Patients The goal of this project is to build a robust prognostic model that will predict length of survival for patients with colorectal cancer. Techniques employed for this purpose include Neural Networks, Case-based reasoning (CBR), Cox's Regression and Regression Tree Induction. Accomplishments have been the development of point estimates of survival from Cox's regression to allow direct comparison of Artificial Intelligence techniques with Cox's results and the development of techniques for building more perspicuous models that are intuitive to medical practitioners. Pre-empting Complications in Diabetic Patients Data, provided by Data Retrieval in General Practice, is currently being used to discover common trends that appear among diabetic patients. These trends can then be used to predict complications in a diabetic's life cycle. This will allow patients who are not at risk to be filtered out and those who are at risk to be carefully monitored. Data mining techniques such as Sequential Pattern Discovery are being employed for this purpose. Risk Assessment in Cardiology This project has been investigating the use of case-based reasoning and the incorporation of fuzzy logic to provide insight into the understanding of the underlying mechanisms of coronary heart disease. By identifying patients at risk early on, it is easier to stop the progression of this disease. A series of qualitative and quantitative information was gathered from middle aged men, such as weight, blood pressure, body fat and family medical histories over a three year time span, to help identify which factors contribute to coronary heart disease. Integrating Data Mining into Medical Audit Systems The aim of this project is to bring Data Mining into mainstream medical audit by integrating it with Medical statistics techniques already in use. This will encourage the practice of Evidence-based medicine, that is becoming a norm in western societies, providing IT support for it. The motivation behind this project is the realization that current Medical statistics techniques fall short of certain objectives and requirements of medical audit.