Inductive Learning Program Research Articles

Creating Decision Trees from Rules using RBDT‐1

Most of the methods that generate decision trees for a specific problem use the examples of data instances in the decision tree–generation process. This article proposes a method called RBDT‐1—rule‐based decision tree—for learning a decision tree from a set of decision rules that cover the data instances rather than from the data instances themselves. The goal is to create on demand a short and accurate decision tree from a stable or dynamically changing set of rules. The rules could be generated by an expert, by an inductive rule learning program that induces decision rules from the examples of decision instances such as AQ‐type rule induction programs, or extracted from a tree generated by another method, such as the ID3 or C4.5. In terms of tree complexity (number of nodes and leaves in the decision tree), RBDT‐1 compares favorably with AQDT‐1 and AQDT‐2, which are methods that create decision trees from rules. RBDT‐1 also compares favorably with ID3 while it is as effective as C4.5 where both (ID3 and C4.5) are well‐known methods that generate decision trees from data examples. Experiments show that the classification accuracies of the decision trees produced by all methods under comparison are indistinguishable.

Prototyping structural description using an inductive learning program

Character recognition systems can contribute tremendously to the advancement of the automation process and can improve the interaction between man and machine in many applications, including office automation, cheque verification and a large variety of banking, business and data entry applications. The main theme of this paper is the automatic recognition of hand-printed Arabic characters using machine learning. Conventional methods have relied on hand-constructed dictionaries which are tedious to construct and difficult to make tolerant to variation in writing styles. The advantages of machine learning are that it can generalize over the large degree of variation between writing styles and recognition rules can be constructed by example. The system was tested on a sample of handwritten characters from several individuals whose writing ranged from acceptable to poor in quality and the correct average recognition rate obtained using cross-validation was 89.65%.

Prototyping structural description using an inductive learning program

Character recognition systems can contribute tremendously to the advancement of the automation process and can improve the interaction between man and machine in many applications, including office automation, check verification and a large variety of banking, business and data entry applications. The main theme of the paper is the automatic recognition of hand-printed Arabic characters using machine learning. Conventional methods have relied on hand-constructed dictionaries which are tedious to construct and difficult to make tolerant to variation in writing styles. The advantages of machine learning are that it can generalize over the large degree of variation between writing styles and recognition rules can be constructed by example. The system was tested on a sample of handwritten characters from several individuals whose writing ranged from acceptable to poor in quality and the correct average recognition rate obtained using cross-validation was 89.65%.

Modelling current parna distribution in a local area

Parna is a wind-blown clay, mobilised from inland Australia as the result of a series of intermittent high wind events during the Quaternary. Parna can be recognised on the basis of colour, texture, distributional patterns, and pedology. Parna deposits have been recorded across a wide area of south eastern Australia and have influenced the local pedology and hydrology. In some cases parna has increased soil sodicity and the potential for dryland salinisation. Predicting its spatial distribution is useful when considering agricultural potential and in assessing the risk and spatial spread of dryland salinity. Here we present the results of modelling to predict its local distribution in an area covering 291 km2 in the Young district of NSW. Two conceptual models of parna deposition and subsequent redistribution were used to develop a current parna distribution map: (a) deposition = f(topography, aspect) after assuming that interactions of rainfall, vegetation, and wind speed were relatively the same at the local scale; (b) removal or retention = f (slope angle, catchment size, slope length) as a representation of the erosive energy of gravity. Five landscape variables, elevation, aspect, slope, flow accumulation, and flow length, were derived from a 20 m digital elevation model (DEM). A training set of parna deposits was established using air photos and field survey from limited exposures in the Young district of NSW. These areas were digitised and converted to a grid of areas of parna and no-parna. This training set for parna and the 5 landscape variable grids were processed in the IDRISI for WINDOWS Geographic Information System (GIS). Spatial relationships between the parna and no-parna deposits and the 5 landscape variables were extracted from this training set. This information was imported into an inductive learning program called KnowledgeSEEKER. A decision tree was built by recursive partitioning of the data set using Chi-squares to categorise variables, and an F test for continuous variables to best replicate the training data classification of ‘parna’ and ‘no-parna’. The rules derived from this process were applied to the study area to predict the occurrence of parna in the broader landscape. Predictions were field checked and the rules adjusted until they best represented the occurrence of parna in the field. The final model showed predictions of parna deposits as follows: (i) higher elevations in the Young landscape were the dominant sites of parna deposits; (ii) thicker deposits of parna occurred on the windward south-west and north-west; (iii) thinner deposits occurred on the leeward side of a central ridge feature; (iv) because the training set concentrated around the major central ridge feature, poorer predictions were obtained on gently undulating country.

Data-driven constructive induction

An inductive learning program's ability to find an accurate hypothesis can depend on the quality of the representation space. The authors have developed a data-driven constructive-induction method that uses multiple operators to improve the representation space. They have applied it to two real-world problems. Constructive-induction integrates ideas and methods previously considered separate: attribute selection, construction, and abstraction. By integrating these methods into AQ17-DCI, they were able to increase predictive accuracy by up to 29% in their test cases.

Evaluation of automatically learned intelligent alarm systems

In this contribution it is investigated whether a combination of mathematical simulation and inductive machine learning can replace the usual knowledge elicitation techniques. To test this a domain was selected for which knowledge based systems had a high performance: intelligent alarm systems. A mathematical model of a breathing circuit and ventilated patient was implemented in PSpice ®. Airway pressure, gas flows and CO 2 concentration were simulated with this model, during normal functioning of the breathing circuit and during several mishaps, for a wide range of simulated patients. With an inductive machine learning program, classification trees were created from the simulated patient data. The classification trees described each breathing circuit mishap in terms of changes in signal feature values with respect to the normal situation and were implemented as alarm system knowledge bases. The alarm systems were tested with data measured at 17 mechanically ventilated animals. During ventilation of the animals several mishaps were introduced. For each animal, 93–100% of all mishaps could be detected correctly by the alarm systems. The false alarm rate ranged on average from one false alarm per h to one false alarm every 2.5 h. It was concluded that the suggested approach to knowledge elicitation was successful.

On Learning Decision Structures

A decision structure is a simple and powerful tool for organizing a decision process. It differs from a conventional decision tree in that its nodes are assigned tests that can be functions of the attributes, rather than single attributes; the branches stemming from a node can be assigned a subset of attribute values rather than a single attribute value (test outcome); and the leaves can be assigned one or more alternative decisions. We describe a methodology for learning decision structures from declarative knowledge expressed in the form of decision rules. The decision rules are generated by an expert, or by an AQ-type inductive learning program (with or without constructive induction). From a given set of rules, one can generate many different decision structures. The proposed methodology generates the one that is most suitable for the given decision-making situation, according to a multicriterion evaluation function. Experiments with a program implementing the proposed methodology have demonstrated its many useful features.

Scaling up inductive learning with massive parallelism

Machine learning programs need to scale up to very large data sets for several reasons, including increasing accuracy and discovering infrequent special cases. Current inductive learners perform well with hundreds or thousands of training examples, but in some cases, up to a million or more examples may be necessary to learn important special cases with confidence. These tasks are infeasible for current learning programs running on sequential machines. We discuss the need for very large data sets and prior efforts to scale up machine learning methods. This discussion motivates a strategy that exploits the inherent parallelism present in many learning algorithms. We describe a parallel implementation of one inductive learning program on the CM-2 Connection Machine, show that it scales up to millions of examples, and show that it uncovers special-case rules that sequential learning programs, running on smaller datasets, would miss. The parallel version of the learning program is preferable to the sequential version for example sets larger than about 10K examples. When learning from a public-health database consisting of 3.5 million examples, the parallel rule-learning system uncovered a surprising relationship that has led to considerable follow-up research.

Geographic Field Data Collection: Using machine learning techniques to verify minimum data requirements for the classification task.

Geographers and environmental scientists prefer to construct Spatial Information System (SIS) decision support from the smallest possible data. This is due to the considerable cost of ground-based surveys for data collection. This paper extends on the work of (Kirkby, 1994; Eklund et al., 1995) and reports on the use of machine learning classifiers to obtain the minimum sample size for ground-based data surveys. The study proposes a method to assess ground-based data collection using machine learning classifiers.In this domain, the inductive learning program C4.5 (Quinlan, 1993) was used to verify that a high performance classifier, better than 95 % classification accuracy on unseen data, can be constructed using 235 sample points in the study area. We compare this result to the magnitude of sample sizes required for back-propagation neural networks (NN) and instance-based learning (IBL) with the same classification accuracy on unseen data. We examine the reasons and implications for these variations for classification accuracy in this domain.

Learning rules to predict rodent carcinogenicity of non-genotoxic chemicals

The results of short-term assays (induction of chromosomal aberrations and sister-chromatid exchanges, oncogenic transformations and cellular toxicity) together with MTD (maximum tolerated dose) values and physical chemical properties of non-genotoxic (i.e. Salmonella non-mutagens) carcinogens and non-carcinogens were submitted to RL, an inductive learning program. RL was able to learn rules that correctly predicted between 70 and 80% of non-genotoxic chemicals. This is a marked improvement over current predictions using only the results of short-term assays and exceeds the predictions of human experts that used the whole spectrum of acute and subchronic toxicity results as well as human knowledge and intuition.

Learning control strategies for chemical processes: a distributed approach

The design of a distributed learning system (DLS) which combines the features of instance-space and hypothesis-space methods is described. This algorithm decomposes a data set of training examples into subsets. After applying an inductive learning program on each subset, it synthesizes the results using a genetic algorithm. It is shown that this parallel distributed approach is more efficient, since each inductive learning program works on only a subset of data. Since the genetic algorithm searches globally in the hypothesis space, this approach gives a more accurate concept description. The implementation of DLS in Common LISP is discussed, and its distributed approach is compared to C4.5 and PLS1 algorithms.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Using the ID3 Algorithm to find Discrepant Diagnoses from laboratory Databases of Thyroid Patients

Rare cases are a central problem when an expert system is constructed from example cases with machine learning techniques. It is difficult to make a decision support system (DSS) to cover all possible clinical cases. An inductive learning program can be used to construct an expert system for detecting cases that differ from routine cases. The ID3 algorithm and the pessimistic pruning algorithm were tested in this study: a DSS was built directly from the data of patient records. A decision tree was generated, and the cases misclassified by the decision tree as compared with the classifications of a clinician were listed on a checklist, which formed the feedback to the clinician. In clinical situations about 5-10% of functional thyroid disorders may be misclassified. At this error level, the method found over 90% of the errors with a specificity of 95%. In simple medical classification tasks this dynamic self-learning system can be used to create a DSS that can assist in the quality control of clinical decision making.

Integrating quantitative and qualitative discovery in the ABACUS system

Most research on inductive learning has been concerned with qualitative learning that induces conceptual, logic-style descriptions from the given facts. In contrast, quantitative learning deals with discovering numerical laws characterizing empirical data. This research attempts to integrate both types of learning by combining newly developed heuristics for formulating equations with the previously developed concept learning method embodied in the inductive learning program AQ11. The resulting system, ABACUS, formulates equations that bind subsets of observed data, and derives explicit, logic-style descriptions stating the applicability conditions for these equations. In addition, several new techniques for quantitative learning are introduced. Units analysis reduces the search space of equations by examining the compatibility of variables' units. Proportionality graph search addresses the problem of identifying relevant variables that should enter equations. Suspension search focusses the search space through heuristic evaluation. The capabilities of ABACUS are demonstrated by several examples from physics and chemistry.