Inductive Learning System Research Articles

From Inductive Learning Towards Interactive Inductive Learning

Growing amount of information in the world encourage the use of automatic data processing techniques that reduce humans routine work. There is a wide range of methods used for machine learning; however inductive learning algorithms are preferable in the systems where understanding of decision making steps and further processing of results is needed, for instance the expert systems, where the rules induced by learning algorithms can be used. As the classification tasks are getting more complicated computer program may not make enough informed decision by itself. In such situations collaborative approach between machine and systems user (expert) would be useful. Inductive learning system learns classification from training examples and uses induced rules for classifying new cases. If a decision cannot be inferred from rules base, a guess is performed. Interactive inductive system in uncertain conditions could ask human for decision and improve its knowledge base with the rule derived from this human-made decision. The paper summarises approaches discussed in related works and classifies them by the phase in inductive learning process in which the human interaction appears. As a result a new approach to interactive inductive system is presented. Conceptual example of topographical map classification using this system is demonstrated.

MULTI-ASPECT MINING IN HEPATITIS DATA

When therapy using interferon medication for chronic hepatitis patients, various conceptual knowledge/rules will benefit for giving a treatment. The paper describes our work on cooperatively using various data mining agents including GDT-RS, learning with ordered information (LOI), and peculiarity oriented mining (POM) in a spiral discovery process with the multi-phase such as pre-processing, rule mining, and post-processing, for multi-aspect analysis of the hepatitis data and meta learning. GDT-RS is an inductive learning system for discovering decision rules. LOI discovers ordering rules and important features. POM finds peculiarity data/rules. Our methodology and experimental results show that the perspective of medical doctors will be changed from a single type of experimental data analysis towards a holistic view, by using our multi-aspect mining approach.

A Self-Organized, Distributed, and Adaptive Rule-Based Induction System

Learning classifier systems (LCSs) are rule-based inductive learning systems that have been widely used in the field of supervised and reinforcement learning over the last few years. This paper employs sUpervised Classifier System (UCS), a supervised learning classifier system, that was introduced in 2003 for classification tasks in data mining. We present an adaptive framework of UCS on top of a self-organized map (SOM) neural network. The overall classification problem is decomposed adaptively and in real time by the SOM into subproblems, each of which is handled by a separate UCS. The framework is also tested with replacing UCS by a feedforward artificial neural network (ANN). Experiments on several synthetic and real data sets, including a very large real data set, show that the accuracy of classifications in the proposed distributed environment is as good or better than in the nondistributed environment, and execution is faster. In general, each UCS attached to a cell in the SOM has a much smaller population size than a single UCS working on the overall problem; since each data instance is exposed to a smaller population size than in the single population approach, the throughput of the overall system increases. The experiments show that the proposed framework can decompose a problem adaptively into subproblems, maintaining or improving accuracy and increasing speed.

Local Dimensionality Reduction and Supervised Learning Within Natural Clusters for Biomedical Data Analysis

Inductive learning systems were successfully applied in a number of medical domains. Nevertheless, the effective use of these systems often requires data preprocessing before applying a learning algorithm. This is especially important for multidimensional heterogeneous data presented by a large number of features of different types. Dimensionality reduction (DR) is one commonly applied approach. The goal of this paper is to study the impact of natural clustering--clustering according to expert domain knowledge--on DR for supervised learning (SL) in the area of antibiotic resistance. We compare several data-mining strategies that apply DR by means of feature extraction or feature selection with subsequent SL on microbiological data. The results of our study show that local DR within natural clusters may result in better representation for SL in comparison with the global DR on the whole data.

A global optimal algorithm for class-dependent discretization of continuous data

This paper presents a new method to convert continuous variables into discrete variables for inductive machine learning. The method can be applied to pattern classification problems in machine learning and data mining. The discretization process is formulated as an optimization problem. We first use the normalized mutual information that measures the interdependence between the class labels and the variable to be discretized as the objective function, and then use fractional programming (iterative dynamic programming) to find its optimum. Unlike the majority of class-dependent discretization methods in the literature which only find the local optimum of the objective functions, the proposed method, OCDD, or Optimal Class-Dependent Discretization, finds the global optimum. The experimental results demonstrate that this algorithm is very effective in classification when coupled with popular learning systems such as C4.5 decision trees and Naive-Bayes classifier. It can be used to discretize continuous variables for many existing inductive learning systems.

Combining abductive reasoning and inductive learning to evolve requirements specifications

The development of requirements specifications inevitably involves modification and evolution. To support modification while preserving particular requirements goals and properties, the use of a cycle composed of two phases: analysis and revision is proposed. In the analysis phase, a desirable property of the system is checked against a partial specification. Should the property be violated, diagnostic information is provided. In the revision phase, the diagnostic information is used to help modify the specification in such a way that the new specification no longer violates the original property. An instance of the above analysis–revision cycle that combines new techniques of logical abduction and inductive learning to analyse and revise specifications, respectively is investigated. More specifically, given an (event-based) system description and a system property, abductive reasoning is applied in refutation mode to verify whether the description satisfies the property and, if it does not, identify diagnostic information in the form of a set of examples of property violation. These (counter) examples are then used to generate a corresponding set of examples of system behaviours that should be covered by the system description. Finally, such examples are used as training examples for inductive learning, changing the system description in order to resolve the property violation. This is accomplished with the use of the connectionist inductive learning and logic programming system—a hybrid system based on neural networks and the backpropagation learning algorithm. A case study of an automobile cruise control system illustrates the approach and provides some early validation of its capabilities.

Using Inductive Learning to Predict Bankruptcy

An emerging trend in organizational computing is using information technology to learn decision knowledge from enterprise data. The primary contribution of this study is the presentation of a sound theory and a comprehensive technique for learning the decision model for predicting bankruptcy. The theory is based on the information contained in cash flow components, which is the foundation of valuation theory, and an analytical system that measures the amount of uncertainty in the cash flow information. The approach links a tree-based inductive learning system that relies on the concept of entropy, with an information system based on the cash flow of a firm. A test of the cash flow approach involves the cash flow components for a sample of 99 failed and 99 non-failed companies. The structural instability of cash flow components generated by an inductive learning system is a serious issue for financial analysts. However, this shortcoming is overcome by using a jackknife procedure to develop a global tree that identifies the most important cash flow components. The final global tree found only 3 cash flow components were needed to classify correctly 89% of the companies as either failed or non-failed. Only a few early studies achieved a higher level of predictive accuracy. The 3 significant cash flow components were dividends, net investment, and net operating cash flow. Using the same data, a probit statistical technique generated a 67.5% predictive accuracy. In summary, the inductive learning results indicate that cash flow components are not only a natural tool for explaining the bankruptcy process, but they provide a high level of predictive accuracy.

Near Knowledge: Inductive Learning Systems in Law

Induction is an interesting model of legal reasoning, since it provides a method of capturing initial states of legal principles and rules, and adjusting these principles and rules over time as the law changes. In this Article I explain how Artificial Intelligence-based inductive learning algorithms work, and show how they have been used in law to model legal domains. I identify some problems with implementations undertaken in law to date, and create a taxonomy of appropriate cases to use in legal inductive inferencing systems. I suggest that inductive learning algorithms have potential in modeling law, but that the artificial intelligence implementations to date are problematic. I argue that induction should be further investigated, since it has the potential to be an extremely useful mechanism for understanding legal domains.

The Connectionist Inductive Learning and Logic Programming System

This paper presents the Connectionist Inductive Learning and Logic Programming System (C-IL2P). C-IL2P is a new massively parallel computational model based on a feedforward Artificial Neural Network that integrates inductive learning from examples and background knowledge, with deductive learning from Logic Programming. Starting with the background knowledge represented by a propositional logic program, a translation algorithm is applied generating a neural network that can be trained with examples. The results obtained with this refined network can be explained by extracting a revised logic program from it. Moreover, the neural network computes the stable model of the logic program inserted in it as background knowledge, or learned with the examples, thus functioning as a parallel system for Logic Programming. We have successfully applied C-IL2P to two real-world problems of computational biology, specifically DNA sequence analyses. Comparisons with the results obtained by some of the main neural, symbolic, and hybrid inductive learning systems, using the same domain knowledge, show the effectiveness of C-IL2P.

Machine learning for map interpretation: An intelligent tool for environmental planning

The design of a user interface integrating instruments for visual and textual representation and image interpretation is a relevant problem when developing an advisory system for environmental planning. Indeed, the user of the system needs a support to the interpretation of maps, that is, a tool that segments maps and automatically associates geometric regions on a map with those semantic labels useful for applying hints and advices suggested by the environmental planning system. In the article, we present the application of symbolic machine learning techniques to the interpretation of maps. Two inductive learning systems, namely, INDUBI/CSL and ATRE, have been used to complete the knowledge base of an expert system for environmental planning. The application described concerns the recognition of four environmental concepts that are relevant for environmental protection. The positive results obtained in two different experiments prove the strength of the adopted approach for the interpretation task.

Learning symbolic descriptions of shape for object recognition in X-ray images

In this paper we describe a method for learning shape descriptions of objects in X-ray images. The descriptions are induced from shape examples using the AQ15c inductive learning system. The method has been experimentally compared to k-nearest neighbor, a statistical pattern recognition technique, the C4.5 decision tree learning program, and a multilayer feed-forward neural network. Experimental results demonstrate strong advantages of the AQ methodology over the other methods. Specifically, the method has higher predictive accuracy and faster learning and recognition rates. AQ's representation language, VL 1, was better suited for this problem, which can be seen by examining the empirical results and the learned rules. The method was applied to the problem of detecting blasting caps in X-ray images of luggage. An intelligent system performing this detection task can be used to assist airport security personnel with luggage screening.

An environmental evaluation learning apprentice system

In engineering problems with a weak domain theory, such as environmental evaluation, machine learning techniques should bring to bear any existing background knowledge so as to guide the knowledge-acquisition process. This paper proposes an Interactive Inductive Learning System (IILS) which can use background knowledge provided by experts to avoid incorrect or incomplete induced heuristics. Through the specification of guidance relations, the expert can force the rule-induction system to focus on a subset of relevant attributes and training instances. A guidance relation consists of a set of attribute constraints, where each constraint instructs IILS on the role of an attribute for the induction of a concept. IILS has been linked to the database of an Environmental Evaluation Support System (EESS) for the induction of impact estimation and comparison heuristics. The induced heuristics are generalizations of past assessments stored in the database and permit the prediction of impact levels for cases not previously encountered by EESS. The integration of IILS transforms EESS into an environmental impact assessment learning apprentice system capable of acquiring and improving evaluation heuristics through its normal use.

Deep assessment of machine learning techniques using patient treatment in acute abdominal pain in children

Learning from patient records may aid knowledge acquisition and decision making. Existing inductive machine learning (ML) systems such us Newld, CN2, C4.5 and AQ15 learn from past case histories using symbolic and/or numeric values. These systems learn symbolic rules (IF… THEN like) which link an antecedent set of clinical factors to a consequent class or decision. This paper compares the learning performance of alternative ML systems with each other and with respect to a novel approach using logic minimization, called LML, to learn from data. Patient cases were taken from the archives of the Paediatric Surgery Clinic of the University Hospital of Crete, Heraklion, Greece. Comparison of ML system performance is based both on classification accuracy and on informal expert assessment of learned knowledge.

Generalization of Clauses under Implication

In the area of inductive learning, generalization is a main operation, and the usual definition of induction is based on logical implication. Recently there has been a rising interest in clausal representation of knowledge in machine learning. Almost all inductive learning systems that perform generalization of clauses use the relation theta-subsumption instead of implication. The main reason is that there is a well-known and simple technique to compute least general generalizations under theta-subsumption, but not under implication. However generalization under theta-subsumption is inappropriate for learning recursive clauses, which is a crucial problem since recursion is the basic program structure of logic programs. We note that implication between clauses is undecidable, and we therefore introduce a stronger form of implication, called T-implication, which is decidable between clauses. We show that for every finite set of clauses there exists a least general generalization under T-implication. We describe a technique to reduce generalizations under implication of a clause to generalizations under theta-subsumption of what we call an expansion of the original clause. Moreover we show that for every non-tautological clause there exists a T-complete expansion, which means that every generalization under T-implication of the clause is reduced to a generalization under theta-subsumption of the expansion.

Class-dependent discretization for inductive learning from continuous and mixed-mode data

Inductive learning systems can be effectively used to acquire classification knowledge from examples. Many existing symbolic learning algorithms can be applied in domains with continuous attributes when integrated with a discretization algorithm to transform the continuous attributes into ordered discrete ones. In this paper, a new information theoretic discretization method optimized for supervised learning is proposed and described. This approach seeks to maximize the mutual dependence as measured by the interdependence redundancy between the discrete intervals and the class labels, and can automatically determine the most preferred number of intervals for an inductive learning application. The method has been tested in a number of inductive learning examples to show that the class-dependent discretizer can significantly improve the classification performance of many existing learning algorithms in domains containing numeric attributes.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Encouraging experimental results on learning CNF

This paper presents results comparing three simple inductive learning systems using different representations for concepts, namely: CNF formulae, DNF formulae, and decision trees. The CNF learner performs surprisingly well. Results on five natural data sets indicates that it frequently trains faster and produces more accurate and simpler concepts.

Encouraging Experimental Results on Learning CNF

This paper presents results comparing three simple inductive learning systems using different representations for concepts, namely: CNF formulae, DNF formulae, and decision trees. The CNF learner performs surprisingly well. Results on five natural data sets indicates that it frequently trains faster and produces more accurate and simpler concepts.

Theoretical and practical considerations of uncertainty and complexity in automated knowledge acquisition

Inductive machine learning has become an important approach to automated knowledge acquisition from databases. The disjunctive normal form (DNF), as the common analytic representation of decision trees and decision tables (rules), provides a basis for formal analysis of uncertainty and complexity in inductive learning. A theory for general decision trees is developed based on C. Shannon's (1949) expansion of the discrete DNF, and a probabilistic induction system PIK is further developed for extracting knowledge from real world data. Then we combine formal and practical approaches to study how data characteristics affect the uncertainty and complexity in inductive learning. Three important data characteristics, namely, disjunctiveness, noise and incompleteness, are studied. The combination of leveled pruning, leveled condensing and resampling estimation turns out to be a very powerful method for dealing with highly disjunctive and inadequate data. Finally the PIK system is compared with other recent inductive learning systems on a number of real world domains.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

INDUSTRIAL EXPERT SYSTEM ACQUIRED BY MACHINE LEARNING

Abstract This paper describes SERP, an expert system for repairing helicopter blades, Although very useful, it cannot evolve automatically with advances in technology and expert knowledge. To address this problem, we have designed an inductive learning system ( MEA) that uses rules or preclassified examples in order to incrementally build production rules. These rules are of the following form: if description of the fault & xrArr; repair and can be used by KEE ( knowledge engineering environment), an expert system generator. Both MEA and SERP use a representation based on structured objects. In this paper some aspects of MEA will be described, and the way MEA can extend andlor improve the iterative dichotomiier ( ID3) algorithm will be explained. MEA has been tested on a real-life application concerning the repairing of helicopter blades. Results show that it is efficient, simple to use, and helpful to the expert.

Hypothesis-Driven Constructive Induction in AQ17-HCI: A Method and Experiments

The proposed method for constructive induction searches for concept descriptions in a representation space that is being iteratively improved. In each iteration, the system learns concept description from training examples projected into a newly constructed representation space, using an Aq algorithm-based inductive learning system (AQ15). The learned description is analyzed to determine desirable problem-oriented modifications of the representation space. These modifications include generating new attributes, removing redundant or insignificant ones, and/or agglomerating attribute values into larger units. New attributes are constructed by assigning names to groups of the best-performing characteristic rules for each decision class, and then are used to define the representation space for the next iteration. This iterative process repeats until the created hypotheses satisfy a stopping criterion. In several experiments on learning discrete functions, the developed AQ17-HCI system consistently outperformed, in terms of the prediction accuracy on new examples, all systems that it was compared to, including the AQ15 rule learning system, GREEDY3 and GROVE decision-list learning systems, and REDWOOD and FRINGE decision-tree learning systems. Although the proposed method was developed for the Aq-based rule learning system, it can potentially be adapted to any other inductive learning system. In this sense, it represents a universal new approach to constructive induction.