Inductive Logic Programming Research Articles

A Logic-based Computational Method for the Automated Induction of Fuzzy Ontology Axioms

Fuzzy Description Logics DLs are logics that allow to deal with structured vague knowledge. Although a relatively important amount of work has been carried out in the last years concerning the use of fuzzy DLs as ontology languages, the problem of automatically managing the evolution of fuzzy ontologies has received very little attention so far. We describe here a logic-based computational method for the automated induction of fuzzy ontology axioms which follows the machine learning approach of Inductive Logic Programming. The potential usefulness of the method is illustrated by means of an example taken from the tourism application domain.

A Survey on Inductive Logic Programming Learning Algorithm

A Survey on Inductive Logic Programming Learning Algorithm

Identifying Audiences of E-Infrastructures - Tools for Measuring Impact

Research evaluation should take into account the intended scholarly and non-scholarly audiences of the research output. This holds too for research infrastructures, which often aim at serving a large variety of audiences. With research and research infrastructures moving to the web, new possibilities are emerging for evaluation metrics. This paper proposes a feasible indicator for measuring the scope of audiences who use web-based e-infrastructures, as well as the frequency of use. In order to apply this indicator, a method is needed for classifying visitors to e-infrastructures into relevant user categories. The paper proposes such a method, based on an inductive logic program and a Bayesian classifier. The method is tested, showing that the visitors are efficiently classified with 90% accuracy into the selected categories. Consequently, the method can be used to evaluate the use of the e-infrastructure within and outside academia.

Gene Function Hypotheses for the Campylobacter jejuni Glycome Generated by a Logic-Based Approach

Increasingly, experimental data on biological systems are obtained from several sources and computational approaches are required to integrate this information and derive models for the function of the system. Here, we demonstrate the power of a logic-based machine learning approach to propose hypotheses for gene function integrating information from two diverse experimental approaches. Specifically, we use inductive logic programming that automatically proposes hypotheses explaining the empirical data with respect to logically encoded background knowledge. We study the capsular polysaccharide biosynthetic pathway of the major human gastrointestinal pathogen Campylobacter jejuni. We consider several key steps in the formation of capsular polysaccharide consisting of 15 genes of which 8 have assigned function, and we explore the extent to which functions can be hypothesised for the remaining 7. Two sources of experimental data provide the information for learning—the results of knockout experiments on the genes involved in capsule formation and the absence/presence of capsule genes in a multitude of strains of different serotypes. The machine learning uses the pathway structure as background knowledge. We propose assignments of specific genes to five previously unassigned reaction steps. For four of these steps, there was an unambiguous optimal assignment of gene to reaction, and to the fifth, there were three candidate genes. Several of these assignments were consistent with additional experimental results. We therefore show that the logic-based methodology provides a robust strategy to integrate results from different experimental approaches and propose hypotheses for the behaviour of a biological system.

Analysis and Comparative Study of Classifiers for Relational Data Mining

As an important task of relational database, relational classification can directly classify the data that involve multiple relations from a relational database and have more advantages than propositional data mining approaches. The information age has provided us with huge data repositories which cannot longer be analyzed manually. Most available existing data mining algorithms looks for pattern in a single relation. To classify data from relational database need of multi-relational classification arise which is used to analyze relational database and used to predict behavior and unknown pattern automatically which include business data, bioinformatics, pharmacology, web mining, credit card fraud detection, disease diagnosis system, computational biology, online retailers. In this paper, we present the several kinds of multi-relational classification methods including Inductive Logic Programming (ILP) based, Associative based multirelational classification, Emerging Patterns based, Relational database based classification approaches and discuss each relational classification approaches, their characteristics, their comparisons and challenging issues in detail.

Guest editorial: special issue on Inductive Logic Programming (ILP 2011)

This special issue is associated with the 21st International Conference of Inductive Logic Programming (ILP 2011) which was held in Cumberland Lodge, Great Windsor Park and marked 20 years since the first ILP workshop in 1991. During this period the conference has developed into the premier forum for work on logic-based machine learning. The submission procedure followed a similar format to that of previous conferences, and was particularly close to that used in ILP 2006. Submissions were requested in two phases. The first phase involved submission of short papers (6 pages) which were then presented at the conference and posted on the website prior to the event itself. In the second phase, reviewers selected papers for long paper submission (15 pages). These were assessed by the same reviewers, who then decided which papers to include in the journal special issue, and which ones in the conference proceedings. The post-conference proceedings of ILP 2011 have been published as Volume 7207 in the Springer Lecture Notes in Artificial Intelligence. A separate latebreaking paper proceedings is being published by Imperial College Press under the title “Latest Advances in Inductive Logic Programming”. In the first phase there were 66 papers. Each paper was reviewed by 3 reviewers. Out of these 31 were invited as long papers. Out of the long paper submissions 5 were selected for the Machine Learning journal special issue and 23 were accepted for the proceedings. In addition, one paper (“Machine Learning a Probabilistic Network of Ecological Interactions”) was nominated by PC referees for the applications prize (the “Michie ILP Application Prize” sponsored by Syngenta Ltd) and one (“k-Optimal: A Novel Approximate Inference Algorithm for ProbLog”) for the theory prize (the “Turing ILP Theory Prize” sponsored by the Machine Learning Journal). The papers in this special issue represent the diversity and vitality in present ILP research.

Automated identification of protein-ligand interaction features using Inductive Logic Programming: a hexose binding case study.

BackgroundThere is a need for automated methods to learn general features of the interactions of a ligand class with its diverse set of protein receptors. An appropriate machine learning approach is Inductive Logic Programming (ILP), which automatically generates comprehensible rules in addition to prediction. The development of ILP systems which can learn rules of the complexity required for studies on protein structure remains a challenge. In this work we use a new ILP system, ProGolem, and demonstrate its performance on learning features of hexose-protein interactions.ResultsThe rules induced by ProGolem detect interactions mediated by aromatics and by planar-polar residues, in addition to less common features such as the aromatic sandwich. The rules also reveal a previously unreported dependency for residues cys and leu. They also specify interactions involving aromatic and hydrogen bonding residues. This paper shows that Inductive Logic Programming implemented in ProGolem can derive rules giving structural features of protein/ligand interactions. Several of these rules are consistent with descriptions in the literature.ConclusionsIn addition to confirming literature results, ProGolem’s model has a 10-fold cross-validated predictive accuracy that is superior, at the 95% confidence level, to another ILP system previously used to study protein/hexose interactions and is comparable with state-of-the-art statistical learners.

Integrating logic-based machine learning and virtual screening to discover new drugs

Investigational Novel Drug Discovery by Example (INDDEx™) is a technology developed to guide hit to lead discovery by learning rules from existing active compounds that link activity to chemical substructure. INDDEx is based on Inductive Logic Programming [1], which learns easily interpretable qualitative logic rules from active ligands that give an insight into chemistry, relate molecular substructure to activity, and can be used to guide the next steps of drug design chemistry. Support Vector Machines weight the rules to produce a quantitative model of structure-activity relationships. Whereas earlier testing [2,3] was performed on single dataset examples, this talk presents the largest and fullest test of the method. The method was benchmarked on the Directory of Useful Decoys (DUD) datasets [4], using the same methodology described in the paper on the assessment of LASSO [5] and DOCK. For each of the DUD datasets, the known active ligands were mixed with all the decoy compounds in DUD, and the retrieval rates of INDDEx and DUD were measured when they were trained on 2, 4, and 8 of the known active ligands (Figure 2). Early retrieved compounds showed high topological differences to molecules used as training data, showing the strength of this method for scaffold hopping. This work was supported by a BBSRC case studentship with Equinox Pharma Ltd (http://www.equinoxpharma.com). Figure 1 Recovery of actives in each of the DUD datasets from all decoys in the DUD, averaged across all 40 datasets.

A new relational Tri-training system with adaptive data editing for inductive logic programming

Relational Tri-training (R-Tri-training for short), as a relational semi-supervised learning system, can effectively exploit unlabeled examples to improve the generalization ability. However, the R-Tri-training may also suffer from the common problem in traditional semi-supervised learning, i.e., the performance is usually not stable for the unlabeled examples often be wrongly labeled and accumulated during the iterative learning process. In this paper, a new Relational Tri-training system named ADE-R-Tri-training (R-Tri-training with Adaptive Data Editing) is proposed. Not only does it employ a specific data editing technique to identify and correct the examples possibly mislabeled throughout the co-labeling iterations, but it also takes an adaptive strategy to decide whether to trigger the editing operation according to different cases. The adaptive strategy consists of five pre-conditional theorems, all of which ensure the iterative reduction of classification error under PAC (Probably Approximately Correct) learning theory. Experiments on well-known benchmarks show that ADE-R-Tri-training can more effectively enhance the performance of the hypothesis learned than R-Tri-training.

Adaptive complex event processing for harmful situation detection

In the field of infrastructures’ surveillance and protection, it is important to make decisions based on activities occurring in the environment and its local context and conditions. In this paper we use an active rule based event processing architecture in order to make sense of situations from the combination of different signals received by the rule engine. However obtaining some high level information automatically is not without risks, especially in sensitive environments, and detection mistakes can happen for various reasons: the signal’s source can be defective, whether it is human—miss-interpretation of the signal—or computed—material malfunction; the aggregation rules can be wrong syntaxically, for example when a rule will never be triggered or a situation never detected; the interpretation given to the combination of signals does not correspond to the reality on the field—because the knowledge of the rule designer is subjective or because the environment evolves over-time—the rules are therefore incorrect semantically. In this paper, a new approach is proposed to avoid the third kind of error sources. We present a hybrid machine learning technique adapted to the complexity of the rules’ representation, in order to create a system more conform to reality. The proposed approach uses a combination of an Association Rule Mining algorithm and Inductive Logic Programming for rule induction. Empirical studies on simulated datasets demonstrate how our method can contribute to sensible systems such as the security of a public or semi-public place.

Error Diagnosis of Chinese Sentences Using Inductive Learning Algorithm and Decomposition-Based Testing Mechanism

This study presents a novel approach to error diagnosis of Chinese sentences for Chinese as second language (CSL) learners. A penalized probabilistic First-Order Inductive Learning (pFOIL) algorithm is presented for error diagnosis of Chinese sentences. The pFOIL algorithm integrates inductive logic programming (ILP), First-Order Inductive Learning (FOIL), and a penalized log-likelihood function for error diagnosis. This algorithm considers the uncertain, imperfect, and conflicting characteristics of Chinese sentences to infer error types and produce human-interpretable rules for further error correction. In a pFOIL algorithm, relation pattern background knowledge and quantized t -score background knowledge are proposed to characterize a sentence and then used for likelihood estimation. The relation pattern background knowledge captures the morphological, syntactic and semantic relations among the words in a sentence. One or two kinds of the extracted relations are then integrated into a pattern to characterize a sentence. The quantized t -score values are used to characterize various relations of a sentence for quantized t -score background knowledge representation. Afterwards, a decomposition-based testing mechanism which decomposes a sentence into background knowledge set needed for each error type is proposed to infer all potential error types and causes of the sentence. With the pFOIL method, not only the error types but also the error causes and positions can be provided for CSL learners. Experimental results reveal that the pFOIL method outperforms the C4.5, maximum entropy, and Naive Bayes classifiers in error classification.

Mining simulation data by rule induction to determine critical source areas of stream water pollution by herbicides

Modelling in environmental sciences is becoming increasingly complex because ever-increasing numbers of processes are combined, thus making model-based decision aids both more relevant but more difficult to develop. Our approach focused on water quality and aimed to identify spatial tree patterns that represent surface flow and pollutant pathways from plot to plot involved in water pollution by herbicides. First, a simulation model predicted herbicide transfer rate, the proportion of applied herbicide that reaches water courses, based on the spatial and temporal distribution of weed-control activities. These predictions were used as a set of learning examples for symbolic learning techniques to induce rules based on qualitative and quantitative attributes and explain two classes of transfer rate. In this study we compared two automatic symbolic learning techniques applied to a set of simulations: (1) a relational-inductive method using the inductive logic programming (ILP) approach to induce spatial tree patterns; and (2) an attribute-value method to induce aggregated attributes of the trees. Twenty-eight and thirty-three rules were learnt by the ILP and attribute-value approaches which explained 81% and 88% of the examples, respectively. The ILP approach provided relevant indicators of plot-to-plot connectivity. The integrated attribute-value approach is simpler and quicker, but the ILP patterns are more useful for stakeholders.

Reasoning on Logic Programs with Annotated Disjunctions

Probabilistic Inductive Logic Programming and Statistical Relational Learning are families of techniques that are exploited in Machine Learning applications to perform advanced tasks in several domains. Every day the size and complexity of such problems increases and advanced, expressive and efficient tools are needed to successfully solve them. The literature proposes several algorithms to cope with these problems, each of them with its own quirks and perks. Among various solutions, Logic Programming with Annotated Disjunctions (LPAD) is one of the more attractive formalisms, thankstotheexpressivenessandreadabilityofitslanguage.Unfortunately,itsmostadvancedimplementationsarelackingefficient features and techniques that have been introduced for other formalisms, such as ProbLog. In this work, after introducing LPADs and an inference algorithm for computing the probability of a query, we investigate four different approximated algorithms, inspired by similar work done in ProbLog. In particular, we present each algorithm and we evaluate its performances on real and artificial datasets. The results show that our approaches have performances that are usually in line with ProbLog. The Monte Carlo algorithm, however, has performances that are better than the exact approach in terms of both the maximum size of the problems and the execution time, with a neglectable loss in the accuracy of the result.

Automatic traffic incident detection based on nFOIL

Traffic incidents inevitably cause traffic delay and deteriorate road safety conditions. Incidents are increasing alongside the fast economic growth. Due to the rampant growth of traffic incidents, developing efficient and effective automated incident detection (AID) techniques has prompted a growing worldwide interest. In this paper, the great efforts on developing a new approach to this problem based on nFOIL, a novel inductive logic programming (ILP), are done. By way of illustration, a simulated traffic data generated from Ayer Rajah Expressway (AYE) highway in Singapore and a real traffic data collected in I-880 freeway in California are used to assess the detection performance of this approach, and performance metrics includes detection rate, false alarm rate, mean time to detection, classification rate and the area under Receiver Operating Characteristic (ROC) curve (AUC). For comparison, we conducted the experiments on neural networks and support vector machine. The experimental results showed that nFOIL is sensitive to the skewed distribution of positive and negative examples in the dataset, and we make use of two different techniques, resampling and ensemble learning, to cope with highly skewed data in the context of ILP classification problems and investigated the effect of them typicality on the performance of AID model. It is concluded that ILP based AID approach are feasible, and have a favorable performance compared to neural networks and support vector machines.

A Study on Classification Approaches across Multiple Database Relations

Classification is an important task in data mining and machine learning, which has been studied extensively and has a wide range of applications. Lots of algorithms have been proposed to build accurate and scalable classifiers. Most of these algorithms can only applied to single “flat“ relations, whereas in the real world most data are stored in multiple tables. As converting data from multiple relations into single flat relation usually causes many problems, development of classification across multiple database relations becomes important. In this paper, we present the several kinds of classification method across multiple database relations including Inductive Logic Programming (ILP), Relational database, Emerging Pattern, Associative approaches and their characteristics, the comparisons in detail.

Evaluation of inductive logic programming for information extraction from natural language texts to support spatial data recommendation services

In this article we analyze a well-known and extensively researched problem: how to find all datasets, on the one hand, and on the other hand only those that are of value to the user when dealing with a specific spatially oriented task. In analogy with existing approaches to a similar problem from other fields of human endeavor, we call this software solution ‘a spatial data recommendation service.’ In its final version, this service should be capable of matching requests created in the user's mind with the content of the existing datasets, while taking into account the user's preferences obtained from the user's previous use of the service. As a result, the service should recommend a list of datasets best suited to the user's needs. In this regard, we consider metadata, particularly natural language definitions of spatial entities, a crucial piece of the solution. To be able to use this information in the process of matching the user's request with the dataset content, this information must be semantically preprocessed. To automate this task we have applied a machine learning approach. With inductive logic programming (ILP) our system learns rules that identify and extract values for the five most frequent relations/properties found in Slovene natural language definitions of spatial entities. The initially established quality criterion for identifying and extracting information was met in three out of five examples. Therefore we conclude that ILP offers a promising approach to developing an information extraction component of a spatial data recommendation service.

ILP turns 20

Inductive Logic Programming (ILP) is an area of Machine Learning which has now reached its twentieth year. Using the analogy of a human biography this paper recalls the development of the subject from its infancy through childhood and teenage years. We show how in each phase ILP has been characterised by an attempt to extend theory and implementations in tandem with the development of novel and challenging real-world applications. Lastly, by projection we suggest directions for research which will help the subject coming of age.

Learning theories for noun-phrase sentiment composition

The work presented here is an approach to Sentiment Analy- sis from a rule-based, compositional perspective. The proposed approach is characterized by three major points: (a) rules are automatically learned from annotated corpora using Inductive Logic Programming and represented as Prolog sets of clauses, (b) the focus is on the noun-phrase (NP) level, and (c) learning is performed on deep-parsed structures. We describe the process of annotating a collection of some 3000 German NPs of medium to quite complex structure, as well as the empirical evaluation of our implementation, in comparison with commonly used classifiers and a handcrafted rule-based system.

Inverse subsumption for complete explanatory induction

Modern explanatory inductive logic programming methods like Progol, Residue procedure, CF-induction, HAIL and Imparo use the principle of inverse entailment (IE). Those IE-based methods commonly compute a hypothesis in two steps: by first constructing an intermediate theory and next by generalizing its negation into the hypothesis with the inverse of the entailment relation. Inverse entailment ensures the completeness of generalization. On the other hand, it imposes many non-deterministic generalization operators that cause the search space to be very large. For this reason, most of those methods use the inverse relation of subsumption, instead of entailment. However, it is not clear how this logical reduction affects the completeness of generalization. In this paper, we investigate whether or not inverse subsumption can be embedded in a complete induction procedure; and if it can, how it is to be realized. Our main result is a new form of inverse subsumption that ensures the completeness of generalization. Consequently, inverse entailment can be reduced to inverse subsumption without losing the completeness for finding hypotheses in explanatory induction.

Introduction to the 27th International Conference on Logic Programming Special Issue

Following the initiative in 2010 taken by the Association for Logic Programming and Cambridge University Press, the full papers accepted for the International Conference on Logic Programming again appear as a special issue of Theory and Practice of Logic Programming (TPLP)—the 27th International Conference on Logic Programming Special Issue. Papers describing original, previously unpublished research and not simultaneously submitted for publication elsewhere were solicited in all areas of logic programming including but not restricted to: Theory: Semantic Foundations, Formalisms, Non- monotonic Reasoning, Knowledge Representation. Implementation: Compilation, Memory Management, Virtual Machines, Parallelism. Environments: Program Analysis, Transformation, Validation, Verification, Debugging, Profiling, Testing. Language Issues: Concurrency, Objects, Coordination, Mobility, Higher Order, Types, Modes, Assertions, Programming Techniques. Related Paradigms: Abductive Logic Programming, Inductive Logic Programming, Constraint Logic Programming, Answer-Set Programming. Applications: Databases, Data Integration and Federation, Software Engineering, Natural Language Processing, Web and Semantic Web, Agents, Artificial Intelligence, Bioinformatics.