INDED: a distributed knowledge-based learning system

Abstract

The INDED (induction-deduction, pronounced "indeed") system performs rule discovery using the techniques of inductive logic programming, and accumulates and handles knowledge using a deductive nonmonotonic reasoning engine. Using the language of logic programming, we use a hypergraph to represent the knowledge base from which rules are mined. Because the hypergraph gets inordinately large in INDED's serial version, we have devised a parallel implementation that creates smaller subhypergraphs. We investigate the integrity and meaning of decomposing data so that many processors can attempt to learn the same global pattern simultaneously (although locally, each discovered pattern is usually unique). Many data decompositions are fallacious and lead to nonsensical discovered rules. Some data, however, exhibits enough mutual exclusivity to render it partitionable among processors. This examination of partitionability of data has been the underlying driving force of this work. A great deal of work has been done in parallelizing unguided discovery of association rules. The novel aspects of our work include the parallelization of both a nonmonotonic reasoning system and an inductive logic programming learner. We describe the schemes we have explored and are exploring in this pursuit. We also present our data-partitioning algorithms that we based on data locality.