Designing for scalability in a knowledge fusion system

Abstract

The knowledge reuse and fusion/transformation (KRAFT) project has defined a generic agent-based architecture to support knowledge fusion — the process of locating and extracting knowledge from multiple, heterogeneous on-line sources, and transforming it so that the union of the knowledge can be applied in problem-solving. KRAFT focuses on knowledge in the form of constraints expressed against an object data model defined by a shared ontology. KRAFT employs three kinds of agent: facilitators locate appropriate on-line sources of knowledge; wrappers transform heterogeneous knowledge to a homogeneous constraint interchange format; mediators fuse the constraints together with associated data to form a dynamically-composed constraint satisfaction problem, which is then passed to an existing constraint solver engine to compute solutions. The KRAFT architecture has been designed to be scalable to large numbers of agents; this paper describes the features of the architecture designed to support scalability. In particular, we examine static techniques that underpin the growth of large-scale KRAFT networks, and dynamic techniques that allow reorganisation of a KRAFT network as it increases in scale.