A framework for verification, validation, and refinement of knowledge bases: The VVR system

Abstract

Verification and validation (V&V) of Knowledge Bases (KBs) are two sides of the same coin: one is intended to assure the structural correctness of the KB, while the other is intended to assure the functional correctness of the domain model embodied in the KB. Knowledge base refinement aims to appropriately revise the KB if a structural or functional error is detected during the V&V process. This paper presents a uniform framework for verification, validation and refinement of KBs represented as sets of production rules, called the VVR system. It incorporates a contradiction-tolerant truth maintenance system (CTMS) for performing both verification and validation analyses, and some simple explanation-based learning techniques for guiding the refinement process. Verification analysis consists of detecting and correcting the main types of structural anomalies: circular rules, redundant rules, inconsistent rules, and inconsistent data, and checks the KB for completeness and violated semantic constraints. In terms of validation, given a set of test cases, the VVR system is capable of detecting and correcting functional errors caused by overgeneralization and/or overspecialization of the KB. If the set of test cases is not available, the VVR system can generate synthetic test cases intended to help the user evaluate KBS performance. © 1994 John Wiley & Sons, Inc.