One step at a time: An efficient approach to query-based ontology debugging

Abstract

When ontologies reach a certain size and complexity, faults such as inconsistencies, unsatisfiable classes or wrong entailments are hardly avoidable. Locating the incorrect axioms that cause these faults is a hard and time-consuming task. Addressing this issue, several techniques for a semi-automatic fault localization in ontologies have been proposed and extensively studied. One class of these approaches involve a human expert who provides answers to system-generated queries about the intended (correct) ontology in order to reduce the possible fault locations. To suggest as informative questions as possible, existing methods draw on various algorithmic optimizations as well as heuristics. However, these computations are often based on certain assumptions about the interacting expert.In this work, we demonstrate that these assumptions might not always be adequate and discuss consequences of their violations. In particular, we characterize a range of expert types with different query answering behavior and show that existing approaches are far from achieving optimal efficiency for all of them. In addition, we find that the cost metric adopted by state-of-the-art techniques might not always be realistic and that a change of metric has a decisive impact on the best choice of query answering strategy. As a remedy, we suggest a new – and simpler – type of expert question that leads to a stable fault localization performance for all analyzed expert types and effort metrics, and has numerous further advantages over existing techniques. Moreover, we present an algorithm which computes and optimizes this new query type in worst-case polynomial time and which is fully compatible with existing concepts (e.g., query selection heuristics) and infrastructure (e.g., debugging user interfaces) in the field.Comprehensive experiments on faulty real-world ontologies attest that the new querying method is substantially and statistically significantly superior to existing techniques both in terms of the number of necessary expert interactions and in terms of the query computation time. We find that relying on the new querying method can save an interacting expert more than 80% of their work, and can reduce the expert’s waiting time for the next query by more than three orders of magnitude. Beside these findings, we demonstrate that the efficiency of existing query-based tools can be significantly boosted by suggesting an appropriate query answering strategy to an expert; we also make recommendations in this regard. Further, we suggest optimal configurations of a debugger for situations where the new type of query is used.Remarkably, the proposed approach is not only applicable to ontologies, but to any monotonic knowledge representation language, and can even be adopted to solve general model-based diagnosis problems expressible using Reiter’s theory.