A Roadmap to Gradually Compare and Benchmark Description Logic Calculi

Abstract

DL reasoners were developed with cutting-edge performance, implementing plenty of specific optimization techniques over tableaux-based methods, which took over the field. However, promising methods may have been neglected in such a scenario, in which the tough competition is often focused on gains through optimizations. Therefore, perhaps there is still room available for “basic research” on DL reasoning. The purpose of this work is to stimulate research on trying out DL calculi other than tableaux. Such endeavors should be carried out by making a careful, detailed comparison between tableaux and other inference methods in a systematic way: first starting with simpler languages (like ALC) without any optimizations. Then gradually including optimizations and comparing them; and continuing these interactive steps: enhancing language expressivity, including optimizations, and testing until reaching the most expressive DL fragments such as SROIQ. The comparison can also be done by in terms memory usage and algorithm asymptotic analysis, with worst and average cases, etc. The rationale is identifying whether there are fragments which are more suitable to certain inference methods, as well as which aspects or constructs (e.g., the costliest combinations, which usually involve inverses, nominals, equalities, etc) are sensitive to which calculus.