Testing LTL formula translation into Büchi automata

Abstract

Model checkers are often used to verify critical systems, and thus a lot of effort should be put on ensuring their reliability. We describe techniques for testing linear temporal logic (LTL) model checker implementations, focusing especially on LTL-to-Buchi automata translation. We propose a randomized testing approach based on the cross-comparison of independent translator implementations, and present methods for test failure analysis. We give experimental results using an automated tool implementing the testing methodology. This method has helped to detect errors in model checking tools such as Spin. We also propose extending LTL model checkers with a simple counterexample validation algorithm.