Ambiguous classes in [formula omitted]-calculi hierarchies

Abstract

A classical result by Rabin states that if a set of trees and its complement are both Büchi definable in the monadic second order logic, then these sets are weakly definable. In the language of μ -calculi, this theorem asserts the equality between the complexity classes Σ 2 ∩ Π 2 and Comp ( Σ 1 , Π 1 ) of the fixed-point alternation-depth hierarchy of the μ -calculus of tree languages. It is natural to ask whether at higher levels of the hierarchy the ambiguous classes Σ n + 1 ∩ Π n + 1 and the composition classes Comp ( Σ n , Π n ) are equal, and for which μ -calculi. The first result of this paper is that the alternation-depth hierarchy of the games μ -calculus—whose canonical interpretation is the class of all complete lattices—enjoys this property. More explicitly, every parity game which is equivalent both to a game in Σ n + 1 and to a game in Π n + 1 is also equivalent to a game obtained by composing games in Σ n and Π n . The second result is that the alternation-depth hierarchy of the μ -calculus of tree languages does not enjoy the property. Taking into account that any Büchi definable set is recognized by a nondeterministic Büchi automaton, we generalize Rabin's result in terms of the following separation theorem: if two disjoint languages are recognized by nondeterministic Π n + 1 automata, then there exists a third language recognized by an alternating automaton in Comp ( Σ n , Π n ) containing one and disjoint from the other. Finally, we lift the results obtained for the μ -calculus of tree languages to the propositional modal μ -calculus: ambiguous classes do not coincide with composition classes, but a separation theorem is established for disjunctive formulas.