Two well-known polytopes whose vertices are indexed by permutations in the symmetric group Sn are the permutohedron Pn and the Birkhoff polytope Bn. We consider polytopes Pn(Π) and Bn(Π), whose vertices correspond to the permutations in Sn avoiding a set of patterns Π. For various choices of Π, we explore the Ehrhart polynomials and h∗-vectors of these polytopes as well as other aspects of their combinatorial structure.For Pn(Π), we consider all subsets Π⊆S3 and are able to provide results in most cases. To illustrate, Pn(123,132) is a Pitman–Stanley polytope, the number of interior lattice points in Pn(132,312) is a derangement number, and the normalized volume of Pn(123,231,312) is the number of trees on n vertices.The polytopes Bn(Π) seem much more difficult to analyze, so we focus on four particular choices of Π. First we show that the Bn(231,321) is exactly the Chan–Robbins–Yuen polytope. Next we prove that for any Π containing {123,312} we have h∗(Bn(Π))=1. Finally, we study Bn(132,312) and B˜n(123), where the tilde indicates that we choose vertices corresponding to alternating permutations avoiding the pattern 123. In both cases we use order complexes of posets and techniques from toric algebra to construct regular, unimodular triangulations of the polytopes. The posets involved turn out to be isomorphic to the lattices of Young diagrams contained in a certain shape, and this permits us to give an exact expression for the normalized volumes of the corresponding polytopes via the hook formula. Finally, Stanley’s theory of (P,ω)-partitions allows us to show that their h∗-vectors are symmetric and unimodal.Various questions and conjectures are presented throughout.
Read full abstract