Extended pattern avoidance

Abstract

A 0–1 matrix is said to be extendably τ-avoiding if it can be the upper left corner of a τ-avoiding permutation matrix. This concept arose in Eriksson and Linusson (Electron. J. Combin. 2 (1995) R6) where the surprising result that the number of extendably 321-avoiding rectangles are enumerated by the ballot numbers was proved. Here we study the other five patterns of length three. The main result is that the six patterns of length three divide into only two cases, no easy symmetry can explain this. Another result is that the Simion–Schmidt–West bijection for permutations avoiding patterns 12 τ and 21 τ works also for extended pattern avoidance. As an application, we use the results on extended pattern avoidance to prove a sequence of refinements on the enumeration of permutations avoiding patterns of length 3. The results and proofs use many properties and refinements of the Catalan numbers.