Analytic Combinatorics of Lattice Paths with Forbidden Patterns: Enumerative Aspects

Abstract

This article presents a powerful method for the enumeration of pattern-avoiding words generated by an automaton or a context-free grammar. It relies on methods of analytic combinatorics, and on a matricial generalization of the kernel method. Due to classical bijections, this also gives the generating functions of many other structures avoiding a pattern (e.g., trees, integer compositions, some permutations, directed lattice paths, and more generally words generated by a push-down automaton). We focus on the important class of languages encoding lattice paths, sometimes called generalized Dyck paths. We extend and refine the study by Banderier and Flajolet in 2002 on lattice paths, and we unify several dozens of articles which investigated patterns like peaks, valleys, humps, etc., in Dyck and Motzkin words. Indeed, we obtain formulas for the generating functions of walks/bridges/meanders/excursions avoiding any fixed word (a pattern). We show that the autocorrelation polynomial of this forbidden pattern (as introduced by Guibas and Odlyzko in 1981, in the context of regular expressions) still plays a crucial role for our algebraic functions. We identify a subclass of patterns for which the formulas have a neat form. En passant, our results give the enumeration of some classes of self-avoiding walks, and prove several conjectures from the On-Line Encyclopedia of Integer Sequences. Our approach also opens the door to establish the universal asymptotics and limit laws for the occurrence of patterns in more general algebraic languages.