Analysis of generalized continued fraction algorithms over polynomials

Abstract

We study and compare natural generalizations of Euclid's algorithm for polynomials with coefficients in a finite field. This leads to gcd algorithms together with their associated continued fraction maps. The gcd algorithms act on triples of polynomials and rely on two-dimensional versions of the Brun, Jacobi–Perron and fully subtractive continued fraction maps, respectively. We first provide a unified framework for these algorithms and their associated continued fraction maps. We then analyse various costs for the gcd algorithms, including the number of iterations and two versions of the bit-complexity, corresponding to two representations of polynomials (the usual and the sparse one). We also study the associated two-dimensional continued fraction maps and prove the invariance and the ergodicity of the Haar measure. We deduce corresponding estimates for the costs of truncated trajectories under the action of these continued fraction maps, obtained thanks to their transfer operators, and we compare the two models (gcd algorithms and their associated continued fraction maps). Proving that the generating functions appear as dominant eigenvalues of the transfer operator allows indeed a fine comparison between the models.