Ternary cyclotomic polynomials having a large coefficient

Abstract

Let Φn(x) denote the nth cyclotomic polynomial. In 1968 Sister Marion Beiter conjectured that an(k), the coefficient of xk in Φn(x), satisfies |an(k)| ≦ (p + 1)/2 in case n = pqr with p < q < r primes (in this case Φn(x) is said to be ternary). Since then several results towards establishing her conjecture have been proved (for example |an(k)| ≦ 3p/4). Here we show that, nevertheless, Beiter's conjecture is false for every p ≧ 11. We also prove that given any ε > 0 there exist infinitely many triples (pj, qj, rj) with p1 < p2 < ⋯ consecutive primes such that |apjqjrj(nj)| > (2/3 – ε)pj for j ≧ 1.