Permutation groups with restricted stabilizers

Abstract

Fix a positive integer d and let Γd be the class of finite groups without sections isomorphic to the alternating group Ad. The groups in Γd were studied by Babai, Cameron and Pálfy in the 1980s and they determined bounds on the order of a primitive permutation group with this property, which have found a wide range of applications. Subsequently, results on the base sizes of such groups were also obtained. In this paper we replace the structural conditions on the group by restrictions on its point stabilizers, and we obtain similar, and sometimes stronger conclusions. For example, we prove that there is a linear function f such that the base size of any finite primitive group with point stabilizers in Γd is at most f(d). This generalizes a recent result of the first author on primitive groups with solvable point stabilizers. For non-affine primitive groups we obtain stronger results, assuming only that stabilizers of c points lie in Γd. We also show that if G is any permutation group of degree n whose c-point stabilizers lie in Γd, then |G|⩽((1+oc(1))d/e)n−1. This asymptotically extends and improves a dn−1 upper bound on |G| obtained by Babai, Cameron and Pálfy assuming G∈Γd.