A normal variety X is called H-spherical for the action of the complex reductive group H if it contains a dense orbit of some Borel subgroup of H. We resolve a conjecture of Hodges–Yong by showing that their spherical permutations are characterized by permutation pattern avoidance. Together with results of Gao–Hodges–Yong this implies that the sphericality of a Schubert variety $$X_w$$ with respect to the largest possible Levi subgroup is characterized by this same pattern avoidance condition.