Abstract We prove several rigidity results about the centralizer of a smooth diffeomorphism, concentrating on two families of examples: diffeomorphisms with transitive centralizer, and perturbations of isometric extensions of Anosov diffeomorphisms of nilmanifolds. We classify all smooth diffeomorphisms with transitive centralizer: they are exactly the maps that preserve a principal fiber bundle structure, acting minimally on the fibers and trivially on the base. We also show that for any smooth, accessible isometric extension $f_{0}\colon M\to M$ of an Anosov diffeomorphism of a nilmanifold, subject to a spectral bunching condition, any $f\in \textrm{Diff}^{\infty }(M)$ sufficiently $C^{1}$-close to $f_{0}$ has centralizer a Lie group. If the dimension of this Lie group equals the dimension of the fiber, then $f$ is a principal fiber bundle morphism covering an Anosov diffeomorphism. Using the results of this paper, we classify the centralizer of any partially hyperbolic diffeomorphism of a $3$-dimensional, nontoral nilmanifold: either the centralizer is virtually trivial, or the diffeomorphism is an isometric extension of an Anosov diffeomorphism, and the centralizer is virtually ${{\mathbb{Z}}}\times{{\mathbb{T}}}$.