AbstractIn PDE‐constrained shape optimization, a lot of computational effort is used to update the geometry at every iteration step. This iterative process can result in loss of element quality and degeneracy of the underlying mesh, which is particularly relevant for applications involving large deformations of some parts of the domain. To avoid remeshing, we model the mesh deformation using the method of mappings. We use an extension equation that maps a boundary control variable to a deformation field defined over the entire domain. By using the nonlinear extension operator proposed in our previous work, we increase the set of reachable shapes, allowing us to model large deformations. We discuss how the choice of parameters of the extension equations affects the mesh quality and we study the influence of the extension factor on the convergence properties of the iterative solvers.