We establish optimal local regularity results for vector-valued extremals and minimizers of variational integrals whose integrand is the squared distance function to a compact set K in matrix space \({{{\mathbb M}^{N \times n}}}\). The optimality is illustrated by explicit examples showing that, in the nonconvex case, minimizers need not be locally Lipschitz. This is in contrast to the case when the set K is suitably convex, where we show that extremals are locally Lipschitz continuous. The results rely on the special structure of the integrand and elementary Cordes–Nirenberg type estimates for elliptic systems.