We study the nonlinear stability of the Cauchy horizon in the interior of extremal Reissner-Nordstr\"om black holes under spherical symmetry. We consider the Einstein-Maxwell-Klein-Gordon system such that the charge of the scalar field is appropriately small in terms of the mass of the background extremal Reissner-Nordstr\"om black hole. Given spherically symmetric characteristic initial data which approach the event horizon of extremal Reissner-Nordstr\"om sufficiently fast, we prove that the solution extends beyond the Cauchy horizon in $C^{0,\frac{1}{2}}\cap W^{1,2}_{loc}$, in contrast to the subextremal case (where generically the solution is $C^0\setminus (C^{0,\frac{1}{2}}\cap W^{1,2}_{loc}))$. In particular, there exist non-unique spherically symmetric extensions which are moreover solutions to the Einstein-Maxwell-Klein-Gordon system. Finally, in the case that the scalar field is chargeless and massless, we additionally show that the extension can be chosen so that the scalar field remains Lipschitz.