Despite the very high efficiency at blue wavelengths, nitride-based light-emitting diodes suffer from low efficiencies in the green and red spectral ranges. Different solutions have been proposed to mitigate the ``green gap,'' including the addition of an ($\mathrm{Al}$,$\mathrm{Ga}$)$\mathrm{N}$ capping layer to the quantum well structure. In this work, we show how the increased polarization field due to such an ($\mathrm{Al}$,$\mathrm{Ga}$)$\mathrm{N}$ capping layer has profound effects on carrier location and recombination probabilities. The proper choice of ($\mathrm{Al}$,$\mathrm{Ga}$)$\mathrm{N}$ composition leads to an increased electron-hole overlap as well as an enhanced confinement in the quantum well region. Therefore, the combination of band structure and electric field engineering can be a promising approach to mitigate the green gap problem.