The quantum mass shift of the soliton in the Skyrme model has been calculated from all nonzero modes. The calculations were carried out using a method applied earlier to the sine-Gordon model. The mass shifts do not depend on the baryonic spin, therefore they are the same for the nucleon and the $\ensuremath{\Delta}$. Our model parameters are the pion decay constant ${F}_{\ensuremath{\pi}}$, with its experimental value, and the pionic mass ${m}_{\ensuremath{\pi}}$, once in the chiral limit ${m}_{\ensuremath{\pi}}=0$ and also with the experimental value. We justify taking for the Skyrme parameter $e$ the experimental value of ${g}_{\ensuremath{\rho}\ensuremath{\pi}\ensuremath{\pi}}$, the mesonic $\ensuremath{\rho}$-pion coupling constant. The results depend on the vibrational energy. In this case an energy cutoff must be introduced. Using renormalization considerations as in the nonlinear $\ensuremath{\sigma}$ model, we have chosen this cutoff to be $e{F}_{\ensuremath{\pi}}=1136$ MeV. Our results (for $e={g}_{\ensuremath{\rho}\ensuremath{\pi}\ensuremath{\pi}}=6.11$ and ${F}_{\ensuremath{\pi}}=186$ MeV) are ${M}_{N}=996$ MeV, ${M}_{\ensuremath{\Delta}}=1593$ MeV for ${m}_{\ensuremath{\pi}}=0$; ${M}_{N}=857$ MeV, ${M}_{\ensuremath{\Delta}}=1641$ MeV for ${m}_{\ensuremath{\pi}}=138$ MeV. The spin dependence and the energy contribution of the coupling term between vibrations and rotations are being considered here only in a qualitative way.