The photoluminescence emission by mesoscopic condensed matter is ultimately dictated by the fine-structure splitting of the fundamental exciton into optically allowed and dipole-forbidden states. In epitaxially grown semiconductor quantum dots, nonradiative equilibration between the fine-structure levels is mediated by bulk acoustic phonons, resulting in asymmetric spectral broadening of the excitonic luminescence. In isolated colloidal quantum dots, spatial confinement of the vibrational motion is expected to give rise to an interplay between the quantized electronic and phononic degrees of freedom. In most cases, however, zero-dimensional colloidal nanocrystals are strongly coupled to the substrate such that the charge relaxation processes are still effectively governed by the bulk properties. Here we show that encapsulation of single colloidal CdSe/CdS nanocrystals into individual organic polymer shells allows for systematic vibrational decoupling of the semiconductor nanospheres from the surroundings. In contrast to epitaxially grown quantum dots, simultaneous quantization of both electronic and vibrational degrees of freedom results in a series of strong and narrow acoustic phonon sidebands observed in the photoluminescence. Furthermore, an individual analysis of more than 200 compound particles reveals that enhancement or suppression of the radiative properties of the fundamental exciton is controlled by the interaction between fine-structure states via the discrete vibrational modes. For the first time, pronounced resonances in the scattering rate between the fine-structure states are directly observed, in good agreement with a quantum mechanical model. The unambiguous assignment of mediating acoustic modes to the observed scattering resonances complements the experimental findings. Thus, our results form an attractive basis for future studies on subterahertz quantum opto-mechanics and efficient laser cooling at the nanoscale.