We study how the viscosity of neutron star (NS) matter affects the distribution of tilt angles ($\chi$) between the spin and magnetic axes in young pulsars. Under the hypothesis that the NS shape is determined by the magnetically-induced deformation, and that the toroidal component of the internal magnetic field exceeds the poloidal one, we show that the dissipation of precessional motions by bulk viscosity can naturally produce a bi-modal distribution of tilt angles, as observed in radio/$\gamma$-ray pulsars, with a low probability of achieving $\chi \sim (20^\circ - 70^\circ)$ if the interior B-field is $\sim (10^{11} - 10^{15})$~G and the birth spin period is $\sim 10 - 300$~ms. As a corollary of the model, the idea that the NS shape is solely determined by the poloidal magnetic field, or by the centrifugal deformation of the crust, is found to be inconsistent with the tilt angle distribution in young pulsars. When applied to the Crab pulsar, with $\chi \sim 45^\circ - 70^\circ$ and birth spin $\sim$ 20 ms, our model implies that: (i) the magnetically-induced ellipticity is $\epsilon_B \gtrsim 3 \times 10^{-6}$; (ii) the measured positive $\dot{\chi} \sim 3.6 \times 10^{-12}$ rad s$^{-1}$ requires an additional viscous process, acting on a timescale $\lesssim 10^4$ yrs. We interpret the latter as crust-core coupling via mutual friction in the superfluid NS interior. One critical implication of our model is a GW signal at (twice) the spin frequency of the NS, due to $\epsilon_B \sim 10^{-6}$. This could be detectable by Advanced LIGO/Virgo operating at design sensitivity.