We study monolayer WSe2 using ultrafast electron diffraction. We introduce an approach to quantitatively extract atomic-site-specific information, providing an element-specific view of incoherent atomic vibrations following femtosecond excitation. Via differences between W and Se vibrations, we identify stages in the nonthermal evolution of the phonon population. Combined with a calculated phonon dispersion, this element specificity enables us to identify a long-lasting overpopulation of specific optical phonons and to interpret the stages as energy transfer processes between specific phonon groups. These results demonstrate the appeal of resolving element-specific vibrational information in the ultrafast time domain.