Ultra-hot rocky super-Earths are thought to be sufficiently irradiated by their host star to melt their surface and allow for long-lasting magma oceans as a result. A number of processes have been proposed to explain how such planets may have retained the primordial hydrogen captured during their formation, while moving inward in the planetary system. The new generation of space telescopes such as the James Webb Space Telescope may provide observations that are precise enough to characterize the atmospheres and perhaps the interiors of such exoplanets. We used a vaporization model that calculates the gas-liquid equilibrium between the atmosphere (including hydrogen) and the magma ocean to compute the elemental composition of a variety of atmospheres with different quantities of hydrogen. We then used the elemental composition in a steady-state atmospheric model (ATMO) to compute the atmospheric structure and generate synthetic emission spectra. With this method, we were able to confirm previous results showing that silicate atmospheres exhibit a thermal inversion, with a notable emission peak of SiO at 9 μm. We compared our method to the literature on the inclusion of hydrogen in the atmosphere to show that hydrogen reduces the thermal inversion because of the formation of H2O, which has a strong greenhouse potential. However, planets that are significantly irradiated by their host star are sufficiently hot to dissociate H2O, thus also allowing them to maintain a thermal inversion. The observational implications are twofold: (1) H2O is more likely to be detected in colder atmospheres and (2) detecting a thermal inversion in hotter atmospheres does not a priori exclude the presence of H (in its atomic form). Due to the impact of H on the overall chemistry and atmospheric structure (and, thus, observations), we emphasize the importance of including volatiles in the calculation of the gas-liquid equilibrium. Finally, we provide a criterion to determine potential targets for observation in light of these findings.