Recent experiments suggest that ground state chemical reactivity can be modified when placing molecular systems inside infrared cavities where molecular vibrations are strongly coupled to electromagnetic radiation. This phenomenon lacks a firm theoretical explanation. Here, we employ an exact quantum dynamics approach to investigate a model of cavity-modified chemical reactions in the condensed phase. The model contains the coupling of the reaction coordinate to a generic solvent, cavity coupling to either the reaction coordinate or a non-reactive mode, and the coupling of the cavity to lossy modes. Thus, many of the most important features needed for realistic modeling of the cavity modification of chemical reactions are included. We find that when a molecule is coupled to an optical cavity it is essential to treat the problem quantum mechanically to obtain a quantitative account of alterations to reactivity. We find sizable and sharp changes in the rate constant that are associated with quantum mechanical state splittings and resonances. The features that emerge from our simulations are closer to those observed in experiments than are previous calculations, even for realistically small values of coupling and cavity loss. This work highlights the importance of a fully quantum treatment of vibrational polariton chemistry.