Previous research on transistor gate oxides reveals a clear link between hydrogen content and oxide breakdown. This has implications for redox-based resistive random access memory (ReRAM) devices, which exploit soft, reversible, dielectric breakdown, as hydrogen is often not considered in modeling or measured experimentally. Here quantitative measurements, corroborated across multiple techniques are reported, that reveal ReRAM devices, whether manufactured in a university setting or research foundry, contain concentrations of hydrogen at levels likely to impact resistance switching behavior. To the knowledge this is the first empirical measurement depth profiling hydrogen concentration through a ReRAM device. Applying a recently-developed Secondary Ion Mass Spectrometry analysis technique enables to measure hydrogen diffusion across the interfaces of SiOx ReRAM devices as a result of operation. These techniques can be applied to a broad range of devices to further understand ReRAM operation. Careful control of temperatures, precursors, and exposure to ambient during fabrication should limit hydrogen concentration. Additionally, using thin oxynitride or TiO2 capping layers should prevent diffusion of hydrogen and other contaminants into devices during operation. Applying these principles to ReRAM devices will enable considerable, informed, improvements in performance.