Temperature mapping by in situ thermoreflectance thermal imaging (TRTI) or midwave infrared spectroscopy has played an important role in understanding the origins of threshold switching and the effect of insulator-metal transitions in oxide-based memrsitive devices. In this study, we use scanning thermal microscopy (SThM) as an alternative thermal mapping technique that offers high spatial resolution imaging (∼100 nm) and is independent of material. Specifically, SThM is used to map the temperature distribution in NbOx-based cross-bar and nanovia devices with Pt top electrodes. The measurements on cross-bar devices reproduce the current redistribution and confinement processes previously observed by TRTI but without the need to coat the electrodes with a material of high thermo-reflectance coefficient (e.g., Au), while those on the nanovia devices highlight the spatial resolution of the technique. The measured temperature distributions are compared with those obtained from physics-based finite-element simulations and suggest that thermal boundary resistance plays an important role in heat transfer between the active device volume and the top electrode.