Observations by the current generation of gravitational-wave detectors have been pivotal in expanding our understanding of the universe. Although tens of exciting compact binary mergers have been observed, neutron star-black hole (NSBH) mergers remained elusive until they were first confidently detected in 2020. The number of NSBH detections is expected to increase with sensitivity improvements of the current detectors and the proposed construction of new observatories over the next decade. In this work, we explore the NSBH detection and measurement capabilities of these upgraded detectors and new observatories using the following metrics: network detection efficiency and detection rate as a function of redshift, distributions of the signal-to-noise ratios, the measurement accuracy of intrinsic and extrinsic parameters, the accuracy of sky position measurement, and the number of early-warning alerts that can be sent to facilitate the electromagnetic follow-up. Additionally, we evaluate the prospects of performing multimessenger observations of NSBH systems by reporting the number of expected kilonova detections with the Vera C. Rubin Observatory and the Nancy Grace Roman Space Telescope. We find that as many as $\mathcal{O}(10)$ kilonovae can be detected by these two telescopes every year, depending on the population of the NSBH systems and the equation of state of neutron stars.