Continuous respiration monitoring is significant for real-life healthcare applications, but realizing it is extremely hard as wearable sensors are cumbersome and contact-free sensors largely fail to tolerate user movements. Meanwhile, tracking users indoors mostly demands user-held devices, while device-free localization can barely tell what and who it tracks. Fortunately, as both contact-free respiration monitoring and device-free localization may rely on Radio-Frequency (RF) sensing, fusing them together creates a novel system capable of continuously tracking users while recovering their fine-grained respiratory waveforms. To this end, we propose BreathCatcher as a continuous human respiration tracking system for indoor applications. To build this system, we employ commercial-grade compact radar pairs to capture RF reflections containing respiratory signals. We then propose a hybrid human respiration and position tracking algorithm to locate and identify respiratory signals from complex RF reflection mixtures. Finally, we design an encoder-decoder deep neural network driven by variational inference to recover fine-grained respiratory waveforms. Essentially, BreathCatcher cannot only obtain respiratory waveforms from multiple walking users, but also identify each user according to the latent properties of the respiratory signals. We evidently demonstrate the accuracy of both tracking and respiration monitoring via experiments involving 12 subjects and 80 man-hour data.