High intensity focused ultrasound (HIFU) can produce cavitation, which requires monitoring for specific applications such as sonoporation, targeted drug delivery or histotripsy. Passive acoustic mapping has been proposed in the literature as a method for monitoring cavitation, but it lacks spatial resolution, primarily in the axial direction, due to the absence of a time reference. This is a common issue with passive imaging compared to standard pulse-echo ultrasound. In order to improve the axial resolution, we propose an adaptation of the Cross spectral Matrix Fitting (CMF) method for passive cavitation imaging, which is based on the resolution of an inverse problem with different regularizations that promote sparsity in the reconstructed cavitation maps: Elastic Net (CMF-ElNet) and sparse Total Variation (CMF-spTV). The results from both simulated and experimental data are presented and compared to state-of-the-art approaches, such as the frequential Delay-and-Sum (DAS) and the frequential Robust Capon Beamformer (RCB). We show the interest of the method for improving the axial resolution, with an axial Full Width Half Maximum (FWHM) divided by 3 and 5 compared to RCB and DAS, respectively. Moreover, CMF based methods improve Contrast-to-Noise Ratio (CNR) by more than 15 dB in experimental conditions compared to RCB. We also show the advantage of the sparse Total Variation prior over Elastic Net when dealing with cloud shaped cavitation sources, that can be assumed as sparse grouped sources.