Estimating photometric redshifts (photo-z) of quasars is crucial for measuring cosmic distances and monitoring cosmic evolution. While numerous point estimation methods have successfully determined photo-z, they often struggle with the inherently ill-posed nature of the problem and frequently overlook significant morphological features in the probability density functions (pdfs) of photo-z, such as calibration and sharpness. To address these challenges, we introduce a cross-modal contrastive learning probabilistic model that employs adversarial training, contrastive loss functions, and a mixture density network to estimate the pdf of photo-z. This method facilitates the conversion between multiband photometric data attributes, such as magnitude and color, and photometric image features, while extracting features invariant across modalities. We utilize the continuous ranked probability score (CRPS) and the probability integral transform (PIT) as metrics to assess the quality of the pdf. Our approach demonstrates robust performance across various survey bands, image qualities, and redshift distributions. Specifically, in a comprehensive data set from the Sloan Digital Sky Survey and the Wide-field Infrared Survey Explorer (WISE) survey, our probabilistic model achieved a CRPS of 0.1187. Additionally, in a combined data set from SkyMapper and WISE, it reached a CRPS of 0.0035. Our probabilistic model also produced well-calibrated PIT histograms for both data sets, indicating nearly uniform distributions. We further tested our approach in classification tasks within the SkyMapper data set. Despite the absence of u, v, and g bands, it effectively distinguished between quasars, galaxies, and stars with an accuracy of 98.96%. This versatile method can be extended to other scenarios, such as analyzing extended sources like galaxies, across different surveys and varying redshift distributions.