Calcium fluoride (CaF2), as a typical brittle optical material, has been applied in various fields due to its excellent optical properties. However, an existing challenge is how to non-destructively characterize the surface integrity of brittle materials, which may easily suffer from subsurface damage (SSD) during machining processes. This paper explores a novel snapshot surface-integrity test method based on single-exposure back-focal-plane (BFP) imaging of CaF2 within ellipsometric parameter spectra corresponding to an extensive range of incident angles. The relationship between the layer thicknesses of surface roughness, SSD, and ellipsometric spectra was analyzed by establishing a four-layer-subsurface optical model combined with quasi-Brewster angle technology. Through polishing CaF2 under different conditions and combining the measurement results of angle-resolved ellipsometry (ARE), it has been demonstrated that the study based on ARE is a successful non-destructive testing method for surface integrity, which could help to further elucidate the surface evolution mechanism of brittle material polishing processes.