The quantitative characterization of the local permittivity of transition metal dichalcogenide (TMD) materials is essential to estimate both their electrostatic properties and further achieve electronic applications. Electrostatic force microscopy (EFM) can achieve local, non-destructive, and high-resolution characterization of the dielectric properties of materials. However, in EFM, the probe geometry and long-range nature of electrostatic forces make the effective probe region that interacts with the specimen complex, posing daunting challenges to accurately measuring the dielectric property. Here, we present a systematic analysis of the impact of probe geometry on the EFM measurement and quantification of the dielectric constant. Then, a corrected methodology to partly remove such a geometric effect is developed. The proposed methodology has been validated by using finite element numerical calculations and successfully applied to quantitatively characterize the dielectric performance of MoS2 and WSe2 nanosheets. The results show that the dielectric constant determined by the corrected method is insensitive to the geometric parameters of the probe, especially the probe height and cantilever size. This work provides a more accurate and universal way to estimate the dielectric constants, and opens new opportunities in the study of the electrostatics of 2D materials that is essential for their promised applications.