The traditional altimetry satellites based on pulse-limited radar altimeter only calculate along-track deflection of the vertical (DOV), which results in poorer precision of the prime vertical component than that of the meridian component and limits the precision of the marine gravity field inversion. We expect an improvement in the higher precision prime vertical component using the Ice, Cloud and land Elevation Satellite 2 (ICESat-2) sea surface height (SSH) data. In this paper, the 2′ × 2′ gridded DOVs derived from along-beam DOVs, cross-beam DOVs, and joint along-cross beam DOVs in the South China Sea (SCS; 0°–23°N, 103°–120°E) are calculated with the weighted least squares method, respectively. The inverse Vening–Meinesz (IVM) formula is applied to derive 2′ × 2′ gravity anomalies over the SCS from ICESat-2-derived gridded DOVs. In addition, the XGM2019e_2159-DOV and SIO V31.1-DOV models are used to assess the precision of the gridded DOVs. The XGM2019e_2159-GRA, SIO V31.1-GRA models, and ship-borne gravity anomalies are also adopted to evaluate the quality of gravity anomalies. The results show that the gridded DOVs calculated by the joint along-cross beam DOVs have the highest precision among the three gridded DOVs determined by ICESat-2. The precision of difference between gravity anomalies derived from the joint along-cross beam DOV and the above verification data are higher than those derived from the along-beam and cross-beam DOVs. We conclude that the joint along-cross beam DOV can effectively improve the precision of the gridded DOV, which is conducive to the inversion of a high-precision marine gravity field.