Accurate bathymetric data is essential for marine, coastal ecosystems, and related studies. In the past decades, a lot of studies were investigated to obtain bathymetric data in shallow waters using satellite remotely sensed data. Satellite multispectral imagery has been widely used to estimate shallow water depths based on empirical models and physics-based models. However, the in-situ water depth information is essential (as the priori) to use the empirical model in a specific area, which limits its application, especially for remote reefs. In this study, the bathymetric maps in shallow waters were produced based on empirical models with only satellite remotely sensed data (i.e., the new ICESat-2 bathymetric points and Sentinel-2 multispectral imagery). The bathymetric points from the spaceborne ICESat-2 lidar were used in place of the in-situ auxiliary bathymetric points to train the classical empirical models (i.e., the linear model and the band ratio model). The bathymetric points were firstly extracted from noisy ICESat-2 raw data photons by an improved point cloud processing algorithm, and then were corrected for bathymetric errors (which were caused by the refraction effect in the water column, the refraction effect on the water surface, and the fluctuation effect on the water surface). With the trained empirical models and Sentinel-2 multispectral images, the bathymetric maps were produced for Yongle Atoll, in the South China Sea and the lagoon near Acklins Island and Long Cay, to the southeast of Bahama with four-date Sentinel-2 images. The bathymetry performance (including the accuracy and consistency of multi-date data) was evaluated and compared with the in-situ measurements. The results indicate that the bathymetric accuracy is well, and the RMSE is lower or close to 10% of the maximum depth for the two models with four-date images in two study areas. The consistency of multi-date data is well with the mean R2 of 0.97. The main novelties of this study are that the accuracy bathymetric points can be obtained from the ICESat-2 raw data using the proposed signal processing and error correction method, and using the ICESat-2 bathymetric points, the satellite multispectral imagery based on empirical models is no longer limited by local priori measurements, which were essential in previous studies. Hence, In the future, with the help of free and open-access satellite data (i.e., ICESat-2 data and Sentinel-2 imagery), this approach can be extended to a larger scale to obtain bathymetric maps in the shallow water of coastal areas, surroundings of islands and reefs, and inland waters.