In unmanned aerial vehicle (UAV)-based time difference of arrival (TDOA) positioning technique, baselines are limited due to communication constraints. In this case, the accuracy is highly sensitive to the TDOA measurements’ error. This article primarily addresses the problem of short-baseline high-precision time synchronization and TDOA measurement. We conducted a detailed analysis of error models in TDOA systems, considering both the time and phase measurement. We utilize the frequency division wireless phase synchronization technique in TDOA systems. Building upon this synchronization scheme, we propose a novel time delay estimation method that relies on phase measurements based on the integer least squares method. The performance of this method is demonstrated through Monte Carlo simulations and outdoor experiments. The standard deviations of synchronization and TDOA measurements in experiments are 1.12 ps and 1.66 ps, respectively. Furthermore, the circular error probable (CEP) accuracy is improved from 0.33%R to 0.02%R, offering support for the practical application of distributed short-baseline high-precision passive location techniques.