We propose a time-domain delay estimator that takes the slope of the best fit line crossing the origin in the instantaneous frequency-phase difference plane as the delay estimate. This formulation differs from existing phase-based estimators in two respects. First, we find the instantaneous frequency at all individual sample points, including large and abrupt spikes caused by destructive interference in the coherent scattering process. This differs from Loupas which finds a smoothed-out center frequency estimate within an observation window. We show that under high signal-to-noise ratio (SNR), the information from these spikes can be properly used. Second, we show that error ought to be considered as the deviation of the phase difference from the best fit line rather than deviation from the averaged phase difference. Without considering instantaneous frequency, phase-based estimators make the following two errors: samples with phase difference far away from the center frequency need not be errors as they naturally have large phase difference when their instantaneous frequency is large; samples with phase difference close to the center frequency may in fact be errors if their instantaneous frequency is large. We derive the Gauss-Markov least-squares best fit line and then propose an iterative variant that removes samples from the line-fitting process if its deviation from the best fit line is sufficiently large. The iterative version can reduce the effect of aliasing for larger delays and also further reduce the root-mean-square error (RMSE) of the estimate. Simulation studies using various bandwidth, SNR, and delay parameters indicate that iterative phase least squares (PLS) begins to outperform correlation phase Loupas at between SNR of 30 dB (for larger bandwidths and larger delays) and 60 dB (for smaller bandwidths and smaller delays). As SNR increases, iterative PLS can reach a 30- to 50-dB increase in performance over correlation phase Loupas with respect to RMSE in the most favorable conditions.