In recent years, MIMO high-frequency surface wave radar (MIMO-HFSWR) plays an increasingly important role in sea surveillance because of its wide-area surveillance capabilities. In signal processing, the coherent targets with the same distance and speed can only be distinguished in the angle dimension. However, HF radar’s angular resolution is poor because of the restrictions of the aperture, which causes aliasing of targets on the range–angle (RA) spectrum. Traditional super-resolution algorithms, such as MUSIC, are also inapplicable because of the targets’ coherence. Therefore, the spatial smoothing algorithm is usually used to realize the decoherence of echo at the cost of array aperture. The loss of array aperture makes the algorithm fail when there are too many targets in the echo. Aiming at this problem, this paper proposes an iterative calculation method (iterative calculation via weight vector orthogonal decomposition, IC-WORD) to estimate the angle of multiple coherent targets. Through the verification of simulation, it is proved that IC-WORD has better performance than the traditional spatial smoothing algorithm. More specifically, IC-WORD can correct spectral peak shift, improve angle measurement accuracy, and still operate stably under the condition of multiple coherent targets. This paper also uses the measured data from one of the most advanced HFSWR stations in China to validate the algorithm, which makes the paper have strong practical significance.