AbstractA modified RELAX algorithm based on iterative coarray‐domain beamforming for fast source direction‐of‐arrival (DOA) estimation with fully augmentable sparse arrays is proposed. The authors exploit the deterministic centralised nature of the noise in the coarray domain and the Hermitian symmetry of the spatial autocorrelation function to efficiently incorporate source number estimation within the iterative framework. In doing so, the proposed algorithm allows low‐complexity, fast DOA estimation of more sources than sensors, without resorting to computationally expensive implementations of source number estimation using information theoretic criteria. Three variants of the proposed algorithm are presented, each differing in terms of the specific method employed for source number estimation. Extensive simulations are performed with a minimum redundancy array to compare and contrast the performance of the three variants in terms of their accuracy in estimating the number of sources in the field‐of‐view of the array. The results demonstrate that the modified RELAX algorithm can provide accurate estimates of the number and directions of sources, especially when the number of uncorrelated sources is equal to or higher than the number of sensors.