At-wavelength metrology of X-ray optics plays a crucial role in evaluating the performance of optics under actual beamline operating conditions, enabling insitu diagnostics and optimization. Techniques utilizing a wavefront random modulator have gained increasing attention in recent years. However, accurately mapping the measured wavefront slope to a curved X-ray mirror surface when the modulator is downstream of the mirror has posed a challenge. To address this problem, an iterative method has been developed in this study. The results demonstrate a significant improvement compared with conventional approaches and agree with offline measurements obtained from optical metrology. We believe that the proposed method enhances the accuracy of at-wavelength metrology techniques, and empowers them to play a greater role in beamline operation and optics fabrication.