Power amplifier (PA) nonlinearity and in-phase and quadrature-phase (IQ) imbalance in the transmitter are of critical concern in modern communication systems due to their significant impact on the transmitter performance. In this paper, in order to eliminate their effects without increasing the complexity at the transmitter, we propose an iterative structure for IQ imbalance and PA nonlinearity joint estimation at the receiver side. The strategic design aims to mitigate the influence of one distortion before estimating the other, ultimately improving the accuracy of estimating each distortion. The proposed approach comprises two primary modules: an IQ imbalance estimation module and a PA nonlinearity estimation module. In addition, there are two supporting modules, namely the PA nonlinearity compensation module and the IQ imbalance reconstruction module, positioned before these two primary modules respectively. Furthermore, we have derived the theoretical Cramér–Rao lower bound (CRLB) for the joint distortion model of IQ imbalance and PA nonlinearity. Finally, extensive experiments are conducted to demonstrate the proposed algorithm's capability in estimating hardware distortions and mitigating them.