The interferogram containing the noises often affects the accuracy of phase retrieval, leading to the degradation of the phase imaging quality. To address this issue, a new interferogram blind denoising (IBD) method based on deep residual learning is proposed. In the presence of unknown noise levels, during the training, the deep residual convolutional neural networks (DRCNN) in the IBD approach is able to remove the latent clean interferogram implicitly, and then gradually establish the residual mapping relation in the pixel-level between the interferogram and the noises. With a well-trained DRCNN model, this algorithm can deal not only with the single-frame interferogram efficiently but also with the multi-frame phase-shifted interferograms collaboratively, while effectively retaining interferogram features related to phase retrieval. Simulation and experimental results demonstrate the feasibility and applicability of the proposed IBD method.