A lightweight infrared image denoising method based on adversarial transfer learning is proposed. The method adopts a generative adversarial network (GAN) framework and optimizes the model through a phased transfer learning strategy. In the initial stage, the generator is pre-trained using a large-scale grayscale visible light image dataset. Subsequently, the generator is fine-tuned on an infrared image dataset using feature transfer techniques. This phased transfer strategy helps address the problem of insufficient sample quantity and variety in infrared images. Through the adversarial process of the GAN, the generator is continuously optimized to enhance its feature extraction capabilities in environments with limited data. Moreover, the generator structure incorporates structural reparameterization technology, edge convolution modules, and progressive multi-scale attention block (PMAB), significantly improving the model's ability to recognize edge and texture features. During the inference stage, structural reparameterization further optimizes the network architecture, significantly reducing model parameters and complexity and thereby improving denoising efficiency. The experimental results of public and real-world datasets demonstrate that this method effectively removes additive white Gaussian noise from infrared images, showing outstanding denoising performance.