Transfer learning of multi-source domain adaptation seems a promising way for fault diagnosis of roller element bearings under variable working conditions. Data imbalance affects the performance of multi-source domain adaptation greatly and is expected to be solved by GAN. However, GAN-based transfer learning diagnosis models suffer pattern collapse and training instability, leading to unsatisfying diagnosis results in practical engineering. This paper proposes a denoising diffusion multi-source domain adaptation model (DDMDA). The proposed model uses diffusion denoising, which has better performance and is simpler to train than GAN, to generate shifted source domains for solving the data imbalance problem. A new noise prediction structure in diffusion denoising named Utrans-net, is constructed to restore the data distribution in the shifted source domain. Also, a multiple-domain discriminator structure is designed to extract features from multiple source domains to solve the issue of variable working conditions. Advanced models are used in this paper to compare with the proposed model for validation. Experimental demonstrations show that the proposed model is superior to the comparison models with satisfying performance.