The electrocardiogram (ECG) signal is susceptible to interference from unknown noises during the acquisition process due to their low frequency and amplitude, resulting in the loss of significant information in the signals. Recent advancements in deep learning models have shown promising results in signal processing. However, these models lack robustness against various types of noise and often overlook the gradient difference between denoised and original signals. In this study, we propose a novel deep learning denoising method based on an attention half instance normalization block (AHIN block) and a gradient difference max loss function (GDM Loss). Our approach consists of two stages: firstly, we input the noisy ECG signal to obtain a denoised version; secondly, we reconstruct the denoised signal by fusing preliminary results from the first stage while correcting waveform distortions caused by initial denoising to minimize information loss. Additionally, we introduce a new loss function that considers differences between slopes of the denoised ECG signal and clean ECG signal. To validate our proposed method's effectiveness, extensive experiments were conducted on both our model architecture and loss function compared with other state-of-the-art methods. Results demonstrate that our approach achieves excellent performance in terms of both signal-to-noise ratio (SNR) and root-mean-square error (RMSE). The proposed noise reduction method improves 8.86%, 12.05% and 10.50% respectively under BW, MA and EM noise.