In digital holographic measurement, when light waves pass through inhomogeneous media or surfaces, speckle noise is generated, resulting in random, granular light and dark spots in the hologram, which greatly reduces the image quality. Therefore, in order to improve the image quality of holographic reconstruction, a noise reduction method based on the BM3D improved convolutional neural network (CNN) is proposed in this paper. Firstly, the similarity and important statistical information between blocks can be obtained by using BM3D. Then, the denoising convolutional neural network (DnCNN) is used to learn the relationship between the noise of a large number of samples and the noise image, and further purify the image to retain the details for a better denoising effect. Finally, a reflective off-axis digital holographic optical path system is constructed to collect the holograms of the test samples, and the reconstructed images are obtained by the Fresnel diffraction method to constitute a dataset with the simulated holographic reconstructed images to validate the proposed method in this paper, compared to the other methods, such as DnCNN, convolutional blind denoising network (CBDNet), BM3D, and Wiener filtering. The experimental results of qualitative and quantitative analyses show that the proposed method combines the advantages of traditional algorithms and deep learning, significantly enhances the robustness of the system, optimizes the denoising performance, and preserves the details of the reconstructed image to the greatest extent.