Being one of the five most commonly used nondestructive testing (NDT) routines, ultrasonic testing (UT) is under fast development in recent years, with more attention being focused on quantitative testing and nondestructive evaluation (NDE). In the evaluation of pressure vessels and piping, UT is utilized not only in manufacturing quality controlling, but also in-service monitoring and residual life prediction, such as the inspection of welded joints, monitoring of crack propagation, evaluation of materials property deterioration. In ultrasonic NDE and quantitative NDT, one of the main factors disturbing the reliability and accuracy of test is noise encountered during inspection. At present, a digitized instrument is increasingly preferred in practice. Considering every step, from wave emitting to digitalization of analog signal, the following noises usually emerge—the electronic circuit noise of instrument; structure noise caused by grain boundaries of material under testing; ringing noise due to oscillation of probe; digital noise of finite-word digital system; pulse noise by virtue of fluctuation of inspection circumstance. Among these, the most serious is the structure noise encountered in the testing of coarse-grained austenite stainless steel, which affects the defect signal, making the least detectable defect size increase. In the present paper the characteristics, detriment and elimination algorithms of electrical noise, pulse noise, ringing noise and structure noise in a digital ultrasonic NDE system are discussed. A physical model of the digital ultrasonic NDE system is established, and noises are classified into different categories from the viewpoint of the model. The characteristics of electrical noise are analyzed; an algorithm of extremum filtering constructed to eliminate the pulse noise; high-pass filter and wavelet packet are employed to process ringing noise; the features of structure noise are studied and it is de-noised by wavelet transform (WT) and wavelet packet transform (WPT). The results obtained from real-world signal show that the electrical noise can be taken approximately as white noise with a Gaussian distribution. The algorithm of extremum filtering can filter the pulse noise without any effect on other information in the signal. Wavelet packet algorithm is more suitable for the elimination of long-term ringing noise than high-pass filter under the condition of less loss of defect echo. Processing the signals of coarse-grained austenite stainless steel samples with defects by use of WT and WPT concludes that as structure noise can be divided into certain frequency bands, generally different from those of defect echoes by WT and WPT, the defect can be pointed out, and the signal-to-noise ratio enhanced substantially after the threshold processing of frequency components of signals followed reconstruction.