Cardiac Auscultation is widely used in the diagnosis of congenital heart disease (CHD) due to its non-invasive and cost-effective procedure. Heart sound analysis can provide effective auxiliary diagnosis information and aid in automatically screening patients. However, there are different signal spectrum characteristics in different auscultation locations, and there are no unified standards in the selection of auscultation locations during the heart sound analysis. This paper addresses the problem of auscultation locations and the selection of spectrum characteristics in the heart sound identification of CHD patients. 385 cases of normal and CHD heart sound signals were used to extract three groups of representative spectral features: Power Spectral Density (PSD), Mel-Frequency Cepstrum Coefficients (MFCCs) and Instantaneous Frequency Cumulative(IFC) from five different auscultation locations which are Aortic area(A), Pulmonic area(P), Tricuspid area(T), Mitral area(M), and Second aortic valve area(E). Significance detections based on p-value and Gaussian kernel support vector machine (SVM) were used to test these spectrum characteristics in five auscultation locations. The results show that the spectrum energy and the difference of auscultation areas concentrated within the frequency range of 20-150Hz. There is no statistical significance in terms of the spectrum characteristics of CHD in area A (p>0.05) compared with the statistical significance in other auscultation areas (p<0.01). The classification performance of the SVM method that uses spectrum characteristics of area E was the best overall. The experiment and analysis results show that, in CHD heart sound recognition, E-area signals have the best recognition effect, signals from TMP areas could be used as a reference, and A-area signals should be used with cautions. This discovery provides a practical guide to the clinical auscultation and signal processing of Congenital Heart Disease.