The prevention and diagnosis of cardiovascular diseases have become one of the primary problems in the medical community since the mortality of this kind of diseases accounts for 31% of global deaths in 2016. Heart sound, which is an important physiological signal of human body, mainly comes from the pulsing of cardiac structures and blood turbulence. The analysis of heart sounds plays an irreplaceable role in early diagnosis of heart disease since they contain a large amount of pathological information about each part of human heart. Heart sounds can be detected and recorded by Phonocardiogram (PCG). As a noninvasive method to detect and diagnose heart disease, PCG signals have been paid more and more attention by researchers. In this paper, a novel envelope extraction model is proposed and used to estimate the cardiac cycle of each PCG signal. We present a strategy combining empirical mode decomposition (EMD) technique and the proposed envelope model to extract the time-domain features. After applying EMD process to each PCG signal, the second intrinsic mode function is chosen for further analysis. Based on the proposed envelope model, the cardiac cycles of PCG signals can be estimated and then the time-domain features can be extracted. Combining with the frequency-domain features and wavelet-domain features, the feature vectors are obtained. Finally, the support vector machine (SVM) classifier is used to detect the normal and abnormal PCG signals. Two public datasets are used to test our framework in this paper. And classification accuracies of more than [Formula: see text] on both datasets show the effectiveness of the proposed model.