Traditionally, heart murmurs are diagnosed through cardiac auscultation, which requires specialized training and experience. The purpose of this study is to predict patients' clinical outcomes (normal or abnormal) and identify the presence or absence of heart murmurs using phonocardiograms (PCGs) obtained at different auscultation points. A semi-supervised model tailored to PCG classification is introduced in this study, with the goal of improving performance using time–frequency deep features. The study begins by investigating the behavior of PCGs in the time–frequency domain, utilizing the Stockwell transform to convert the PCG signal into two-dimensional time–frequency maps (TFMs). A deep network named AlexNet is then used to derive deep feature sets from these TFMs. In feature reduction, redundancy is eliminated and the number of deep features is reduced to streamline the feature set. The effectiveness of the extracted features is evaluated using three different classifiers using the CinC/Physionet challenge 2022 dataset. For Task I, which focuses on heart murmur detection, the proposed approach achieved an average accuracy of 93%, sensitivity of 91%, and F1-score of 91%. According to Task II of the CinC/Physionet challenge 2022, the approach showed a clinical outcome cost of 5290, exceeding the benchmark set by leading methods in the challenge.