Cardiovascular diseases are one of those diseases which should be considered to be serious health threats for humans since today it is seen that majority of deaths are caused by heart disorders. There are advanced techniques such as electrocardiograph (ECG), computerized tomography in order to detect heart disorders. However, these methods are usually costly and time-consuming. Besides, auscultating the heart sounds by doctors for the detection of deformation of the heart sounds is an alternative method. Nevertheless, analysis of heart sounds performed by auscultating the patient mostly depends on the practitioner’s expertise and skills. Therefore, record and analysis of heart sounds via computers is a preferred method. In this paper, a new heart sound classification model is proposed based on Local Binary Pattern (LBP) and Local Ternary (LTP) Pattern features and deep learning. In the first step of the model, One Dimensional Local Binary Pattern (1D-LBP) and One Dimensional Local Ternary Pattern (1D-LTP) are used to extract features from signals. In the second step, the features obtained as a result of the application of 1D-LBP and 1D-LTP are combined and a hybrid feature vector is obtained from a signal. In the third step, ReliefF based feature selection method is used to identify the most relevant features. Finally, the most discriminative features are given as input to the designed One Dimensional Convolutional Neural Network (1D-CNN) in order to complete the classification process. Experiments are made with the help of two popular datasets that are used in the context to determine the efficiency of distinct techniques. These datasets are PASCAL and PhysioNet 2016. We obtain 91.66% and 91.78% accuracy rates for the PASCAL and PhysioNet 2016 datasets respectively. As a consequence, when we compare our results that we obtain from the proposed method in this study with those of the up-to-date methods’ results used in the literature, it is obvious that our technique surpasses them according to the classification accuracy rate.