Human Activity Recognition (HAR) is an eminent area of research due to its extensive scope of applications in remote health monitoring, sports, smart home, and many more. Smartphone-based HAR systems use sensor data to infer human physical activities. Extraction of pertinent and essential features to identify human activities is a crucial but challenging task. Researchers continuously endeavor to extract pertinent and non-redundant features without compromising the classification accuracy. Smartphone sensor data generates high dimensional feature sets to recognize human physical activities. This work aims to build an efficient HAR model that extracts the most relevant features from the 3-axial accelerometer and gyroscope signal data and enhances the HAR system’s classification accuracy without data loss using time–frequency domain features. After feature extraction, we apply the Guided Regularized Random Forest (GRRF) feature selection method to determine the most pertinent and non-redundant features to reduce the time to recognize the human activities efficiently. After selecting the most relevant features, Support Vector Machine (SVM), Naive Bayes (NB), Random Forest (RF), and Decision Tree (DT) are used to identify various human physical activities. Using two different datasets, we investigate that GRRF selects relevant feature sets compared to two benchmark feature selection methods such as Relief-F and GRF, without compromising the recognition accuracy. This integration model with GRRF shows improved performance using all methods mentioned above. Our proposed strategy achieves higher accuracy values of 99.10% and 99.30% for SVM using two different datasets.