Precise partial discharge (PD) detection is a key factor in anticipating insulation failures. The continuous efforts of researchers have led to the design of a variety of algorithms focusing on PD pattern classification. However, the trade-off between features taken up for classification and the detection rate continues to pose considerable challenges in terms of feature selection from acquired data, increased computing time, and so on. In this article, a Hypergraph (HG) based improved Random Forest (RF) algorithm by employing the Recursive Feature Elimination (RFE) algorithm (HG-RF-RFE), has been developed for PD source classification. HG representation of data is considered for obtaining statistical features, which turn out to be a subset of a set of all hyper edges called Hyper statistical features (Helly, Non-Helly, and Isolated hyper edges). HG-RF-RFE takes hyper statistical features and hyper edges as features for classification. The algorithm’s efficiency is tested against noise-free PD data obtained from SASTRA High Voltage Laboratory, and large-sized noisy PD data obtained from High-Voltage Research and Test Laboratory at Universidad Tecnica Federico Santa Maria (LIDAT). The robustness of the proposed algorithm is tested with both time and phase domain PD features using the Mathews Correlation Coefficient (MCC), harmonic mean-based feature Score (F1 Score) as evaluation metrics, and by k-fold validation technique. The proposed HG-RF-RFE achieved 98.8% accuracy with minimal features and significantly reduces computation time without compromising accuracy. It is worth mentioning that the HG-RF-RFE technique is superior to many state of the art algorithms in terms of feature elimination and classification accuracy.