In the field of brain-computer interface (BCI) based on motor imagery (MI), multi-channel electroencephalography (EEG) data is commonly utilized for MI task recognition to achieve sensory compensation or precise human-computer interaction. However, individual physiological differences, environmental variations, or redundant information and noise in certain channels can pose challenges and impact the performance of BCI systems. In this study, we introduce a channel selection method utilizing Hybrid-Recursive Feature Elimination (H-RFE) combined with residual graph neural networks for MI recognition. This channel selection method employs a recursive feature elimination strategy and integrates three classification methods, namely random forest, gradient boosting, and logistic regression, as evaluators for adaptive channel selection tailored to specific subjects. To fully exploit the spatiotemporal information of multi-channel EEG, this study employed a graph neural network embedded with residual blocks to achieve precise recognition of motor imagery. We conducted algorithm testing using the SHU dataset and the PhysioNet dataset. Experimental results show that on the SHU dataset, utilizing 73.44% of the total channels, the cross-session MI recognition accuracy is 90.03%. Similarly, in the PhysioNet dataset, using 72.5% of the channel data, the classification result also reaches 93.99%. Compared to traditional strategies such as selecting three specific channels, correlation-based channel selection, mutual information-based channel selection, and adaptive channel selection based on Pearson coefficients and spatial positions, the proposed method improved classification accuracy by 34.64%, 10.8%, 3.25% and 2.88% on the SHU dataset, and by 46.96%, 5.04%, 5.81% and 2.32% on the PhysioNet dataset, respectively.