The aim of this study is to determine the optimal carbonization conditions for carbon hollow fiber membranes for hydrogen purification. To this end, it is a prerequisite to obtain the best permselectivity of the resultant membrane. Three different carbonization atmospheres (vacuum, helium flow, and argon flow) were examined, while their effects on the changes in the microstructure and gas permeation properties of the as-synthesized carbon membranes were elucidated using a pure gas permeation test, coupled with an array of characterizations, including Raman spectroscopy and XPS. Moreover, the issue of vacuum degree was addressed to maintain the best reproducibility of the carbon membrane. The results revealed that CHFM-HV exhibited the best separation capability for H2/CH4 of 516.90, thereby indicating distinct hydrogen purification characteristics. Such good performance was driven by the attractive synergism of the ordered microstructure and larger pore volume. Notably, the recovery of permselectivity for CHFM-HV reached 90% after four months, thus suggesting its aging resistance. Hence, the demonstrated approach is efficient for altering the microstructure of the carbon membrane, thereby strengthening the separation performance and long-term stability. Overall, it is suitable for broad applications of diverse polymer precursors and prolonged the lifespan of the as-prepared carbon membrane.