A novel hydroxyl-containing polyetherimide (named BAHPPF-ODPA) with highly aromatic backbone and contorted chain conformation was designed and synthesized as the precursor to fabricate carbon molecular sieve membranes (CMSMs) for H2/N2 separation. The structural evolution from polymeric precursor to carbon during pyrolysis was investigated using characterization techniques combined with pyrolysis simulation. And the relationship between membrane structure formation and their separation performance was explored here. Results indicate that the generation and evolution of pyrolysis products including formed fragments and gases, which depend on the precursor chemistry and pyrolysis conditions, exhibited a significant influence on microstructure and performance of derived CMSMs. Designing precursors with an aromatic backbone and contorted structure and precisely tuning pyrolysis temperature allows the fabrication of CMSMs with controllable micropore structure and excellent H2 separation performance. The BAHPPF-ODPA derived CMSM pyrolyzed at 600 °C exhibited the high H2 permeability of about 3500 Barrer and the superior H2/N2 selectivity of more than 400 were obtained after 750 h of long-term mixed gas (75:25 mixture of H2/N2) testing, showing promising application prospects. The experimental and simulation results obtained here would be beneficial to understand the CMS formation and reasonable to design the precursor molecular structure for fabrication of high-performance CMSMs.