Proton exchange membranes (PEMs) are the core components of electrochemical hydrogen energy production, storage and conversion devices. It is a great challenge to develop PEMs with a combination of high proton conductivity, minimal H2 crossover and mechanical robustness for achieving high efficiency and long-term stability in these devices. Here, a co-assembled PEM composed of ionic covalent organic framework (iCOF) nanosheets and functionalized graphene oxide (GO) nanosheets was presented. The iCOF (TpPa-SO3H) nanosheets and polydopamine-modified graphene oxide (DGO) nanosheets were co-assembled into TpPa-SO3H/DGO mixed nanosheets with ultrahigh aspect ratio, which were readily further processed into TpPa-SO3H/DGO co-assembled membranes. The abundant and uniformly distributed –SO3H groups on TpPa-SO3H nanosheets and the acid-base pairs formed between TpPa-SO3H and DGO nanosheets promoted proton conduction significantly. Additionally, the dense structured DGO nanosheets inhibited H2 crossover remarkably, and the multiple interactions between TpPa-SO3H and DGO nanosheets enhanced mechanical strength. The resultant TpPa-SO3H/DGO co-assembled membrane exhibited an ultrahigh proton conductivity of 916.4 mS cm−1 (80 °C, 100% RH), an unprecedently low H2 crossover of 9.7 barrer and a sufficient tensile stress of 87.8 MPa. The enhanced proton conduction and H2 barrier property led to enhanced performance in both electrochemical hydrogen compression and single hydrogen fuel cells.