Fabrication of thinner, mechanically, and chemically stable separators is desirable and can maximize the energy density of lithium batteries. Covalent organic frameworks (COFs) as functional porous materials with tunable structures and designable properties are alternative materials for the construction of battery separators, but their powder state limits the possibility of further processing into ultrathin membranes. In this work, we developed a side-chain engineering strategy to tune the interlayer distance of COFs to facilitate exfoliation processing by mild mechanical force. A large number of uniform micron-scale nanosheets can be obtained by ball-milling of COF gel, and COF–C16/PE composite membrane (∼9 μm thick) with good chemical and mechanical stability can be fabricated through a vacuum-assisted self-assembly process. The COF–C16/PE composite membrane assembled Li–LiFePO4 batteries delivered cycle stability over 850 cycles at 1.0C with a discharge capacity of 128 mAh/g. Furthermore, the COF–C16/PE composite membrane can serve as an efficient separator to inhibit polysulfide shuttle, and the capacity retained 580 mAh/g after 100 cycles at 0.2C in Li–S batteries. We further implemented such a membrane in a quasi-solid battery system, which also demonstrated stable cycling. This work opens up a new avenue for fabricating thin and stable membranes for high-energy-density lithium batteries.