Rechargeable zinc–air batteries have attracted considerable attention in the industries due to their high energy density, affordability, and safety. However, these batteries still have limitations in their lifetime because of the issue of zincate ion (Zn(OH)42−) crossover. To suppress this issue, a new PVA-KOH-based gel polymer electrolyte membrane incorporating zeolitic imidazolate framework-8 (ZIF-8) nanoparticles was developed as a selective membrane. The influence of different-sized ZIF-8 nanoparticles synthesized with three different zinc salts on thermal, physical, and electrochemical properties was investigated. The PVA-KOH/ZIF-8 membrane significantly reduced the diffusion coefficient of zincate ions to 3.27 × 10−6, 2.01 × 10−6, and 1.99 × 10−6 cm2 min−1 by the incorporation of ZIF-8, synthesized with Zn(NO3)2, Zn(OAc)2, and ZnSO4, respectively. Because of the pore apertures of ZIF-8 that are equivalent to Zn(OH)42−, the PVA-KOH/ZIF-8 selective membrane could block Zn(OH)42−and allowed OH− to pass through the membrane. Moreover, a large external hydrophilic surface was achieved by the influence of smaller-sized particles, which improved the retention of electrolytes. As a result, the ionic conductivity of the PVA-KOH/ZIF-8 membrane increased up to 164.27 mS cm−1 when added 10 wt% ZIF-8, synthesized by ZnSO4. Furthermore, this membrane showed the highest value of battery life cycle of 89 cycles at a current density 10 mA cm−2 and the discharge capacity of 396 mAh g−1. These results indicated that the development of a PVA-KOH/ZIF-8 membrane has a high potential and suitability for use as a new class of separators in rechargeable zinc–air batteries.