We report that low-temperature ozone calcination allowed MFI type polycrystalline zeolite membranes to maximize their p-/o-xylene separation factor (as high as ca. 2000) by suppressing defect formation. Conventional high-temperature calcination and rapid thermal processing, which provide poor and marked p-/o-xylene separation abilities, respectively, were used for comparison. The corresponding defect structures were quantitatively analyzed by image processing of fluorescence confocal optical microscopy images combined with membrane permeation modeling, revealing the main defects (grain boundary defects and cracks) and their tortuosity, porosity, and size. To the best of our knowledge, we, for the first time, demonstrated that in contrast to common belief, the minor portion of wider cracks instead of the major portion of narrower grain boundary defects determined the final permeation rates. Specifically, the MFI membrane prepared by high-temperature calcination contained many grain boundary defects (narrow; ca. 1 nm) and few cracks (wide; ca. 20 nm) that accounted for ca. 0.1% and 99.8%, respectively, of the slowly permeating o-xylene molar flux. In contrast, the ozone-treated MFI membranes, which only possessed grain boundary defects, achieved the high p-/o-xylene separation performance, underlining the need for the selective reduction in the number of wider cracks rather than ubiquitous, narrow grain boundary defects.