CRISPR–Cas9 as a programmable genome editing tool is hindered by off-target DNA cleavage1–4, and the underlying mechanisms by which Cas9 recognizes mismatches are poorly understood5–7. Although Cas9 variants with greater discrimination against mismatches have been designed8–10, these suffer from substantially reduced rates of on-target DNA cleavage5,11. Here we used kinetics-guided cryo-electron microscopy to determine the structure of Cas9 at different stages of mismatch cleavage. We observed a distinct, linear conformation of the guide RNA–DNA duplex formed in the presence of mismatches, which prevents Cas9 activation. Although the canonical kinked guide RNA–DNA duplex conformation facilitates DNA cleavage, we observe that substrates that contain mismatches distal to the protospacer adjacent motif are stabilized by reorganization of a loop in the RuvC domain. Mutagenesis of mismatch-stabilizing residues reduces off-target DNA cleavage but maintains rapid on-target DNA cleavage. By targeting regions that are exclusively involved in mismatch tolerance, we provide a proof of concept for the design of next-generation high-fidelity Cas9 variants.