Magnetic $3d$ ions doped into wide-gap oxides show signatures of room-temperature ferromagnetism, although their concentration is two orders of magnitude smaller than that in conventional magnets. The prototype of these exceptional materials is Co-doped ZnO, for which an explanation of the room-temperature ferromagnetism is still elusive. Here we demonstrate that magnetism originates from ${\text{Co}}^{2+}$ oxygen-vacancy pairs with a partially filled level close to the ZnO conduction-band minimum. The magnetic interaction between these pairs is sufficiently long ranged to cause percolation at moderate concentrations. However, magnetically correlated clusters large enough to show hysteresis at room temperature already form below the percolation threshold and explain the current experimental findings. Our work demonstrates that the magnetism in ZnO:Co is entirely governed by intrinsic defects and a phase diagram is presented. This suggests a recipe for tailoring the magnetic properties of spintronics materials by controlling their intrinsic defects.