Generic higher derivative theories are believed to be fundamentally unphysical because they contain Ostrogradsky ghosts. We show that within complex classical mechanics it is possible to construct higher derivative theories that circumvent the Ostrogradsky theorem and have a real energy spectrum that is bounded from below. The complex theory can be canonically quantised. The resulting quantum theory does not suffer from the kinetic instability and maintains the usual probabilistic interpretation without violating the correspondence principle. As a proof of concept, we construct a class of stable interacting complex higher derivative theories and present a concrete example. This consistent and canonical framework allows us to analyse the previous attempts to avoid ghosts that use non-canonical quantisation schemes, such as the Lee–Wick theories, Dirac–Pauli quantisation or PT-symmetric quantum mechanics. The key to understand the would-be ghosts in any kinetically stable higher derivative theory is to accept the complex system behind it.