The optical phase of the driving field in the process of high harmonic generation and the coherence properties of the harmonics are fundamental concepts in attosecond physics. Here, we consider driving the process by incoherent classical and nonclassical light fields exhibiting an undetermined optical phase. With this, we introduce the notion of quantum optical coherence into high harmonic generation and show that high harmonics can be generated from incoherent radiation despite having a vanishing electric field. We explicitly derive the quantum state of the harmonics when driven by carrier-envelope phase unstable fields and show that the generated harmonics are incoherent and exhibit zero electric field amplitudes. We find that the quantum state of each harmonic is diagonal in its photon number basis, but nevertheless has the exact same photon statistics as the widely considered coherent harmonics. From this, we conclude that assuming coherent harmonic radiation can originate from a preferred ensemble fallacy. These findings have profound implications for attosecond experiments and how to infer the harmonic radiation properties. Published by the American Physical Society 2024