Direct-conversion radio receivers can offer highly integrated low-cost hardware solutions for cognitive radio (CR) devices. Such receivers are, however, also very sensitive to various radio frequency (RF) impairments such as IQ imbalance, which can considerably limit the spectrum sensing capabilities. Most of the existing spectrum sensing studies in literature assume an ideal RF receiver and hence neglect the impacts of such practical RF hardware limitations. In this article, we study energy detection (ED) based spectrum sensing in both single-channel and multi-channel direct-conversion receiver scenarios impaired by IQ imbalance. With complex Gaussian primary user (PU) signal models, we first derive the detection and false alarm probabilities in closed-form for both receiver scenarios. The analytical results, confirmed through extensive simulations, show that while the single-channel receiver scenario is fairly robust to IQ imbalance, the wideband multi-channel sensing receiver is very sensitive to the image channel crosstalk induced by IQ imbalance. More specifically, it is shown that the false alarm probability of multi-channel energy detection increases significantly, compared to ideal RF receiver case, and the exact performance depends on the image channel power level and IQ imbalance values. In order to prevent such degradation in the ability to identify free spectrum, a waveform level interference cancellation method is then proposed to mitigate the image channel crosstalk. The optimum cancellation coefficient yielding interference-free signal is first derived, being also complemented with a practical coefficient sample estimator. An explicit condition is also derived for which the proposed cancellation scheme deploying the practical coefficient sample estimator provides performance gain in the sensing decisions compared to uncompensated energy detection. Extensive computer simulations with various signal and imbalance conditions are provided which demonstrate that the proposed enhanced energy detection can suppress the image channel crosstalk efficiently, yielding detection and false alarm probabilities essentially identical to those of an ideal RF receiver.