In this paper, we investigate the effects of in-phase/quadrature-phase imbalance (IQI) on the performance of different modulation schemes under multipath fading channels. In particular, a comprehensive framework for the analysis of coherent and noncoherent modulation with IQI is proposed. Specifically, new moment generating function (MGF) expressions for both point-to-point and multiple antenna systems are derived for the cases of transmitter IQI, receiver IQI, and joint transmitter/receiver IQI. Based on the derived MGFs, new analytical expressions for the corresponding symbol error rate (SER) are derived and are subsequently used to provide deep insights into the effects of IQI on the system performance. It is shown that, while in some cases the SER performance degradation due to IQI is marginal, in other cases, this impairment is more pronounced. Accordingly, IQI compensation is of paramount importance as it can guarantee a reliable communication link. This is particularly important in demanding communication technologies where reduction of latency and complexity are essential.