AbstractThis paper presents a simplified mathematical approach to evaluate the performance of any given circular constellation of 16‐level quadrature amplitude modulation (16‐QAM) in terms of symbol error rate (SER). Following this approach, with the aim to work with memoryless nonlinear satellite channels, a model is derived as a generalized form for both linear and nonlinear channels in the presence of down link additive white Gaussian noise (AWGN). The analysis provides means to calculate the optimal ring ratio (RR) and phase difference (PD) for several possible candidates of 16‐QAM circular constellations. The effects of RR and PD on the SER performance are investigated in the analysis. To overcome the nonlinear distortion, data pre‐distortion is taken into account in the study. The paper gives a general procedure for data pre‐distortion implementation for all circular 16‐QAM constellations. The analytical formulation has been extended for total degradation (TD) performance measure as a function of input back‐off (IBO) of the nonlinear amplifier. A SER performance‐comparison between different constellations for 16‐QAM systems has also been presented in this paper. The analytical results are validated by simulation. Copyright © 2006 John Wiley & Sons, Ltd.