Adopting crushed coal gangue as aggregates for new concrete production could be a sustainable alternative to natural aggregates and could reduce landfill space. For the structural applications of innovative concrete, the elastic modulus is a key parameter influencing its serviceability limit state. Prediction models are presented herein for the elastic modulus of concrete prepared with coarse coal gangue aggregates (CGAs), including spontaneous-combustion coal gangue aggregate (SCGAs) and rock coal gangue aggregate (RCGAs). Experimental data from our previous studies and other literature were first collected to quantify the effects of the replacement ratio of CGAs, as well as the density and compressive strength of CGA concrete on the elastic modulus of the resulting concrete. Based on our previous experimental data, empirical models were proposed and validated against 150 groups of tested data. The obtained results indicated that the replacement ratio of CGAs is the major factor, and there were up to 57% and 41% reductions in the elastic modulus of the resulting concrete prepared with 100% SCGAs and RCGAs, respectively; by considering the replacement ratio, compressive strength and density of concrete, the newly proposed models in this study estimated the elastic modulus of both SCGA and RCGA concrete accurately.