As an efficient method for utilizing coal gangue (CG), concrete incorporating coal gangue as coarse aggregate has significantly reduced the reliance on natural aggregates, offering substantial environmental and economic benefits. In this study, coal gangue concrete was prepared with coal gangue replacement rates of 0, 20, 40, 60, 80, and 100%, and mechanical tests under unconfined compression were conducted to evaluate the stress-strain behavior and failure mechanism of coal gangue coarse aggregate concrete (CGC). Utilizing scanning electron microscope (SEM) microscopic characterization, the microscopic failure mechanism of CGC was further elucidated. With increased coal gangue replacement, the CGC's uniaxial compression failure mode shifts from shear to longitudinal splitting failure. The slope, peak stress and elastic modulus of the stress–strain curve's rising section are negatively correlated with the coal gangue content, while the falling section's slope, peak strain and ultimate strain are positively correlated. Next, building upon the classical constitutive model, we adjust the constitutive parameters utilizing the uniaxial compressive strength and coal gangue content. Finally, we introduce a predictive model for the CGC's constitutive compressive behavior across various content levels. There is a notably high agreement between the model and experimental data.