The current constitutive model and damage evolution law of crumb rubber concrete (CRC) were obtained by fitting and changing parameters based on the normal concrete model. However, this model does not accurately reflect the characteristics of the material. In this paper, we studied the energy dissipation in the failure process of CRC to derive the constitutive model and damage evolution law of CRC based on the energy method. Four substitution rates of 5%, 10%, 15%, and 20% were selected, and the rubber concrete prism was prepared by replacing the natural fine aggregate with the same volume of crumb rubber aggregate. After that, uniaxial compressive tests were conducted. The energy lost due to the damage was calculated and analyzed, and the energy method was used to establish the damage evolution law and damage model of the crumb rubber concrete. The results demonstrated that the Guo Zhenhai damage model, which is based on the energy method, can more effectively explain the crumb rubber concrete stress–strain full curve, and the energy consumed as a result of the damage exhibits a growing and then reducing pattern with the increase in rubber doses. When the energy-based method is used, the Guo Zhenhai damage evolution model is similar to the damage evolution law calculated using the SIR damage evolution model. During uniaxial compression damage, rubber concrete with various rubber dosages demonstrated varying energy absorption in different deformation phases. When the rubber particle content was 10%, the energy absorption capacity of the specimen was 6.9% higher than that of normal concrete.