To investigate the mechanical properties and a compressive constitutive model of recycled concrete under sulfate attack considering the effects of multiple factors, two waste concrete strengths (i.e., C30 and C40), four replacement ratios of recycled coarse aggregates (i.e., 0, 30%, 50% and 100%), and two water–cement ratios (i.e., 0.50 and 0.60) were considered in this study, and a total of 32 recycled concrete specimens were designed and tested. The results indicated that the failure processes and patterns of recycled concrete were not significantly influenced by the replacement ratio of recycled coarse aggregates, the waste concrete strength, the water–cement ratio, or sulfate attack. The higher the replacement ratio of recycled coarse aggregates and the water–cement ratio and the lower the waste concrete strength, the more obvious the reduction in cubic compressive strength, with a maximum reduction of 38.48%. A prediction model for the cubic compressive strength of recycled concrete under sulfate attack was proposed. The higher the replacement ratio of recycled coarse aggregates and the water–cement ratio and the lower the waste concrete strength, the more significant the reduction in axial compressive strength, with a maximum reduction of 37.82%. A prediction model for the axial compressive strength of recycled concrete under sulfate attack was established. A compressive constitutive model of recycled concrete under sulfate attack considering the effects of the replacement ratio of recycled coarse aggregates, the waste concrete strength, and the water–cement ratio was established. The pore structure of recycled concrete was significantly destroyed by the expansion stress generated by Na2SO4 crystals: a large number of Na2SO4 crystals were attached to the surface of concrete matrix, and the concrete matrix became loose. The research results can provide a theoretical basis and data support for engineering applications of recycled concrete.