Large-volume concrete structures in the saline areas of Western China often have cold joint structural surfaces due to the improper handling of construction joints, dividing them into old and new layers of concrete. In this sulfate-rich environment, the shear strength of concrete with cold joint and the evolution pattern of its composition with the sulfate dry–wet cycles are particularly important in assessing the quality of the interface between old and new concrete bond surfaces. Therefore, in this paper, direct shear tests and microscopic scans were performed on concrete specimens with cold joint at different pouring age intervals and numbers of sulfate dry–wet cycles. The mechanism of the effect of both on the shear strength and failure mode of concrete was analyzed. Results show that the sulfate dry–wet cycles of erosion and the age interval have significant effects on the shear strength of the cold with cold joint and their components. As the number of sulfate dry–wet cycles increased, the shear dilation of concrete specimens with cold joints under direct shear action was reduced, the internal friction angle initially increased and then decreased for specimens with a short pouring interval (0–0.25 day), and the reduction of the internal friction angle of the specimens with a long pouring age interval (0.25–28 days) was nonlinear. The shear dilation of the specimen increased, and the internal friction angle decreased nonlinearly with the increase of the pouring interval. In addition, the failure criterion of specimens transformed from the bilinear Mohr–Coulomb criterion before erosion to the linear Mohr–Coulomb criterion under the action of sulfate dry–wet cycles.