To explore the key inhibitory mechanism of external chloride ions on concrete sulfate attack, long-term partial- and full-immersion experiments of concrete in chloride solutions and chloride and sulfate composite solutions were performed. As corrosion time begins, visual inspection, compressive strength, and ultrasonic velocity were periodically observed or tested. Moreover, the micro-morphology, the contents of Cl and S elements in concrete, and the transport speeds of Cl− and SO42− ions through concrete were investigated by using scanning electron microscope, X-ray fluorescence spectrum, and chemical analysis. Chloride ions neither have corrosive effects on concrete nor effects on concrete physical and mechanical properties, which is the prerequisite for chloride ions in inhibiting concrete sulfate attack. In the case of combined solutions, the transport speed of Cl− ions into concrete is evidently higher than that of SO42− ions. Then, chloride ions enter concrete early and form Friedel's salt with AFm or with C3A and Ca(OH)2 in concrete, which inhibits the transport of SO42− ions into concrete and reduces the formation of expansive corrosion products of ettringite or gypsum. Thus, the sulfate attack on plain concrete is alleviated. Based on the inhibitory mechanism of Cl− ions on concrete sulfate attack, the presence of Cl− ions cannot completely avoid concrete sulfate attack but can partially reduce sulfate risk on concrete.