Cement-free concrete has a superior physical performance, such as in its strength and durability, compared to OPC concrete; however, it has the disadvantage of large shrinkage. Large shrinkage can cause cracks due to shrinkage stress in the long term. In this study, a shrinkage reducing agent (SRA) was used to reduce the shrinkage of cement-free mortar; its content was increased from 0.0 to 1.5%. For an SRA content of 1.0%, a calcium sulfoaluminate (CSA) expansive additive (EA) (2.5, 5.0, and 7.5%) was added. To calculate the shrinkage stress of cement-free mortar using the SRA and EA, the compressive strength, elastic modulus, and total and autogenous shrinkage were measured. The unit shrinkage stress of cement-free mortar was obtained by multiplying the elastic modulus by the length change and accumulated to obtain the shrinkage stress acting on the mortar according to the age. The shrinkage stress of cement-free mortar showed different tendencies as the age increased. At early ages, the shrinkage rate of the mortar occupied a large proportion of the shrinkage stress. In the long term, the shrinkage stress was significantly affected by the elastic modulus. As a result, SRA was found to be effective in reducing the shrinkage stress by decreasing both the elastic modulus and shrinkage. However, EA increased the shrinkage stress over the long term due to an increase in the elastic modulus even though it compensated for early-ages shrinkage.