This paper proposes a topology optimization method to design self-support structures for metal additive manufacturing. Constraining the process-induced residual stresses is considered with this method to avoid part failures of cracking, delamination, or warpage. Specifically, an inherent strain method-based finite element analysis model is presented to simulate the complex mechanical behavior during the additive manufacturing process. Then, according to the inherent strain-based fast simulation model, the residual stress constrained self-support topology optimization algorithm is formulated, and more importantly, the critical sensitivity information is derived through the adjoint analysis and validated with the finite difference method. Finally, the proposed method is applied to several 2D benchmark examples to demonstrate the effectiveness on residual stress and distortion control. The influences of different RAMP interpolation parameters, residual stress upper limits, and printing directions on the residual stress-constrained design effect are thoroughly explored. A comprehensive discussion is presented at the end to summarize the numerical phenomena.