A hybrid metal-additive manufacturing (AM) process that combines laser-based powder bed fusion (PBF-LB) with interlayer burnishing is investigated using a comprehensive modeling framework to provide new insights into how the inhomogeneous microstructure and residual stress from the laser powder bed fusion process affect the induced residual stress field that evolves during interlayer burnishing. Researchers have recently studied changes in microstructure resulting from similar hybrid metal-additive processes, however, it was only hypothesized that the resulting microstructure has some influence on the induced residual stress. In addition, researchers have numerically investigated the influence of burnishing/rolling process parameters on induced stress but neglected the effects of microstructure, thereby making homogeneous, isotropic assumptions. Such practice inhibits the prediction of microstructure-driven anisotropy that can exist in the inhomogeneous fused layer. This paper parametrically examines the influence of microstructure modeling, inherent residual stress mapping, and environment temperature on the induced residual stress during the hybrid metal-additive process. The demonstrated modeling framework incorporates inherent residual stresses that emerge from the laser powder bed fusion process, as well as the predicted microstructure, in a subsequent burnishing simulation to elucidate their individual and combined influences on the burnishing-induced residual stress. Findings reveal that modeling an inhomogeneous PBF-LB microstructure introduces an anisotropic distribution of plastic strain and residual stress along the burnished surface; a periodicity in planar stress components along the treated surface coincides with the PBF-LB scan lines. Effects of inherent residual stress on the burnishing-induced residual stress is less significant, but nonetheless observable. Elevated temperatures not only reduce the magnitude of compressive residual stress induced but also result in less variation of residual stress component magnitudes predicted along scan lines and hatch spaces. The presented framework offers new insights into the decoupled influences of microstructure and PBF-LB residual stress on burnishing-induced stresses that are not distinguishable via experimental techniques. However, trends in averaged residual stress through the depth of the specimen, as well as surface hardness magnitudes after burnishing show good agreements, respectively, with X-ray diffraction and microindentation measurements documented in the literature.