It is well-known that the 9–12%Cr martensitic heat-resistant steels own excellent mechanical properties; yet, in the case of aging embrittlement mechanism, little has been documented with carbides effect. In this study, to understand the aging embrittlement mechanism in steel 10Cr12Ni3Mo2VN, we focused on high-temperature short-term aging embrittlement by investigating the relationship between the heat treatment, microstructures, and aging embrittlement. It is mainly interested in the effect of carbides on the impact toughness of the steel sample after a heat treatment. The results show that the steel appears a reversible short-term aging embrittlement at temperatures within 500–600℃. It was found that carbides play an important role in the aging embrittlement, especially the M7C3-type carbides. Carbides formed at boundaries cause boundaries weakening and induce tempered martensite matrix to fracture in quasi-cleavage mode, which leads to the aging embrittlement. It was also found that the aging embrittlement can be eliminated by re-tempering at a higher temperature due to the transformation of carbides from M7C3-type to M23C6-type. Aging embrittlement arises again but becomes weak after a re-aging treatment. In addition, by refining prior austenite grains, increasing temper temperature and reducing carbon content, it can improve the resistance to aging embrittlement in the sample as refining grains increase the resistance to brittle fracture; what’s more, it could reduce the precipitation of carbides at boundaries during aging by increasing temper temperature and reducing carbon content.