The existing infrared target simulation system with a total internal reflection (TIR) prism has the problem of low imaging contrast ratio, which will seriously affect the quality of the simulated image. This study proposes a design method of optimized TIR prism (OTIR) based on Snell's law in medium-wave infrared (MWIR) to solve the problem. The radiation theory is used to construct the constraint model of the OTIR prism in the MWIR target simulation system. Further, this study investigates the influence of different states of the digital micromirror device on the beam direction and derives the design equation of the OTIR prism composed of three prisms based on Snell's law. Finally, the designed OTIR prism is simulated and experimentally verified. The simulated results show that the OTIR prism of the compact MWIR target simulation system can enhance the contrast ratio. The experimental results show that the output contrast ratio of the simulation system at 700 K is about 298:1. In the specified temperature range, the contrast ratio of the infrared target simulation system increases with the increase of the light source temperature. Thus, the OTIR prism has the function of improving the contrast ratio of MWIR target simulation system.