Infrared imaging systems with a wide detection scale, excellent precision, and high resolution are playing an increasingly important role in many fields. Compared with uncooled infrared detectors, the cooled infrared detectors have the advantages of high precision, long detection distance, and a high signal-to-noise ratio due to the low working temperature. In this paper, a cooled off-axis three-mirror mid-wave infrared (MWIR) optical system with a wide field of view (FOV), long focal length, and large relative aperture is designed. We first establish the initial coaxial three-mirror system based on the third-order Seidel aberration theory and then make the aperture stop offset to eliminate central obscuration. During the process of optimization, the FOV of the system is expanded stepwise, and it is not biased to avoid the extra aberration. In order to increase the optimizing degrees of freedom to improve the aberration elimination ability of the system, we apply the XY polynomial surface on the tertiary mirror (TM). After optimization, we finally obtain a cooled off-axis three-mirror MWIR optical system with a FOV of 5.3∘×4.4∘, a focal length of 500 mm, and a relative aperture of 1/2. The image quality of our system is excellent, and the aperture stop is between the focal plane and the TM, and coaxial with the focal plane, meeting the conditions of matching with the cold stop of the infrared detector. The tolerance analysis is also carried out, and the result shows that the system meets the requirements of practical fabrication and alignment.