Lightweight, noiseless, and constant low-temperature refrigeration is crucial for cooled infrared (IR) detectors, especially short-wave-infrared (SWIR) detectors. However, some SWIR detectors, such as HgCdTe photovoltaic detectors are working below 210 K. For this application, multi-stage thermoelectric coolers (TEC) are preferable to single-stage thermoelectric coolers because they can generate larger temperature differences. This paper presents a refrigeration design that integrates SWIR and TEC into a vacuum Dewar package and combines them with a microchannel heat sink (MHS) to improve the detection performance and better responsiveness of SWIR. Two different refrigeration models, SWIR + TEC + AHS (air-cooled heat sink) and SWIR + TEC + MHS, were established to evaluate the refrigeration performance of the proposed SWIR + TEC + MHS scheme using the finite element method. The influences of multi-stage TEC leg length and current, inlet water temperature, and water velocity on the refrigeration performance of the TEC are analyzed. The results show that the SWIR + TEC + MHS design scheme has the best comprehensive performance and minimum volume. The influences of all the factors are obvious while the current has the most important effect on the performance. The temperature of SWIR reached 201.69 K at standard working conditions. It shows that the cooling scheme can not only meet the needs of flexible deployment of SWIR but also provide better thermal security. The proposed cooling design has an excellent prospect for short-wave-infrared detectors.