Interference imaging spectroscopy combines modern imaging technology with spectral technology, holding significant importance for object imaging and spectral detection. This article introduces the principle of an adjustable wave plate polarization interferometric imaging spectrometer. The example design specifications are set for an observation wavelength range of 450–780 nm and a maximum resolution of 2 nm at 450 nm, with a 0.5 in detector as the base for calculating the specific dimensions of the Soleil–Babinet compensator. An investigation was conducted on the issues of nonuniform sampling, as well as three types of mechanical errors: flatness, wedge angle tolerance, and optical axis orientation accuracy. Emphasis was placed on discussing the impact of these errors on the instrument’s optical path difference and spectral reconstruction accuracy. This research provides theoretical guidance for the design and engineering of this miniaturized imaging spectrometer.