As one of several main error sources, polarization effect of the front optical systems has an important impact on polarimetric accuracy of imaging polarimeters. In order to know and mitigate the impact from the error source, an analysis method based on Jones matrix and Pauli spin matrices is reviewed and a typical example of an uncoated transmission spherical interface is given to investigate the function mechanism of the impact from optical systems on polarization measurement. Through investigating, we found that the same optical interface has different impacts when it is located in different locations in the light path, and the rotationally symmetric interface unexpectedly has a non-rotationally symmetric polarization effect in the system coordinate. Research on the comparison of the two methods of coating depolarizing film and polarization compensator indicates both of them have complementary advantages but the former is a better choice at present from the engineering application point of view.