The nonlinearity of heterodyne interferometers is caused not only by polarization ellipticity or nonorthogonality of the laser beams. The imperfect optical parts, especially the polarizing beam splitter (PBS), can also contribute. Past researchers pointed out that the nonlinearity error due to the leakage of the PBS could be generally neglected. Because the laser beam has to pass the PBS at least two times and its leakage component is mostly filtered before being mixed at the receiver. This paper has proved this viewpoint experimentally. It also shown how to distinguish and measure the nonlinearity errors caused by the two reasons experimentally.