As freeform surfaces are widely used in off-axis optical systems with large apertures, large fields of view, and long focal lengths, the polarization effect caused by the system cannot be ignored. For this in mind, a polarization aberration analysis method for freeform optical system with fringe Zernike polynomial is proposed. An analytical model for the polarization aberration in this case is constructed. The influence of the surface shape on the polarization aberration is investigated. Our analysis shows that the distribution of phase aberration, diattenuation and retardance at the exit pupil introduced by the freeform surface in the system is related to the entrance pupil, incident field angle, and also closely dependent on the surface sag of the freeform surface. The introduction of freeform surfaces causes the system polarization aberration distribution to lose rotational symmetry. Finally, a freeform surface multi-angle polarization characteristic test platform was built to test the validity of the theoretical model. The polarization aberration characteristic data of the incident mirror at different angles of view at different positions were obtained and analyzed. The measured results show that the relative errors of phase aberration, diattenuation and retardance are all less than 7.2%, which is basically consistent with the theoretical simulation analysis results. The research in this paper provides a theoretical basis for guiding the design of an off-axis freeform surface polarization imaging optical system, and provides theoretical guidance for improving the polarization aberration calibration of a freeform surface optical system.