Due to the process defects and imperfection of drivers, permanent magnet synchronous motors (PMSM) are problematic to control. There is still a lack of effective high-performance control methods for inertial stabilized platforms based on PMSM currently. At present, the most frequently used method is sliding mode control (SMC), but traditional sliding mode control cannot overcome the contradiction between high performance and system chattering. In order to solve this problem and improve the system reliability and pointing accuracy, a new approach law for the sliding mode controller is proposed in this paper. In view of the large periodic torque ripple in PMSM, an iterative learning controller (ILC) is introduced to compensate for the disturbance. Based on these, aimed at suppressing all kinds of real-time disturbances in the working environment of the system, the extended state observer (ESO) is brought into the servo system to observe the lumped disturbance of the system, and the total disturbance observed is compensated into the sliding mode controller, so as to better suppress the system chattering and enhance the system’s ability of resisting external disturbance. Experiments are carried out on an inertial stabilization platform based on DSP + CPLD. The final experiments verify that the SMC with the new approach, combined with ILC and ESO, is of outstanding performance when compared with the traditional proportional integral (PI) + disturbance observer (DOB) control scheme.