In the field of robotic polishing, achieving uniform material removal typically involves addressing the issue of constant contact force control. However, multi-source external disturbances in the polishing scenarios of complex workpiece surfaces can severely affect the robot’s force control accuracy. To enhance the responsiveness and disturbance rejection capabilities of robots in the compliant polishing process, this paper proposes an adaptive admittance controller with practical finite-time stability. A virtual control input is introduced into the basic admittance control framework in light of the state space theory, aiming to provide flexibility for common adaptive law designs. On this basis, a robust sliding mode control (SMC) algorithm is proposed to suppress external disturbances. The force tracking error is theoretically proven to achieve finite-time convergence when applying the proposed control strategy. Experimental results across various polishing scenarios demonstrate that, compared with the existing admittance control strategies, the proposed method can reduce fluctuations of the polishing force and improve the surface quality, thus verifying its effectiveness.