As the tracking mode based on color probability distribution in Continuously Adaptive Mean-SHIFT can not describe object’s nonrigid deformation precisely, an efficient robust face tracking method is proposed, which integrates object’s moving state estimation into Kalman particle filter, and combines Kalman particle filter with CamShift tracking algorithm. Experimental results show that the method presented has good robustness to size and angle variation, rapid movement, partial and fully occlusion of face.