Existing scale estimation methods mainly use fixed-size tracking box for target tracking. However, when the moving direction of the target changes from near to far or from far to near, the size of the current tracking box cannot adapt to the scale change. In order to overcome the disadvantage of invariant size of tracking box in the scale change, this paper proposes a scale offset estimation method based on correlation filter by combining the state of previous frame and current frame of the target. Select the scale proposal box in the scale layer, and adjust the position and size of the actual tracking box in real time according to the size of the proposal box, target position and scale offset. In this paper, OTB-100 is used as the dataset. The obtained tracking results show that our algorithm Ours has better tracking performance than the existing others tracking algorithm in scale variation, and it has better tracking effect and stronger robustness in drift and occlusion events.