Recently discriminative correlation filter (DCF) based methods have gained much popularity for their excellent performance and high efficiency. However, most of them perform poorly in long-term tracking as they are not equipped with an effective mechanism to evaluate the quality of tracking results and correct tracking errors. To resolve such issue, this paper proposes a long-term tracking method, which consists of two components, including tracking-by-detection and re-detection. The tracking-by-detection part is built upon the DCF framework by incorporating a contour constraint map, which could identify non-target samples and refine the tracking results efficiently in presence of some challenging situations, such as deformation and occlusion. Benefited by our proposed re-detection strategy, the missing/occluded target could be captured immediately after it reappears. Moreover, a re-detected result is allowed to replace the original tracking result only when it owns a performance gap against the original one, which can reduce the risk of wrong substitution and well enhance the long-term tracking robustness. Extensive experiments on OTB2015, Temple-Color, UAV20L and VOT-LT2018 show that the proposed long-term tracking method outperforms state-of-the-art hand-crafted based methods, and even some deep learning based methods.