Although discriminative correlation filter-based (DCF) trackers have shown excellent performance in recent years, it still has the problem of model drift. A generic approach alleviating model drift is proposed for DCF tracking. Specific correlation filters are learned in feature channels and the combination of response maps depends on the confidence level of corresponding channels. Model drift is eliminated by allowing contaminated correlation filters to retrogress to the ones that are not influenced by impure samples. As a result, correlation filters are purified by neglecting the impure part of them. Meanwhile, both the confidence level of detection and the variant level of interframe feature space are considered during online update of correlation filters. Our approach is experimented on both OTB-50 and OTB-100 datasets. Especially on OTB-100, our approach outperforms the baseline fast discriminative scale space tracking by 6.9%, 8.1%, and 5.6% in mean distance precision, mean overlap precision, and the area-under-the-curve score, respectively. Moreover, the competitive performance is presented by comparing with other state-of-the-art tracking algorithms.