In heterogeneous scenes with latent non-deterministic states, template features determine the performance of the Siamese network-based trackers, however, background noise is easily introduced in the search area matching process, which causes the trackers lack sufficient discrimination. To solve this problem, this paper proposes an object tracking algorithm based on the cyclic memory module. Firstly, the extracted primary features are mapped to the cyclic spatial memory network, the spatial memory unit is used to store the interference components. On this basis, the cyclic structure is used to filter out the interference components and refine the template features. Then a channel correlation network is introduced into the proposed method, which can mine semantic complementary information and capture global contextual relevance by modeling the positive and the negative correlation between the channels. The experimental results on tracking benchmarks such as OTB50, OTB100, UAV123, VOT2018, GOT10k, and LaSOT have shown that the proposed method outperforms recent representative algorithms in tracking accuracy in challenging scenarios.