Visual tracking is a challenging task in computer vision. Correlation filter (CF) based visual tracking algorithm has become an attractive tracking technique, while there are still some limitations. Existing CF-based tracking algorithms are vulnerable to the influence of the surrounding background, and the usage of fixed scale template may easily lead to tracking failure. To overcome the above limitations, we propose a CFs-based visual tracking algorithm, CFs with Gabor energy filter, variable-scale template and features fusion (CFGVF). In CFGVF, the Gabor energy filter is firstly adopted to preprocess every frame of the image sequence, which largely eliminates the influence of illumination variation. Then, the Gabor energy, Histogram of Oriented Gradient and color naming features are integrated to enhance the ability of dealing with significant appearance variations such as deformation and motion blur. Furthermore, we propose a variable-scale template method to estimate the scale of the target object. Finally, an online updating schema is adopted to reduce the interference of surrounding background and occlusion. Experimental results on the visual tracking benchmark dataset OOTB show that the performance and robustness of the CFGVF tracker outperform those of several state-of-the-art trackers.