The traditional visual multi-object tracking methods based on the Gaussian mixture probability hypothesis density filter are generally not well adapted for tracking the targets in the complex scenarios, where there are a large number of unknowable newborn objects and occluded objects, even some missing objects cannot be associated with their previous trajectories when they are redetected. An improved visual multi-object tracking algorithm is proposed by integrating an improved efficient convolution operator of the correlation filter and the Gaussian mixture probability hypothesis density filter. First, a similarity matrix based on the intersection-of-union is proposed for classifying the objects of survival objects, newborn objects, and then the improved efficient convolution operator method is employed to further identify whether the objects disappear or are missing. Moreover, the feature pyramid similarity is proposed to update the objects for enhancing the tracking accuracy. Finally, compared with some challenging methods on some challenging video sequences from publicly available MOT17 dataset, the proposed Gaussian mixture probability hypothesis density–feature pyramid similarity—efficient convolution operator* method has a good performance on detecting the newborn objects, occluded objects, blurring objects and re-identifying the missing objects with higher multiple object tracking accuracy.