Determining if there is any coal left in the coal mine skip unloading is a challenging problem. We propose a new object tracking method to provide new ideas to solve the problem of whether there is residual coal in the skip unloading. In this study, a fast vision-based measurement framework about new diamond search normalized cross-correlation (NDS-NCC), including a movement prediction part and a template tracking part, is used. Firstly, a diamond search method with fast adaptive coarse localization is proposed to overcome the time-consuming problem of NCC template matching process. Secondly, we incorporate a template updating strategies that allows targets to be tracked consistently. The performance of the NDS-NCC algorithm was tested on the bench and at the coal mine site for balancing cylinders and wire rope tracking. The results show that the NDS-NCC algorithm has better efficiency (7 ms/frame) and accuracy (0.4512 px) in the coal mine field application.