This paper presents a new method for tracking moving objects that have colour variations due to illumination, which uses a maximum colour distance on the mean shift framework. Conventional colour-based mean shift methods show good results when tracking non-rigid moving objects. However, they do not provide accurate results when the initial colour distribution of the object disappears. Our method uses the maximum colour distance to represent the objects. If a colour distance can be defined as the geometric distance between two colour points in a colour space, then the maximum colour distance is defined as the maximum value among all of the colour distances. The physical illumination model, under assumptions, guarantees that the maximum colour distance is independent of ambient lighting and other illumination with an identical solid angle. The objective of our method is to provide robust real-time object tracking with large colour variation in objects whose colour changes due to external illumination. The implementation of this new algorithm shows effective tracking results with a complete object colour change over time. The validation of our approach is illustrated by the comparison of experimental results obtained using the methods described above.