In this study, the authors propose a novel and effective short track speed skating tracking system. Aimed at several challenging tracking problems of short track skating: long‐time occlusion and complex group situations, size variations, similar or identical uniforms, fast motion speed, quick orientation changes etc.; a novel fuzzy model is proposed to initialise contours of skaters even they wear the same kind of uniform. Then, a novel approach for multiple object tracking is developed to track skaters reliably. The experimental results demonstrate that the proposed system can solve the above challenging problems very effectively. The information provided by the proposed system, including trajectories, velocity analysis and two‐dimensional reconstruction animations, is valuable to broadcasters, athletes and coaches alike. The proposed tracking algorithm can obtain better results than that by the other eight state‐of‐the‐art trackers in short track speed skating tracking.