When designing, developing, and testing medium-voltage (MV) and high-voltage (HV) switchgears, it is of utmost importance to analyze the movements of their mechanical parts, such as drive trains, contact nozzles, etc. This ensures the safety of switchgear operations and enables detecting and predicting issues that could lead to component damage, power interruptions, reduced efficiency, or even switch failure. Conventional invasive measuring setups, including encoders and laser distance measurements, are often difficult or expensive to use, due to undesirable environmental properties such as high temperatures and/or high voltages, or dimensional constraints often encountered in testing laboratories, compact substations or confined equipment rooms. Video object tracking can be a viable solution in such conditions, allowing for the extraction of the trajectory of mechanical parts of an object under analysis. This paper proposes a novel kernel-based algorithm with adaptive feature extraction for precise colour-based object tracking through video processing. The implemented method is based on the principles of the CamShift algorithm, augmented with fusion with the Kalman filter for continuous estimation and prediction of the object’s position based on the available measurements while reducing sensitivity to noise. Beyond precise tracking of the object of interest, the algorithm automatically adapts the mask used in the standard CamShift algorithm by extracting and processing the features of the selected object. This approach exhibits flexibility and robustness in adverse industrial environments. It supports modifications according to user preferences and represents a cost-effective alternative to conventional methods, while performing real-time processing. Experimental results underscore this noninvasive approach is flexible, highly robust, and enables tracking of coloured markers even in challenging conditions like poor lighting, significant blur, and low frame rates.