Development of a teat sensing system for robotic milking by combining thermal imaging and stereovision technique

Abstract

The robotic application of milking cluster to the udder of a cow in a rotary high capacity group milking system is a major challenge in automated milking. Application time and reliability are the main constraints. Manual application by an operator of a rotary milking system is of the order of 10s and 100% reliable. Existing commercial automatic milking systems employ laser scanning technology. The teat cups are applied to each teat individually and the process can take up to 2min. In order to achieve a more rapid simultaneous application of the four cups, the three dimensional locations of the four teats must be known in real time. In this paper, an innovative multimodal vision system combining optical stereovision and thermal imaging is developed. The two technologies were combined and calibrated into one vision system. Algorithms of detection of the teat and determination of their three dimensional position were also developed. Using a dummy thermal udder, laboratory tests were performed for various situations and the system was evaluated from the point of view of accuracy and robustness. Results showed that the system could locate accurately in less than one second the three dimensional position coordinates of the four teats. This speed of detection is much faster than any existing technology employed with automatic milking and could be an alternative approach of sensing the teats, which can increase the yield of rotary milking system. In terms of robustness, the system achieved promising results by retrieving the position of the teats with challenging configurations including touching and overlapping teats. Further optimisation is proposed to increase the robustness prior to in-situ trials with real cows. The overall results demonstrated that it is possible to increase the efficiency of the stereovision technique for teat location by introducing thermal imaging.