Don’t get lost in the crowd: Graph convolutional network for online animal tracking in dense groups

Abstract

Tracking the behaviour of animals in group-housed situations is a critical area of study for precision livestock farming, but it poses challenges in diverse and crowded environments. Existing methods often struggle with false positives and false negatives due to the complexities of these scenarios. In this study, we propose a robust computer vision algorithm for long-term (>10mins) animal tracking, with a primary focus on group-housed pigs. Our method addresses the limitations of current approaches by effectively handling errors from the detector in challenging environments. Our approach integrates Graph Convolutional Networks.(GCNs) with deep learning-based object detection techniques. We represent the data as a graph structure, with nodes corresponding to multiple animal detections over several frames. Edges connect thedetections that do not appear in the same frame. The model's objective is to perform edge classification, where each edge is associated with a scalar representing the probability of being the same object. To enhance the robustness of edge classification, we combine the classifier withan ensemble predictor, enabling accurate tracking of animal identitiesfor extended periods. Notably, our graph convolutional model achievesaccurate re identification of non-consecutive animal detections inreal-time without the need for trajectory prediction or ID association. We evaluate our method by comparing its performance against othertracking methods, including DeepSORT—a state-of-the-art animal trackingmethod that relies on a Kalman filter, a deep appearance descriptor, and the Hungarian algorithm for trajectory prediction and ID association.Our comparison demonstrates the advantages of our method, as it (1)improves the IDF1 score, indicating enhanced ID association accuracy, by 1.72%; (2) mitigates 93% of errors resulting from ID-switch anddeviation; (3) extends the tracking duration of each animal byapproximately 66% in challenging conditions; and (4) achieves aprocessing speed that is 1000 times faster than DeepSORT when operatingat 22fps with the detector. The potential applications of our method extend to various aspects of livestock management. By accurately tracking individual animals, we can monitor behaviours such as feeding, drinking, and aggressive interactions. This information can lead to improvements in animal welfare, resource allocation, and breeding strategies. Follow-ups of this research are available at: https://gitlab.kuleuven.be/m3-biores/public/m3pig.