Basic motion behaviour recognition of dairy cows based on skeleton and hybrid convolution algorithms

Abstract

Accurate and rapid recognition of the basic motion behaviours of dairy cows is the key to intelligent perception of their health status. As spatiotemporal data with a long time range, a 3D convolution kernel is more suitable for feature extraction of dairy cows’ basic motion behaviours. Based on image features, the traditional 3D convolutional neural network (CNN) requires many parameters and insufficient depth, and the robustness is always poor. To accurately recognize basic motion behaviours (walking, standing, and lying) of cows, in this research, basic motion behaviours based on cow skeletons and a mixed convolution algorithm were proposed. The depth of the 3D CNN was increased by connecting a deep 2D convolution in series connection after each 3D convolution, and a parallel 2D convolution was added on the basis of the series. Then, the 3D and 2D feature maps were correlated to share spatial information. Simultaneously, the key point information of the cow’s skeleton corresponding to the frame was added in the form of a heat map in the parallel 2D convolution feature. While increasing the depth of the 3D convolutional network, it effectively controlled the number of model parameters and robustness. Three hundred cow videos containing the three specific motion behaviours were selected for testing. The results showed that after 5-fold cross validation, the final classification ACC of this method was 91.80%, which was 3.40% higher than that of a mixed 3D/2D convolutional tube (MiCT). To verify the robustness of the method, a gamma transform was applied to adjust the image brightness to simulate real brightness changes. Under different brightness, the accuracy of this method had a maximum offset of 6.40%, which was significantly lower than that of temporal segment networks (TSNs) and MiCT. Furthermore, the classification ACC fluctuated slightly even when adding different degrees of random noise to the cow skeleton. All of the results showed that the proposed method was effective for the classification of walking, standing, and lying behaviours of cows and can be used to identify the basic motion behaviours of cows.