Automatically recognizing four-legged animal behaviors to enhance welfare using spatial temporal graph convolutional networks

Abstract

Automatically recognizing animal behaviors in zoos and in national natural reserves can provide valuable insight into their welfare for facilitating scientific decision-making processes in animal management. Due to the difficulty of capturing massive amounts of animal video footage, a few existing methods have identified the behaviors of several different animal species in static images, but little is known about video-based animal behavior recognition. An animal's behavior is carried out in consecutive frames rather than in a single image; thus, image-based animal behavior recognition methods have low recognition accuracy. To address this dilemma, we not only construct the first skeleton-based dynamic multispecies dataset (Animal-Skeleton) but also propose a novel scheme that automatically designs the best spatial-temporal graph convolutional network (GCN) architecture with neural architecture search (NAS) to perform animal behavior recognition, named Animal-Nas for short. This is the first time that GCNs with NAS have been introduced into the animal behavior recognition task. To alleviate the trial-and-error cost of manually designing the network structure, we turn to NAS and design a novel search space with graph-based cells. Furthermore, we adopt a differentiable architecture search strategy to automatically search the cost-efficient spatial-temporal graph convolutional network structure. To evaluate the performance of the proposed model, we conduct extensive experiments on Animal-Skeleton datasets from three perspectives: model accuracy, parameter amount and stability. The results show that our model can achieve state-of the-art performance with fewer parameters.