Hyperpointnet for Point Cloud Sequence-Based 3D Human Action Recognition

Abstract

Point cloud sequence-based 3D action recognition achieves impressive performance and efficiency. Conventional approaches for modeling point cloud sequences usually perform cross-frame spatio-temporal local encoding, thus resulting in intensive computation and mutual interference between spatial and temporal information extracting. In this work, to avoid spatio-temporal local encoding, we propose a strong parallelized point cloud sequence network referred to as HyperpointNet for 3D action recognition. HyperpointNet is composed of two serial modules, i.e., a hyperpoint sequence embedding module and a hyperpoint sequence encoding module. In the hyperpoint sequence embedding module, employing static point cloud modeling methods, the point cloud sequence is abstracted into a new point data type named hyper-point sequence. In the hyperpoint sequence encoding module, a temporal PointNet (TPN) layer is designed to model the hyperpoint sequence. Extensive experiments conducted on two public datasets show that HyperpointNet outperforms state-of-the-art approaches.