FGDA-GS: Fast guided decision attack based on gradient signs for skeletal action recognition

Abstract

The black-box adversarial attacks for skeletal action recognition currently construct high-quality adversarial examples by sacrificing queries and perturbation imperceptibility. To overcome the above drawbacks, we propose a novel fast guided decision attack based on gradient signs for skeletal action recognition (FGDA-GS) scheme. Firstly, to reduce the queries, we guide the adversary to attack along the gradient direction obtained by sampling in a custom Gaussian distribution and design a gradient direction estimation method based on successful history information queries, which can quickly determine the updated direction of the effective perturbation without a large query. Secondly, to ensure the imperceptibility of the adversarial examples, we design a manifold projection function matching the first-order derivatives based on joint position constraints, joint angle constraints, and bone-length constraints to generate complete and natural skeletal adversarial examples that are invisible to the human eye. Finally, comprehensive experimental results demonstrate that FGDA-GS can generate strong-natured, highly effective adversarial examples with significantly reduced query. Specifically, FGDA-GS achieves a 100% attack success rate on different datasets and models with reduced queries by hundreds to thousands and reduced time by more than half. Particularly in the untargeted attack setting, compared to BASAR, FGDA-GS outperforms in terms of average joint position deviation, average joint acceleration deviation, average joint angle acceleration deviation, and average bone-length deviation percentage for the ST-GCN model on the NTU dataset, with percentage reductions of 91.13%, 91.81%, 80.60%, and 81.08%, respectively. We also conducted experiments on defense methods to validate our proposed scheme's effectiveness further.