Keypoint-enhanced adaptive weighting model with effective frequency channel attention for driver action recognition

Abstract

Traffic accidents caused by distracted driving have become a threat to people’s lives and properties, so it is necessary to recognize driver actions for early warning effectively. Since it is sometimes impossible to distinguish similar activities only by global driving image features, we explicitly extract the action-related keypoint features and propose a keypoint-enhanced model for classification. Specifically, we construct an effective frequency channel attention module to generate discriminative global representations. Considering the diversity of keypoint information, we design an adaptive weighted residual bottleneck to make the model weights of the input keypoint features dynamic. Furthermore, we propose a keypoint-guided conditional computation module. Under the guidance of keypoint features, the expert weights generated by conditional computation enable the model to adapt to different categories of driving images. Essentially, the model generates keypoint-enhanced attention to scale the classification feature channels. We also propose a model backbone training strategy combining self-supervised and supervised contrastive learning so that the model can achieve better results without large-scale labeled driver behavior data.