Passenger non-driving related tasks detection using a light weight neural network based on human prior knowledge and soft-hard feature constraints

Abstract

Many evidences prove that the development of autonomous cars increases the probability of motion sickness when passengers perform non-driving related tasks. Making the autonomous cars know what the passenger is doing is necessary to adjustment optimal strategy of vehicle motion style and alleviate motion sickness used the human–machine interaction method automatically. For autonomous cars, passengers’ non-drive related task detecting is a non-main task, and it is necessary to reduce the occupation of computing resources, the current study did not address this issue. Therefore, we have choose several types of behaviors based on current motion sickness research literature to improve motion sickness from a human–computer interaction perspective and build a dataset, then, a lightweight model for the detection of passenger non-driving related tasks is proposed. In order to reduce computational load, feature redundancies and improve the accuracy, the soft-hard features constraints method is proposed based on the human prior knowledge. We annotated the key features regions of the images and make the network learn the labeled regions used a two-layer feature constraint structure, which allow the model to learn labeled features earlier, and the label smooth regularization method is used to improve the robustness. The proposed model used the windowed attention mechanism to calculate the importance of each neuron in the feature map and reduce the number of parameters. Finally, we used a visualization method visualize features related to passenger tasks. The results show that the proposed model has an accuracy of 0.9710 with 2.5 M Params, and 1.15G FLOPs, compared with other models, the proposed lightweight model has the highest accuracy.