A behavioral conditional diffusion probabilistic model for human motion modeling in multi-action mixed human-robot collaboration

Abstract

Collision avoidance is priority for human-robot collaboration (HRC). Human motions have stochastic, making it difficult for robots to recognize humans. To establish accurate human motion models is important to recognize humans. In the collaboration process, the action categories may suddenly and continuously switch to adapt to complex production tasks, namely, the stochastic of human motions, and we call this collaboration process as multi-action mixed HRC. The previous HRC rarely considered the stochastic of human motions. However, the stochastic in multi-action mixed HRC brings great challenges to human motion modeling. On the one hand, the same observed motions may develop into different action categories, the motion distribution need to be predicted. On the other hand, the connected motions between two action categories are missing, and need to be predicted. To address these two problems, this study introduced diffusion probabilistic models to capture the stochastic of human motions. However, the current diffusion probabilistic models cannot realize the high-accuracy human motion prediction. Most of them did not consider the condition distribution in diffusion process, and did not comprehensively capture the spatiotemporal characteristics between human and robot. To solve the above problems, a behavioral conditional probabilistic diffusion model (BCDPM) is proposed. The BCDPM model can capture the stochastic of human motions to model the human motion distribution. A mask mechanism is combined with diffusion prior generated by BCDPM model to predict the missing connected motions. The BCDPM model consists of the designed constrain network, the denoising network, and enhanced memory graph neural network. The constrain network can generate the condition distribution, the denoising network and enhanced memory graph neural network can capture the short term and long term spatiotemporal characteristics, respectively. Moreover, the overlapping motion primitives are explicitly encoded into the implicit BCDPM model to mine commonalities of motions. The results show the proposed method can obtain excellent accuracy in prediction and outstanding F1 score in collision detection experiment.