Intra-class variations with deep learning-based gait analysis: A comprehensive survey of covariates and methods

Abstract

Gait recognition is an essential biometric technique that recognizes humans at a distance through their unique walking style. In the present era of deep learning, automated gait analysis has become easier with an increase in processing power. However, the recognition accuracy is affected by many covariates such as clothing conditions, carrying objects, varying viewing angles, occlusion, walking speed variations, and thus, it remains a challenging problem. For this complex problem, huge datasets are required to train for given conditions and predict new situations; thus, deep learning is preferred. In this review paper, we categorize various gait covariates which have been recently handled. There are various approaches, but the most effective approach is deep learning; hence in this paper, we include the most used deep learning approaches for each covariate condition found in the literature. Further, we highlight open problem areas handling these covariates and offer some suggestions about their better handling. Based on the review and our understanding of all the gait pipelines employed in deep learning, we have suggested a comprehensive and universal deep learning pipeline that can handle most gait covariates rather than customized deep learning pipelines. The methods of handling gait covariates are summarized according to appearance, pose, and sensors. A comprehensive comparison of reviewed approaches for real-time scenarios in terms of their novelty, benefits, and limitations is then offered, which led us to identify open research problems related to gait covariates. In the end, the paper concludes with the challenges identified and prospects.