Gait Recognition System Research Articles

An autonomous decision-making framework for gait recognition systems against adversarial attack using reinforcement learning

Gait identification based on Deep Learning (DL) techniques has recently emerged as biometric technology for surveillance. We leveraged the vulnerabilities and decision-making abilities of the DL model in gait-based autonomous surveillance systems when attackers have no access to underlying model gradients/structures using a patch-based black-box adversarial attack with Reinforcement Learning (RL). These automated surveillance systems are secured, blocking the attacker’s access. Therefore, the attack can be conducted in an RL framework where the agent’s goal is determining the optimal image location, causing the model to perform incorrectly when perturbed with random pixels. Furthermore, the proposed adversarial attack presents encouraging results (maximum success rate = 77.59%). Researchers should explore system resilience scenarios (e.g., when attackers have no system access) before using these models in surveillance applications.

A flexible, stretchable and triboelectric smart sensor based on graphene oxide and polyacrylamide hydrogel for high precision gait recognition in Parkinsonian and hemiplegic patients

Intelligent gait recognition system plays an important role in the field of identity recognition, physical training and medical diagnostics. In clinical medicine, no definitive diagnostic tool has been developed for the diagnosis of Parkinson’s disease and hemiplegia. Thus, there is an urgent need to develop an effective and portable human-machine interaction system to monitor and recognize these symptoms. Herein, a self-powered strain sensor based on graphene oxide-polyacrylamide (GO-PAM) hydrogels is reported to monitor subtle human motions, including gait movements. The sensor can be used as a triboelectric nanogenerator (TENG) to collect mechanical energy. The output power of the TENG based on the 0.02 wt% GO-PAM hydrogel was up to 26 mW, which was 2.2 times that of the pure PAM hydrogel film. The capability of the TENG in powering electrical devices was demonstrated by lighting up 353 light-emitting diodes (LEDs) and powering an electronic thermometer. Besides, a wearable in-shoe monitoring system was designed which includes a flexible insole, a data processing module and a PC interface developed using Python. Among the models with different algorithms, the system with the artificial neural network (ANN) exhibits the highest recognition accuracy of 99.5 % and 98.2 % for human daily-life gait and pathological gait, respectively. This system provides a more convenient option for human gait monitoring and recognition, which can be used for a wide range of medical applications such as early diagnosis, rehabilitation evaluation and treatment of patients.

Gait phases recognition based on lower limb sEMG signals using LDA-PSO-LSTM algorithm

Gait phases are widely used in exoskeleton movement control. Surface electromyography (sEMG) is predictive and plays an important role in gait phase recognition. The purpose of this study is to improve the stability and accuracy of gait recognition methods based on the sEMG signals of lower limbs. First, we presented a LDA-PSO-LSTM algorithm based on feature combination selection and verified its recognition accuracy through experiments. LDA-PSO-LSTM had an average recognition rate of94.89% and a maximum accuracy of 97.02%. Second, we tested and compared the recognition accuracy of LDA-LSTM (92.17%). Experiments showed that the PSO optimization model had good recognition performance. Finally, we compared LDA-LSTM with all classifier combinations and concluded that the LDA-LSTM method has the highest recognition rate among a series of method combinations. The results indicated that LDA-PSO-LSTM as a classification model has apparent advantages in gait recognition. LDA-PSO-LSTM provides more accurate gait phase results for lower limb exoskeleton control. This method is beneficial to the development of the exoskeleton gait recognition system.

A multi-head pseudo nodes based spatial–temporal graph convolutional network for emotion perception from GAIT

In the recent years, the emotion recognition task has attracted a lot of attention in the field of human–computer interaction. Most existing research typically uses aural-visual analysis, which has been an effective approach to capturing emotional features. However, aural-visual signals are difficult to notice, in comparison to other human representations such as gait in remote situations. Recently, human gait can be effectively recognized in more complex backgrounds because of the advancement of Graph Convolutional Networks (GCNs). According to the anatomy of the human body, central torso joints play a key role in GCNs-based human gait recognition systems, instead of the body’s marginal limb joints. As a result, there is a major issue of receptive field imbalance. In this study, we propose a method for perceiving emotions based on the human gait skeleton. We present a multi-head pseudo nodes strategy to alleviate the receptive field imbalance problem and capture the non-local dependencies among different joints. The strategy employs a series of extra nodes that link to all physical human body joints and obtain global information from different feature spaces. The results of the experiments on a public emotion-gait dataset demonstrate that our proposed method outperforms existing skeleton-based methods. Further, to verify the effectiveness of our method, we use publicly available human action recognition datasets. Our results show that our method significantly improves performance in comparison to other baseline methods.

Wi-PIGR: Path Independent Gait Recognition With Commodity Wi-Fi

Wi-Fi based gait recognition has many potential applications. However, the gait information derived from Wi-Fi changes with the walking path. This makes the human identification through gait really challenging, the existing Wi-Fi based gait recognition systems require the subject walking along a predetermined path. This path dependence restriction impedes Wi-Fi based gait recognition from being widely used. In this paper, a path independent gait recognition system for a single subject, Wi-PIGR, is proposed. In Wi-PIGR, the subject is identified through the gait regardless of the walking path. Specifically, an extra receiver is introduced to get CSI data in orthogonal directions. A series of signal processing techniques are proposed to eliminate the differences among signals introduced by walking along the arbitrary paths and generate a high quality path independent signal spectrogram. Furthermore, a deep learning approach is integrated into the feature extraction. The experiment results in typical indoor environment demonstrate the superior performance of Wi-PIGR, with the average recognition accuracy of 77.15 percent, when the number of subjects is 50.

A Deep Learning Approach for Biometric Security in Video Surveillance System Using Gait

Video surveillance systems and biometrics inclusion play a significant part in various applications like a criminal investigation, medical rehabilitation, virtual reality, etc. Human Gait is a popular biometric where the individual is differentiated by unique limb actions and special ground reaction force. The unique movement of limb actions is called gait, and a record of 2-D aggregate floor response force through one walking cycle is called Cumulative Foot Pressure Images (CFPI). Both gait and cumulative foot pressure images can be acquired simultaneously of the same person during walking under a surveillance system for human identification. Accurate gait recognition is highly impactful for most applications and a major challenge for researchers due to various external factors like different shoes, mood, clothes, injuries etc., affects the individual gait. The novel system addresses the accuracy issue and proposes two models using the Deep Convolution Neural Network (DCNN) architecture on a large standard database, CASIA-D, containing gait pose and CFPI images of the same person. First, the model of the DCNN is trained using unique Gait Energy Image (GEI) features which reduce the computational time compared to other types of feature sets. The second model is prepared using the CFPI features. Experimentation has been carried out to evaluate the performance of these models with different optimization methods and activation functions and has proven that the DCNN model is far superior in building an accurate gait recognition system on large standard datasets.

Gait Recognition Using Spatio-Temporal Information of 3D Point Cloud via Millimeter Wave Radar

Gait recognition is one of the crucial methods in identity recognition, which has a wide range of applications in many fields, such as smart home, smart office, and health monitoring. The camera is the most mainstream traditional solution. But the camera is difficult to maintain stable performance in the dark, low light, and bad weather conditions. In addition, privacy leakage is also one of the important issues that people worry about. In contrast, as the latest research progress in gait recognition, millimeter wave radar can not only protect people’s privacy, but also maintain normal perception performance in dark conditions. In this paper, we propose a system for gait recognition named MTPGait using spatio-temporal information via millimeter wave radar. We specially design a neural network that can extract multiscale spatio-temporal features along space and time dimensions of 3D point cloud concisely and efficiently. We use LSTM to design the context flow of local and global time and space, fusing local and global spatio-temporal features. In addition, we construct and release a millimeter wave radar 3D point cloud data set, which consists of 960-minute gait data of 40 volunteers. Using the data set, we evaluate the system and compare it with four state-of-the-art algorithms. The experimental results show that MTPGait is able to achieve 96.7% recognition accuracy in a single-person scene on fixed route and 90.2% recognition accuracy when two people coexist, while none of the existing methods is more than 90% recognition accuracy in either scenario.

Utilizing Spatio Temporal Gait Pattern and Quadratic SVM for Gait Recognition

This study aimed to develop a vision-based gait recognition system for person identification. Gait is the soft biometric trait recognizable from low-resolution surveillance videos, where the face and other hard biometrics are not even extractable. The gait is a cycle pattern of human body locomotion that consists of two sequential phases: swing and stance. The gait features of the complete gait cycle, referred to as gait signature, can be used for person identification. The proposed work utilizes gait dynamics for gait feature extraction. For this purpose, the spatio temporal power spectral gait features are utilized for gait dynamics captured through sub-pixel motion estimation, and they are less affected by the subject’s appearance. The spatio temporal power spectral gait features are utilized for a quadratic support vector machine classifier for gait recognition aiming for person identification. Spatio temporal power spectral preserves the spatiotemporal gait features and is adaptable for a quadratic support vector machine classifier-based gait recognition across different views and appearances. We have evaluated the gait features and support vector machine classifier-based gait recognition on a locally collected gait dataset that captures the effect of view variance in high scene depth videos. The proposed gait recognition technique achieves significant accuracy across all appearances and views.

Exploiting vulnerability of convolutional neural network-based gait recognition system

Exploiting vulnerability of convolutional neural network-based gait recognition system

A Faster R-CNN and recurrent neural network based approach of gait recognition with and without carried objects

Gait recognition is the identification of any person from his/her walking pattern. Walking pattern of each individual is unique and cannot be replicated by others. But, gait recognition is very difficult if any object is carried by any individual. This article proposes a novel computer vision based method of gait recognition both with and without carried objects (COs) using Faster region convolutional neural network (R-CNN) based architecture. To the best of my knowledge, this is the first investigation based on faster R-CNN for gait recognition in the literature. The Faster R-CNN detects and extracts the pedestrian only in all the frames of the video irrespective of the pedestrian is carrying any object or not. Deep convolutional layers have then been used to generate the feature vector from the walking pattern of the pedestrian generated comprising all the frames. The generated feature vectors from various walking patterns have then been studied using two different versions of recurrent neural network (RNN) namely, long–short term memory (LSTM) and bidirectional long–short term memory (BLSTM) to recognize the walking patterns. To the best of my knowledge, the present investigation uses the BLSTM variant of RNN classifier to recognize the walking patterns for the first time in the literature. The performance of the proposed system has been tested on four widely used public datasets—OU-ISIR Large Population Gait database with real-life carried object (OU-LP-Bag), the OU-ISIR Gait database Treadmill dataset B (OUTD-B), OU-ISIR Large Population Gait database with Age (OULP-Age), and CASIA Gait database B (CASIA-B). The experimental results demonstrate that the proposed gait recognition system outperforms the existing state-of-the-art results.

Hybrid Optimized GRU-ECNN Models for Gait Recognition with Wearable IOT Devices.

With the advent of the Internet of Things (IoT), human-assistive technologies in healthcare services have reached the peak of their application in terms of diagnosis and treatment process. These devices must be aware of human movements to provide better aid in clinical applications as well as the user's daily activities. In this context, real-time gait analysis remains to be key catalyst for developing intelligent assistive devices. In addition to machine and deep learning algorithms, gait recognition systems have significantly improved in terms of high accuracy recognition. However, most of the existing models are focused on improving gait recognition while ignoring the computational overhead that affects the accuracy of detection and even remains unsuitable for real-time implementation. In this research paper, we proposed a hybrid gated recurrent unit (GRU) based on BAT-inspired extreme convolutional networks (BAT-ECN) for the effective recognition of human activities using gait data. The gait data are collected by implanting the wearable Internet of Things (WIoT) devices invasively. Then, a novel GRU and ECN networks are employed to extract the spatio-temporal features which are then used for classification to realize gait recognition. Extensive and comprehensive experimentations have been carried out to evaluate the proposed model using real-time datasets and also other benchmarks such as whuGait and OU-ISIR datasets. To prove the excellence of the proposed learning model, we have compared the model's performance with the other existing hybrid models. Results demonstrate that the proposed model has outperformed the other learning models in terms of high gait classification and less computational overhead.

Gait Analysis for Surveillance

Gait refers to the patterns of movements the limbs make when walking. Human gaits refer to the variety of ways in which an individual can move, either naturally or as a result of training. The gait of each individual is as unique as their voice. On the basis of this knowledge, Machine Learning (ML) algorithms have been developed for gait recognition. Computer Vision (CV) techniques have facilitated the development of a wide range of approaches for identifying people by their movements in videos using both natural biometric characteristics (the human skeleton, silhouette, changes during walking) and abstractions. A gait recognition system identifies the human body based on its shape and the way it moves. A machine-learning system can recognize a person even if their face is hidden, turned away from the camera, or concealed behind a mask. An algorithm analyzes a person's silhouette, height, speed, and walking pattern to identify him or her. Gait recognition technology acquires data from multiple sources, such as video cameras and motion sensors. Data from these sources are then processed by a number of algorithms. Gait is recognized, data is processed, contours and silhouettes are detected, and individual features are segmented, according to the algorithm. After this, the feature extraction algorithm takes effect - this is what differentiates one gait from another. There are many different algorithmic requirements, and these algorithms can vary. Some algorithms, for example, are designed to process video information, while others employ sensor data. Because each gait is distinct, the identification algorithms are always confronted with new data. The system will assess future data better if it detects more gait variants. Assume the program compares two gaits that are highly similar. The algorithms for pattern recognition and silhouette segmentation have been trained to separate the tiny details and enter them into the database. This enables for more accurate gait categorization and improved results in the future.

A Lightweight System Towards Viewing Angle and Clothing Variation in Gait Recognition

In this paper, the authors have proposed a computationally efficient, robust, and lightweight system for gait recognition. The proposed system contains two main stages: In the first stage, a classification network identifies optical flow corners in the normalized silhouette and calculates the distances traveled in every viewpoint which is further used by a regression model to identify the viewing angle. In the second stage, a feature extraction network computes the gait energy image (GEI) for every viewpoint and then uses principal component analysis (PCA) to extract low dimensional feature vectors from these GEI images. Finally, a multi-layer perceptron model is trained using the extracted principal components for every viewing angle. The performance of a system is comprehensively evaluated on the CASIA B and OULP gait dataset. The experimental results demonstrate the superior performance of a proposed system in viewing angle classification (100% accuracy), gait recognition (100% accuracy in normal walk), computational efficiency, robustness to clothing, and viewing angle variation.

Deep Learning and Kurtosis-Controlled, Entropy-Based Framework for Human Gait Recognition Using Video Sequences

Gait is commonly defined as the movement pattern of the limbs over a hard substrate, and it serves as a source of identification information for various computer-vision and image-understanding techniques. A variety of parameters, such as human clothing, angle shift, walking style, occlusion, and so on, have a significant impact on gait-recognition systems, making the scene quite complex to handle. In this article, we propose a system that effectively handles problems associated with viewing angle shifts and walking styles in a real-time environment. The following steps are included in the proposed novel framework: (a) real-time video capture, (b) feature extraction using transfer learning on the ResNet101 deep model, and (c) feature selection using the proposed kurtosis-controlled entropy (KcE) approach, followed by a correlation-based feature fusion step. The most discriminant features are then classified using the most advanced machine learning classifiers. The simulation process is fed by the CASIA B dataset as well as a real-time captured dataset. On selected datasets, the accuracy is 95.26% and 96.60%, respectively. When compared to several known techniques, the results show that our proposed framework outperforms them all.

Wearable Gait Recognition Systems Based on MEMS Pressure and Inertial Sensors: A Review

Gait is a basic characteristic of human motion. Different gaits are usually associated with different body functions. Gait recognition has wide applications in clinical medicine, rehabilitation training, posture recognition, and other fields. At present, many wearable systems based on MEMS pressure and inertial sensors have been developed and used for gait recognition. However, there has been a lack of a comprehensive summary and comparison of these systems from the perspectives of their hardware compositions, working principles, algorithm models, and applications. This review aims to promote the development of wearable gait recognition devices. First, sensor technologies for gait measurement are summarized from the perspectives of their working principles and characteristics. Then, these technologies are compared in terms of the performance of algorithms for data preprocessing, cycle and phase segmentation, and gait recognition. Next, the applications of MEMS sensor-based gait recognition systems in various fields are summarized. Finally, some limitations of existing wearable gait recognition systems and their future directions are discussed.

Anonymization of Human Gait in Video Based on Silhouette Deformation and Texture Transfer

These days, a lot of videos are uploaded onto web-based video sharing services such as YouTube. These videos can be freely accessed from all over the world. On the other hand, they often contain the appearance of walking private people, which could be identified by silhouette-based gait recognition techniques rapidly developed in recent years. This causes a serious privacy issue. To avoid it, this paper proposes a method for anonymizing the appearance of walking people, namely human gait, in video. In the proposed method, we first crop human regions from all frames in an input video and binarize them to get their silhouettes. Next, we slightly deform the silhouettes from the aspects of static body shape and dynamic walking rhythm so that the person in the input video cannot be correctly identified by gait recognition techniques. After that, the textures of the original human regions are transferred onto the deformed silhouettes. We achieve this by a displacement field-based approach, which is training-free and thus robust to a variety of clothes. Finally, the anonymized human regions with the transferred textures are filled back into the input video. In the results of our experiments, we successfully degraded the accuracy of CNN-based gait recognition systems from 100% to 1.57% in the lowest case without yielding serious distortion in the appearance of the human regions, which demonstrated the effectiveness of the proposed method.

<P Class="Elsarticletitle"><Span Lang="En-Us">Model-Based Gait Recognition Using Graph Network on Very Large Population Database<O:P></O:P></Span></P>

At present, the existing gait recognition systems are focusing on developing methods to extract robust gait feature from silhouette images and they indeed achieved great success. However, gait can be sensitive to appearance features such as clothing and carried items. Compared with appearance-based method, model-based gait recognition is promising due to the robustness against these variations. In recent years, with the development of human pose estimation, the difficulty of model-based gait recognition methods has been mitigated. In this paper, to resist the increase of subjects and views variation, local features are built and a siamese network is proposed to maximize the distance of samples from the same subject. We leverage recent advances in action recognition to embed human pose sequence to a vector and introduce Spatial-Temporal Graph Convolution Blocks (STGCB) which has been commonly used in action recognition for gait recognition. Experiments on the very large population dataset named OUMVLP-Pose and the popular dataset, CASIA-B, show that our method archives some state-of-the-art (SOTA) performances in model-based gait recognition. The code and models of our method are available at https://github.com/timelessnaive/Gait-for-Large-Dataset after being accepted.

Multiview Gait Recognition on Unconstrained Path Using Graph Convolutional Neural Network

Human gait recognition is a valuable biometric trait with vast applications in security domain. In most situations, the gait data is collected while the subject walks straight. Thus, the performance of the gait recognition system degrades when the subject changes walking direction. Previous gait recognition research was predominantly conducted for constrained paths, which limited the system’s robustness and applicability. This paper introduces a novel approach for gait recognition which aims to recognize subjects walking along an unconstrained path. A graph neural network-based method is proposed for gait recognition along unconstrained path. The input of the architecture is the body joint coordinates and adjacency matrix representing the skeleton joints. Furthermore, a residual connection is incorporated to produce a smoothened output of the input feature. This graph neural network model utilizes the kinematic relationships of the body joints as well as spatial and temporal features. The findings demonstrate that the proposed method outperformed other state-of-the-art gait recognition methods on unconstrained paths. Multi-view Gait AVA and CASIA-B dataset are used to evaluate the efficacy of the proposed method.

Real-world smartphone-based gait recognition

As the smartphone and the services it provides are becoming targets of cybercrime, it is critical to secure smartphones. However, it is important security controls are designed to provide continuous and user-friendly security. Amongst the most important of these is user authentication, where users have experienced a significant rise in the need to authenticate to the device and individually to the numerous apps that it contains. Gait authentication has gained attention as a mean of non-intrusive or transparent authentication on mobile devices, capturing the information required to verify the authenticity of the user whilst the person is walking. Whilst prior research in this field has shown promise with good levels of recognition performance, the results are constrained by the gait datasets utilised being based upon highly controlled laboratory-based experiments which lack the variability of real-life environments. This paper introduces an advanced real-world smartphone-based gait recognition system that recognises the subject within real-world unconstrained environments. The proposed model is applied to the uncontrolled gait dataset, which consists of 44 users over a 7–10 day capture – where users were merely asked to go about their daily activities. No conditions, controls or expectations of particular activities were placed upon the participants. The experiment has modelled four types of motion normal walking, fast walking and down and upstairs for each of the users. The evaluation of the proposed model has achieved an equal error rate of 11.38%, 11.32%, 24.52%, 27.33% and 15.08% for the normal, fast, down and upstairs and all activities respectively. The results illustrate, within an appropriate framework, that gait recognition is a viable technique for real-world use.

Gait Recognition System: A Survey

Human identification by gait has created a great deal of interest in computer vision community due to its advantage of inconspicuous recognition at a relatively far distance. This paper provides a comprehensive survey of recent developments on gait recognition approaches. The survey emphasizes on three major issues involved in a general gait recognition system, namely gait image representation, feature dimensionality reduction and gait classification. It also gives, a review of the available public gait datasets. The concluding discussions outline a number of research challenges and provide promising future directions for the field. In this article presents a survey of the work done in gait analysis for re-identification in the past decade, looking at the main approaches, datasets, and evaluation methodologies also discussed.