Human Gait Recognition Research Articles

DCapsNet: Deep capsule network for human activity and gait recognition with smartphone sensors

Recently, deep neural networks are used to recognize human activity/gait through mobile sensors which have attracted a great attention. Although the existing deep neural networks that perform automatic feature extraction have achieved desirable performance, their classification accuracy needs to be improved. In this paper, a deep neural network that combines a set of convolutional layers and capsule network is proposed. The proposed architecture named DCapsNet is suited to automatically extract the activity or gait features through built in sensors and classify them. The convolutional layers of the DCapsNet are more suitable for processing temporal sequences and provide scalar outputs but not the equivariance. The capsule network (CapsNet) is then trained by a dynamic routing algorithm to capture the equivariance having a magnitude and orientation, which increases the efficiency of the model classification. The performance of the proposed model is evaluated on four public datasets: two HAR datasets (UCI-HAR and WISDM) and two gait datasets (WhuGAIT). The recognition accuracy of the proposed model for the UCI-HAR and WISDM datasets are 97.92 % and 99.30 %, respectively, and for the WhuGAIT Dataset #1 and Dataset #2 are 94.75 % and 97.16 %, respectively. Experimental results show that the proposed model achieves the highest recognition accuracy over the reported results of the state-of-the-art models.

Machine learning models and methods for human gait recognition

The paper explores the challenge of human identification through gait recognition within biometric identification systems. Itoutlines the essential criteria for human biometric features, discusses primary biometric characteristics, and their application in biometric identification systems. The paper also examines the feasibility of utilizing gait as a biometric identifier, emphasizing its advantages, such as not requiring the upfront provision of personal biometric information and specialized equipment.The authors conduct an analysis of existing scientific literature in the field of gait recognition, categorizing gait recognition methodsinto template-based and non-template-based approaches. Throughout their research, they identify the key issues and challenges that researchers face in this domain, along with the prevailing trends in human gait recognition within biometric identification systems.Additionally, the paper introduces a method for person identification based on gait, utilizing the Histogram of Oriented Gradients and the Sum Variance Haralick texture features. It involves transforming input video into a series of images depicting the gait silhouette, creating a Gait Energy Image (GEI) by combining these gait silhouettes throughout a gait cycle, and translating the GEI into the Gait Gradient Magnitude Image (GGMI). The subsequent step involves extracting recommended gait characteristics from the GGMIs of participants included in a dataset.To preprocess the collected characteristics, Principal Component Analysis (PCA) is applied, reducing the dimensions that may negatively impact classification robustness, thereby enhancing overall performance. In the final step, a K-Nearest Neighbors (KNN) classifier is employed to categorize the characteristics obtained from a specific dataset.The proposed novel feature vector in the paper demonstrates increased reliability and effectively captures spatial variations in gait patterns. Notably, it reduces the dimensionality of the feature vector from 3780×1 to 63×1, resulting in decreased computational complexity in the gait recognition system. Experimental evaluations on the CASIA A and CASIA B datasets reveal that the proposed approach outperforms other HOG-based methods in most scenarios, with the exception of situations involving frontal images.

A deep learning approach for human gait recognition from time-frequency analysis images of inertial measurement unit signal

Biomechanical analysis using deep learning has been increasingly used in recent studies to identify human activity. Wearable sensor data from inertial measurement units (IMUs) is widely used for recognizing human activity, but has several drawbacks owing to its high volume and diversity. To overcome these issues, the time-domain and power spectral characteristics of IMU data can be extracted using digital signal processing (DSP) methods. Our research aimed to investigate time-frequency analysis (TFA) methods for classifying the spatio-temporal gait characteristics of physical walking performed by healthy subjects. In this study, open-source biomechanical sensor signal dataset was used. The DSP step was first carried out by segmenting IMU data from the four body segments of 22 healthy subjects, and then by applying Continuous Wavelet Transform (CWT) and Short Time Fourier Transform (STFT) methods. Moreover, the resultants of linear accelerometer signals were applied in a similar manner. The image datasets obtained from this step were applied to a deep convolutional neural network (CNN) model to classify human walking speed (WS) into three classes: fast, normal, and slow. The performance of the 2D-CNN model and the impact of DSP methods using IMU data were evaluated. In conclusion, the highest test classification results demonstrated that STFT-all (85.9%), CWT-all (79.3%), and CWT-resAcc (76.3%) based CNN models present a remarkably precise and easy-to-implement classification problem, with the highest test accuracy, when all IMU channels are subjected to STFT. The classification accuracies of 2D-CNN models were compared to commonly used ML models. The Deep CNN model is a useful gait evaluation tool for healthy subjects. Furthermore, it can enable the diagnosis and phase assessment of gait abnormalities and detect gait biomarkers in rehabilitative wearables.

Gait Recognition Based on Minirocket with Inertial Measurement Units

Gait recognition is one of the key technologies for exoskeleton robot control. The recognition accuracy and robustness of existing gait recognition methods cannot well meet the needs of real-time control. There is still room for improvement in fine-grained gait recognition. In this regard, this paper proposes a gait recognition method based on the MiniRocket and inertial measurement units. In this paper, a human lower limb posture information collection device is developed to collect ten kinds of gait data of human lower limbs (walking, standing, running, backing off, going upstairs, going downstairs, going uphill, going downhill, stand at ease and squat). The MiniRocket algorithm was used to build a human gait recognition model, and the effects of algorithm parameters and the size of the window and shift on the performance of gait recognition were discussed, and user-independent experiments and user-dependent experiments were carried out, respectively, and compared with four algorithms of TST, TCN, RNN and LSTM. The experimental results show that the MiniRocket algorithm has an average recognition accuracy of 94.87% and 97.67% in the user-independent experiment and the user-dependent experiment, which is better than the other four algorithms. And the effectiveness of the method in the IMUs-based human gait recognition problem is shown, which provides some implications for fine-grained gait recognition.

A multi-view human gait recognition using hybrid whale and gray wolf optimization algorithm with a random forest classifier

Gait recognition has become one of the furthest promising behavioral biometric techniques for identifying individuals. Gait recognition models are capable of identifying humans at a distance based on their walking manner without their permission or interference. However, it is usually noted that the performance of a gait recognition approach will drop drastically in the presence of covariates such as carrying conditions, clothing conditions, and variations in the view angle. Therefore, it is necessary to develop a robust gait recognition system in order to identify the most significant gait features. In this paper, we introduced a recently developed hybrid whale and gray wolf optimization algorithm (WGWOA) for determining the optimal subset of gait features by combining the advantages of whale optimization and gray wolf optimization techniques. Moreover, we employed principal component analysis (PCA) to extract the essential gait features from the gradient gait energy image (GGEI) and random forest (RF) approach to classify the optimal gait features. The proposed method has been assessed on the publicly available largest multi-view CASIA-B and OU-MVLP benchmark datasets. Experimental results indicate that the proposed model achieved an accuracy of 99.25%, 98.39%, and 97.97% under normal, carrying a bag and wearing coat walking conditions, respectively on the CASIA-B dataset. Furthermore, the proposed model achieved an accuracy of 97.63% under normal conditions on the OU-MVLP dataset. The comparative results also confirm that the proposed algorithm is superior to the contemporary approaches.

TS[formula omitted]HGRNet: A paradigm of two stream best deep learning feature fusion assisted framework for human gait analysis using controlled environment in smart cities

Industrial advancements and large financial stakes are rousing to grasp the aims of smart cities. Their key aim is to increase efficiency and citizens’ quality of life. Human Gait Recognition (HGR) is an important application of smart cities in which a person is recognized with the help of walking styles. The main advantage of HGR is an individual’s inconspicuous distant gait data acquisition. It is being utilized to mitigate security threats at airports, banks, embassies & various corporate vicinities. In this paper, a deep learning method is proposed for HGR. The proposed methodology utilizes a large Multiview gait database CASIA-B comprising video sequences. Initial video processing in frames precedes the implementation of the pre-trained deep model DenseNet-201. Reutilizing the same for gait recognition using Transfer Learning (TL) yields the target model, which further acts as the source for feature extraction from the global average pool layer. Feature extraction as a dominant step is responsible here for accurate gait recognition. Poor lighting, varying view angles, diverse clothing conditions, carrying luggage, etc., are some constraints that adversely affect systems’ efficiency. To tackle this situation, we use two parallel techniques in the process as- improved BAT algorithm (IBAT) and Entropy selection. We use a Canonical Correlation Analysis (CAA) to fuse these carefully selected features into a matrix finally. Final gait recognition is achieved by passing these fused features from the Softmax classifier. The proposed technique reaches 96.13% and 95.2% accuracy by utilizing all angles of the CASIA-B and CASIA-C datasets. The authenticity of this work is cross-examined with the existing methods.

Wearable Triboelectric Nanogenerator with Ground-Coupled Electrode for Biomechanical Energy Harvesting and Sensing.

Harvesting biomechanical energy for electricity as well as physiological monitoring is a major development trend for wearable devices. In this article, we report a wearable triboelectric nanogenerator (TENG) with a ground-coupled electrode. It has a considerable output performance for harvesting human biomechanical energy and can also be used as a human motion sensor. The reference electrode of this device achieves a lower potential by coupling with the ground to form a coupling capacitor. Such a design can significantly improve the TENG's outputs. A maximum output voltage up to 946 V and a short-circuit current of 36.3 μA are achieved. The quantity of the charge that transfers during one step of an adult walking reaches 419.6 nC, while it is only 100.8 nC for the separate single-electrode-structured device. In addition, using the human body as a natural conductor to connect the reference electrode allows the device to drive the shoelaces with integrated LEDs. Finally, the wearable TENG is able to perform motion monitoring and sensing, such as human gait recognition, step count and movement speed calculation. These show great application prospects of the presented TENG device in wearable electronics.

A comprehensive survey on deep-learning based gait recognition for humans in the COVID-19 pandemic

Human gait recognition is a biometric technique that has been utilized for security purposes for the last decade. Gait recognition is an appealing biometric modality that aims to identify individuals based on the way they walk. The outbreak of the novel coronavirus (COVID-19), has spread across the world. The number of people infected with COVID-19 is rising rapidly throughout the world. Even though some vaccines for this pandemic have been developed to minimize the effects of COVID-19, deep learning-based gait recognition techniques have shown themselves to be an effective tool for identifying the individuals wearing face mask in COVID-19 pandemic. These techniques play an important part in reducing the rate of COVID-19 spreading throughout the world in the context of the COVID-19 pandemic. Deep learning methods are currently dominating the state-of-the-art in gait recognition and have fostered real-world applications. The main objective of this paper is to provide a comprehensive overview of recent advancements in gait recognition with deep learning, including datasets, test protocols, state-of-the-art solutions, challenges, and future research directions. The purpose of this discussion is to identify current challenges that need to be addressed as well as to suggest some directions for future research that could be explored.

HPO Using Dwarf Mongoose Optimization in the GAN Model for Human Gait Recognition

HPO Using Dwarf Mongoose Optimization in the GAN Model for Human Gait Recognition

AutoFi: Toward Automatic Wi-Fi Human Sensing via Geometric Self-Supervised Learning

Wi-Fi sensing technology has shown superiority in smart homes among various sensors for its cost-effective and privacy-preserving merits. It is empowered by channel state information (CSI) extracted from Wi-Fi signals and advanced machine learning models to analyze motion patterns in CSI. Many learning-based models have been proposed for kinds of applications, but they severely suffer from environmental dependency. Though domain adaptation methods have been proposed to tackle this issue, it is not practical to collect high-quality, well-segmented, and balanced CSI samples in a new environment for adaptation algorithms, but randomly captured CSI samples can be easily collected. In this article, we first explore how to learn a robust model from these low-quality CSI samples, and propose AutoFi, an annotation-efficient Wi-Fi sensing model based on a novel geometric self-supervised learning algorithm. The AutoFi fully utilizes unlabeled low-quality CSI samples that are captured randomly, and then transfers the knowledge to specific tasks defined by users, which is the first work to achieve cross-task transfer in Wi-Fi sensing. The AutoFi is implemented on a pair of Atheros Wi-Fi APs for evaluation. The AutoFi transfers knowledge from randomly collected CSI samples into human gait recognition and achieves state-of-the-art performance. Furthermore, we simulate cross-task transfer using public data sets to further demonstrate its capacity for cross-task learning. For the UT-HAR and Widar data sets, the AutoFi achieves satisfactory results on activity recognition and gesture recognition without any prior training. We believe that AutoFi takes a huge step toward automatic Wi-Fi sensing without any developer engagement. Our codes have been included in <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/xyanchen/Wi-Fi-CSI-Sensing-Benchmark</uri> .

Gait recognition methods in the task of biometric human identification

This article focuses on defining the problem of solving the problem of human identification by means of gait recognition in bi-ometric identification systems.In order to determine the prospects of using gait recognition methods for human identification, a gen-eralized model of a biometric identification system was considered, the main modules of the system were identified and a brief de-scription of each module was provided.Next, the basic requirements for human biometric features were identified, the main bio-metric features were considered, and the features of their use in biometric identification systems were determined.The issue of using gait as a biometric identifier was also considered. It has been determined that the use of human gait allows to get rid of two main obstacles in the construction of biometric identification systems: users are not required to provide personal biometric information in advance, and the system does not require specialized equipment.Also, the issue of multi-view gait recognition was considered. Mul-ti-view gait recognition involves capturing gait data from different angles and using this data to improve recognition accuracy.This approach has shown great promise in challenging scenarios such as low lighting conditions.Next, we analyzed scientific works in the field of gait recognition. It was determined that gait recognition methods can be divided into template-based and non-template-based methods.Template-based methods are aimed at obtaining patterns of torso or leg movements, i.e. they usually focus on the dynamics of movement in space or on spatio-temporal methods. Non-template-based methods consider shape and its features as more relevant characteristics, i.e., human recognition areperformed using measurements that reflect the shape of the person.Next, we consider the use of different datasets in the process of training and testing human gait recognition methods. The main datasets were identified and their characteristics and features were collected.We considered the presence of various characteristics in the datasets, as well as the means of representing information about human gait.The researchhas identified the main problems and challenges facing research-ers in this area, as well as the main trends in the field of human gait recognition in biometric identification systems.

Performance Analysis for Human Gait Recognition from different Viewpoints

Human gait, a new biometric aimed at recognizing people based on how they walk, has become increasingly important in visual surveillance application. However, one camera at a single view point gait data has commonly been explored, this has not always sufficient enough in the environment of deployment. This research proposes gait analysis as a solution for subjects’ identification across a network of cameras from different viewpoints. Gait signature of a person is created from Temporal and spatial metrics extracted from modal, such as length of trunk, shin and deviation in the limb angles or the amplitude of a person’s walking pattern and these are transformed into a self-similarity matrix. The method of spatio-temporal correlation is to detect the human gait in successive video sequences. Performance evaluation of the system was carried out with CMU (Carnegie Mellon University) Motion of Body (MoBo) database. The results reveal that gait performance analysis of the proposed system is possible even without knowing the position of the camera or the stance of the subject. This research shows that the derived gait parameters suggest that gait may be successfully employed for individuals' identification with cameras surveillance in different view point scenario which resulted in an average recognition rate of 73.3% for persons walking from right camera viewpoints, 80.0% from persons walking from the left camera viewpoint, 60.0% from persons walking from the rare camera viewpoint and 66.67% from persons walking from the front camera viewpoint. This implies that the system performs better from the left camera viewpoint.

Model-based and model-free deep features fusion for high performed human gait recognition.

In the last decade, the need for a non-contact biometric model for recognizing candidates has increased, especially after the pandemic of COVID-19 appeared and spread worldwide. This paper presents a novel deep convolutional neural network (CNN) model that guarantees quick, safe, and precise human authentication via their poses and walking style. The concatenated fusion between the proposed CNN and a fully connected model has been formulated, utilized, and tested. The proposed CNN extracts the human features from two main sources: (1) human silhouette images according to model-free and (2) human joints, limbs, and static joint distances according to a model-based via a novel, fully connected deep-layer structure. The most commonly used dataset, CASIA gait families, has been utilized and tested. Numerous performance metrics have been evaluated to measure the system quality, including accuracy, specificity, sensitivity, false negative rate, and training time. Experimental results reveal that the proposed model can enhance recognition performance in a superior manner compared with the latest state-of-the-art studies. Moreover, the suggested system introduces a robust real-time authentication with any covariate conditions, scoring 99.8% and 99.6% accuracy in identifying casia (B) and casia (A) datasets, respectively.

A Fusion-Assisted Multi-Stream Deep Learning and ESO-Controlled Newton–Raphson-Based Feature Selection Approach for Human Gait Recognition

The performance of human gait recognition (HGR) is affected by the partial obstruction of the human body caused by the limited field of view in video surveillance. The traditional method required the bounding box to recognize human gait in the video sequences accurately; however, it is a challenging and time-consuming approach. Due to important applications, such as biometrics and video surveillance, HGR has improved performance over the last half-decade. Based on the literature, the challenging covariant factors that degrade gait recognition performance include walking while wearing a coat or carrying a bag. This paper proposed a new two-stream deep learning framework for human gait recognition. The first step proposed a contrast enhancement technique based on the local and global filters information fusion. The high-boost operation is finally applied to highlight the human region in a video frame. Data augmentation is performed in the second step to increase the dimension of the preprocessed dataset (CASIA-B). In the third step, two pre-trained deep learning models-MobilenetV2 and ShuffleNet-are fine-tuned and trained on the augmented dataset using deep transfer learning. Features are extracted from the global average pooling layer instead of the fully connected layer. In the fourth step, extracted features of both streams are fused using a serial-based approach and further refined in the fifth step by using an improved equilibrium state optimization-controlled Newton-Raphson (ESOcNR) selection method. The selected features are finally classified using machine learning algorithms for the final classification accuracy. The experimental process was conducted on 8 angles of the CASIA-B dataset and obtained an accuracy of 97.3, 98.6, 97.7, 96.5, 92.9, 93.7, 94.7, and 91.2%, respectively. Comparisons were conducted with state-of-the-art (SOTA) techniques, and showed improved accuracy and reduced computational time.

Review of Human-exoskeleton Control Strategy for Lower Limb Rehabilitation Exoskeleton

The research on exoskeleton robots has been widely carried out for many years around the world, especially the development of new-style lower limb exoskeletons for rehabilitation and assistance is one of the key research directions. The focus on the control system of lower limb exoskeletons for rehabilitation is discussed. Based on the public literature in recent years, it is summarized from three aspects, i.e., movement mode switching, human gait recognition and human-exoskeleton interaction control. Finally, the technical issues of the current lower limb rehabilitation exoskeleton control strategy are discussed. The future development prospects and research directions of the lower limb rehabilitation exoskeleton are prospected, and some suggestions on how to achieve a more efficient and accurate control are given.

Secure Gait Recognition-Based Smart Surveillance Systems Against Universal Adversarial Attacks

Currently, the internet of everything (IoE) enabled smart surveillance systems are widely used in various fields to prevent various forms of abnormal behaviors. The authors assess the vulnerability of surveillance systems based on human gait and suggest a defense strategy to secure them. Human gait recognition is a promising biometric technology, but one significantly hindered because of universal adversarial perturbation (UAP) that may trigger system failure. More specifically, in this research study, the authors emphasize on sample convolutional neural network (CNN) model design for gait recognition and assess its susceptibility to UAPs. The authors compute the perturbation as non-targeted UAPs, which trigger a model failure and lead to an inaccurate label to the input sample of a given subject. The findings show that a smart surveillance system based on human gait analysis is susceptible to UAPs, even if the norm of the generated noise is substantially less than the average norm of the images. Later, in the next stage, the authors illustrate a defense mechanism to design a secure surveillance system based on human gait.

HGRBOL2: Human gait recognition for biometric application using Bayesian optimization and extreme learning machine

The goal of gait recognition is to identify a person from a distance based on their walking style using a visual camera. However, the covariates such as a walk with carrying a bag and a change in clothes impact the recognition accuracy. This paper proposed a framework for human gait recognition based on deep learning and Bayesian optimization. The proposed framework includes both sequential and parallel steps. In the first step, optical flow-based motion regions are extracted and utilized to train the fine-tuned EfficentNet-B0 deep model. Simultaneously, we proposed a video frame enhancement technique. The same model is also separately trained on enhanced video frames. Instead of selecting static hyperparameters during the training process, Bayesian Optimization is employed in this work. Following that, two models have obtained: the motion frames model and the original frames model. Features are extracted from both models and fused using a proposed parallel approach named-Sq-Parallel Fusion (SqPF). The Tiger optimization algorithm was then improved and named Entropy controlled Tiger optimization (EVcTO). The best features are finally classified using an extreme learning machine (ELM) classifier. The experiment was carried out on two publicly available datasets, CASIA B and CASIA C, and yielded average accuracy of 92.04 and 94.97%, respectively. The proposed framework outperforms other deep learning-based networks in terms of accuracy.

Ensemble of Heterogeneous Base Classifiers for Human Gait Recognition.

Human gait recognition is one of the most interesting issues within the subject of behavioral biometrics. The most significant problems connected with the practical application of biometric systems include their accuracy as well as the speed at which they operate, understood both as the time needed to recognize a particular person as well as the time necessary to create and train a biometric system. The present study made use of an ensemble of heterogeneous base classifiers to address these issues. A Heterogeneous ensemble is a group of classification models trained using various algorithms and combined to output an effective recognition A group of parameters identified on the basis of ground reaction forces was accepted as input signals. The proposed solution was tested on a sample of 322 people (5980 gait cycles). Results concerning the accuracy of recognition (meaning the Correct Classification Rate quality at 99.65%), as well as operation time (meaning the time of model construction at <12.5 min and the time needed to recognize a person at <0.1 s), should be considered as very good and exceed in quality other methods so far described in the literature.

Occlusion-Aware Human Mesh Model-Based Gait Recognition

Partial occlusion of the human body caused by obstacles or a limited camera field of view often occurs in surveillance videos, which affects the performance of gait recognition in practice. Existing methods for gait recognition against occlusion require a bounding box or the height of a full human body as a prerequisite, which is unobserved in occlusion scenarios. In this paper, we propose an occlusion-aware model-based gait recognition method that works directly on gait videos under occlusion without the above-mentioned prerequisite. Specifically, given a gait sequence that only contains non-occluded body parts in the images, we directly fit a skinned multi-person linear (SMPL)-based human mesh model to the input images without any pre-normalization or registration of the human body. We further use the pose and shape features extracted from the estimated SMPL model for recognition purposes, and use the extracted camera parameters in the occlusion attenuation module to reduce intra-subject variation in human model fitting caused by occlusion pattern differences. Experiments on occlusion samples simulated from the OU-MVLP dataset demonstrated the effectiveness of the proposed method, which outperformed state-of-the-art gait recognition methods by about 15% rank-1 identification rate and 2% equal error rate in the identification and verification scenarios, respectively.

Human Gait Recognition Based on Sequential Deep Learning and Best Features Selection

Human Gait Recognition Based on Sequential Deep Learning and Best Features Selection