Gait Energy Image Research Articles

Multi-view gait recognition using NMF and 2DLDA

View Transformation Model(VTM) is extensively employed in multi-view gait recognition. However, there still exists decline of matching accuracy among view transformation procedures. Particularly, the loss grows rapidly with the increase of the disparity of views. In the face of this difficulty, firstly, Non-negative Matrix Factorization(NMF) is introduced to obtain local structured features of human body to compensate accuracy loss. Moreover, 2D Linear Discriminant Analysis(2DLDA) is applied to improve classification ability by projecting features into a discriminant space. In the end, gait features, the Gait Energy Images(GEIs), is strengthened as 2D Enhanced GEI(2D-EGEI) by using the reconstruction of 2D Principal Component Analysis(2DPCA). Compared with the state-of-the-art, proposed method significantly outperforms the others. Furthermore, the comparisons of two deep learning methods is evaluated as well. Experimental outcomes show that the proposed method provides an alternative way to obtain the approximative outcomes compared with the deep learning methods.

Multi‐perspective gait recognition based on classifier fusion

Gait recognition has been well known as a promising biometric, which is non-offensive and can identify a person from a distance. In this study, a novel ensemble learning framework for gait recognition, namely multi-perspective gait recognition based on classifier fusion is proposed. Firstly, by utilising bidirectional optical flow, a new algorithm for gait feature extraction is presented, which adaptively extracts the dynamic gait characteristics of walking persons. Secondly, two base classifiers, namely the support vector machine and the hidden Markov model, are trained using the extracted dynamic gait features and traditional gait energy images separately. Thirdly, a novel algorithm is presented for combining two types of base gait classifiers together on the decision level. Finally, the proposed framework by two experiments on the well-known CASIA and OU-ISIR gait databases is evaluated, respectively, and demonstrate the advantages of the proposed methods in comparison with others.

Cross-view Gait Recognition by Discriminative Feature Learning.

Recently, deep learning based cross-view gait recognition becomes popular owing to the strong capacity of convolutional neural networks (CNNs). Current deep learning methods often rely on loss functions used widely in the task of face recognition, e.g., contrastive loss and triplet loss. These loss functions have the problem of hard negative mining. In this paper, a robust, effective and gait-related loss function, called angle center loss (ACL), is proposed to learn discriminative gait features. The proposed loss function is robust to different local parts and temporal window sizes. Different from center loss which learns a center for each identity, the proposed loss function learns multiple sub-centers for each angle of the same identity. Only the largest distance between the anchor feature and the corresponding crossview sub-centers is penalized, which achieves better intra-subject compactness. We also propose to extract discriminative spatialtemporal features by local feature extractors and a temporal attention model. A simplified spatial transformer network is proposed to localize the suitable horizontal parts of the human body. Local gait features for each horizontal part are extracted and then concatenated as the descriptor. We introduce long-short term memory (LSTM) units as the temporal attention model to learn the attention score for each frame, e.g., focusing more on discriminative frames and less on frames with bad quality. The temporal attention model shows better performance than the temporal average pooling or gait energy images (GEI). By combing the three aspects, we achieve the state-of-the-art results on several cross-view gait recognition benchmarks.

Unsupervised GEI-Based Gait Disorders Detection From Different Views.

Early detection of gait disorders may provide a safer living for elderly people. In this paper, we propose an automatic method for detecting gait disorders using RGB or RGBD camera (e.g., MS Kinect, Asus Xtion PRO). We use Gait Energy Image (GEI) as our main feature that can be computed from different views. Our method depends on computing GEI, learning the representative features from the GEI using convolutional autoencoder, and using anomaly detection method for detecting abnormal gait. We applied the proposed method on two different public datasets that include normal and abnormal gait from different views. Experimental results show that our method achieves high accuracy in detecting different gait disorders from different views, which makes it general to be applied to home environment and adds a step towards convenient in-home automatic health care services.

Gait identification by convolutional neural networks and optical flow

Non-interactive biometric systems have gained an enormous interest from computer vision researchers as they provide more efficient and reliable ways of identification and authorization from a distance. Gait and face recognition are types of non-interactive biometric systems without users’ cooperation with the surveillance system. On the contrary to face recognition, gait recognition can manage low-resolution and low-brightness images. It aims to know the individuals based on their style and way of walking. Gait recognition has numerous applications in several domains, such as healthcare monitoring, security systems, and surveillance systems for indoor and outdoor activities. Yet, gait recognition performance is frequently deteriorated by some variety of factors, such as viewing angle variations, and clothing changes. Recently, deep learning models have been employed efficiently in gait recognition systems. They are more generic, since the feature construction process is completely automated. This paper presents gait features measured automatically in the midst of walking for the recognition system. To extract these features from a video of a moving object, two vital modules are used, namely the motion detection and tracking, and the feature extraction. Accordingly, the principal module serves to distinguish the walking style in an image sequence or video. A background subtraction technique is executed to fragment the movement of the background, and the moving area related to the spatial silhouette is correctly tracked and segmented. The second module “Feature Extraction” is used to extract the features from the sequence of silhouette images. The gait cycle is calculated from the shape changes of the silhouettes, and it is used to construct a small sequence of Gait Energy Images (GEI). The optical flow of the GEI is measured to extract only the moving parts and exclude the static ones. Finally, the Convolution Neural Network (CNN) is fed with the optical flow output to build unique features. These features are used for neural network training, and evaluation is performed on popular gait benchmark datasets. The obtained results reveal an accuracy level of 95% with more resistance to view and probe changes.

Robust gait recognition using hybrid descriptors based on Skeleton Gait Energy Image

Gait features have been widely applied in human identification. The commonly-used representations for gait recognition can be roughly classified into two categories: model-free features and model-based features. However, due to the view variances and clothes changes, model-free features are sensitive to the appearance changes. For model-based features, there is great difficulty in extracting the underlying models from gait sequences. Based on the confidence maps and the part affinity fields produced by a two-branch multi-stage CNN network, a new model-based representation, Skeleton Gait Energy Image (SGEI), has been proposed in this paper. Another contribution is that a hybrid representation has been produced, which uses SGEI to remedy the deficiency of model-free features, Gait Energy Image (GEI) for instance. The experimental performances indicate that our proposed methods are more robust to the cloth changes, and contribute to increasing the robustness of gait recognition in the unconstrained environments with view variances and clothes changes.

Gait-based age progression/regression: a baseline and performance evaluation by age group classification and cross-age gait identification

Gait is believed to be an advanced behavioral biometric that can be perceived at a large distance from a camera without subject cooperation and hence is favorable for many applications in surveillance and forensics. However, appearance differences caused by human aging may significantly reduce the performance of gait recognition. Modeling the aging process on gait features is one of the possible solutions to this problem, and it may inspire more potential applications, such as finding lost children and examining health status. To the best of our knowledge, this topic has not been studied in the literature. Motivated by the fact that aging effects are mainly reflected in the shape and appearance deformations of the gait feature, we propose a baseline algorithm for gait-based age progression and regression using a generic geometric transformation between different age groups, in conjunction with the gait energy image, which is an appearance-based gait feature frequently used in the gait analysis community, to render gait aging and reverse aging effects simultaneously. Various evaluations were conducted through gait-based age group classification and cross-age gait identification to validate the performance of the proposed method, in addition to providing several insights for future research on the subject.

Real-time and robust multiple-view gender classification using gait features in video surveillance

It is common to view people in real applications walking in arbitrary directions, holding items, or wearing heavy coats. These factors are challenges in gait-based application methods because they significantly change a person's appearance. This paper proposes a novel method for classifying human gender in real time using gait information. The use of an average gait image (AGI), rather than a gait energy image (GEI), allows this method to be computationally efficient and robust against view changes. A viewpoint (VP) model is created for automatically determining the viewing angle during the testing phase. A distance signal (DS) model is constructed to remove any areas with an attachment (carried items, worn coats) from a silhouette to reduce the interference in the resulting classification. Finally, the human gender is classified using multiple view-dependent classifiers trained using a support vector machine. Experiment results confirm that the proposed method achieves a high accuracy of 98.8% on the CASIA Dataset B and outperforms the recent state-of-the-art methods.

Person identification from partial gait cycle using fully convolutional neural networks

Gait as a biometric property for person identification plays a key role in video surveillance and security applications. In gait recognition, normally, gait feature such as Gait Energy Image (GEI) is extracted from one full gait cycle. However in many circumstances, such a full gait cycle might not be available due to occlusion. Thus, the GEI is not complete giving rise to a degrading in gait-based person identification rate. In this paper, we address this issue by proposing a novel method to identify individuals from gait feature when a few (or even single) frame(s) is available. To do so, we propose a deep learning-based approach to transform incomplete GEI to the corresponding complete GEI obtained from a full gait cycle. More precisely, this transformation is done gradually by training several auto encoders independently and then combining these as a uniform model. Experimental results on two public gait datasets, namely OULP and Casia-B demonstrate the validity of the proposed method in dealing with very incomplete gait cycles.

Coupled Bilinear Discriminant Projection for Cross-View Gait Recognition

A problem that hinders good performance of general gait recognition systems is that the appearance features of gaits are more affected-prone by views than identities, especially when the walking direction of the probe gait is different from the register gait. This problem cannot be solved by traditional projection learning methods because these methods can learn only one projection matrix, and thus for the same subject, it cannot transfer cross-view gait features into similar ones. This paper presents an innovative method to overcome this problem by aligning gait energy images (GEIs) across views with the coupled bilinear discriminant projection (CBDP). Specifically, the CBDP generates the aligned gait matrix features for two views with two sets of bilinear transformation matrices, so that the original GEIs’ spatial structure information can be preserved. By iteratively maximizing the ratio of inter-class distance metric to intra-class distance metric, the CBDP can learn the optimal matrix subspace where the GEIs across views are aligned in both horizontal and vertical coordinates. Therefore, the CBDP is also able to avoid the under-sample problem. We also theoretically prove that the upper and lower bounds of the objective function sequence of the CBDP are both monotonically increasing, so the convergence of the CBDP is demonstrated. In the terms of accuracy, the comparative experiments on the CASIA (B) and OU-ISIR gait databases show that our method is superior to the state-of-the-art cross-view gait recognition methods. More impressively, encouraging performance is obtained by our method even in matching a lateral-view gait with a frontal-view gait.

Gait recognition with cross-domain transfer networks

Abstract This paper proposes a novel cross-domain transfer networks (CDTN) that is employed for multi-view gait recognition. CDTN consists of two VGAN layers and a GAN unit. The VGAN layer merges variational autoencoder (VAE) into generative adversarial network (GAN) by replacing the generator of GAN with VAE. Gait energy images (GEIs) are taken as the input of the VGAN layer, and then discriminative loss, reconstruction loss and Kullback–Leibler divergence are adopted to optimize CDTN synchronously. Two VGAN layers are connected by a GAN unit that takes the two gait samples collected under different views as input. After optimization, the output of generator is decoded by a decoder, the decoding results are taken as the output of CDTN. Finally, extensive experiments are conducted on two famous gait datasets. Compare with the state-of-the-art methods, CDTN achieves better gait recognition accuracies, such as 94.78% under single view angle and 93.68% under multi-view angles, which outperforms the existing methods by a significant margin. The results indicate that CDTN is effective for improving the accuracy of multi-view gait recognition. Besides, CDTN provides an important reference for solving the similar problems.

High Speed Image Searching For Human Gait Feature Selection

In this paper, a new patch distribution feature (i.e., referred to as Gabor-PDF) is used for human gait recognition. We represent each gait energy image (GEI) as a set of local augmented Gabor features, which concatenate the Gabor features extracted from different scales and different orientations. A global Gaussian mixture model (GMM) (i.e., referred to as the universal back- ground model) with the local augmented Gabor features from all the gallery GEIs; then, each gallery or probe GEI is further expressed as the normalized parameters of an image-specific GMM adapted from the global GMM. Observing that one video is naturally represented as a group of GEIs, also a new classification method called locality-constrained group sparse representation (LGSR) to classify each probe video by minimizing the weighted l1,2 mixed-norm-regularized reconstruction error with respect to the gallery videos. In contrast to the standard group sparse representation method that is a special case of LGSR, the group sparsity and local smooth sparsity constraints are both enforced in LGSR. The same LGSR algorithm is used for both color images and for content based image retrieval (CBIR

Multi-Task GANs for View-Specific Feature Learning in Gait Recognition

Gait recognition is of great importance in the fields of surveillance and forensics to identify human beings since gait is the unique biometric feature that can be perceived efficiently at a distance. However, the accuracy of gait recognition to some extent suffers from both the variation of view angles and the deficient gait templates. On one hand, the existing cross-view methods focus on transforming gait templates among different views, which may accumulate the transformation error in a large variation of view angles. On the other hand, a commonly used gait energy image template loses temporal information of a gait sequence. To address these problems, this paper proposes multi-task generative adversarial networks (MGANs) for learning view-specific feature representations. In order to preserve more temporal information, we also propose a new multi-channel gait template, called period energy image (PEI). Based on the assumption of view angle manifold, the MGANs can leverage adversarial training to extract more discriminative features from gait sequences. Experiments on OU-ISIR, CASIA-B, and USF benchmark data sets indicate that compared with several recently published approaches, PEI + MGANs achieves competitive performance and is more interpretable to cross-view gait recognition.

On the selection of spatiotemporal filtering with classifier ensemble method for effective gait recognition

In this paper, we improve the performance of gait recognition by modeling human’s motion with spatiotemporal gait features. Since existing methods often use average of silhouettes, i.e., gait energy image to model the gait, temporal information of walking may not be preserved under covariate factors. To handle such features in different conditions, we study the gait model from energy viewpoint. In the proposed method, energy of a gait, i.e., spatiotemporal feature, is derived from a newly designed filtering approach and the energies within a period will be aggregated into a single template that is called gait spatiotemporal image. The required features are truly extracted from spatial and temporal impulse responses that are redesigned and optimized for the gait. Moreover, to recognize the gait under covariate factors, a hybrid decision-level classifier based on random subspace method has been utilized for the given templates. Experimental results on well-known public datasets demonstrate the efficacy of our model. The proposed gait recognition system achieves the recognition rate of 72.25% for Rank1 and 85.64% for Rank5 on the USF dataset that is improved by at least 2% in Rank1 and 0.3% in Rank5 with respect to recent template-based methods.

The Effect of Using Projective Cameras on View-Independent Gait Recognition Performance

Gait as a biometric can be used to identify subjects at a distance and thus it receives great attention from the research community for security and surveillance applications. One of the challenges that affects gait recognition performance is view variation. Much work has been done to tackle this challenge. However, the majority of the work assumes that gait silhouettes are captured by affine cameras where only the height of silhouettes changes and the difference in viewing angle of silhouettes in one gait cycle is relatively small. In this paper, we analyze the variation in gait recognition performance when using silhouettes from projective cameras and from affine cameras with different distance from the center of a walking path. This is done by using 3D models of walking people in the gallery set and 2D gait silhouettes from independent (single) cameras in the probe set. Different factors that affect matching 3D human models with 2D gait silhouettes from single cameras for view-independent gait recognition are analyzed. In all experiments, we use 258 multi-view sequences belong to 46 subjects from Multi-View Soton gait dataset. We evaluate the matching performance for 12 different views using Gait Energy Image (GEI) as gait features. Then, we analyze the effect of using different camera configurations for 3D model reconstruction, the GEI from cameras with different settings, the upper and lower body parts for recognition and different GEI resolutions. The results illustrate that low recognition performance is achieved when using gait silhouettes from affine cameras while lower recognition performance is obtained when using gait silhouettes from projective cameras.

Gait Verification System Through Multiperson Signature Matching for Unobtrusive Biometric Authentication

The unobtrusive nature of gait facilitates the development of optimal biometric authentication systems. Recent approaches on video-analytic gait authentication show excellent results but their implementations are threshold-based which trade off a set amount of FAR (false acceptance rate) to produce an acceptable FRR (false rejection rate). The proposed multiperson signature mapping (MSM) approach overcomes this drawback with a design that substantially decreases the FAR of the authentication system without having to increase the FRR. This technique removes the need of an empirically adjusted threshold. The state-of-the-art algorithms mostly prefer the nearest neighbor (NN) classifier where the Euclidean distance calculated from the extracted feature hyperplane is taken as the similarity measure. Our study proves that the Bayes’ rule applied over the extracted feature set provides a much better performance compared to the conventional NN approach. The MSM is applied on top of template-based gait recognition algorithms to produce an efficient gait authentication system. The method is evaluated on four different gait templates including the popular Gait Energy Image (GEI) and its variation with the genetic template segmentation (GTS). The study analyzes the performance across different clothing and carrying conditions. The deployment of the gait authentication system for practical application is explained in detail. Experimental results with the CASIA-B gait database depict the potential of our proposed approach.

Human gait recognition using GEI-based local multi-scale feature descriptors

Human gait recognition is a biometric technique for persons identification based on their walking manner. This paper proposes a novel gait recognition approach capable of selecting information characteristics for human identification under different conditions including normal walking, carrying a bag and wearing a clothing for different angles of view; thereby enhancing the recognition accomplishment. The proposed approach relies on two feature extraction methods based on multi-scale feature descriptors including Multi-scale Local Binary Pattern (MLBP) and Gabor filter bank, through Spectra Regression Kernel Discriminant Analysis (SRKDA) reduction algorithm. The proposed features are extracted locally from two Region of Interest (ROIs) representing the dynamic areas in the Gait Energy Image (GEI). The experiments conducted on CASIA and USF Gait databases have shown that the suggested methods achieve better recognition performances up to 92% in terms of identification rate at rank-1 than the existing similar and recent state-of-the-art methods.

Learning Efficient Spatial-Temporal Gait Features with Deep Learning for Human Identification.

The integration of the latest breakthroughs in bioinformatics technology from one side and artificial intelligence from another side, enables remarkable advances in the fields of intelligent security guard computational biology, healthcare, and so on. Among them, biometrics based automatic human identification is one of the most fundamental and significant research topic. Human gait, which is a biometric features with the unique capability, has gained significant attentions as the remarkable characteristics of remote accessed, robust and security in the biometrics based human identification. However, the existed methods cannot well handle the indistinctive inter-class differences and large intra-class variations of human gait in real-world situation. In this paper, we have developed an efficient spatial-temporal gait features with deep learning for human identification. First of all, we proposed a gait energy image (GEI) based Siamese neural network to automatically extract robust and discriminative spatial gait features for human identification. Furthermore, we exploit the deep 3-dimensional convolutional networks to learn the human gait convolutional 3D (C3D) as the temporal gait features. Finally, the GEI and C3D gait features are embedded into the null space by the Null Foley-Sammon Transform (NFST). In the new space, the spatial-temporal features are sufficiently combined with distance metric learning to drive the similarity metric to be small for pairs of gait from the same person, and large for pairs from different persons. Consequently, the experiments on the world's largest gait database show our framework impressively outperforms state-of-the-art methods.

Comparative Study of Gait Gender Identification using Gait Energy Image (GEI) and Gait Information Image (GII)

Identifying gender from the pedestrian video is one crucial key to study demographics in such areas. With current video surveillance technology, identifying gender from a distance is possible. This research proposed the utilization of computer vision to identify gender based on their walking gait. The data feature used to determine gender based on their walking gait divided into five parts, namely the head, chest, back, waist & buttocks, and legs. Two different methods are used to perform the real-time gender gait recognition process, i.e., Gait Energy Image (GEI) and Gait Information Image (GII), while the Support Vector Machine (SVM) method used as the data classifier. The experimental results show that the process of identifying gender based on walking with GEI method is 55% accuracy and GII method is 60% accuracy. From these results, it can conclude that the method GII with SVM classifier has the best accuracy in the process of gender classification

Multi-View Gait Recognition Based on a Spatial-Temporal Deep Neural Network

This paper proposes a novel spatial–temporal deep neural network (STDNN) that is applied to multi-view gait recognition. The STDNN comprises a temporal feature network (TFN) and a spatial feature network (SFN). In TFN, a feature sub-network is adopted to extract the low-level edge features of gait silhouettes. These features are input to the spatial–temporal gradient (STG) network that adopts a STG unit and a long short-term memory unit to extract the STG features. In SFN, the spatial features of gait sequences are extracted by multilayer convolutional neural networks from a gait energy image. The SFN is optimized by classification loss and verification loss jointly, which makes inter-class variations larger than intra-class variations. After training, the TFN and the SFN are employed to extract temporal and spatial features, respectively, which are applied to multi-view gait recognition. Finally, the combined predicted probability is adopted to identify individuals by the differences in their gaits. To evaluate the performance of the STDNN, extensive evaluations are carried out based on the CASIA-B, OU-ISIR, and CMU MoBo data sets. The best recognition scores achieved by STDNN are 95.67% under an identical view, 93.64% under a cross-view, and 92.54% under a multi-view. State-of-the-art approaches are compared with the STDNN in various situations. The results show that the STDNN outperforms the other methods and demonstrates the great potential of the STDNN for practical applications in the future.