Gait Image Research Articles

Machine vision-based gait scan method for identifying cognitive impairment in older adults.

Early identification of cognitive impairment in older adults could reduce the burden of age-related disabilities. Gait parameters are associated with and predictive of cognitive decline. Although a variety of sensors and machine learning analysis methods have been used in cognitive studies, a deep optimized machine vision-based method for analyzing gait to identify cognitive decline is needed. This study used a walking footage dataset of 158 adults named West China Hospital Elderly Gait, which was labelled by performance on the Short Portable Mental Status Questionnaire. We proposed a novel recognition network, Deep Optimized GaitPart (DO-GaitPart), based on silhouette and skeleton gait images. Three improvements were applied: short-term temporal template generator (STTG) in the template generation stage to decrease computational cost and minimize loss of temporal information; depth-wise spatial feature extractor (DSFE) to extract both global and local fine-grained spatial features from gait images; and multi-scale temporal aggregation (MTA), a temporal modeling method based on attention mechanism, to improve the distinguishability of gait patterns. An ablation test showed that each component of DO-GaitPart was essential. DO-GaitPart excels in backpack walking scene on CASIA-B dataset, outperforming comparison methods, which were GaitSet, GaitPart, MT3D, 3D Local, TransGait, CSTL, GLN, GaitGL and SMPLGait on Gait3D dataset. The proposed machine vision gait feature identification method achieved a receiver operating characteristic/area under the curve (ROCAUC) of 0.876 (0.852-0.900) on the cognitive state classification task. The proposed method performed well identifying cognitive decline from the gait video datasets, making it a prospective prototype tool in cognitive assessment.

AI in evaluating ambulation of stroke patients: severity classification with video and functional ambulation category scale

ABSTRACT Background The evaluation of gait function and severity classification of stroke patients are important to determine the rehabilitation goal and the level of exercise. Physicians often qualitatively evaluate patients’ walking ability through visual gait analysis using naked eye, video images, or standardized assessment tools. Gait evaluation through observation relies on the doctor’s empirical judgment, potentially introducing subjective opinions. Therefore, conducting research to establish a basis for more objective judgment is crucial. Objective To verify a deep learning model that classifies gait image data of stroke patients according to Functional Ambulation Category (FAC) scale. Methods Gait vision data from 203 stroke patients and 182 healthy individuals recruited from six medical institutions were collected to train a deep learning model for classifying gait severity in stroke patients. The recorded videos were processed using OpenPose. The dataset was randomly split into 80% for training and 20% for testing. Results The deep learning model attained a training accuracy of 0.981 and test accuracy of 0.903. Area Under the Curve(AUC) values of 0.93, 0.95, and 0.96 for discriminating among the mild, moderate, and severe stroke groups, respectively. Conclusion This confirms the potential of utilizing human posture estimation based on vision data not only to develop gait parameter models but also to develop models to classify severity according to the FAC criteria used by physicians. To develop an AI-based severity classification model, a large amount and variety of data is necessary and data collected in non-standardized real environments, not in laboratories, can also be used meaningfully.

Development of a screening AI model for Parkinson's disease using gait image sequences

Development of a screening AI model for Parkinson's disease using gait image sequences

Machine Learning for Depression Detection on Web and Social Media

Depression, a significant psychiatric disorder, affects individuals' physical well-being and daily functioning. This focused analysis provides a comprehensive exploration of contemporary research conducted between 2012 and 2023 that delves into the utilization of sophisticated machine learning methodologies aimed at identifying correlates of depression within social media content. Our study meticulously dissects various data sources and performs a comprehensive examination of different machine learning algorithms cited in the researched articles and literature, aiming to pinpoint an approach that can enhance detection accuracy. Furthermore, we have scrutinized the use of varied data from social media platforms and pinpointed emerging trends, notably spotlighting novel applications of artificial neural networks for image processing and classification, along with advanced gait image models. Our results offer essential direction for future research focused on enhancing detection precision, acting as a valuable reference for academic and industry scholars in this field.

Nonlinear Modeling Analysis of Stabilization Behavior of Robotic Gait Control System Based on Image Processing Techniques

Abstract Existing robot gait control methods have problems such as high gait energy consumption and difficulty in generating optimal gait strategies. This paper designs a robot gait automatic control system based on the X86 platform and robot sensor interface. The main board of the robot controller is designed based on CISC, which ensures high stability and anti-interference capabilities. The information provided by ultrasonic sensors and infrared sensors is used to adjust and execute the robot’s trajectory, action sequence, and gait. To extract the target in the gait image, the frame difference algorithm is employed, and the phase and amplitude factors are collected after the Radon and Fourier-Mellin transform to identify gait characteristics for robot gait recognition. The robot’s two-dimensional spatial dynamics model is constructed continuously, and its dynamics equations are derived. The residual fusion technique is used to combine image data with sensor data. A system for stabilizing gait control has been designed. The tracking error rate of the robot is not more than ±10% in general, and a variety of gait patterns can be used to cross the obstacles, which verifies the effectiveness of the designed system for realizing the gait control and performance of the robot.

Gait image spatio-temporal restoration network and its application under occlusion conditions

Gait image spatio-temporal restoration network and its application under occlusion conditions

Accurate metaheuristic deep convolutional structure for a robust human gait recognition

<span lang="EN-US">Gait recognition has become a developing technology in various security, industrial, medical, and military applications. This paper proposed a deep convolutional neural network (CNN) model to authenticate humans via their walking style. The proposed model has been applied to two commonly used standardized datasets, Chinese Academy of Sciences (CASIA) and Osaka University-Institute of Scientific and Industrial Research (OU-ISIR). After the silhouette images have been isolated from the gait image datasets, their features have been extracted using the proposed deep CNN and the traditional ones, including AlexNet, Inception (GoogleNet), VGGNet, ResNet50, and Xception. The best features were selected using genetic, grey wolf optimizer (GWO), particle swarm optimizer (PSO), and chi-square algorithms. Finally, recognize the selected features using the proposed deep neural network (DNN). Several performance evaluation parameters have been estimated to evaluate the model’s quality, including accuracy, specificity, sensitivity, false negative rate (FNR), and training time. Experiments have demonstrated that the suggested framework with a genetic feature selector outperforms previous selectors and recent research, scoring accuracy values of 99.46% and 99.09% for evaluating the CASIA and OU-ISIR datasets, respectively, in low time (19 seconds for training).</span>

Cross-View Gait Recognition Method Based on Multi-Teacher Joint Knowledge Distillation.

Aiming at challenges such as the high complexity of the network model, the large number of parameters, and the slow speed of training and testing in cross-view gait recognition, this paper proposes a solution: Multi-teacher Joint Knowledge Distillation (MJKD). The algorithm employs multiple complex teacher models to train gait images from a single view, extracting inter-class relationships that are then weighted and integrated into the set of inter-class relationships. These relationships guide the training of a lightweight student model, improving its gait feature extraction capability and recognition accuracy. To validate the effectiveness of the proposed Multi-teacher Joint Knowledge Distillation (MJKD), the paper performs experiments on the CASIA_B dataset using the ResNet network as the benchmark. The experimental results show that the student model trained by Multi-teacher Joint Knowledge Distillation (MJKD) achieves 98.24% recognition accuracy while significantly reducing the number of parameters and computational cost.

Research Method of Discontinuous-Gait Image Recognition Based on Human Skeleton Keypoint Extraction.

As a biological characteristic, gait uses the posture characteristics of human walking for identification, which has the advantages of a long recognition distance and no requirement for the cooperation of subjects. This paper proposes a research method for recognising gait images at the frame level, even in cases of discontinuity, based on human keypoint extraction. In order to reduce the dependence of the network on the temporal characteristics of the image sequence during the training process, a discontinuous frame screening module is added to the front end of the gait feature extraction network, to restrict the image information input to the network. Gait feature extraction adds a cross-stage partial connection (CSP) structure to the spatial-temporal graph convolutional networks' bottleneck structure in the ResGCN network, to effectively filter interference information. It also inserts XBNBlock, on the basis of the CSP structure, to reduce estimation caused by network layer deepening and small-batch-size training. The experimental results of our model on the gait dataset CASIA-B achieve an average recognition accuracy of 79.5%. The proposed method can also achieve 78.1% accuracy on the CASIA-B sample, after training with a limited number of image frames, which means that the model is more robust.

Detecting depression tendency based on deep learning and multi-sources data

In the past decade, cases of depression have been increasingly reported. While early detection offers excellent advantages in reducing these cases, it still faces several challenges. Firstly, the traditional diagnosis of depression is time-consuming and expensive. Secondly, detecting depression by deep learning that solely relies on single source data leads to low prediction accuracy. Thirdly, there is a lack of established linkage datasets based on individuals’ multi-source information for detecting depression tendencies. To address these limitations, the study collected multi-source linkage datasets, including social network text data, human gait image data, and human gait key point data to establish three deep learning models: the social network model based on BERT, the gait image model based on 3D convolutional, and the gait key point model based on LSTM, for predicting depression tendencies. An integrated model was then created by adopting the Majority Voting method for the ensemble learning of these three models. The experimental results demonstrated that the accuracy of the integrated model (91.51 %) was significantly improved compared to the three models mentioned before and related models based on single source data, highlighting the necessity and effectiveness of the proposed method based on multi-sources data. Finally, this study offers practical implications for the development of self-testing Apps as well as the improvement of work efficiency for psychological clinicians.

Regional Time-Series Coding Network and Multi-View Image Generation Network for Short-Time Gait Recognition.

Gait recognition is one of the important research directions of biometric authentication technology. However, in practical applications, the original gait data is often short, and a long and complete gait video is required for successful recognition. Also, the gait images from different views have a great influence on the recognition effect. To address the above problems, we designed a gait data generation network for expanding the cross-view image data required for gait recognition, which provides sufficient data input for feature extraction branching with gait silhouette as the criterion. In addition, we propose a gait motion feature extraction network based on regional time-series coding. By independently time-series coding the joint motion data within different regions of the body, and then combining the time-series data features of each region with secondary coding, we obtain the unique motion relationships between regions of the body. Finally, bilinear matrix decomposition pooling is used to fuse spatial silhouette features and motion time-series features to obtain complete gait recognition under shorter time-length video input. We use the OUMVLP-Pose and CASIA-B datasets to validate the silhouette image branching and motion time-series branching, respectively, and employ evaluation metrics such as IS entropy value and Rank-1 accuracy to demonstrate the effectiveness of our design network. Finally, we also collect gait-motion data in the real world and test them in a complete two-branch fusion network. The experimental results show that the network we designed can effectively extract the time-series features of human motion and achieve the expansion of multi-view gait data. The real-world tests also prove that our designed method has good results and feasibility in the problem of gait recognition with short-time video as input data.

Multi gait recognition using Clustering based Faster Regions-Convolutional Neural Network

Gait recognition is the process of recognizing a person based on their walking style. Each person’s walking gait is distinctive and cannot be imitated by others. However, the walking motion of a person will be changed based on their behaviour but their walking pattern doesn’t change. In this paper, a novel Clustering based Faster RCNN has been proposed to identify the single, double and multi-gait. The gait images from the publicly available dataset are pre-processed using Multi scale Retinex (MSR) to reduce the noise artifacts. The Faster RCNN is used for extracting the relevant features from the gait images via the two modules namely CNN and RPN. The CNN layers extract the most relevant features as feature maps and RPN is used for creating the bounding boxes for the extracted features. Fuzzy K-means clustering is used to group the features based on their labels, and it specifies the features acquired using CNN and RPN as input. Finally, the Fast RCNN is employed for classifying the gait images into suspicious and non-suspicious walking pattern. The proposed Clustering based Faster RCNN net achieves the high accuracy rate of 98.74% and 99.19% for suspicious and non-suspicious walking pattern respectively. The proposed Clustering based Faster RCNN model was compared with other traditional models like CNN, U-net, Fab net and Fast R-CNN. The proposed Clustering based Faster RCNN model improves the overall accuracy of 8.86% 33.77% 3.12% and 5.48% better than mmGait, LSTM Net, STDNN and RNN respectively.

A Deep-Learning Approach for Identifying a Drunk Person Using Gait Recognition

Various accidents caused by alcohol consumption have recently increased in prevalence and have become a huge social problem. There have been efforts to identify drunk individuals using mobile devices; however, it is difficult to apply this method to a large number of people. A promising approach that does not involve wearing any sensors or subject cooperation is a markerless, vision-based method that only requires a camera to classify a drunk gait. Herein, we first propose a markerless, vision-based method to determine whether a human is drunk or not based on his or her gait pattern. We employed a convolutional neural network to analyze gait patterns with image augmentation depending on gait energy images. Gait images captured through a camera allow a complex neural network to detect the human body shape accurately. It is necessary for removing the background behind human shape in the gait image because it disrupts the detection algorithm. A process of conversion into gait energy images and augmenting image data is then applied to the dataset of the gait images. A total of 20 participants participated in the experiment. They were required to walk along a line both with and without wearing the Drunk Busters Goggles, which were intended to collect sober and drunk gait images. Validation accuracy for the recognition of a drunk state in 20 persons was approximately 74.94% under optimal conditions. If the present approach fulfills its promise, we can prevent safety accidents due to alcohol, thus decreasing its burden on industries and society.

Gait Image Classification Using Deep Learning Models for Medical Diagnosis

Gait refers to a person’s particular movements and stance while moving around. Although each person’s gait is unique and made up of a variety of tiny limb orientations and body positions, they all have common characteristics that help to define normalcy. Swiftly identifying such characteristics that are difficult to spot by the naked eye, can help in monitoring the elderly who require constant care and support. Analyzing silhouettes is the easiest way to assess and make any necessary adjustments for a smooth gait. It also becomes an important aspect of decision-making while analyzing and monitoring the progress of a patient during medical diagnosis. Gait images made publicly available by the Chinese Academy of Sciences (CASIA) Gait Database was used in this study. After evaluating using the CASIA B and C datasets, this paper proposes a Convolutional Neural Network (CNN) and a CNN Long Short-Term Memory Network (CNN-LSTM) model for classifying the gait silhouette images. Transfer learning models such as MobileNetV2, InceptionV3, Visual Geometry Group (VGG) networks such as VGG16 and VGG19, Residual Networks (ResNet) like the ResNet9 and ResNet50, were used to compare the efficacy of the proposed models. CNN proved to be the best by achieving the highest accuracy of 94.29%. This was followed by ResNet9 and CNN-LSTM, which arrived at 93.30% and 87.25% accuracy, respectively.

FCML-gait: fog computing and machine learning inspired human identity and gender recognition using gait sequences.

Security threats are always there if the human intruders are not identified and recognized well in time in highly security-sensitive environments like the military, airports, parliament houses, and banks. Fog computing and machine learning algorithms on Gait sequences can prove to be better for restricting intruders promptly. Gait recognition provides the ability to observe an individual unobtrusively, without any direct cooperation or interaction from the people, making it very attractive than other biometric recognition techniques. In this paper, a Fog Computing and Machine Learning Inspired Human Identity and Gender Recognition using Gait Sequences (FCML-Gait) are proposed. Internet of things (IoT) devices and video capturing sensors are used to acquire data. Frames are clustered using the affinity propagation (AP) clustering technique into several clusters, and cluster-based averaged gait image(C-AGI) feature is determined for each cluster. For training and testing of datasets, sparse reconstruction-based metric learning (SRML) and Speeded Up Robust Features (SURF) with support vector machine (SVM) are applied on benchmark gait database ADSC-AWD having 80 subjects of 20 different individuals in the Fog Layer to improve the processing. The performance metrics, for instance, accuracy, precision, recall, F-measure, C-time, and R-time have been measured, and a comparative evaluation of the projected method with the existing SRML technique has been provided in which the proposed FCML-Gait outperforms and attains the highest accuracy of 95.49%.

Multiview Gait Recognition Based on Slack Allocation Generation Adversarial Network

Through the in-depth study of biological characteristics, the gait information of each person is found to be unique. Therefore, gait characteristics can be used to identify a person’s identity information. However, pedestrian gait information is often affected by various interference factors such as viewing angle, dress, and carrying objects. To solve this problem, the idea of perspective transformation is proposed to transform gait images from different perspectives and different states to gait images under standard conditions on one side. Generative adversarial network (GAN) is adopted for perspective transformation. Moreover, in order to solve the difficulty and instability to converge characteristics of GAN in the training process, we introduced a slack module to adjust the shortcomings of the GAN. Experiments show that under normal walking conditions, our method achieves an average accuracy of 98.32% on the CASIA-B gait dataset.

Athlete Gait Feature Recognition Method Based on Multisource Sensing Information

Aiming at the problem of low accuracy of two-dimensional gait recognition at present, a gait feature recognition method based on multisource sensing information is proposed. The multisource sensing information is combined to collect the athlete’s gait characteristics, collect the single frame gait image sequence of the human lower limbs during the movement, and extract the human body’s three-dimensional feature data during human walking by using the body structure and multisource sensing information, so as to realize the separation of the athlete’s gait image background. Finally, it is confirmed by experiments that the recognition rate of athlete gait feature recognition method based on multisource sensing information is significantly improved.

A competent multimodal recognition using imperfect region based face and gait cues using Median-LBPF and Median-LBPG based PCA followed by LDA

Today, all conventional computer based multimodal biometric system for human recognition that combines face and gait cues have primarily focused on recognition from perfect face and gait images. There are circumstances in which perfect face and gait images may not be obtainable which means probe images are imperfect. This paper proposes new methods Median Local Binary Pattern of Face image (Median-LBPF) and Gait image (Median-LBPG) to extract the features of imperfect face and gait images efficiently for better recognition. Initially the given imperfect face and gait images are divided into six overlapped regions called top, bottom, left, right, vertical center, horizontally center overlapped half images. The features of these six overlapped regions of imperfect face and gait images in the spatial domain are extracted by using Median-LBPF and Median-LBPG. Subsequently the dimensionality of the feature sets are reduced by a two stage feature reduction algorithms Principal Component Analysis (PCA) followed by Linear Discriminant Analysis (LDA). Next for classification, Euclidean distance measure is used to calculate the minimum of minimum distance between the six overlapped regions of given imperfect face and gait probe images and the corresponding regions of all six overlapped regions in the training sets separately called face decision and gait decision. Finally the face decision and gait decision are fused at decision level for recognition. The proposed methods are tested by using publically available data sets ORL face and CASIA gait. The experimental results show that features of a region of face and gait images are adequate for recognition and its average recognition performance is same as perfect face and gait images.

Dynamic Lower-Limb Alignment Focusing on Gait Stability

Introduction: Prosthetic limbs must be developed with proper alignment to facilitate safe and efficient gait patterns. This study aimed to identify factors impacting clinically proper gait patterns by objectively evaluating them in persons with lower-limb amputations who use prosthetic limbs. Materials and Methods: This non-experimental descriptive study assessed 58 persons with amputation who used prosthetic limbs. The mechanical axis angle (MAA) of the lower limb during the heel contact, midstance, and toe-off phases and the angle between the tube and floor during the midstance phase were measured using coronal plane gait images. We also investigated whether the MAA and tube angle during the midstance phase have a multimodal distribution. In case of multimodal distributions, we tested for significant between-group differences in patient characteristics. Results: The MAA and tube angle in the coronal plane during the midstance phase had a bimodal distribution (mean 0°). There was a significant difference in the duration of prosthetic limb use between the MAA Conclusions: In this study, prosthetic alignment during the midstance phase had a bimodal distribution. In both groups, deviations in the lower limb MAA were aligned to be approximately 3°.

Gait Recognition System: A Survey

Human identification by gait has created a great deal of interest in computer vision community due to its advantage of inconspicuous recognition at a relatively far distance. This paper provides a comprehensive survey of recent developments on gait recognition approaches. The survey emphasizes on three major issues involved in a general gait recognition system, namely gait image representation, feature dimensionality reduction and gait classification. It also gives, a review of the available public gait datasets. The concluding discussions outline a number of research challenges and provide promising future directions for the field. In this article presents a survey of the work done in gait analysis for re-identification in the past decade, looking at the main approaches, datasets, and evaluation methodologies also discussed.