Human Tracking Method Research Articles

Design and implementation of deep learning-based object detection and tracking system

Many human tracking methods by deep learning rely on powerful computing resources. For embedded platforms with limited resources, efficient use of resources is a priority. In this paper, we design an object detection and tracking system based on deep learning methods. We propose an efficient system with software and hardware design. We apply the framework of Vitis AI and its Deep Learning Processing Unit using a hardware/software co-design approach. This approach capitalizes on a higher-level acceleration design framework, where the convolutional models can be updated more flexibly and rapidly. This design approach not only provides a fast design flow but also has good performance in terms of throughput. We facilitate the design and accelerate the object detection model YOLO v3 to achieve higher throughput and energy efficiency. Our tracking method achieves a 1.27x improvement in processing speed with the addition of a single-object tracker. Our proposed human tracking methods can achieve better performance than the others in precision with the same test sequences.

A human hybrid tracking and localization method for mixed reality simulation of complex system

In order to obtain stable and high-precision motion tracking in mixed reality (MR) simulation of the complex system, an optical-inertial hybrid tracking method is proposed, which combines the accuracy of optical motion capture equipment and the robustness of inertial motion capture equipment, and can reduce the occlusion problem of entity scene on the premise of ensuring the localization accuracy. The method has been applied to the ergonomic evaluation of large civil aircraft development processes, and the effectiveness and usability of the method have been verified.

Gesture Tracking and Recognition Algorithm for Dynamic Human Motion Using Multimodal Deep Learning

To address the problems of the traditional human motion gesture tracking and recognition methods, such as poor tracking effect, low recognition accuracy, high frame loss rate, and long-time cost, a dynamic human motion gesture tracking and recognition algorithm using multimode deep learning was proposed. Firstly, the collected human motion images are repaired in the three-dimensional (3D) environment, and the multimodal 3D human motion model is reconstructed using the processed images. Secondly, according to the results of model reconstruction, the camera gesture and other parameters of the keyframe are used to construct the target tracking optimization function so as to achieve the purpose of accurate tracking of human motion. Finally, for multimodal human motion gesture learning, a convolutional neural network (CNN) is developed. The trained CNN is utilized to complete dynamic human motion recognition after convolutional and pooling calculations. The results demonstrate that the proposed algorithm is effective in tracking human motion gestures. The average recognition accuracy is 96%, the average frame loss rate is 8.8%, the time cost is low, and the proposed algorithm has a high F-measure and much lower power consumption than other algorithms, indicating that the proposed algorithm is effective.

A Lightweight Passive Human Tracking Method Using Wi-Fi.

Target tracking is a critical technique for localization in an indoor environment. Current target-tracking methods suffer from high overhead, high latency, and blind spots issues due to a large amount of data needing to be collected or trained. On the other hand, a lightweight tracking method is preferred in many cases instead of just pursuing accuracy. For this reason, in this paper, we propose a Wi-Fi-enabled Infrared-like Device-free (WIDE) method for target tracking to realize a lightweight target-tracking method. We first analyze the impact of target movement on the physical layer of the wireless link and establish a near real-time model between the Channel State Information (CSI) and human motion. Secondly, we make full use of the network structure formed by a large number of wireless devices already deployed in reality to achieve the goal. We validate the WIDE method in different environments. Extensive evaluation results show that the WIDE method is lightweight and can track targets rapidly as well as achieve satisfactory tracking results.

Proposal of a method for human tracking of inverted two-wheeled vehicles

Proposal of a method for human tracking of inverted two-wheeled vehicles

Development of a user-following mobile robot with a stand-up assistance function

In this paper, a user-following mobile robot which tracks and follows the user, offering stand-up assistance function is proposed. The proposed robot plays the role of a chair where the user can sit on, and offers a stand-up assistance function compensating the lack of muscle strength. In the proposed robot, a sensing method for buttocks recognition using a depth sensor is proposed. By measuring the distance from the user’s buttocks, the walking state is recognized and the tracking is performed at a fixed distance. As an approach to realize the tracking function, a human tracking method for mobile robots using PD control is constructed. According experimental study, usefulness of the proposed mobile robot with the function of user-following and stand-up assistance is confirmed. The user recognition method and the tracking method using PD control are confirmed effective. With the proposed robot system, improvement in welfare field can be expected.

Extraction of Human Motion Information from Digital Video Based on 3D Poisson Equation

Based on the 3D Poisson equation, this paper extracts the features of the digital video human body action sequence. By solving the Poisson equation on the silhouette sequence, the time and space features, time and space structure features, shape features, and orientation features can be obtained. First, we use the silhouette structure features in three-dimensional space-time and the orientation features of the silhouette in three-dimensional space-time to represent the local features of the silhouette sequence and use the 3D Zernike moment feature to represent the overall features of the silhouette sequence. Secondly, we combine the Bayesian classifier and AdaBoost classifier to learn and classify the features of human action sequences, conduct experiments on the Weizmann video database, and conduct multiple experiments using the method of classifying samples and selecting partial combinations for training. Then, using the recognition algorithm of motion capture, after the above process, the three-dimensional model is obtained and matched with the model in the three-dimensional model database, the sequence with the smallest distance is calculated, and the corresponding skeleton is outputted as the results of action capture. During the experiment, the human motion tracking method based on the university matching kernel (EMK) image kernel descriptor was used; that is, the scale invariant operator was used to count the characteristics of multiple training images, and finally, the high-dimensional feature space was mapped into the low-dimensional to obtain the feature space approximating the Gaussian kernel. Based on the above analysis, the main user has prior knowledge of the network environment. The experimental results show that the method in this paper can effectively extract the characteristics of human body movements and has a good classification effect for bending, one-foot jumping, vertical jumping, waving, and other movements. Due to the linear separability of the data in the kernel space, fast linear interpolation regression is performed on the features in the feature space, which significantly improves the robustness and accuracy of the estimation of the human motion pose in the image sequence.

Body Shape-Guided Motion Retargeting to Reduce Effort on Human to Humanoid Landmark Placements

Motion retargeting reduces the animator’s efforts to create robot motion by adapting human motion. However, it still requires a number of manual landmark placements to achieve satisfactory whole-body retargeted motion. Therefore, to reduce efforts on placing landmarks for corresponding minor body parts, this paper proposes a whole-body motion retargeting method for a general humanoid robot that considers the body shape similarity in addition to traditional landmark-based similarity. An additional strategy that matches the volumetric distribution of the body shape between a human and a robot is presented as guidance to handle redundancy from fewer landmarks and to force consistent outcomes from ambiguous landmark placements. The kinematically constrained Gaussian mixture model, originally used as a volumetric model-based human tracking method, is adapted and modified to manage both the shape and the landmarks in the proposed method. The shape and landmark similarity metrics are respectively introduced, and the overall similarity metric is defined by composing the sum of both metrics with weighting coefficients to control the balance between two policies by animators. Then the expectation-maximization based optimization is utilized to calculate target robot angles with human demonstration frame by frame. Experimental results validate the effectiveness of body shape matching, controllability through weighting coefficients, noise robustness on self-retargeting, and generality on applying different humanoid robots.

Automatic multiple human tracking using an adaptive hybrid GMM based detection in a crowd

For a visual surveillance in a crowd, multiple human tracking is essential and of course a challenging task. Real-world applications require multiple cameras to capture crowd scenes so that a keen tracking is observed. Automatic tracking in a crowded environment is very important criteria for the surveillance. Accurate and real-time tracking in a crowd, the number of people present in the public places and shopping mall are some of the vital information for monitoring traffic violations. To provide human safety and security, surveillance like theft prevention and automated checkout provides the necessary consumer information to the managers. The conventional tracking algorithm does not handle the complex background, multi-view points, various illumination changes and severe occlusion occurring in a crowd. The above problem can be effectively handled by using the proposed Adaptive Hybrid Multiple Human Tracking (AHMHT) method. The proposed work utilizes the Adaptive Hybrid Gaussian Mixture Model (AHGMM) (Karpagavalli and Ramprasad, International Journal of Multimedia Tools and Application 76(12):14129–14149, 12) detected output, so that, the proposed algorithm tracks all the blobs in each frame on the basis of motion information along with the width and height information of exact blob. The experimental results demonstrate that the proposed method performs well compared to other methods. The multiple human tracking rates are improved with maximum of 91% using the proposed frame work compared with other methods. The proposed method is efficient in terms of computational time (CT) using an adaptive hybrid tracking.

Depth and thermal information fusion for head tracking using particle filter in a fall detection context

The security of elderly people living alone is a major issue. A system that detects anomalies can be useful for both individual and retirement homes. In this paper, we present an adaptive human tracking method built on particle filter, using depth an

Human motion tracking under indoor and outdoor surveillance system

This paper gives an overview of the various potential applications involved in human motion tracking. Also, a review of some relevant algorithms in this area are summarised based on three main components: the human target extraction, the features extraction, and the motion model. In addition, a proposed generative and discriminative human tracking method is presented. This proposed method is based on the hybridisation of Kalman filter (KF) and the support vector machines (SVMs). Numerous experiments illustrate the effectiveness of the proposed method against several state of the art trackers. These experiments are applied on several real world image sequences with various challenges that make the tracker vulnerable which are: partial and full occlusions, illumination changes, scale variation, out-of-plane, rotation, motion blur, low resolution, deformation, fast motion and background clutter.

Automatic multiple moving humans detection and tracking in image sequences taken from a stationary thermal infrared camera

Several Particle Filter (PF)-based methods for human tracking in thermal IR image sequences have been proposed in the literature. Unfortunately, the majority of these methods are developed for tracking only a single human. Moreover, this human is manually pre-selected in the first frame of the image sequence, which is not practical for the real case of intelligent and efficient video surveillance system that needs tracking more than one human and without any external operator intervention. To contribute to addressing this need, in this paper, we propose a novel PF-based method that detects and tracks multiple moving humans using a thermal IR camera, without prior knowledge about their number and initial locations in the monitored scene. This method consists of three main parts. In the first one, all the moving objects are extracted from the image sequence by using the Gaussian Mixture Model (GMM) and then, for each extracted object, a combined shape, appearance, spatial and temporal-based similarity function that allows us to detect a human without any prior training of a mathematical model is calculated. The second part consists in tracking the human previously detected by using a PF and an adaptive combination of spatial, intensity, texture and motion velocity cues. In each cue, a model for the detected human is created, and when new observations arrive in the next frames, the similarity distances between each created model and the observed moving regions are calculated. The human tracking is achieved by combining individual similarity distances using adaptive weights, into a PF algorithm. The third part is devoted to detect and handle occlusions by using simple heuristic rules and grayscale Vertical Projection Histogram (VPH). Each part of the proposed method was separately tested on a set of real-world thermal IR image sequences containing background clutters, appearance and disappearance of multiple moving objects, occlusions, illumination and scales changes. A comparative study with several state-of-the-art methods has shown that the proposed method performs consistently better in terms of Center Location Error (CLE) and the Success Rate (SR), and it can also run at speed of about 15 Frame Per Second per human, which is considerably enough for real-time applications.

A cascaded CNN model for multiple human tracking and re-localization in complex video sequences with large displacement

Human tracking and localization play a crucial role in many applications like accident avoidance, action recognition, safety and security, surveillance and crowd analysis. Inspired by its use and scope, we introduced a novel method for human tracking (one or many) and re-localization in a complex environment with large displacement. The model can handle complex background, variations in illumination, changes in target pose, the presence of similar target and appearance (pose and clothes), the motion of target and camera, occlusion of the target, background variation, and massive displacement of the target. Our model uses three convolutional neural network based deep architecture and cascades their learning such that it improves the overall efficiency of the model. The first network learns the pixel level representation of small regions. The second architecture uses these features and learns the displacement of a region with its category between moved, not-moved, and occluded classes. Whereas, the third network improves the displacement result of the second network by utilizing the previous two learning. We also create a semi-synthetic dataset for training purpose. The model is trained on this dataset first and tested on a subset of CamNeT, VOT2015, LITIV-tracking and Visual Tracker Benchmark database without training with real data. The proposed model yield comparative results with respect to current state-of-the-art methods based on evaluation criteria described in Object Tracking Benchmark, TPAMI 2015, CVPR 2013 and ICCV 2017.

An improved CNN framework for detecting and tracking human body in unconstraint environment

Human tracking and localization play a crucial role in many applications like accident avoidance, action recognition, safety and security, surveillance and crowd analysis. Inspired by its use and scope, we introduced a novel method for human tracking (one or many) and re-localization in a complex environment with large displacement. The model can handle complex background, variations in illumination, changes in target pose, the presence of similar target and appearance (pose and clothes), the motion of target and camera, occlusion of the target, background variation, and massive displacement of the target. Our model uses three convolutional neural network based deep architecture and cascades their learning such that it improves the overall efficiency of the model. The first network learns the pixel level representation of small regions. The second architecture uses these features and learns the displacement of a region with its category between moved, not-moved, and occluded classes. Whereas, the third network improves the displacement result of the second network by utilizing the previous two learning. We also create a semi-synthetic dataset for training purpose. The model is trained on this dataset first and tested on a subset of CamNeT, VOT2015, LITIV-tracking and Visual Tracker Benchmark database without training with real data. The proposed model yield comparative results with respect to current state-of-the-art methods based on evaluation criteria described in Object Tracking Benchmark, TPAMI 2015, CVPR 2013 and ICCV 2017.

Satellite Articulation Tracking Using Computer Vision

Autonomous on-orbit satellite servicing benefits from an inspector satellite that can gain as much information as possible about the primary satellite. This includes performance of articulated objects such as solar arrays, antennas, and sensors. A method is developed for tracking the articulation of the primary satellite in real time, given an articulated model using silhouettes extracted from inspection imagery. Additionally, a physical satellite model is built and used to collect stop-motion images of simulated articulated motion over an inspection route in a simulated space illumination environment. The images are used as measurements for the silhouette articulation tracking method, and successful tracking is demonstrated qualitatively. Finally, a human pose tracking algorithm is modified for tracking satellite articulation, demonstrating the applicability of human tracking methods to satellite articulation tracking when an articulated model is available. The presented methods have all been demonstrated to be capable of tracking articulated motion.

Human tracking from single RGB-D camera using online learning

Human detection and tracking is an essential component in several robotics applications, especially in indoor built environments where humans and robots are expected to coexist or collaborate. Due to their low cost and capability to capture both color and depth data, RGB-D cameras have shown significant promise in human detection and tracking for robotic applications. In this paper, a new human tracking method is proposed to detect and track a specific individual from a single RGB-D sensor using online learning classifiers with no ground plane assumption. Given a previous target human position, a candidate sampling method is designed to find potential positive samples while negative samples are obtained by a random sampling process. The kernelized Support Vector Machine (SVM) is employed as the online classifier to recognize the target human and updated using both the positive and negative examples. The experimental results on six RGB-D videos of a public dataset demonstrate that the proposed method achieves higher success rates compared to a 2D tracker and a 3D human detection method at a frame rate of 3.8 fps, and is capable of efficiently retrieving the target human following intermittent occlusion.

Saliency Detection Based on the Combination of High-Level Knowledge and Low-Level Cues in Foggy Images

A key issue in saliency detection of the foggy images in the wild for human tracking is how to effectively define the less obvious salient objects, and the leading cause is that the contrast and resolution is reduced by the light scattering through fog particles. In this paper, to suppress the interference of the fog and acquire boundaries of salient objects more precisely, we present a novel saliency detection method for human tracking in the wild. In our method, a combination of object contour detection and salient object detection is introduced. The proposed model can not only maintain the object edge more precisely via object contour detection, but also ensure the integrity of salient objects, and finally obtain accurate saliency maps of objects. Firstly, the input image is transformed into HSV color space, and the amplitude spectrum (AS) of each color channel is adjusted to obtain the frequency domain (FD) saliency map. Then, the contrast of the local-global superpixel is calculated, and the saliency map of the spatial domain (SD) is obtained. We use Discrete Stationary Wavelet Transform (DSWT) to fuse the cues of the FD and SD. Finally, a fully convolutional encoder–decoder model is utilized to refine the contour of the salient objects. Experimental results demonstrate that the presented model can remove the influence of fog efficiently, and the performance is better than 16 state-of-the-art saliency models.

GPS 기반 옥외용 로봇의 인간 추종 방법

GPS 기반 옥외용 로봇의 인간 추종 방법

LRFとビジョンの併用による群衆通り抜け時における人追跡手法の開発

LRFとビジョンの併用による群衆通り抜け時における人追跡手法の開発

三つのモデルの統合と適応的学習による群衆中の人追跡

Human tracking in crowded scenes is a challenging problem because of frequent occlusion and the presence of similar regions. In this paper, we propose an online human tracking method which can handle occlusion and targets with similar regions. Our method compares the target region with a surrounding region and targets with similar regions at current frame. In addition, we also compare the target region at current and previous frames. We reduce the probabilities of uncommon colors at current and previous frames, and the tracking accuracy is improved. The effectiveness of the proposed method has been demonstrated by comparison with state-of-the-art trackers on the PETS2009 dataset.