Pedestrian Tracking Research Articles

Enhancing Pedestrian Group Detection and Tracking Through Zone-Based Clustering

Enhancing Pedestrian Group Detection and Tracking Through Zone-Based Clustering

Multi-Panda Tracking

Multi-Panda Tracking (MPT) is a video-based tracking task for panda individuals, which is conducive to the observation and measurement of distribution and status of pandas. Different from tracking general objects such as pedestrians and vehicles, MPT is extremely challenging due to the indistinguishable appearances and diversified postures of pandas. In this case, existing tracking methods cannot appropriately tackle with the excessive occlusion between different panda individuals, hence suffering from identity switch, missing and inaccurate detections. To address these problems, we propose a simple yet effective MPT framework in the tracking-by-detection paradigm, which is benefited both from a short-term prediction filtering module and a discriminative feature learning network. In particular, the short-term prediction filtering module introduces similarity learning to enhance the temporal consistency among detections, which is capable of supplementing the missing detections and discarding false positive detections. Besides, the discriminative feature learning network leverages a two-branch network to learn both local and global discriminative features, so as to distinguish different panda individuals with a very similar appearance with a subtle difference. To evaluate the proposed method, we annotate a large-scale MPT dataset, named PANDA2021, which is particularly challenging due to the similar appearance and dramatic occlusion between panda individuals. Experiments on PANDA2021 demonstrate that the proposed MPT method significantly outperforms the competing methods. Moreover, experimental results on pedestrian tracking dataset MOT16 further demonstrate that the proposed MPT method achieves comparative performance with competing methods.

Multi-object pedestrian tracking method based on improved high resolution neural network

Multi-object pedestrian tracking method based on improved high resolution neural network

Virtual reality study of the effects of railway crossing safety technologies on pedestrian behaviour

This study reports on the description of an evaluation of safety systems at Pedestrian Track Crossings (PTC) in French train stations installed to cross over tracks when no other installation is deployed. Two complementary systems - a physical chicane positioned before the crossing zone and a ringing alarm - have been proposed in addition to the existing flashing pictogram. A study, combining a quantitative experimental setup using virtual reality to measure the efficacy of the systems, and a qualitative ergonomic analysis method to complete the results, was proposed to 100 participants. Our results allow us to measure the effect of safety systems on participants’ crossing behaviour, gaze, and kinematic. It appears that the combination with the ringing alarm led to more safety behaviours. Moreover, qualitative data gave new perspectives to improve the system tested and make PTC safer.

Stable and real-time pedestrian tracking method based on improved DeepSORT under complex background

Stable and real-time pedestrian tracking method based on improved DeepSORT under complex background

A Training Smartphone Application for the Simulation of Outdoor Blind Pedestrian Navigation: Usability, UX Evaluation, Sentiment Analysis.

Training blind and visually impaired individuals is an important but often neglected aspect of Assistive Technology solutions (ATs) that can benefit from systems utilizing multiple sensors and hardware devices. Training serves a dual purpose as it not only enables the target group to effectively utilize the ATs but, also, helps in improving their low acceptance rate. In this paper, we present the design, implementation, and validation of a smartphone-based training application. It is a form of immersive system that enables users to learn the features of an outdoor blind pedestrian navigation application and, simultaneously, to help them develop long-term Orientation and Mobility (O&M) skills. The system consists of an Android application leveraging, as data sources, an external high-accuracy GPS sensor for real-time pedestrian mobility tracking, a second custom-made device attached to traffic lights for identifying their status, and an ultra-sonic sensor for detecting near-field obstacles on the navigation path of the users. The training version running as an Android application employs route simulation with audio and haptic feedback, is functionally equivalent to the main application, and was used in the context of specially designed user-centered training sessions. A Usability and User Experience (UX) evaluation revealed the positive attitude of the users towards the training version as well as their satisfaction with the skills acquired during their training sessions (SUS = 69.1, UEQ+ = 1.53). Further confirming the positive attitude was the conduct of a Recursive Neural Network (RNN)-based sentiment analysis on user responses with a score of 3 on a scale from 0 to 4. Finally, we conclude with the lessons learned and the proposal of general design guidelines concerning the observed lack of accessibility and non-universal interfaces.

Multiple Pedestrian Tracking in Dense Crowds Combined with Head Tracking

In order to reduce the negative impact of severe occlusion in dense scenes on the performance degradation of the tracker, considering that the head is the highest and least occluded part of the pedestrian’s entire body, we propose a new multiobject tracking method for pedestrians in dense crowds combined with head tracking. For each frame of the video, a head tracker is first used to generate the pedestrians’ head movement tracklets, and the pedestrians’ whole body bounding boxes are detected at the same time. Secondly, the degree of association between the head bounding boxes and the whole body bounding boxes are calculated, and the Hungarian algorithm is used to match the above calculation results. Finally, according to the matching results, the head bounding boxes in the head tracklets are replaced with the whole body bounding boxes, and the whole body motion tracklets of the pedestrians in the dense scene are generated. Our method can be performed online, and experiments suggested that our method effectively reduces the negative effects of false negatives and false positives on the tracker caused by severe occlusion in dense scenes.

Design and implementation of Pedestrian detection based on Zynq APSoc

Pedestrian recognition is an important content of intelligent security surveillance video processing system. In order to meet the application requirements of miniaturization and real-time security system, a moving target recognition system based on APSoc platform is designed. The system uses ov5640 camera as the video collector, zynq7020 as the development platform, and uses the soft and hard co-design to realize each function module. Finally, the pedestrian image acquisition and real-time display system is successfully completed. The test results show that the system can effectively and stably track pedestrians and moving objects within a certain range of distance, and can be displayed in real time, miniaturized and low power consumption, which can be further applied in the field of security.

Multidomain Joint Learning of Pedestrian Detection for Application to Quadrotors

Pedestrian detection and tracking are critical functions in the application of computer vision for autonomous driving in terms of accident avoidance and safety. Extending the application to drones expands the monitoring space from 2D to 3D but complicates the task. Images captured from various angles pose a great challenge for pedestrian detection, because image features from different angles tremendously vary and the detection performance of deep neural networks deteriorates. In this paper, this multiple-angle issue is treated as a multiple-domain problem, and a novel multidomain joint learning (MDJL) method is proposed to train a deep neural network using drone data from multiple domains. Domain-guided dropout, a critical mechanism in MDJL, is developed to self-organize domain-specific features according to neuron impact scores. After training and fine-tuning the network, the accuracy of the obtained model improved in all the domains. In addition, we also combined the MDJL with Markov decision-process trackers to create a multiobject tracking system for flying drones. Experiments are conducted on many benchmarks, and the proposed method is compared with several state-of-the-art methods. Experimental results show that the MDJL effectively tackles many scenarios and significantly improves tracking performance.

Online Multiple-Pedestrian Tracking With Detection-Pair-Based Graph Convolutional Networks

The typical Internet of Things application, unattended driving systems, will need the ability to recognize relevant traffic participants and detect dangerous situations ahead of time. An important component of these systems is one that is able to distinguish pedestrians and track their motion to make intelligent driving decisions. This article develops a high-accuracy multiple pedestrian tracking algorithm which is vital for intelligent transportation. Here, we use the off-the-shelf detectors and explore the benefits of modeling pedestrian interactions, such as the interaction of two pedestrians simultaneously matched to two pedestrians in another frame, for robust detection association. Explicitly studying interactions is nontrivial. Previous works often manually selected interacting detections (or “tracklets”) to simplify the association process. In this article, we propose a novel association method based on deep graph convolutional affinity networks (DGCANs) and extend detection-level interactions to the association-level, which treats a potential association of a detection pair as a node in the graph, and explicitly modeling the interactions among potential associations. Specifically, with the novel node, two corresponding edges are readily designed to model the compatible and colliding interactions between related associations. Our proposed method, by redefining nodes and edges, enables us to blend sufficient interaction cues from appearance and motion and learns a robust affinity measure in an end-to-end fashion. Using the Hungarian algorithm as an online tracker, our method archives state-of-the-art performance on benchmark data sets 2-D MOT15, MOT16, and MOT17.

Dynamic representation-based tracker for long-term pedestrian tracking with occlusion

This paper presents a dynamic representation-based tracker (DRT) to handle occlusions in the long-term pedestrian tracking of a single target. In our DRT, an adaptive representation network (ARN) is first constructed to extract multiple features, including classical features such as appearance and pose as well as some vector-format deep features. These features are then stacked to form a dynamic representation so as to convert the target tracking into a matching problem between the target features and candidate features, where the Euclidean distance (ED) and locality-constrained linear coding (LLC) are used as measurements in the decision-making. Next, the target state is determined through a voting procedure according to the feature matching error. Finally, a pose supervised module (PSM) and an IOU filtering module (IFM) are applied, respectively, to refine the target state and to filter out some invalid candidate targets that have been detected. Experimental results on public benchmark datasets show that our DRT is quite robust to complex environments with long-term pedestrian occlusions, and outperforms several existing state-of-the-arts trackers as it produces the best performance on both the pedestrian tracking dataset with occlusion (PTDO) and the pedestrian tracking dataset with occlusion plus (PTDO Plus).

Real-time pedestrian pose estimation, tracking and localization for social distancing.

The corona virus pandemic has introduced limitations which were previously not a cause for concern. Chief among them are wearing face masks in public and constraints on the physical distance between people as an effective measure to reduce the virus spread. Visual surveillance systems, which are common in urban environments and initially commissioned for security surveillance, can be re-purposed to help limit the spread of COVID-19 and prevent future pandemics. In this work, we propose a novel integration technique for real-time pose estimation and multiple human tracking in a pedestrian setting, primarily for social distancing, using CCTV camera footage. Our technique promises a sizeable increase in processing speed and improved detection in very low-resolution scenarios. Using existing surveillance systems, pedestrian pose estimation, tracking and localization for social distancing (PETL4SD) is proposed for measuring social distancing, which combines the output of multiple neural networks aided with fundamental 2D/3D vision techniques. We leverage state-of-the-art object and pose estimation algorithms, combining their strengths, for increase in speed and improvement in detections. These detections are then tracked using a bespoke version of the FASTMOT algorithm. Temporal and analogous estimation techniques are used to deal with occlusions when estimating posture. Projective geometry along with the aforementioned posture tracking is then used to localize the pedestrians. Inter-personal distances are calculated and locally inspected to detect possible violations of the social distancing rules. Furthermore, a "smart violations detector" is employed which estimates if people are together based on their current actions and eliminates false social distancing violations within groups. Finally, distances are intuitively visualized with the right perspective. All implementation is in real time and is performed on Python. Experimental results are provided to validate our proposed method quantitatively and qualitatively on public domain datasets using only a single CCTV camera feed as input. Our results show our technique to outperform the baseline in speed and accuracy in low-resolution scenarios. The code of this work will be made publicly available on GitHub at https://github.com/bilalze/PETL4SD.

Smartphone-Based Pedestrian NLOS Positioning Based on Acoustics and IMU Parameter Estimation

This paper proposes an integrated positioning algorithm for mobile devices and achieves long-term and high-accuracy indoor pedestrian tracking under severe non-Line-of-Sight (NLOS) scenarios. The traditional fusion method for hand-held devices lacks zero-speed correction and cannot clear the accumulated error of the Pedestrian Dead Reckoning (PDR). Secondly, the PDR algorithm also requires user privacy data for high positioning accuracy. Hence, we propose a customized model with acoustic and PDR through self-updating parameters with two novel fusing strategies: Kalman Filter with Least-Square (KFLS) and Kalman Filter with Bayesian Parameter Estimation (KFBPE), which utilize numerical feedback and Bayesian distribution, respectively. Experiments with Huawei Mate 9 show that both methods above can effectively eliminate the outlier resulting from severe signal loss, regardless of hand-holding gestures, with no individual privacy data required. Extensive experimental results demonstrate that the proposed methods are more efficient for NLOS and perform much better than the baselines of traditional fusion frameworks like the standard Kalman Filter. KFBPE has a relatively smoother tracking result, which guarantees an average positioning accuracy of up to 25 cm under the circumstance of nearly thirty percent acoustic signal loss (or NLOS) at the same time.

End-to-End Network for Pedestrian Detection, Tracking and Re-Identification in Real-Time Surveillance System.

Surveillance video has been widely used in business, security, search, and other fields. Identifying and locating specific pedestrians in surveillance video has an important application value in criminal investigation, search and rescue, etc. However, the requirements for real-time capturing and accuracy are high for these applications. It is essential to build a complete and smooth system to combine pedestrian detection, tracking and re-identification to achieve the goal of maximizing efficiency by balancing real-time capture and accuracy. This paper combined the detector and Re-ID models into a single end-to-end network by introducing a new track branch to YOLOv5 architecture for tracking. For pedestrian detection, we employed the weighted bi-directional feature pyramid network (BiFPN) to enhance the network structure based on the YOLOv5-Lite, which is able to further improve the ability of feature extraction. For tracking, based on Deepsort, this paper enhanced the tracker, which uses the Noise Scale Adaptive (NSA) Kalman filter to track, and adds adaptive noise to strengthen the anti-interference of the tracking model. In addition, the matching strategy is further updated. For pedestrian re-identification, the network structure of Fastreid was modified, which can increase the feature extraction speed of the improved algorithm by leaps and bounds. Using the proposed unified network, the parameters of the entire model can be trained in an end-to-end method with the multi-loss function, which has been demonstrated to be quite valuable in some other recent works. Experimental results demonstrate that pedestrians detection can obtain a 97% mean Average Precision (mAP) and that it can track the pedestrians well with a 98.3% MOTA and a 99.8% MOTP on the MOT16 dataset; furthermore, high pedestrian re-identification performance can be achieved on the VERI-Wild dataset with a 77.3% mAP. The overall framework proposed in this paper has remarkable performance in terms of the precise localization and real-time detection of specific pedestrians across time, regions, and cameras.

Online multi-object tracking with δ-GLMB filter based on occlusion and identity switch handling

• In this paper, a novel method of online multi-object tracking (MOT) is proposed. • The proposed method is based on the delta Generalized Labeled Multi-Bernoulli ( δ -GLMB) filter. • The main contributions of the paper are handling ID switch and addressing occlusion and miss-detection issues. • Experimental results show the superiority of the proposed method in handling MOT issues. In this paper, we propose an online multi-object tracking (MOT) method based on the delta Generalized Labeled Multi-Bernoulli ( δ -GLMB) filter framework to address occlusion and miss-detection issues and recover identity switch (ID switch). Along with the principal δ -GLMB filter that performs multi-object tracking, we propose a one-step δ -GLMB filter to handle occlusion and miss-detection. The one-step δ -GLMB filter is non-iterative and only requires current measurements. The filter is based on a proposed measurement-to-reappeared track association method and addresses MOT issues by incorporating all occluded and miss-detected objects. We introduce a novel similarity metric to apply in the measurement-to-reappeared track association process to define the weight of hypothesized reappeared tracks. To ensure the track consistency, we also extend the principal δ -GLMB filter to efficiently recover switched IDs using the cardinality density, size, and visual features of the hypothesized tracks. In addition, we perform an ablation study to demonstrate the contribution of the main parts of the proposed method. We evaluate the proposed method on well-known and publicly available test datasets focused on pedestrian tracking. Note that our proposed method is online and not based on the learning paradigm. So it does not use any additional source of information such as private detections and pre-trained networks. Despite that, we achieved a reliable performance in multiple persons tracking at complex scenes by applying occlusion/miss-detection and ID switch handlers. Experimental results show that the proposed tracker performs better or at least at the same level of the state-of-the-art online and offline MOT methods.

Deep-Learning-Based Inertial Odometry for Pedestrian Tracking Using Attention Mechanism and Res2Net Module

Pedestrian dead reckoning is a challenging task due to the low-cost inertial sensor error accumulation. Recent research has shown that deep learning methods can achieve impressive performance in handling this issue. In this letter, we propose inertial odometry using a deep-learning-based velocity estimation method. The deep neural network based on Res2Net modules and two convolutional block attention modules is leveraged to restore the potential connection between the horizontal velocity vector and raw inertial data from a smartphone. Our network is trained using only 50% of the public inertial odometry dataset (RoNIN) data. Then, it is validated on the RoNIN testing dataset and another public inertial odometry dataset (OXIOD). Compared with the traditional step-length and heading-system-based algorithm, our approach decreases the absolute translation error (ATE) by 76%–86%. In addition, compared with the state-of-the-art deep learning method (RoNIN), our method improves its ATE by 6%–31.4%.

Directional Statistics-Based Deep Metric Learning for Pedestrian Tracking and Re-Identification

Multiple Object Tracking (MOT) is the problem that involves following the trajectory of multiple objects in a sequence, generally a video. Pedestrians are among the most interesting subjects to track and recognize for many purposes such as surveillance, and safety. In recent years, Unmanned Aerial Vehicles (UAV’s) have been viewed as a viable option for monitoring public areas, as they provide a low-cost method of data collection while covering large and difficult-to-reach areas. In this paper, we present an online pedestrian tracking and re-identification framework based on learning a compact directional statistic distribution (von-Mises-Fisher distribution) for each person ID using a deep convolutional neural network. The distribution characteristics are trained to be invariant to clothes appearances and to transformations including rotation, translation, and background changes. Learning a vMF for each ID helps simultaneously in measuring the similarity between object instances and re-identifying the pedestrian’s ID. We experimentally validated our framework on standard publicly available dataset, which we used as a case study.

A Low-Cost Foot-Placed UWB and IMU Fusion-Based Indoor Pedestrian Tracking System for IoT Applications

Among existing wireless and wearable indoor pedestrian tracking solutions, the ultra-wideband (UWB) and inertial measurement unit (IMU) sensors are the popular options due to their accurate and globally referenced positioning, and low-cost and compact size, respectively. However, the UWB position accuracy is compromised by the indoor non-line of sight (NLOS) and the IMU estimation suffers from orientation drift as well as requiring position initialization. To overcome these limitations, this paper proposes a low-cost foot-placed UWB and IMU fusion-based indoor pedestrian tracking system. Our data fusion model is an improved loosely coupled Kalman filter with the inclusion of valid UWB observation detection. In this manner, the proposed system not only adjusts the consumer-grade IMU's accumulated drift but also filters out any NLOS instances in the UWB observation. We validated the performance of the proposed system with two experimental scenarios in a complex indoor environment. The root mean square (RMS) positioning accuracy of our data fusion model is enhanced by 60%, 53%, and 27% compared to that of the IMU-based pedestrian dead reckoning, raw UWB position, and conventional fusion model, respectively, in the single-lap NLOS scenario, and by 70%, 34%, and 12%, respectively, in the multi-lap LOS+NLOS scenario.

FMCW-MIMO Radar-Based Pedestrian Trajectory Tracking Under Low- Observable Environments

Pedestrian trajectory perception is always a challenging task for autonomous urban driving since the weak echoes from pedestrians may be masked by strong background clutters, for example, road infrastructures and other vehicles. As a consequence, this article presents a frequency-modulated continuous wave-multiple input multiple output (FMCW-MIMO) radar-based trajectory tracking method to realize continuous pedestrian detection and tracking under low-observable environments. More specifically, the keystone transform-constant false alarm ratio (KT-CFAR) is first proposed to maximize the output signal-to-interference plus noise ratio (SINR) depending on the time–space correlations of a sequence of consecutive echoes, for the sake of improving the pedestrian detection probability under low-observable conditions. Then, the multiple signal classification (MUSIC) algorithm is employed to provide a high azimuth resolution to improve the multitarget resolving ability. After that, a pedestrian trajectory tracking framework is constructed by means of Bayesian theory and sequential Monte Carlo (SMC) sampling. Finally, simulation and experimental results show that the proposed method can obtain high-precision pedestrian trajectory tracking under low SINR conditions compared with conventional Kalman filter (KF)-based tracking methods.

Pedestrian Gait Information Aided Visual Inertial SLAM for Indoor Positioning Using Handheld Smartphones

Simultaneous localization and mapping (SLAM) is currently a widely used technology for indoor positioning. Many studies have focused on the smartphone-based localization and navigation for its portability and feature-rich sensors. But due to the low-quality sensors and complex environments, current SLAM-based positioning technology using smartphones still poses great challenges, such as inherent cumulative global drift and potential divergence facing texture-less indoor regions. For the regular gait characteristics of pedestrians when naturally walking, the gait motion model can certainly provide effective observation of the pedestrian motion state. We propose a visual-inertial odometry (VIO) assisted by pedestrian gait information for smartphone-based indoor positioning. This work mainly builds two additional state constraints, pedestrian velocity, and step displacement, obtained by the pedestrian dead reckoning (PDR) algorithm for the visual-inertial tracking system. For each step, the corresponding residual term of step length and velocity constraints is constructed and added to the cost function for nonlinear sliding-window optimization. Furthermore, the step displacement is applied again in the four-degree-of-freedom (4-DOF) graph-based optimization to refine the trajectory. VIO system also assists the PDR algorithm in mode switching, to improve the accuracy of gait information by applying the adaptive step length formula. Field experiments were conducted, and the results indicate that with the aiding of the pedestrian gait information, the accuracy and robustness of the visual-inertial pedestrian tracking system using smartphones have been significantly improved. Compared with the state-of-the-art algorithm monocular visual-inertial navigation system (VINS-MONO), our method improves the accuracy by 54.6% on average in our field tests in challenging environments.