Pedestrian Tracking Research Articles

Enhancing Availability of Autonomous Shuttle Services: A Conceptual Approach toward Challenges and Opportunities

<div>Autonomous vehicle technologies have become increasingly popular over the last few years. One of their most important application is autonomous shuttle buses that could radically change public transport systems. In order to enhance the availability of shuttle service, this article outlines a series of interconnected challenges and innovative solutions to optimize the operation of autonomous shuttles based on the experience within the Shuttle Modellregion Oberfranken (SMO) project. The shuttle shall be able to work in every weather condition, including the robustness of the perception algorithm. Besides, the shuttle shall react to environmental changes, interact with other traffic participants, and ensure comfortable travel for passengers and awareness of VRUs. These challenging situations shall be solved alone or with a teleoperator’s help. Our analysis considers the basic sense–plan–act architecture for autonomous driving. Critical components like object detection, pedestrian tracking, trajectory planning, and teleoperation are analyzed, and concepts for overcoming the challenges mainly in these components are presented.</div>

Novel calibration method for improved UWB sensor distance measurement in the context of application for 3D analysis of human movement

Integrated UWB and MIMU sensor systems have become popular for pedestrian tracking and indoor localization, since this facilitates data fusion that improves position estimation accuracy by exploiting the complementary nature of their error sources. Integrated UWB/MIMU sensors also have great potential in only on-body use for 3D analysis of human movement, as with MIMU sensors alone accurate direct estimation of (relative) body segment position is not possible. For this, a position estimation accuracy with errors smaller than 1 cm is deemed required. The lowest position estimating error with integrated UWB/MIMU systems, reported so far, is around 5 cm. The main accuracy limiting factors were found to be the systematic errors in the distance estimates from the UWB sensor. Multiple reported attempts to calibrate for these systematic errors failed to achieve the desired accuracy. This article presents a novel distance-bias calibration method that minimizes the residual systematic distance estimate errors using multiple sensors in a swarm configuration. Validation was performed against synthetic reference data and against reference data measured with an optical motion tracking system. Significantly reduced systematic distance estimate errors (≤0.5 cm) were found. These results promise to facilitate significantly better position estimates in future UWB/MIMU data fusion.

Fg-ORKA: fast and gridless reconstruction of moving and deforming objects in multidimensional data

Abstract Identifying and tracking objects over multiple observations is a frequent task in many applications. Traffic monitoring requires the tracking of vehicles or pedestrians in video data and geophysical exploration relies on identifying seismic wave fronts from data of multiple sensors, only to mention two examples. In many cases, the object changes its shape or position within the given data from one observation to another. Vehicles can change their position and angle relative to the camera while seismic waves have different arrival times, frequencies, or intensities depending on the sensor position. This complicates the task at hand.

In a previous work, the authors presented a new algorithm to solve this problem - Object reconstruction using K-approximation (ORKA). This algorithm is hindered by two conflicting limitations: the tracked movement is limited by the sampling grid while the complexity increases exponentially with the resolution. We introduce an iterative variant of the ORKA algorithm that is able to overcome this conflict. We also give a brief introduction on the original ORKA algorithm. Knowledge of the previous work is thus not required.

We give theoretical error bounds and a complexity analysis which we validate with several numerical experiments. Moreover, we discuss the influence of different parameter choices in detail. The results clearly show that the iterative approach can outperform ORKA in both accuracy and efficiency. On the example of video processing we show that the new method can be applied where the original algorithm is too time and memory intensive. Furthermore, we demonstrate on seismic exploration data that we are now able to recover much finer details on the wave front movement then before.

Efforts to reduce illegal parking in Lampung Province

Purpose: The formulation of the problem in this study is how land is prohibited for parking and the efforts to suppress illegal parking in Lampung Province. Research Methodology: The method used in this study was descriptive and qualitative. Results: The results of this study are There are 10 no-parking areas that must be obeyed, including: Bends, shoulders or a bridge, On pedestrian or bicycle tracks, Near traffic lights, On main roads, Facing or close to other stopped vehicles, Within 6 meters (20 feet) of an intersection, or within 9 meters (30 feet) of a bus stop, unless it is damaged. Then, they do not stop or park 3 m (10 feet) on the other side of a fire hydrant, or which may interfere with the access of firefighting vehicles to the hydrant, along slippery roads, on overpasses, tunnels, or on the side of the road leading to the overpass or tunnel, on grass verge or road shoulder. The efforts of the Lampung Province Transportation Agency to deal with illegal parking constraints are as follows: The Department of Transportation must perform mapping, good parking management not only helps in traffic management but also increases local revenue (PAD), the importance of awareness from the community and the government, the government must be present and provide guidance and ensure that parking is managed properly, there are sanctions for violators of indiscriminate parking, but the emphasis is more on education and guidance than sanctions alone.

MM_Fast_RCNN_ResNet: Construction of Multimodal Faster RCNN Inception and ResNet V2 for Pedestrian Tracking and detection

Pedestrian identification and tracking is a crucial duty in smart building monitoring. The development of sensors has led to architects' focus on smart building design. The image distortions caused by numerous external environmental factors present a significant problem for pedestrian recognition in smart buildings. It is difficult for machine learning algorithms and other conventional filter-based image classification methods, such as histograms of oriented gradient filters, to function efficiently when dealing with many input photos of pedestrians. Deep learning algorithms are now performing substantially better when processing an enormous amount of image data. This article evaluates a novel multimodal classifier-based pedestrian identification method. The proposed method is Multimodal Faster RCNN Inception and ResNet V2 (MM Fast RCNN ResNet). The collected attributes address a tracking problem and establish the foundation for several object recognition tasks (novelty). Our method's neural network is regularized, and the feature representation is automatically adjusted to the detection assignment, resulting in high accuracy (superior to the proposed method). The proposed method is assessed using the PenFudan dataset and contemporary techniques regarding several factors. It is discovered that the recommended MM Fast RCNN ResNet obtains precision, recall, FPPI, FPPW, and average precision of 0.9057, 0.8629, 0.0898, and 0.0943.

Retraction Note: Dark infrared night vision imaging proposed work for pedestrian detection and tracking

Retraction Note: Dark infrared night vision imaging proposed work for pedestrian detection and tracking

CCBA-NMS-YD: A Vehicle Pedestrian Detection and Tracking Method Based on Improved YOLOv7 and DeepSort

In this paper, we propose a vehicle pedestrian detection and tracking method based on the improved YOLOv7 and DeepSort algorithms. We aim to improve the quality of vehicle pedestrian detection and tracking, addressing the challenges that current commercially available autonomous driving technologies face in complex and changing road traffic situations. First, the NMS (non-maximum suppression) algorithm in YOLOv7 is replaced with a modified Soft-NMS algorithm to ensure that targets can be accurately detected at high densities, and second, the CCBA (coordinate channel attention module) attention mechanism is incorporated to improve the feature extraction and perception capabilities of the network. Finally, a multi-scale feature network is introduced to extract features of small targets more accurately. Finally, the MobileNetV3 lightweight module is introduced into the feature extraction network of DeepSort, which not only reduces the number of model parameters and network complexity, but also improves the tracking performance of the target. The experimental results show that the improved YOLOv7 algorithm improves the average detection accuracy by 3.77% compared to that of the original algorithm; on the MOT20 dataset, the refined DeepSort model achieves a 1.6% increase in MOTA and a 1.9% improvement in MOTP; in addition, the model volume is one-eighth of the original algorithm. In summary, our model is able to achieve the desired real-time and accuracy, which is more suitable for autonomous driving.

Understanding pedestrian movement using urban sensing technologies: the promise of audio-based sensors

While various sensors have been deployed to monitor vehicular flows, sensing pedestrian movement is still nascent. Yet walking is a significant mode of travel in many cities, especially those in Europe, Africa, and Asia. Understanding pedestrian volumes and flows is essential for designing safer and more attractive pedestrian infrastructure and for controlling periodic overcrowding. This study discusses a new approach to scale up urban sensing of people with the help of novel audio-based technology. It assesses the benefits and limitations of microphone-based sensors as compared to other forms of pedestrian sensing. A large-scale dataset called ASPED is presented, which includes high-quality audio recordings along with video recordings used for labeling the pedestrian count data. The baseline analyses highlight the promise of using audio sensors for pedestrian tracking, although algorithmic and technological improvements to make the sensors practically usable continue. This study also demonstrates how the data can be leveraged to predict pedestrian trajectories. Finally, it discusses the use cases and scenarios where audio-based pedestrian sensing can support better urban and transportation planning.

Robust pedestrian multi-object tracking in the intelligent bus environment

Abstract Pedestrian multi-object tracking algorithms aim to maintain identity information of pedestrians by comparing the similarity between trajectories and detections, predicting pedestrian motion trajectories. However, within the context of intelligent bus, challenges arise due to factors such as passenger growth and vehicle vibrations, rendering existing pedestrian multi-object tracking algorithms less accurate. Therefore, this paper proposes an intelligent bus terminal robust pedestrian multi-object tracking algorithm named pure motion (PM), which can consistently and stably track pedestrians. The proposed algorithm employs several key strategies. Firstly, it optimizes trajectory prediction by adapting the aspect ratio of the prediction box based on pedestrian movement, automatically adjusting its shape, and selecting velocity weight coefficients according to different tracking targets. Secondly, it decomposes the homography matrix to acquire motion components and correct predicted results under motion conditions. Subsequently, the algorithm leverages the similarity between detection results and trajectories to retain high-confidence detections, eliminating low-confidence ones associated with background, thereby reducing false negatives and enhancing trajectory coherence. Futhermore, the introduction of detection confidence into trajectory updates to enhances the precision of measurement noise. The proposed algorithm underwent testing in intelligent bus driving scenarios, including turns, waiting for traffic lights, emergency braking, and approaching bus stops. The tracking accuracy on the MOT17-13-val dataset reaches 81.8. The results demonstrate that PM significantly improves the robustness of pedestrian multi-object tracking algorithms in the environment of intelligent bus.

A Local-and-Global Attention Reinforcement Learning Algorithm for Multiagent Cooperative Navigation.

The cooperative navigation algorithm is the crucial technology for multirobot systems to accomplish autonomous collaborative operations, and it is still a challenge for researchers. In this work, we propose a new multiagent reinforcement learning algorithm called multiagent local-and-global attention actor-critic (MLGA2C) for multiagent cooperative navigation. Inspired by the attention mechanism, we design the local-and-global attention module to dynamically extract and encode critical environmental features. Meanwhile, based on the centralized training and decentralized execution (CTDE) paradigm, we extend a new actor-critic method to handle feature encoding and make navigation decisions. We also evaluate the proposed algorithm in two cooperative navigation scenarios: static target navigation and dynamic pedestrian target tracking. The multiple experimental results show that our algorithm performs well in cooperative navigation tasks with increasing agents.

Research on Multi-Modal Pedestrian Detection and Tracking Algorithm Based on Deep Learning

Research on Multi-Modal Pedestrian Detection and Tracking Algorithm Based on Deep Learning

A novel approach for reliable pedestrian trajectory collection with behavior-based trajectory reconstruction for urban surveillance systems

Collecting reliable pedestrian trajectories in pedestrian behavior analysis, trajectories broken by frame sampling and trajectories crossing in multi-object conditions often hinder their performance of existing pedestrian tracking models. Despite attempts to address these issues by performing detection and tracking simultaneously using deep learning algorithms, previous methods still struggle with errors such as mistaking a single pedestrian for multiple pedestrians. We propose a novel approach to efficiently collect and correct pedestrian trajectories with minimized practical errors in multi-object conditions for urban surveillance systems. Our system utilizes a single vision sensor to automatically collects trajectories of multiple pedestrians and employ simple, low-computational algorithms, particularly the Deep simple online real-time tracking (Deep SORT) method, to calibrate the trajectories from tracking-by-detection models. Additionally, our system identifies and merges broken pedestrian trajectories, treating them as potential single trajectories, while considering their spatiotemporal ranges. We evaluate the proposed system by implementing it on real testbed video footage. Our method significantly improves practical errors and achieves more accurate pedestrian trajectories compared to existing models, and exhibits robust characteristics, effectively handling complex situations such as occlusions and crowds.

Synthetic Data for Video Surveillance Applications of Computer Vision: A Review

In recent years, there has been a growing interest in synthetic data for several computer vision applications, such as automotive, detection and tracking, surveillance, medical image analysis and robotics. Early use of synthetic data was aimed at performing controlled experiments under the analysis by synthesis approach. Currently, synthetic data are mainly used for training computer vision models, especially deep learning ones, to address well-known issues of real data, such as manual annotation effort, data imbalance and bias, and privacy-related restrictions. In this work, we survey the use of synthetic training data focusing on applications related to video surveillance, whose relevance has rapidly increased in the past few years due to their connection to security: crowd counting, object and pedestrian detection and tracking, behaviour analysis, person re-identification and face recognition. Synthetic training data are even more interesting in this kind of application, to address further, specific issues arising, e.g., from typically unconstrained image or video acquisition conditions and cross-scene application scenarios. We categorise and discuss the existing methods for creating synthetic data, analyse the synthetic data sets proposed in the literature for each of the considered applications, and provide an overview of their effectiveness as training data. We finally discuss whether and to what extent the existing synthetic data sets mitigate the issues of real data, highlight existing open issues, and suggest future research directions in this field.

A structural description of pedestrian movement behavior in multiple surveillance videos

This study purpose of recording pedestrians' activities in all surveillance videos and then convert these records into structured descriptions to analyze a large number of surveillance videos. This paper proposes a four-stage processing method. The first stage uses convolutional neural network (CNN) to detect objects in the surveillance video and classify them. The second stage finds the correlation of pedestrians in continuous images and describes multiple objects' trajectories in the surveillance video. The third stage calculates the similarity of the pedestrian object sets of the surveillance video at different times as the basis for pedestrian re-identification. In the fourth stage, extract the walking path of the pedestrian in each surveillance video. Each pedestrian's actions in each surveillance video, including whether to stand, walking path, direction, clothing color and walking time. To construct a correlation table of pedestrian motion in multiple surveillance videos.This experiment uses two image datasets for experimentation and analysis to evaluate this method's effectiveness for pedestrian movement in multiple surveillance videos. The image datasets are the Multi-Camera Object Tracking (MCT) Challenge and PETS 2009 Benchmark Data. This experiment includes pedestrian detection, pedestrian identification, and multi-pedestrian tracking of pedestrians in single and multiple surveillance cameras. The results show that in the pedestrian detection experiment, the convolutional neural network can effectively solve the problems caused by static objects, light changes, similar colors, crowded pedestrians, and incomplete objects. In the pedestrian recognition and re-recognition experiments, the accuracy is reduced due to the influence of low-resolution surveillance videos. This detection method of the pedestrian is more accurate than traditional dynamic object detection methods. After the pedestrian tracking and re-identification are completed, the pedestrian relationships in multiple surveillance cameras are integrated, and a pedestrian behavior association table is built for surveillance videos.

PTDS CenterTrack: pedestrian tracking in dense scenes with re-identification and feature enhancement

PTDS CenterTrack: pedestrian tracking in dense scenes with re-identification and feature enhancement

A simple transformer-based baseline for crowd tracking with Sequential Feature Aggregation and Hybrid Group Training

Tracking pedestrians in crowded scenes is a challenging task. Existing transformer-based tracking methods integrate detection and tracking into a unified model, which simplifies the tracking process. However, these methods also introduce complicated attention mechanisms that increase the model complexity and cost. To address this issue, we propose SOTTrack, a simple online transformer-based method for crowd tracking. Our method enhances feature learning and training strategies without sacrificing simplicity and efficiency. Specifically, we introduce the Sequential Feature Aggregation (SFA) module and the Hybrid Group Training (HGT) approach. The SFA module fuses features from sequential images to improve the temporal consistency of visual features within short time intervals. The HGT approach assigns different queries to multiple guided tasks, such as label assignment and de-noising, which are only used during training and do not incur any inference cost. We evaluate our method on the MOT17 and MOT20 datasets and demonstrate its competitive performance.

Pedestrian tracking method based on S-YOFEO framework in complex scene

A real-time stable multi-target tracking method based on the enhanced You Only Look Once-v8 (YOLOv8) and the optimized Simple Online and Realtime Tracking with a Deep association metric (DeepSORT) for multi-target tracking (S-YOFEO) is proposed with the aim of addressing the issue of target ID transformation and loss caused by the increase of practical background complexity. For the purpose of further enhancing the representation of small-scale features, a small target detection head is first introduced to the detection layer of YOLOv8 in this paper with the aim of collecting more detailed information by increasing the detection resolution of YOLOv8. Secondly, the Omni-Scale Network (OSNet) feature extraction network is implemented to enable accurate and efficient fusion of the extracted complex and comparable feature information, taking into account the restricted computational power of DeepSORT’s original feature extraction network. Again, a novel adaptive forgetting Kalman filter algorithm (FSA) is devised to enhance the precision of model prediction and the effectiveness of parameter updates to adjust to the uncertain movement speed and trajectory of pedestrians in real scenarios. Following that, an accurate and stable association matching process is obtained by substituting Efficient-Intersection over Union (EIOU) for Complete-Intersection over Union (CIOU) in DeepSORT to boost the convergence speed and matching effect during association matching. Last but not least, One-Shot Aggregation (OSA) is presented as the trajectory feature extractor to deal with the various noise interferences in the complex scene. OSA is highly sensitive to information of different scales, and its one-time aggregation property substantially decreases the computational overhead of the model. According to the trial results, S-YOFEO has made some developments as its precision can reach 78.2% and its speed can reach 56.0 frames per second (FPS).

Privacy-preserving pedestrian tracking with path image inpainting and 3D point cloud features

Tracking pedestrian flow in large public areas is vital, yet ensuring privacy is paramount. Traditional visual-based tracking systems are raising concerns for potentially obtaining persistent and permanent identifiers that can compromise individual identities. Moreover, in areas such as the vicinity of restrooms, any form of data acquisition capturing human behavior should be refrained from, making it also crucial to appropriately address and complement these blind spots for a comprehensive analysis of pedestrian movement in the entire area. In this paper, we present our pedestrian tracking algorithm using distributed 3D LiDARs (Light Detection and Ranging), which capture pedestrians as 3D point clouds, omitting identifiable features. Our system bridges blind spots by leveraging historical movement data and 3D point cloud features, complemented by a generative diffusion model to predict trajectories in unseen areas. In a large-scale testbed with 70 LiDARs, the system achieved a 0.98 F-measure, highlighting its potential as a leading privacy-preserving tracking solution.

AODiMP‐TIR: Anti‐occlusion thermal infrared targets tracker based on SuperDiMP

AbstractTo address the issue of tracking drift and failures in thermal infrared (TIR) tracking tasks caused by target occlusion, this study proposes an anti‐occlusion TIR target tracker named AODiMP‐TIR. This approach involves an anti‐occlusion strategy that relies on target occlusion status determination and trajectory prediction. This enables the prediction of the target's current position when it is identified as occluded, ensuring swift recapture upon reappearance. A criterion is introduced for occlusion status determination based on the classification response map of SuperDiMP. Additionally, a trajectory mapping module designed to decouple target motion from camera motion is presented, enhancing the precision of trajectory prediction. Comparative experiments with other state‐of‐the‐art trackers are conducted on the large‐scale high‐diversity thermal infrared object tracking benchmark (LSOTB‐TIR), LSOTB‐TIR100, and thermal infrared pedestrian tracking benchmark (PTB‐TIR) datasets. The results indicate that the AODiMP‐TIR performs well across all three datasets, particularly exhibiting outstanding performance in occlusion sequences. Furthermore, ablation study experiments confirm the effectiveness of the anti‐occlusion strategy, occlusion determination criterion and trajectory mapping module.

Enhancing Pedestrian Tracking in Autonomous Vehicles by Using Advanced Deep Learning Techniques

Effective collision risk reduction in autonomous vehicles relies on robust and straightforward pedestrian tracking. Challenges posed by occlusion and switching scenarios significantly impede the reliability of pedestrian tracking. In the current study, we strive to enhance the reliability and also the efficacy of pedestrian tracking in complex scenarios. Particularly, we introduce a new pedestrian tracking algorithm that leverages both the YOLOv8 (You Only Look Once) object detector technique and the StrongSORT algorithm, which is an advanced deep learning multi-object tracking (MOT) method. Our findings demonstrate that StrongSORT, an enhanced version of the DeepSORT MOT algorithm, substantially improves tracking accuracy through meticulous hyperparameter tuning. Overall, the experimental results reveal that the proposed algorithm is an effective and efficient method for pedestrian tracking, particularly in complex scenarios encountered in the MOT16 and MOT17 datasets. The combined use of Yolov8 and StrongSORT contributes to enhanced tracking results, emphasizing the synergistic relationship between detection and tracking modules.