Accurate Pedestrian Detection Research Articles

PED-AI: Pedestrian Detection for Autonomous Vehicles using YOLOv5

Pedestrian detection is crucial for autonomous vehicles, surveillance, and pedestrian safety. This abstract introduces a novel pedestrian detection method using the YOLOv5 algorithm, known for its real-time object detection prowess. The approach aims to enhance pedestrian detection accuracy across diverse lighting conditions. Methodologically, the process involves data preparation, YOLOv5 model training, and subsequent evaluation. The architecture of YOLOv5, which employs anchor boxes and a single-pass convolutional neural network, allows for quick and accurate pedestrian identification. YOLOv5's design, which includes anchor boxes and a single-pass convolutional neural network, enables speedy and accurate pedestrian recognition. Study tests confirm the efficacy of the YOLOv5-based approach. In the first scenario, the model detected pedestrians in daylight with 75% accuracy, but it also produced 11 false negatives or a 25% miss. Although Scenario 2's accuracy was higher at 85%, there were still 11 false negatives, which suggested that there was a persistent detection gap. In spite of these outcomes, the YOLOv5 model demonstrates the possibility of accurate pedestrian detection in real-world settings. While it greatly improves applications like self-driving cars and pedestrian safety, lowering false negatives remains a primary goal for increasing overall accuracy. The investigation's findings show that YOLOv5 can function in a variety of lighting conditions, but also highlight the necessity for further work in order to meet stringent detection requirements.

Towards comprehensive understanding of pedestrians for autonomous driving: Efficient multi-task-learning-based pedestrian detection, tracking and attribute recognition

Safe interaction with pedestrians is important for autonomous vehicles in urban environments. Comprehensive understanding of pedestrian behavior is a challenge for autonomous driving, as different pedestrian attributes have different characteristics and there is not yet a dataset providing all attribute labels. In this paper, we design an efficient multi-task model for pedestrian detection, tracking and attribute recognition to fully mine information in multiple tasks and multiple datasets to reduce redundant image processing computation. Multiple training datasets are utilized to take full advantage of the available partially-labeled data. The model consists of two stages. The first stage is to detect and track pedestrians, using only low-resolution images to save computational resources. Accurate pedestrian detection is achieved in the bird's-eye-view (BEV), while pedestrian tracking is implemented on the basis of 3D bounding boxes. The second stage is image-based multi-attribute detection. This stage uses the detection and tracking results and some pre-processed information from the first stage along with high-resolution images. Multi-attributes of the pedestrian are identified, such as pedestrian's pose, age, gender, and motion direction. With a carefully-designed multi-task system that shares necessary feature maps among multiple tasks and reuses computation results, we can solve pedestrian detection, tracking, and multi-attribute recognition problems in an efficient framework. This leads to more accurate decisions and more efficient computations. The innovation of the model is to accomplish a simultaneous detection of multiple pedestrian attributes for autonomous driving, while fully sharing the features in the model to avoid repeated computation, as well as utilizing multiple partially-labeled datasets for the pedestrian detection, tracking and attribute recognition model training.

Multi-Object Pedestrian Tracking Using Improved YOLOv8 and OC-SORT.

Multi-object pedestrian tracking plays a crucial role in autonomous driving systems, enabling accurate perception of the surrounding environment. In this paper, we propose a comprehensive approach for pedestrian tracking, combining the improved YOLOv8 object detection algorithm with the OC-SORT tracking algorithm. First, we train the improved YOLOv8 model on the Crowdhuman dataset for accurate pedestrian detection. The integration of advanced techniques such as softNMS, GhostConv, and C3Ghost Modules results in a remarkable precision increase of 3.38% and an mAP@0.5:0.95 increase of 3.07%. Furthermore, we achieve a significant reduction of 39.98% in parameters, leading to a 37.1% reduction in model size. These improvements contribute to more efficient and lightweight pedestrian detection. Next, we apply our enhanced YOLOv8 model for pedestrian tracking on the MOT17 and MOT20 datasets. On the MOT17 dataset, we achieve outstanding results with the highest HOTA score reaching 49.92% and the highest MOTA score reaching 56.55%. Similarly, on the MOT20 dataset, our approach demonstrates exceptional performance, achieving a peak HOTA score of 48.326% and a peak MOTA score of 61.077%. These results validate the effectiveness of our approach in challenging real-world tracking scenarios.

Pedestrian Detection Method Based on Two-Stage Fusion of Visible Light Image and Thermal Infrared Image

Pedestrian detection has important research value and practical significance. It has been used in intelligent monitoring, intelligent transportation, intelligent therapy, and automatic driving. However, in the pixel-level fusion and the feature-level fusion of visible light images and thermal infrared images under shadows during the daytime or under low illumination at night in actual surveillance, missed and false pedestrian detection always occurs. To solve this problem, an algorithm for the pedestrian detection based on the two-stage fusion of visible light images and thermal infrared images is proposed. In this algorithm, in view of the difference and complementarity of visible light images and thermal infrared images, these two types of images are subjected to pixel-level fusion and feature-level fusion according to the varying daytime conditions. In the pixel-level fusion stage, the thermal infrared image, after being brightness enhanced, is fused with the visible image. The obtained pixel-level fusion image contains the information critical for accurate pedestrian detection. In the feature-level fusion stage, in the daytime, the previous pixel-level fusion image is fused with the visible light image; meanwhile, under low illumination at night, the previous pixel-level fusion image is fused with the thermal infrared image. According to the experimental results, the proposed algorithm accurately detects pedestrian under shadows during the daytime and low illumination at night, thereby improving the accuracy of the pedestrian detection and reducing the missed rate and false rate in the detection of pedestrians.

Wireless Sensor Network Based Real-Time Pedestrian Detection and Classification for Intelligent Transportation System

Pedestrian safety has become a critical consideration in developing society especially road traffic, an intelligent transportation need of the hour is the solution left. India tops the world with 11% of global road accidents. With this data, we have moved in the direction of computer vision applications for efficient and accurate pedestrian detection for intelligent transportation systems (ITS). The important application of this research is robot development, traffic management and control, unmanned vehicle driving (UVD), intelligent monitoring and surveillance system, and automatic pedestrian detection system. Much research has focused on pedestrian detection, but sustainable solution-driven research must still be required to overcome road accidents. We have proposed a wireless sensor network-based pedestrian detection system that classifies the real-time set of pedestrian activity and samples the reciprocally received signal strength (RSS) from the sensor node. We applied a histogram of oriented gradient (HOG) descriptor algorithm K-nearest neighbor, decision tree and linear support vector machine to measure the performance and prediction of the target. Also, these algorithms have performed a comparative analysis under different aspects. The linear support vector machine algorithm was trained with 481 samples. The performance achieves the accuracy of 98.90%and has accomplished superior results with a maximum precision of 0.99, recall of 0.98, and F-score of 0.95 with 2% error rate. The model’s prediction indicates that it can be used in the intelligent transportation system. Finally, the limitation and the challenges discussed to provide an outlook for future research direction to perform effective pedestrian detection.

Cascaded information enhancement and cross-modal attention feature fusion for multispectral pedestrian detection

Multispectral pedestrian detection is a technology designed to detect and locate pedestrians in Color and Thermal images, which has been widely used in automatic driving, video surveillance, etc. So far most available multispectral pedestrian detection algorithms only achieved limited success in pedestrian detection because of the lacking take into account the confusion of pedestrian information and background noise in Color and Thermal images. Here we propose a multispectral pedestrian detection algorithm, which mainly consists of a cascaded information enhancement module and a cross-modal attention feature fusion module. On the one hand, the cascaded information enhancement module adopts the channel and spatial attention mechanism to perform attention weighting on the features fused by the cascaded feature fusion block. Moreover, it multiplies the single-modal features with the attention weight element by element to enhance the pedestrian features in the single-modal and thus suppress the interference from the background. On the other hand, the cross-modal attention feature fusion module mines the features of both Color and Thermal modalities to complement each other, then the global features are constructed by adding the cross-modal complemented features element by element, which are attentionally weighted to achieve the effective fusion of the two modal features. Finally, the fused features are input into the detection head to detect and locate pedestrians. Extensive experiments have been performed on two improved versions of annotations (sanitized annotations and paired annotations) of the public dataset KAIST. The experimental results show that our method demonstrates a lower pedestrian miss rate and more accurate pedestrian detection boxes compared to the comparison method. Additionally, the ablation experiment also proved the effectiveness of each module designed in this paper.

A New UAV-Based Social Distance Detector for COVID-19 Outbreaks Reduction, Using IoT, Computer Vision and Deep Learning Technologies

Nowadays, we are living in a dangerous environment and our health system is under the threatened causes of Covid19 and other diseases. The people who are close together are more threatened by different viruses, especially Covid19. In addition, limiting the physical distance between people helps minimize the risk of the virus spreading. For this reason, we created a smart system to detect violated social distance in public areas as markets and streets. In the proposed system, the algorithm for people detection uses a pre-existing deep learning model and computer vision techniques to determine the distances between humans. The detection model uses bounding box information to identify persons. The identified bounding box centroid's pairwise distances of people are calculated using the Euclidean distance. Also, we used jetson nano platform to implement a low-cost embedded system and IoT techniques to send the images and notifications to the nearest police station to apply forfeit when it detects people’s congestion in a specific area. Lastly, the suggested system has the capability to assist decrease the intensity of the spread of COVID-19 and other diseases by identifying violated social distance measures and notifying the owner of the system. Using the transformation matrix and accurate pedestrian detection, the process of detecting social distances between individuals may be achieved great confidence. Experiments show that CNN-based object detectors with our suggested social distancing algorithm provide reasonable accuracy for monitoring social distancing in public places, as well.

Deep Learning-Based Pedestrian Detection in Autonomous Vehicles: Substantial Issues and Challenges

In recent years, autonomous vehicles have become more and more popular due to their broad influence over society, as they increase passenger safety and convenience, lower fuel consumption, reduce traffic blockage and accidents, save costs, and enhance reliability. However, autonomous vehicles suffer from some functionality errors which need to be minimized before they are completely deployed onto main roads. Pedestrian detection is one of the most considerable tasks (functionality errors) in autonomous vehicles to prevent accidents. However, accurate pedestrian detection is a very challenging task due to the following issues: (i) occlusion and deformation and (ii) low-quality and multi-spectral images. Recently, deep learning (DL) technologies have exhibited great potential for addressing the aforementioned pedestrian detection issues in autonomous vehicles. This survey paper provides an overview of pedestrian detection issues and the recent advances made in addressing them with the help of DL techniques. Informative discussions and future research works are also presented, with the aim of offering insights to the readers and motivating new research directions.

An Efficient Pedestrian Detection for Realtime Surveillance Systems Based on Modified YOLOv3

Pedestrian detection is an important branch of object detection due to its various applications. It plays a vital role in many fields such as intelligent surveillance systems. The recognition, identification and tracking modules of surveillance are based on efficient and accurate pedestrian detection. Our paper proposes an efficient model to solve real-time pedestrian detection with high accuracy based on modified ShuffleNet and YOLOv3 models. We provide a method to pick the dimensions and number of anchor boxes for predicting bounding boxes accurately. Then we use two improved shuffle units to lightweight the backbone of YOLOv3, which reduces the 67.5% floating point operations per second (FLOPs) and 65.1% parameters. We validate our model on CrowdHuman detection data set and get 62.7 mAP for face and 62.0 mAP person with 0.748 average IOU. Our network processes images in real-time at 186.1 frames per second for network and 12.5 frames per second for the entire model on CrowdHuman.

Pedestrian Detection Method Based on Recurrent Convolutional Neural Networks

To improve the accuracy of pedestrian detection in videos, a pedestrian detection method based on a recurrent convolutional neural network was proposed. The method used the recurrent convolutional neural network to combine the context information among several consecutive images in the video for implementing accurate pedestrian detection. Firstly, a convolutional neural network was used to extract several groups of feature maps of corresponding input images. Secondly, according to the image order, these groups of feature maps were successively input to the recurrent convolutional neural network to form a mask indicating the pedestrian locations. Finally, the pedestrian detection result of the current image was obtained by estimating the bounding boxes on the mask. The experimental results showed that the proposed method can achieve the accurate pedestrian detection results on three datasets (ETH, CUHK, PETS 2007) when compared with other pedestrian detection methods.

MAF-YOLO: Multi-modal attention fusion based YOLO for pedestrian detection

Achieving the rapid and precise detection of pedestrians in natural environments is crucial for many applications of artificial intelligence systems. However, accurate pedestrian detection at nighttime is challenging due to the low-luminous density and low-resolution of infrared images, and the detection speed is required. In this paper, a real-time pedestrian detection method using a novel multi-modal attention fusion YOLO (MAF-YOLO) was proposed. Firstly, a multi-modal feature extraction module based on the compressed Darknet53 framework was built to adapt the nighttime pedestrian detection and ensure efficiency. The features were extracted from both modalities and then fused by a modal weighted fusion module. Secondly, we defined a loss function and regenerated the size of the anchor box based on the K-means clustering algorithm to improve the detection speed and the robustness of small objects. Finally, a dual attention module was applied to acquire more semantic features from small objects with low-resolution. The experimental results on KAIST and OSU Color-Thermal datasets corroborated the effectiveness of the proposed MAF-YOLO. The proposed methods could also be utilized to facilitate the performance of other pedestrian detection algorithms.

Real-Time and Accurate UAV Pedestrian Detection for Social Distancing Monitoring in COVID-19 Pandemic.

Coronavirus Disease 2019 (COVID-19) is a highly infectious virus that has created a health crisis for people all over the world. Social distancing has proved to be an effective non-pharmaceutical measure to slow down the spread of COVID-19. As unmanned aerial vehicle (UAV) is a flexible mobile platform, it is a promising option to use UAV for social distance monitoring. Therefore, we propose a lightweight pedestrian detection network to accurately detect pedestrians by human head detection in real-time and then calculate the social distancing between pedestrians on UAV images. In particular, our network follows the PeleeNet as backbone and further incorporates the multi-scale features and spatial attention to enhance the features of small objects, like human heads. The experimental results on Merge-Head dataset show that our method achieves 92.22% AP (average precision) and 76 FPS (frames per second), outperforming YOLOv3 models and SSD models and enabling real-time detection in actual applications. The ablation experiments also indicate that multi-scale feature and spatial attention significantly contribute the performance of pedestrian detection. The test results on UAV-Head dataset show that our method can also achieve high precision pedestrian detection on UAV images with 88.5% AP and 75 FPS. In addition, we have conducted a precision calibration test to obtain the transformation matrix from images (vertical images and tilted images) to real-world coordinate. Based on the accurate pedestrian detection and the transformation matrix, the social distancing monitoring between individuals is reliably achieved.

A Pedestrian Detection Algorithm Based on Score Fusion for Multi-LiDAR Systems.

Pedestrian detection plays an essential role in the navigation system of autonomous vehicles. Multisensor fusion-based approaches are usually used to improve detection performance. In this study, we aimed to develop a score fusion-based pedestrian detection algorithm by integrating the data of two light detection and ranging systems (LiDARs). We first evaluated a two-stage object-detection pipeline for each LiDAR, including object proposal and fine classification. The scores from these two different classifiers were then fused to generate the result using the Bayesian rule. To improve proposal performance, we applied two features: the central points density feature, which acts as a filter to speed up the process and reduce false alarms; and the location feature, including the density distribution and height difference distribution of the point cloud, which describes an object’s profile and location in a sliding window. Extensive experiments tested in KITTI and the self-built dataset show that our method could produce highly accurate pedestrian detection results in real-time. The proposed method not only considers the accuracy and efficiency but also the flexibility for different modalities.

Accurate Image-Based Pedestrian Detection With Privacy Preservation

In this paper, we propose an accurate pedestrian detection scheme with privacy preservation (PPPD) based on pedestrian images. By utilizing the vector homomorphic encryption (VHE), private linear-transforming matrices are ingeniously designed to enable arbitrary permutation operations for the encrypted vectors. In this way, the feature vector extraction of the histogram of oriented gradient (HOG) can be efficiently performed over the encrypted pedestrian images. Furthermore, due to the encrypted inner product calculations supported by VHE, an encrypted kernel matrix is constructed to generate multiple encrypted kernels (i.e., linear, polynomial, and Gaussian kernels). The pedestrian detection model based on the supported vector machine (SVM) can be securely trained over the encrypted kernels. With the proposed scheme, the extracted features of pedestrian images are not necessary to be returned to the image owner for decryption, such that the communication costs can be significantly reduced. In addition, the privacy of the whole process in pedestrian detection can also be guaranteed. Extensive experiments are conducted over multiple pedestrian datasets, and it is demonstrated that PPPD can achieve high accuracy of pedestrian detection with lower computation and communication overhead compared with the existing schemes.

A Unified Multi-Task Learning Architecture for Fast and Accurate Pedestrian Detection

We present a unified multi-task learning architecture for fast and accurate pedestrian detection. Different from existing methods which often focus on either a new loss function or architecture, we propose an improved multi-task convolutional neural network learning architecture to effectively and efficiently interfuse the task of pedestrian detection and semantic segmentation. To achieve this, we integrate a lightweight semantic segmentation branch to Faster R-CNN detection framework that enables end-to-end hard parameter sharing in order to boost the detection performance and maintain computational efficiency as follows. Firstly, a Semantic Segmentation to Feature Module (SS2FM) refines the convolutional features in RPN stage by integrating the features generated from the semantic segmentation branch. Secondly, a Semantic Segmentation to Confidence Module (SS2CM) refines the classification confidence in RPN stage by fusing it with the semantic segmentation confidence. We also introduce an effective anchor matching point transform to alleviate the problem of feature misalignment for heavily occluded pedestrians. The proposed unified multi-task learning architecture lends itself well to more robust pedestrian detection in diverse scenarios with negligible computation overhead. In addition, the proposed architecture can achieve high detection performance with low resolution input images, which significantly reduces the computational complexity. Experiment results on CityPersons and Caltech datasets show that our method is the fastest among all state-of-the-art pedestrian detection methods while exhibiting competitive detection performance.

Adversarial Training-Based Hard Example Mining for Pedestrian Detection in Fish-Eye Images

Since fish-eye cameras are popular in the intelligent transportation systems, accurate pedestrian detection in fish-eye images becomes more and more critical in low-speed scenarios. Usually, big data based training is the key for detectors to handle the distortion problem in fish-eye images. Especially, hard examples are more important for detectors. They have more complex features and are hard to recognize. In conventional methods, fish-eye images are collected and labeled manually. These methods are expensive and labor-intense. More importantly, these methods are still hard to collect abundant hard examples since they are rare in reality. This work proposes the Distortion Generation Network, which generates generous fish-eye images automatically using only small samples. Moreover, the Adversarial Distortion Generation Network is proposed to mine hard examples via adversarial training. These hard examples benefit detectors to be more robust to seriously distorted objects in fish-eye images. Experiments with the ETH, the KITTI, the GM-ATCI and real fish-eye datasets demonstrate that the proposed methods achieve higher accuracy than conventional methods in pedestrian detection in fish-eye images.

A novel multi-view pedestrian detection database for collaborative Intelligent Transportation Systems

Recent advances in machine-learning, especially in deep neural networks have significantly accelerated the development and deployment of transport-oriented intelligent designs with increasingly high efficiency. While these technologies are exceptionally promising toward revolutionizing our current mobility and reducing the number of road accidents, the way to safe Intelligent Transportation Systems (ITS) remains long. Since pedestrians are the most vulnerable road users, designing accurate pedestrian detection methods is a priority task. However, traditional monocular pedestrian detection methods are limited, especially in occlusion handling. Hence, a collaborative perception scheme in which vehicles no longer restrict their input data to their immediate embedded sensors and rather exploit data from remote sensors is necessary to achieve a more comprehensive environment perception. In this work, we propose a novel public dataset: Infrastructure to Vehicle Multi-View Pedestrian Detection Database (I2V-MVPD) that combines synchronized images from both a mobile camera embedded in a car and a static camera in the road infrastructure. We also propose a new multi-view pedestrian detection framework based on collaborative intelligence between vehicles and infrastructure. Our results show a significant improvement in detection performance over monocular detection.

SYOLO: An Efficient Pedestrian Detection

Pedestrian detection is an important branch of object detection. It plays a vital role in many fields such as intelligent monitoring systems. The premise of the pedestrian recognition algorithm in the application of the industrial scene is accurate pedestrian detection. Our paper proposes a model to solve real-time pedestrian detection with high accuracy base on YOLO v3. We provide a method to select the size and number of anchor boxes for predicting bounding boxes accurately. Then we use a modified shuffle unit to lightweight the backbone of YOLO v3, which reduces the 67.3% FLOPs and 65.1% parameters. We train and validate our model on CrowdHuman detection dataset, SYOLO gets 62.7 mAP for face and 62.0mAP person with 0.748 average IOU. Our network processes images in real-time at 185.8 FPS for network and 12.3 FPS for the entire model on CrowdHuman.

Accurate Pedestrian Detection by Human Pose Regression.

Pedestrian detection with high detection and localization accuracy is increasingly important for many practical applications. Due to the flexible structure of the human body, it is hard to train a template-based pedestrian detector that achieves a high detection rate and a good localization accuracy simultaneously. In this paper, we utilize human pose estimation to improve the detection and localization accuracy of pedestrian detection. We design two kinds of pose-indexed features that can considerably improve the discriminability of the detector. In addition to employing a two-stage pipeline to carry out these two tasks, we unify pose estimation and pedestrian detection into a cascaded decision forest in which they can cooperate sufficiently. To prevent irregular positive examples, such as truncated ones, from distracting the pedestrian detection and the pose regression, we clean the positive training data by realigning the bounding boxes and rejecting the wrong positive samples. Experimental results on the Caltech test dataset demonstrate the effectiveness of our proposed method. Our detector achieves 11.1% MR-2, outperforming all existing detectors without using the convolutional neural network (CNN). Moreover, our method can be assembled with other detectors based on CNNs to improve detection and localization performance. By collaborating with the recent CNN-based method, our detector achieves 5.5% MR-2 on the Caltech test dataset, outperforming the state-of-the-art methods.

Training Cascade Compact CNN With Region-IoU for Accurate Pedestrian Detection

Recently, pedestrian detection has made significant advances benefiting from the region-based convolutional neural networks (R-CNN). However, training R-CNN with a holistic intersection over union (IoU) always brings many flawed positive samples. This paper introduces a strict matching metric, which is beneficial to selecting well-aligned positive samples. Specifically, this matching metric is defined on a set of region-IoUs instead of a holistic IoU, which considers the alignments of different part regions in a whole bounding box simultaneously. A positive sample matches a ground truth only if all its region-IoUs are bigger than a threshold. Secondly, an improved negative example selection strategy using both the classification and localization information is proposed to mine hard negative examples, which can further suppress the false positive detections near the pedestrians. Based on the proposed sample selection strategy, a cascade compact convolutional neural network (CC-CNN) is proposed for accurate pedestrian detection. Each stage of the CC-CNN is constructed with a compact network that only consists of a small number of parameters, thus making the detector suitable to be implemented on onboard embedded systems. Experimental results on two widely used pedestrian datasets demonstrate that the proposed training strategy and the CC-CNN based detector can effectively improve the detection rate and the localization accuracy using fewer parameters.