Occluded Pedestrian Detection Research Articles

On the Detection of Vulnerable Road Users—Simulating Limits of an FMCW Radar

The complexity of urban traffic environments poses unique challenges for automotive radar sensors. Limited range and Doppler resolution hinder the ability to distinguish closely spaced objects and detect vulnerable road users, particularly pedestrians. Therefore, it is important to know the limits of FMCW radars. To assess these limitations, the WinProp channel simulator is used to model and simulate traffic scenarios regarding the detection of the range, velocity, and SNR of targets. To validate WinProp as a simulation tool for traffic scenarios, a real traffic scenario is configured and the simulation results compared to the measurement results in order to demonstrate the suitability of WinProp for simulating traffic scenarios and evaluating radar detection performance. The close agreement between the simulation and measurement results provides validation for WinProp’s accuracy in modeling detections and signal processing within traffic scenarios. To showcase the limitations of FMCW radars, two scenarios with challenging detection conditions are simulated: an occluded pedestrian between parked vehicles and a densely populated road. A monostatic setup with a single patch antenna and a 16 × 16 patch antenna array is evaluated, using chirp bandwidths of 1 GHz and 4 GHz. The simulations have shown that although it is feasible to detect an occluded pedestrian by multipath propagation, there is still a need in improving the detection performance when there is no multipath from an occluded pedestrian. Furthermore, the results indicate that higher bandwidth improves target separation but is insufficient for occluded pedestrian detection without multipath. However, with some targets, even the range resolution at 4 GHz was not sufficient, which required a separation in the Doppler dimension, emphasizing the need for overall resolution improvement of FMCW radars. Future work should focus on developing more complex road user models and simulating a broader range of scenarios to comprehensively evaluate radar performance in road traffic environments.

Key points and visible part fusion attention network for occluded pedestrian detection in traffic environments

Key points and visible part fusion attention network for occluded pedestrian detection in traffic environments

Reconciling global and local optimal label assignments for heavily occluded pedestrian detection

Reconciling global and local optimal label assignments for heavily occluded pedestrian detection

Improved YOLOv3 Integrating SENet and Optimized GIoU Loss for Occluded Pedestrian Detection.

Occluded pedestrian detection faces huge challenges. False positives and false negatives in crowd occlusion scenes will reduce the accuracy of occluded pedestrian detection. To overcome this problem, we proposed an improved you-only-look-once version 3 (YOLOv3) based on squeeze-and-excitation networks (SENet) and optimized generalized intersection over union (GIoU) loss for occluded pedestrian detection, namely YOLOv3-Occlusion (YOLOv3-Occ). The proposed network model considered incorporating squeeze-and-excitation networks (SENet) into YOLOv3, which assigned greater weights to the features of unobstructed parts of pedestrians to solve the problem of feature extraction against unsheltered parts. For the loss function, a new generalized intersection over unionintersection over groundtruth (GIoUIoG) loss was developed to ensure the areas of predicted frames of pedestrian invariant based on the GIoU loss, which tackled the problem of inaccurate positioning of pedestrians. The proposed method, YOLOv3-Occ, was validated on the CityPersons and COCO2014 datasets. Experimental results show the proposed method could obtain 1.2% MR-2 gains on the CityPersons dataset and 0.7% mAP@50 improvements on the COCO2014 dataset.

Dual-mimic learning for occluded pedestrian detection

It is still a great challenge to detect pedestrians in crowded scenes. Recent works mainly focus on enhancing the feature representation of the visible region. Although this strategy is effective, it fails to fully exploit the appearance information contained in the full-body region. To this end, we propose a Dual-Mimic learning model, which consists of the occluded to unoccluded (Occ-Unocc) and the full-body to visible (Full-Vis) mimic learning branches. Both branches can reduce the intra-class feature variances of pedestrians by different mimicking strategies. Besides, we define an occlusion attribute to describe the occlusion pattern of each detection. Based on the occlusion attribute, we propose the Occlusion-aware NMS (Occ-NMS), which can effectively remain the occluded instance with low confidence. We conduct extensive experiments on standard benchmarks of CityPersons and Caltech, and our method achieves a new state-of-the-art performance at 36.34% MR−2 and 25.07% MR−2 on heavy occlusion sets.

Replacing the human driver: An objective benchmark for occluded pedestrian detection

Early detection of vulnerable road users is a crucial requirement for autonomous vehicles to meet and exceed the object detection capabilities of human drivers. One of the most complex outstanding challenges is that of partial occlusion where a target object is only partially available to the sensor due to obstruction by another foreground object. A number of leading pedestrian detection benchmarks provide annotation for partial occlusion, however each benchmark varies greatly in their definition of the occurrence and severity of occlusion. Research demonstrates that a high degree of subjectivity is used to classify occlusion level in these cases and occlusion is typically categorized into 2–3 broad categories such as “partially” and “heavily” occluded. In addition, many pedestrian instances are impacted by multiple inhibiting factors which contribute to non-detection such as object scale, distance from camera, lighting variations and adverse weather. This can lead to inaccurate or inconsistent reporting of detection performance for partially occluded pedestrians depending on which benchmark is used. This research introduces a novel, objective benchmark for partially occluded pedestrian detection to facilitate the objective characterization of pedestrian detection models. Characterization is carried out on seven popular pedestrian detection models for a range of occlusion levels from 0%–99% to demonstrate the impact of progressive levels of partial occlusion on pedestrian detectability. Results show that the proposed benchmark provides more objective, fine grained analysis of pedestrian detection algorithms than the current state of the art.

Butterfly network: a convolutional neural network with a new architecture for multi-scale semantic segmentation of pedestrians.

The detection of multi-scale pedestrians is one of the challenging tasks in pedestrian detection applications. Moreover, the task of small-scale pedestrian detection, i.e., accurate localization of pedestrians as low-scale target objects, can help solve the issue of occluded pedestrian detection as well. In this paper, we present a fully convolutional neural network with a new architecture and an innovative, fully detailed supervision for semantic segmentation of pedestrians. The proposed network has been named butterfly network (BF-Net) because of its architecture analogous to a butterfly. The proposed BF-Net preserves the ability of simplicity so that it can process static images with a real-time image processing rate. The sub-path blocks embedded in the architecture of the proposed BF-Net provides a higher accuracy for detecting multi-scale objective targets including the small ones. The other advantage of the proposed architecture is replacing common batch normalization with conditional one. In conclusion, the experimental results of the proposed method demonstrate that the proposed network outperform the other state-of-the-art networks such as U-Net + + , U-Net3 + , Mask-RCNN, and Deeplabv3 + for the semantic segmentation of the pedestrians.

Cascade Transformer Decoder Based Occluded Pedestrian Detection With Dynamic Deformable Convolution and Gaussian Projection Channel Attention Mechanism

Occluded pedestrian detection is very challenging in computer vision, because the pedestrians are frequently occluded by various obstacles or persons, especially in crowded scenarios. In this paper, an occluded pedestrian detection method is proposed under a basic DEtection TRansformer (DETR) framework. Firstly, Dynamic Deformable Convolution (DyDC) and Gaussian Projection Channel Attention (GPCA) mechanism are proposed and embedded into the low layer and high layer of ResNet50 respectively, to improve the representation capability of features. Secondly, Cascade Transformer Decoder (CTD) is proposed, which aims to generate high-score queries, avoiding the influence of low-score queries in the decoder stage, further improving the detection accuracy. The proposed method is verified on three challenging datasets, namely CrowdHuman, WiderPerson, and TJU-DHD-pedestrian. The experimental results show that, compared with the state-of-the-art methods, it can obtain a superior detection performance.

Sequential Attention-Based Distinct Part Modeling for Balanced Pedestrian Detection

Despite pedestrian detectors having made significant progress by introducing convolutional neural networks, their performance still suffers degradation, especially in occlusion scenes with more false positives (FPs) and false negatives (FNs). To alleviate the problem, we propose a novel Sequential Attention-based Distinct Part Modeling (SA-DPM) for balanced pedestrian detection. It takes one step further in constructing more robust representation that supports detection with fewer FNs and FPs. Specifically, the Sequential Attention serves as one internal perception process that captures several distinct part areas step by step from each pedestrian proposal (full-body). Different from the previous either-or feature selection, the following Joint Learning attempts to seek a reasonable trade-off between part and full-body features, and combines both features for more accurate classification and regression. Evaluation on the widely used pedestrian datasets including Caltech and Citypersons shows that the proposed SA-DPM achieves promising performance for both non-occluded and occluded pedestrian detection tasks, especially on Caltech Heavy Occlusion set, which yields a new state-of-the-art miss rate by 30.18% and outperforms the second best detector by 6.32%.

PRNet++: Learning towards generalized occluded pedestrian detection via progressive refinement network

Pedestrian detection has achieved significant progress in recent years. Though promising results have been obtained on standard pedestrians, it remains challenging to detect pedestrians in various occlusion situations. In this paper, we propose Progressive Refinement Network (PRNet), a novel single-stage detector for occluded pedestrian detection. Inspired by human’s progressive process on annotating occluded pedestrians, PRNet perform sequential refinement in a single-stage detection framework by three phases: finding anchors of visible parts with high confidence, calibrating these anchors with a full body template derived from occlusion statistics, and adjusting the calibrated anchors to target full-body regions. Unlike conventional methods that utilize predefined anchors directly for full-body estimation, the proposed confidence-aware anchor calibration offers an adaptive anchor initialization for detection with occlusions, while helps reduce the gap between visible-part and full-body detection. We also propose an occlusion loss, which automatically up-weights heavily occluded pedestrian samples. In addition, a Receptive Field Backfeed (RFB) module is introduced to diversify receptive fields in early layers that commonly fire only on visible parts or small-size full-body regions. To further learn generalized representations of pedestrians in different occlusion states, we propose to establish a new single-stage detector with dual-stream architecture namely PRNet++. An Easy-branch and a Hard-branch are designed to learn complementary representation that are more robust to various occlusions. Moreover, the generalization ability of models for other domains are critical in real-world applications, but it has not attracted too much attention. To address this issue, we introduce the unsupervised domain adaptation setting. To handle occlusion situations in unknown domain better, we especially design a Dynamic Iterative adaptation strategy and a Multi-experts adaptation strategy for our occlusion-aware detectors PRNet and PRNet++. Extensive supervised within-dataset experiments and unsupervised domain adaptation experiments were performed on CityPersons, Caltech, ECP-night, KITTI and INRIA datasets, which can validate the effectiveness of our proposed methods in occluded pedestrian detection.

Semantic enhanced guide feature reconstruction for occluded pedestrian detection

Semantic enhanced guide feature reconstruction for occluded pedestrian detection

Mask-Guided Attention Network and Occlusion-Sensitive Hard Example Mining for Occluded Pedestrian Detection.

Pedestrian detection relying on deep convolution neural networks has made significant progress. Though promising results have been achieved on standard pedestrians, the performance on heavily occluded pedestrians remains far from satisfactory. The main culprits are intra-class occlusions involving other pedestrians and inter-class occlusions caused by other objects, such as cars and bicycles. These result in a multitude of occlusion patterns. We propose an approach for occluded pedestrian detection with the following contributions. First, we introduce a novel mask-guided attention network that fits naturally into popular pedestrian detection pipelines. Our attention network emphasizes on visible pedestrian regions while suppressing the occluded ones by modulating full body features. Second, we propose the occlusion-sensitive hard example mining method and occlusion-sensitive loss that mines hard samples according to the occlusion level and assigns higher weights to the detection errors occurring at highly occluded pedestrians. Third, we empirically demonstrate that weak box-based segmentation annotations provide reasonable approximation to their dense pixel-wise counterparts. Experiments are performed on CityPersons, Caltech and ETH datasets. Our approach sets a new state-of-the-art on all three datasets. Our approach obtains an absolute gain of 10.3% in log-average miss rate, compared with the best reported results on the heavily occluded HO pedestrian set of the CityPersons test set. Code and models are available at: https://github.com/Leotju/MGAN.

Context attention network for occluded pedestrian detection

Pedestrian detection in occluded scenes has always been a thorny problem in computer vision. In this case, due to the large difference in scale of occluded pedestrians and low visibility, it usually brings great challenges to detection. To solve this problem, this paper proposes a model structure for pedestrian occlusion detection, which improves the pedestrian detection method based on anchor-free. Specifically, we introduce a structure for extracting multi-scale context information to learn a better feature representation, and a channel attention module on each decoder layer to provide global context as a guidance of low-level features to select category localization details. Experimental results show that this method achieves 41.93% of MR-2 on the occlusion subset of Caltech pedestrian dataset, which is better than other contrast detectors.

PSC-Net: learning part spatial co-occurrence for occluded pedestrian detection

Detecting pedestrians, especially under heavy occlusion, is a challenging computer vision problem with numerous real-world applications. This paper introduces a novel approach, termed as PSC-Net, for occluded pedestrian detection. The proposed PSC-Net contains a dedicated module that is designed to explicitly capture both inter and intra-part co-occurrence information of different pedestrian body parts through a graph convolutional network (GCN). Both inter and intra-part co-occurrence information contribute towards improving the feature representation for handling varying level of occlusions, ranging from partial to severe occlusions. Our PSC-Net exploits the topological structure of pedestrian and does not require part-based annotations or additional visible bounding-box (VBB) information to learn part spatial co-occurrence. Comprehensive experiments are performed on three challenging datasets: CityPersons, Caltech, and CrowdHuman datasets. Particularly, in terms of log-average miss rates and with the same backbone and input scale as those of the state-of-the-art MGAN, the proposed PSC-Net achieves absolute gains of 4.0% and 3.4% over MGAN on the heavy occlusion subsets of CityPersons and Caltech test sets, respectively.

PedHunter: Occlusion Robust Pedestrian Detector in Crowded Scenes

Pedestrian detection in crowded scenes is a challenging problem, because occlusion happens frequently among different pedestrians. In this paper, we propose an effective and efficient detection network to hunt pedestrians in crowd scenes. The proposed method, namely PedHunter, introduces strong occlusion handling ability to existing region-based detection networks without bringing extra computations in the inference stage. Specifically, we design a mask-guided module to leverage the head information to enhance the feature representation learning of the backbone network. Moreover, we develop a strict classification criterion by improving the quality of positive samples during training to eliminate common false positives of pedestrian detection in crowded scenes. Besides, we present an occlusion-simulated data augmentation to enrich the pattern and quantity of occlusion samples to improve the occlusion robustness. As a consequent, we achieve state-of-the-art results on three pedestrian detection datasets including CityPersons, Caltech-USA and CrowdHuman. To facilitate further studies on the occluded pedestrian detection in surveillance scenes, we release a new pedestrian dataset, called SUR-PED, with a total of over 162k high-quality manually labeled instances in 10k images. The proposed dataset, source codes and trained models are available at https://github.com/ChiCheng123/PedHunter.

Attention guided neural network models for occluded pedestrian detection

Occluded pedestrian detection has been a research difficulty in computer vision for a long time. The conventional approach to solve this problem is to learn partial detectors, which can be properly integrated for occlusion handling. However, the efficiency of this type of methods is limited in practical applications since the partial detectors can not cover all occlusion patterns. In this paper, an attention guided neural network model (AGNN) is proposed for the occlusion handling of pedestrian detections, which is inspired by the approaches of sentiment classification. Firstly, a fixed-size window slides on a still image without overlapping to generate a set of sub-images. Secondly, a convolutional neural network is employed to extract the high-level features from resulting sub-images. Then, the attention network performs local feature weighting, from which the features representing the body parts of pedestrians are selected. Finally, the feature sequences are classified by recurrent neural network in proper order based on the weighted results. In addition, we explore different mechanisms of attention guidance on the detector for the detections. Compared with the state-of-the-art methods on two standard pedestrian datasets, experimental results demonstrate the comparable performance of our approach in terms of miss rates.

Occluded Pedestrian Detection Based on Depth Vision Significance in Biomimetic Binocular

Pedestrian detection and tracking has become an important field in the field of computer vision research. However, the existing pedestrian detection algorithms have some problems, such as low accuracy and poor stability due to the similar background and overlapped occlusion interference. Therefore, an occluded pedestrian detection method based on binocular vision is proposed in this paper. We simulate the recognition of human brain and use the deep learning network MobileNet to detect and locate the initial pedestrians. Then, binocular depth is introduced as visual salience prior information, which solves the problem of identifying pedestrians with similar background and occlusion. The experimental results show that our pedestrian detection framework greatly improves the pedestrian error detection under similar background and occlusion conditions.

Multi-label learning of part detectors for occluded pedestrian detection

Despite recent progress of pedestrian detection, it remains a challenging problem to detect pedestrians that are partially occluded due to the uncertainty and diversity of partial occlusion patterns. Following a commonly used framework of handling partial occlusions by part detection, we propose a multi-label learning approach to jointly learn part detectors to capture partial occlusion patterns. The part detectors share a set of decision trees which are learned and combined via boosting to exploit part correlations and also reduce the computational cost of applying these part detectors for pedestrian detection. The learned decision trees capture the overall distribution of all the parts. When used as a pedestrian detector individually, our part detectors learned jointly show better performance than their counterparts learned separately in different occlusion situations. For occlusion handling, several methods are explored to integrate the part detectors learned by the proposed approach. Context is also exploited to further improve the performance. The proposed approach is applied to hand-crafted channel features and features learned by a deep convolutional neural network, respectively. Experiments on the Caltech and CUHK datasets show state-of-the-art performance of our approach for detecting occluded pedestrians, especially heavily occluded ones.

Detection scheme for a partially occluded pedestrian based on occluded depth in lidar–radar sensor fusion

Object detections are critical technologies for the safety of pedestrians and drivers in autonomous vehicles. Above all, occluded pedestrian detection is still a challenging topic. We propose a new detection scheme for occluded pedestrian detection by means of lidar–radar sensor fusion. In the proposed method, the lidar and radar regions of interest (RoIs) have been selected based on the respective sensor measurement. Occluded depth is a new means to determine whether an occluded target exists or not. The occluded depth is a region projected out by expanding the longitudinal distance with maintaining the angle formed by the outermost two end points of the lidar RoI. The occlusion RoI is the overlapped region made by superimposing the radar RoI and the occluded depth. The object within the occlusion RoI is detected by the radar measurement information and the occluded object is estimated as a pedestrian based on human Doppler distribution. Additionally, various experiments are performed in detecting a partially occluded pedestrian in outdoor as well as indoor environments. According to experimental results, the proposed sensor fusion scheme has much better detection performance compared to the case without our proposed method.