Vision-based Pedestrian Detection Research Articles

Illumination and Temperature-Aware Multispectral Networks for Edge-Computing-Enabled Pedestrian Detection

Accurate and efficient pedestrian detection is crucial for the intelligent transportation system regarding pedestrian safety and mobility, e.g., Advanced Driver Assistance Systems, and smart pedestrian crosswalk systems. Among all pedestrian detection methods, the vision-based detection method is demonstrated to be the most effective in previous studies. However, the existing vision-based pedestrian detection algorithms still have two limitations that restrict their implementations, those being real-time performance as well as the resistance to the impacts of environmental factors, e.g., low illumination conditions. To address these issues, this study proposes a lightweight Illumination and Temperature-aware Multispectral Network (IT-MN) for accurate and efficient pedestrian detection. The proposed IT-MN is an efficient one-stage detector. For accommodating the impacts of environmental factors and enhancing the sensing accuracy, thermal image data is fused by the proposed IT-MN with visual images to enrich useful information when visual image quality is limited. In addition, an innovative and effective late fusion strategy is also developed to optimize the image fusion performance. To make the proposed model implementable for edge computing, the model quantization is applied to reduce the model size by 75% while shortening the inference time significantly. The proposed algorithm is evaluated by comparing it with the selected state-of-the-art algorithms using a public dataset collected by in-vehicle cameras. The results show that the proposed algorithm achieves a low miss rate and inference time at 14.19% and 0.03 seconds per image pair on GPU. Besides, the quantized IT-MN achieves an inference time of 0.21 seconds per image pair on the edge device, demonstrating the potentiality of deploying the proposed model on edge devices as a highly efficient pedestrian detection algorithm.

Robust pedestrian detection for driver assistance systems using machine learning

Vision-based pedestrian detection is a challenge task for a variety of applications such as driving assistance systems, especially in case of insufficient illumination. Effective fusion of complementary information acquired by multispectral images (visible and infrared) allows robust pedestrian detection under various lighting conditions (e.g., day and nighttime). In this paper, we propose a multispectral pedestrian detection approach that combines visible and infrared images. Firstly, an Otsu thresholding is applied to infrared images to detect hot spots most likely representing a pedestrian, after applying some morphological operations to enhance the original image and compensate for clothing-based distortions. The significant regions of interest obtained in the infrared image are mapped into corresponding visible image. Secondly, multispectral aggregated channel features are used with a thermal discrete cosine transform, as descriptor combined with a support vector machine (SVM) classifier. Our approach is evaluated on the KAIST multispectral dataset to prove its efficiency.

Coarse-to-fine Method for Vision-based Pedestrian Traffic Light Detection

Pedestrian traffic light detection is an important technique of the navigation system for the visually impaired during road crossing. In this paper, a three-stage coarse-to-fine method ...

Multispectral pedestrian detection based on deep convolutional neural networks

Vision-based pedestrian detection that can last all day is crucial in advanced driver-assistance systems (ADAS), autonomous vehicles and video surveillance. Based on the fact that the wavelength of human body radiation falls around 9.3 μm, thermal images have distinctive advantages for pedestrian detection in the nighttime. With the recent success of convolutional neural networks (CNNs) in the vision community, how to properly explore information in color and thermal images using CNNs-based methods attracts the attention of researchers. Previous CNN-based multispectral pedestrian detectors focus on the design of network architectures. The main contributions of this paper are as follows. First, pixel-level image fusion are also extensively evaluated based on a state-of-the-art CNNs-based framework in the daytime and nighttime for pedestrian detection. Second, the effective strategies to combine pixel-level fusion methods and CNN-fusion architectures are studied based on extensive experimental results. Two combination strategies are designed and an exhaustive experimental analysis is performed to evaluate different combinations for all-day pedestrian detection. Extensive results based on a multispectral pedestrian benchmark show that some pixel-level image fusion methods can achieve similar or even better performance than CNN-fusion architectures, which emphasizes the importance of pixel-level fusion in CNN-based pedestrian detectors. The combination of both can usually more properly exploit multispectral information and further boost detection performance.

Computer vision and deep learning techniques for pedestrian detection and tracking: A survey

Abstract Pedestrian detection and tracking have become an important field in the computer vision research area. This growing interest, started in the last decades, might be explained by the multitude of potential applications that could use the results of this research field, e.g. robotics, entertainment, surveillance, care for the elderly and disabled, and content-based indexing. In this survey paper, vision-based pedestrian detection systems are analysed based on their field of application, acquisition technology, computer vision techniques and classification strategies. Three main application fields have been individuated and discussed: video surveillance, human-machine interaction and analysis. Due to the large variety of acquisition technologies, this paper discusses both the differences between 2D and 3D vision systems, and indoor and outdoor systems. The authors reserved a dedicated section for the analysis of the Deep Learning methodologies, including the Convolutional Neural Networks in pedestrian detection and tracking, considering their recent exploding adoption for such a kind systems. Finally, focusing on the classification point of view, different Machine Learning techniques have been analysed, basing the discussion on the classification performances on different benchmark datasets. The reported results highlight the importance of testing pedestrian detection systems on different datasets to evaluate the robustness of the computed groups of features used as input to classifiers.

Robust Pedestrian Classification Based on Hierarchical Kernel Sparse Representation.

Vision-based pedestrian detection has become an active topic in computer vision and autonomous vehicles. It aims at detecting pedestrians appearing ahead of the vehicle using a camera so that autonomous vehicles can assess the danger and take action. Due to varied illumination and appearance, complex background and occlusion pedestrian detection in outdoor environments is a difficult problem. In this paper, we propose a novel hierarchical feature extraction and weighted kernel sparse representation model for pedestrian classification. Initially, hierarchical feature extraction based on a CENTRIST descriptor is used to capture discriminative structures. A max pooling operation is used to enhance the invariance of varying appearance. Then, a kernel sparse representation model is proposed to fully exploit the discrimination information embedded in the hierarchical local features, and a Gaussian weight function as the measure to effectively handle the occlusion in pedestrian images. Extensive experiments are conducted on benchmark databases, including INRIA, Daimler, an artificially generated dataset and a real occluded dataset, demonstrating the more robust performance of the proposed method compared to state-of-the-art pedestrian classification methods.

A Pedestrian-Detection Method Based on Heterogeneous Features and Ensemble of Multi-View–Pose Parts

Vision-based pedestrian detection remains a challenging task, so far. The detection performance often suffers from the various appearances of pedestrians, the illumination changes, and the possible partial occlusions. Aiming at resolving these challenges, in this paper, a new linear kernel function is proposed to effectively combine two heterogeneous features, i.e., histogram of oriented gradient and local binary pattern, which enhances the pedestrian description ability to illumination conditions and cluttered background. Then, a novel multi-view–pose part ensemble (MVPPE) detector is proposed, in order to better handle pedestrian variability, views, and partial occlusions. Experimental results in public data sets demonstrate that the proposed feature combination method significantly improves the description capabilities of pedestrian features. Compared with the existing multipart ensemble approaches, the proposed MVPPE detector boosts higher detection accuracy.

Combining Multiple Cues for Pedestrian Detection in Crowded Situations

This paper proposes a vision-based pedestrian detection in crowded situations based on a single camera. The main idea behind our work is to fuse multiple cues so that the major challenges, such as occlusion and complex background facing in the topic of crowd detection can be successfully overcome. Based on the assumption that human heads are visible, circle Hough transform (CHT) is applied to detect all circular regions and each of which is considered as the head candidate of a pedestrian. After that, the false candidates resulting from complex background are firstly removed by using template matching algorithm. Two proposed cues called head foreground contrast (HFC) and block color relation (BCR) are incorporated for further verification. The rectangular region of every detected human is determined by the geometric relationships as well as foreground mask extracted through background subtraction process. Three videos are used to validate the proposed approach and the experimental results show that the proposed method effectively lowers the false positives at the expense of little detection rate.

Stereo regions-of-interest selection for pedestrian protection: A survey

Vision-based pedestrian detection for intelligent vehicles applications is a crucial and active research area due to the essential benefits in terms of reducing the number of accidents involving pedestrians and vehicles. During the last decade a considerable amount of research studies have been proposed, filling the gap between prototypes and commercial implementations. Pedestrian detection systems can be roughly divided into three main different sub-parts: Region Of Interest – ROI – selection, classification and tracking. Previous surveys have covered the literature in a holistic way. An example would be, analyzing all the solutions proposed for all the stages and including higher level analysis, but in most cases they give more emphasis to the classification stage. Due to the difficulty of this detection task, the variety of solutions, sensor configurations (monocular/stereo; visible/infrared) available in the literature, we propose to break down the variability of the problem by providing exhaustive review of one specific stage: stereo-based ROI selection. ROI selection is a key component that has to be designed to provide generic obstacles at lowest false negative rate and maintain a low number of false positives. The number of missed pedestrians has to be approximately equal to 0 since a pedestrian missed by the ROI selection stage would not be detected in further stages. In addition, the number of non-pedestrians obstacles should be as low as possible to reduce both the number of false alarms and the computational costs of further stages. In contrast to monocular approaches, stereo ROI selection determines the relative distance between the pedestrian and the vehicle, assuring that the reported candidates are related with real physical objects. The stereo-based ROI selection step is also divided into different components that are independently analyzed, increasing visibility for future proposals and developments. Discussion is finally presented highlighting the current problems for obtaining a global overview of the actual performance of the different approaches and analyzing future trends.

Error Analysis in a Stereo Vision-Based Pedestrian Detection Sensor for Collision Avoidance Applications

This paper presents an analytical study of the depth estimation error of a stereo vision-based pedestrian detection sensor for automotive applications such as pedestrian collision avoidance and/or mitigation. The sensor comprises two synchronized and calibrated low-cost cameras. Pedestrians are detected by combining a 3D clustering method with Support Vector Machine-based (SVM) classification. The influence of the sensor parameters in the stereo quantization errors is analyzed in detail providing a point of reference for choosing the sensor setup according to the application requirements. The sensor is then validated in real experiments. Collision avoidance maneuvers by steering are carried out by manual driving. A real time kinematic differential global positioning system (RTK-DGPS) is used to provide ground truth data corresponding to both the pedestrian and the host vehicle locations. The performed field test provided encouraging results and proved the validity of the proposed sensor for being used in the automotive sector towards applications such as autonomous pedestrian collision avoidance.

LIDAR and vision-based pedestrian detection system

A perception system for pedestrian detection in urban scenarios using information from a LIDAR and a single camera is presented. Two sensor fusion architectures are described, a centralized and a d...

Combination of Feature Extraction Methods for SVM Pedestrian Detection

This paper describes a comprehensive combination of feature extraction methods for vision-based pedestrian detection in Intelligent Transportation Systems. The basic components of pedestrians are first located in the image and then combined with a support-vector-machine-based classifier. This poses the problem of pedestrian detection in real cluttered road images. Candidate pedestrians are located using a subtractive clustering attention mechanism based on stereo vision. A components-based learning approach is proposed in order to better deal with pedestrian variability, illumination conditions, partial occlusions, and rotations. Extensive comparisons have been carried out using different feature extraction methods as a key to image understanding in real traffic conditions. A database containing thousands of pedestrian samples extracted from real traffic images has been created for learning purposes at either daytime or nighttime. The results achieved to date show interesting conclusions that suggest a combination of feature extraction methods as an essential clue for enhanced detection performance