Neural Networks For Pedestrian Detection Research Articles

Multispectral interaction convolutional neural network for pedestrian detection

Fusion of multispectral data in object detection is inevitable in order to cover various environments. However, there is still insufficient research on how to fuse information between two-stream multispectral networks. This paper proposes a novel multispectral interaction convolutional neural network (MICNN) for fusing information between multispectral networks. Unlike existing fusion methods that do not have interactions between multispectral networks, the proposed MICNN reflects information from each multispectral network in the training process. The MICNN is a simple way of forcing interactions by exchanging weights of feature maps between multispectral networks. It does not require any parameters to learn, and does not need additional computations or modifications to the network structure. We verified the effectiveness of the MICNN with the KAIST multispectral pedestrian dataset and YU far-infrared (FIR) pedestrian dataset.

Weak segmentation supervised deep neural networks for pedestrian detection

Semantic segmentation has been used successfully as a complementary information source in pedestrian detection. However, it requires accurate pixel-level semantic segmentation annotations for training, but it is extremely time-consuming to obtain these. In this work, we solve this problem by using weak segmentation masks automatically generated by depth images. This enables joint semantic segmentation and pedestrian detection with only ground truth bounding boxes for training. We show that this joint training boosts the performance of the pedestrian detector. Moreover, we show that fusing the outputs of the classification network and the generated segmentation masks leads to a further detection performance improvement. Extensive experiments have been conducted on three RGBD pedestrian datasets to demonstrate the effectiveness of our proposed method. As a byproduct, we also obtain pedestrian segmentation results of good quality, without using pixel-level segmentation annotations during training.

A Parallel Convolutional Neural Network for Pedestrian Detection

Pedestrian detection is a crucial task in many vision-based applications, such as video surveillance, human activity analysis and autonomous driving. Recently, most of the existing pedestrian detection frameworks only focus on the detection accuracy or model parameters. However, how to balance the detection accuracy and model parameters, is still an open problem for the practical application of pedestrian detection. In this paper, we propose a parallel, lightweight framework for pedestrian detection, named ParallelNet. ParallelNet consists of four branches, each of them learns different high-level semantic features. We fused them into one feature map as the final feature representation. Subsequently, the Fire module, which includes Squeeze and Expand parts, is employed for reducing the model parameters. Here, we replace some convolution modules in the backbone with Fire modules. Finally, the focal loss is led into the ParallelNet for end-to-end training. Experimental results on the Caltech–Zhang dataset and KITTI dataset show that: Compared with the single-branch network, such as ResNet and SqueezeNet, ParallelNet has improved detection accuracy with fewer model parameters and lower Giga Floating Point Operations (GFLOPs).

Pedestrian detection using Doppler radar and LSTM neural network

<span lang="EN-US">Integration of radar systems as primary sensor with deep learning algorithms in driver assist systems is still limited. Its implementation would greatly help in continuous monitoring of visual blind spots from incoming pedestrians. Hence, this study proposes a single-input single-output based Doppler radar and long short-term memory (LSTM) neural network for pedestrian detection. The radar is placed in monostatic configuration at an angle of 45 degree from line of sight. Continuous wave with frequency of 1.9 GHz are continuously transmitted from the antenna. The returning signal from the approaching subjects is characterized by the branching peaks higher than the transmitted frequency. A total of 1108 spectrum traces with Doppler shifts characteristics is acquired from eight volunteers. Another 1108 spectrum traces without Doppler shifts are used for control purposes. The traces are then fed to LSTM neural network for training, validation and testing. Generally, the proposed method was able to detect pedestrian with 88.9% accuracy for training and 87.3% accuracy for testing.</span>

Multi‐receptive field graph convolutional neural networks for pedestrian detection

Although great progress has been achieved for general object detection with the advent of deep learning, there are still several limitations when applied to pedestrian detection task. In this article, the authors mainly consider the following three problems. (i) most deep learning-based methods are time-consuming and require large storage space, making them unsuitable for real-world applications; (ii) the scale of person varies greatly, which decreases the pedestrian detection performance; (iii) as pedestrians may be occluded by vehicles or other persons, it makes pedestrian detection more challenging. In response to these limitations, this article proposes a multi-receptive field-based framework for single-shot pedestrian detection. Authors' contributions can be summarised as follows. First, to reduce the computational complexity of the detector, the authors design the multi-receptive pooling pyramid module, which reduces the computation cost of the detector and improves its performance. Next, body parts graphs are built on the learned convolutional neural networks (CNN) features, and a graph CNN is used to mine the relationships of the graph nodes. Finally, experiments are conducted on two public pedestrian detection datasets to demonstrate the effectiveness of the proposed method.

A Training Method of Convolution Neural Network for Illumination Robust Pedestrian Detection

In this article, the authors propose a new training method of convolution neural networks for pedestrian detection under the illumination of robust environments of ADAS (Advanced Driver Assistance System). This training method was aimed at proposing a method to increase the recognition rate in a system that classifies objects by receiving distorted images in real time as the ADAS, by using the Convolution Neural Network (CNN). The proposed method used images with an increased distortion level by setting gamma to 0.7, and the conventional method was experimented with images with a gamma set to 1. In this article, the authors experiment with the comparison of the conventional training method using the preprocessing accelerator and the proposed training method using the gamma variation. In this study, pedestrian images with a distorted illumination intensity were used in training and then the accuracy of pedestrian classification was tested with normal images and distorted images as test images. The proposed method shows an error rate of 9.8%, which was improved by 1.2% in accuracy.

Part-Level Convolutional Neural Networks for Pedestrian Detection Using Saliency and Boundary Box Alignment

Pedestrians in videos have a wide range of appearance factors, such as body poses, occlusions, and complex backgrounds, which make their detection difficult. Moreover, a proposal shift problem causes the loss of body parts, such as head and legs in pedestrian detection, which further degrades the detection accuracy. In this paper, we propose part-level convolutional neural networks (CNNs) for pedestrian detection using saliency and boundary box (BB) alignment. The proposed network consists of two subnetworks: detection and alignment. In the detection subnetwork, we use saliency to remove false positives, such as lamp posts and trees, by combining a fully convolutional network and a class activation map to extract deep features. Subsequently, we adopt the BB alignment on detection proposals in the alignment subnetwork to overcome the proposal shift problem by applying the part-level CNN to recall the lost body parts. The experimental results on various datasets demonstrate that the proposed method remarkably improves accuracy in pedestrian detection and outperforms the existing state-of-the-art techniques.

Fusion of multispectral data through illumination-aware deep neural networks for pedestrian detection

Multispectral pedestrian detection has received extensive attention in recent years as a promising solution to facilitate robust human target detection for around-the-clock applications (e.g., security surveillance and autonomous driving). In this paper, we demonstrate illumination information encoded in multispectral images can be utilized to boost the performance of pedestrian detection significantly. A novel illumination-aware weighting mechanism is present to depict illumination condition of a scene accurately. Such illumination information is incorporated into two-stream deep convolutional neural networks to learn multispectral human-related features under different illumination conditions (daytime and nighttime). Moreover, we utilized illumination information together with multispectral data to generate more accurate semantic segmentation which is used to supervise the training of pedestrian detector. Putting all of the pieces together, we present an effective framework for multispectral pedestrian detection based on multi-task learning of illumination-aware pedestrian detection and semantic segmentation. Our proposed method is trained end-to-end using a well-designed multi-task loss function and outperforms state-of-the-art approaches on KAIST multispectral pedestrian dataset.

A complete person re-identification model using Kernel-PCA-based Gabor-filtered hybrid descriptors

Person re-identification is a challenging problem in computer vision. Lots of research interest is observed in this area over the past few years. A model for complete person re-identification can prove useful in this direction. Use of convolutional neural networks for pedestrian detection can improve the accuracy of detection to a larger extent. Deriving a descriptor which is invariant to the changes in the illumination, background and the pose can make the difference in the re-identification process. The predominant part of our work focuses on building a robust descriptor which can tackle such challenges. We have concentrated on building a descriptor by employing appearance-based features extracted both at local and global levels. Further, the dimensionality of the descriptor is reduced using kernel PCA. Distance metric learning algorithms are used to evaluate the descriptor on three major benchmark datasets. We propose a complete person re-identification system which involves both pedestrian detection and person re-identification. Major contributions of this work are to detect pedestrians from surveillance videos using CNN-based learning and to generate a kernel-PCA-based spatial descriptor and evaluate the descriptor using known distance metric learning methods on benchmark datasets.

Deep Convolutional Neural Network for Pedestrian Detection with Multi-Levels Features Fusion

Pedestrian detection aims to localize and recognize every pedestrian instance in an image with a bounding box. The current state-of-the-art method is Faster RCNN, which is such a network that uses a region proposal network (RPN) to generate high quality region proposals, while Fast RCNN is used to classifiers extract features into corresponding categories. The contribution of this paper is integrated low-level features and high-level features into a Faster RCNN-based pedestrian detection framework, which efficiently increase the capacity of the feature. Through our experiments, we comprehensively evaluate our framework, on the Caltech pedestrian detection benchmark and our methods achieve state-of-the-art accuracy and present a competitive result on Caltech dataset.

Image-Based Multi-Target Tracking through Multi-Bernoulli Filtering with Interactive Likelihoods.

We develop an interactive likelihood (ILH) for sequential Monte Carlo (SMC) methods for image-based multiple target tracking applications. The purpose of the ILH is to improve tracking accuracy by reducing the need for data association. In addition, we integrate a recently developed deep neural network for pedestrian detection along with the ILH with a multi-Bernoulli filter. We evaluate the performance of the multi-Bernoulli filter with the ILH and the pedestrian detector in a number of publicly available datasets (2003 PETS INMOVE, Australian Rules Football League (AFL) and TUD-Stadtmitte) using standard, well-known multi-target tracking metrics (optimal sub-pattern assignment (OSPA) and classification of events, activities and relationships for multi-object trackers (CLEAR MOT)). In all datasets, the ILH term increases the tracking accuracy of the multi-Bernoulli filter.

Deep Convolutional Neural Networks for pedestrian detection

Pedestrian detection is a popular research topic due to its paramount importance for a number of applications, especially in the fields of automotive, surveillance and robotics. Despite the significant improvements, pedestrian detection is still an open challenge that calls for more and more accurate algorithms. In the last few years, deep learning and in particular Convolutional Neural Networks emerged as the state of the art in terms of accuracy for a number of computer vision tasks such as image classification, object detection and segmentation, often outperforming the previous gold standards by a large margin. In this paper, we propose a pedestrian detection system based on deep learning, adapting a general-purpose convolutional network to the task at hand. By thoroughly analyzing and optimizing each step of the detection pipeline we propose an architecture that outperforms traditional methods, achieving a task accuracy close to that of state-of-the-art approaches, while requiring a low computational time. Finally, we tested the system on an NVIDIA Jetson TK1, a 192-core platform that is envisioned to be a forerunner computational brain of future self-driving cars.