Pedestrian Detection Algorithm Research Articles

Pedestrian detection based on YOLOv3 multimodal data fusion

Multi-sensor fusion has essential applications in the field of target detection. Considering the current actual demand for miniaturization of on-board computers for driverless vehicles, this paper uses the multimodal data YOLOv3 (MDY) algorithm for pedestrian detection on embedded devices. The MDY algorithm uses YOLOv3 as the basic framework to improve pedestrian detection accuracy by optimizing anchor frames and adding small target detection branches. Then the algorithm is accelerated by using TensorRT technology to improve the real-time performance in embedded devices. Finally, a hybrid fusion framework is used to fuse the LIDAR point cloud data with the improved YOLOv3 algorithm to compensate for the shortcomings of a single sensor and improve the detection accuracy while ensuring speed. The improved YOLOv3 improves AP by 6.4% and speed by 11.3 FPS over the original algorithm. The MDY algorithm achieves better performance on the KITTI dataset. To further verify the feasibility of the MDY algorithm, an actual test was conducted on an unmanned vehicle with Jetson TX2 embedded device as the on-board computer within the campus scenario, and the results showed that the MDY algorithm achieves 90.8% accuracy under real-time operation and can achieve adequate detection accuracy and real-time performance on the embedded device.

Vehicle and Pedestrian Detection Algorithm Based on Lightweight YOLOv3-Promote and Semi-Precision Acceleration

Aiming at the shortcomings of the current YOLOv3 model, such as large size, slow response speed, and difficulty in deploying to real devices, this paper reconstructs the target detection model YOLOv3, and proposes a new lightweight target detection network YOLOv3-promote: Firstly, the G-Module combined with the Depth-Wise convolution is used to construct the backbone network of the entire model, and the attention mechanism is introduced and added to perform weighting operations on each channel to get more key features and remove redundant features, thereby strengthening the identification ability of feature network model’s to distinguish target objects among background; Secondly, in order to delete some less important channels to achieve the effect of compressing the model size and improving the calculation speed, the size of the scaling factor gamma in the batch normalization layer is used; Finally, based on NVIDIA’s TensorRT framework model conversion and half-precision acceleration were carried out, and the accelerated model was successfully deployed on the embedded platform Jetson Nano. The performed KITTI experimental results show that the inference speed of our proposed method is about 5 times that of the original model, the parameter volume is reduced to one tenth, the mAP is increased from 86.1% of the original model to 93.1%, and the FPS reaches 25.5fps, realizing the requirements of real-time detection with high precision.

Research on pedestrian detection based on multi-level fine-grained YOLOX algorithm

PurposeThe purpose of the study is to address the problems of low accuracy and missed detection of occluded pedestrians and small target pedestrians when using the YOLOX general object detection algorithm for pedestrian detection. This study proposes a multi-level fine-grained YOLOX pedestrian detection algorithm.Design/methodology/approachFirst, to address the problem of the original YOLOX algorithm in obtaining a single perceptual field for the feature map before feature fusion, this study improves the PAFPN structure by adding the ResCoT module to increase the diversity of the perceptual field of the feature map and divides the pedestrian multi-scale features into finer granularity. Second, for the CSPLayer of the PAFPN, a weight gain-based normalization-based attention module (NAM) is proposed to make the model pay more attention to the context information when extracting pedestrian features and highlight the salient features of pedestrians. Finally, the authors experimentally determined the optimal values for the confidence loss function.FindingsThe experimental results show that, compared with the original YOLOX algorithm, the AP of the improved algorithm increased by 2.90%, the Recall increased by 3.57%, and F1 increased by 2% on the pedestrian dataset.Research limitations/implicationsThe multi-level fine-grained YOLOX pedestrian detection algorithm can effectively improve the detection of occluded pedestrians and small target pedestrians.Originality/valueThe authors introduce a multi-level fine-grained ResCoT module and a weight gain-based NAM attention module.

Global Vibration Comfort Evaluation of Footbridges Based on Computer Vision.

The vibration comfort evaluation is a control standard other than strength and deflection, but the general comfort evaluation method only considers the response of the mid-span position and does not consider the difference in the vibration response of different positions at the same time. It is crucial to study how pedestrians actually feel when they walk on footbridges. The computer vision-based vibration comfort evaluation method is a novel method with advantages, such as noncontact and long-distance. In this study, a computer vision-based method was used to evaluate the global vibration comfort of footbridges under human-induced excitation. The improved Lucas–Kanade optical flow method is used for multitarget displacement identification of footbridges. Additionally, the YOLOv5 algorithm for pedestrian detection is used to obtain the position information of pedestrians on the footbridges. Then, according to the pedestrian position information, the structural responses of different pedestrian positions corresponding to time periods are extracted from the displacement responses of each point, and they are combined to obtain the structural global displacement. The global acceleration can be obtained by calculating the global displacement. The rms value can be calculated based on the global acceleration and compared with the standard for comfort evaluation. The global comfort evaluation method is validated by pedestrian walking experiments with different frequencies on a laboratory footbridge. The experimental results show that the computer vision-based global comfort evaluation method for footbridges is feasible and is a more specific and real-time comfort evaluation method.

Comparison of Pedestrian Detectors for LiDAR Sensor Trained on Custom Synthetic, Real and Mixed Datasets.

Deep learning algorithms for object detection used in autonomous vehicles require a huge amount of labeled data. Data collecting and labeling is time consuming and, most importantly, in most cases useful only for a single specific sensor application. Therefore, in the course of the research which is presented in this paper, the LiDAR pedestrian detection algorithm was trained on synthetically generated data and mixed (real and synthetic) datasets. The road environment was simulated with the application of the 3D rendering Carla engine, while the data for analysis were obtained from the LiDAR sensor model. In the proposed approach, the data generated by the simulator are automatically labeled, reshaped into range images and used as training data for a deep learning algorithm. Real data from Waymo open dataset are used to validate the performance of detectors trained on synthetic, real and mixed datasets. YOLOv4 neural network architecture is used for pedestrian detection from the LiDAR data. The goal of this paper is to verify if the synthetically generated data can improve the detector’s performance. Presented results prove that the YOLOv4 model trained on a custom mixed dataset achieved an increase in precision and recall of a few percent, giving an F1-score of 0.84.

Retraction Note: Infrared pedestrian detection algorithm based on multimedia image recombination and matrix restoration

Retraction Note: Infrared pedestrian detection algorithm based on multimedia image recombination and matrix restoration

A semi-automatic motion-constrained Graph Cut algorithm for Pedestrian Detection in thermal surveillance videos.

This article presents a semi-automatic algorithm that can detect pedestrians from the background in thermal infrared images. The proposed method is based on the powerful Graph Cut optimisation algorithm which produces exact solutions for binary labelling problems. An additional term is incorporated into the energy formulation to bias the detection framework towards pedestrians. Therefore, the proposed method obtains reliable and robust results through user-selected seeds and the inclusion of motion constraints. An additional advantage is that it enables the algorithm to generalise well across different databases. The effectiveness of our method is demonstrated on four public databases and compared with several methods proposed in the literature and the state-of-the-art. The method obtained an average precision of 98.92% and an average recall of 99.25% across the four databases considered and outperformed methods which made use of the same databases.

Little-YOLOv4: A Lightweight Pedestrian Detection Network Based on YOLOv4 and GhostNet

Nowadays, the areas such as intelligent-assisted driving, intelligent monitoring, pedestrian analysis, and attitude detection are developing rapidly. Inseparable from these fields is pedestrian detection technology. In order to deploy pedestrian detection algorithms on devices with limited hardware resources, a recognition algorithm that takes into account both performance and speed is required. Thus, a Little-YOLOv4 network structure is proposed, where the GhostNet is used to extract image features and the PANet is ameliorated by adding BiFPN path fusion which can integrate richer semantic features and preserve spatial information. The DO-DConv and DSC take place of standard convolution and the ReLU6 replaces the Leaky ReLU, which reduce the computational cost. The squeeze-and-excitation network is added to YOLOv4 head network, which could greatly reduce the interference information. The pedestrian detection results show that the mAP is 90.11% and the FPS is 79 by using Little-YOLOv4. The Little-YOLOv4 network structure could achieve a good compromise between algorithmic accuracy and speed.

Towards High Accuracy Pedestrian Detection on Edge GPUs.

Despite the rapid development of pedestrian detection algorithms, the balance between detection accuracy and efficiency is still far from being achieved due to edge GPUs (low computing power) limiting the parameters of the model. To address this issue, we propose the YOLOv4-TP-Tiny based on the YOLOv4 model, which mainly includes two modules, two-dimensional attention (TA) and pedestrian-based feature extraction (PFM). First, we integrate the TA mechanism into the backbone network, which increases the attention of the network to the visible area of pedestrians and improves the accuracy of pedestrian detection. Then, the PFM is used to replace the original spatial pyramid pooling (SPP) structure in the YOLOv4 to obtain the YOLOv4-TP algorithm, which can adapt to different sizes of people to obtain higher detection accuracy. To maintain detection speed, we replaced the normal convolution with a ghost network with a TA mechanism, resulting in more feature maps with fewer parameters. We constructed a one-way multi-scale feature fusion structure to replace the down-sampling process, thereby reducing network parameters to obtain the YOLOv4-TP-Tiny model. The experimental results show that the YOLOv4-TP-tiny has 58.3% AP and 31 FPS in the winder person pedestrian dataset. With the same hardware conditions and dataset, the AP of the YOLOv4-tiny is 55.9%, and the FPS is 29.

Real and Pseudo Pedestrian Detection Method with CA-YOLOv5s Based on Stereo Image Fusion.

With the development of convolutional neural networks, the effect of pedestrian detection has been greatly improved by deep learning models. However, the presence of pseudo pedestrians will lead to accuracy reduction in pedestrian detection. To solve the problem that the existing pedestrian detection algorithms cannot distinguish pseudo pedestrians from real pedestrians, a real and pseudo pedestrian detection method with CA-YOLOv5s based on stereo image fusion is proposed in this paper. Firstly, the two-view images of the pedestrian are captured by a binocular stereo camera. Then, a proposed CA-YOLOv5s pedestrian detection algorithm is used for the left-view and right-view images, respectively, to detect the respective pedestrian regions. Afterwards, the detected left-view and right-view pedestrian regions are matched to obtain the feature point set, and the 3D spatial coordinates of the feature point set are calculated with Zhengyou Zhang’s calibration method. Finally, the RANSAC plane-fitting algorithm is adopted to extract the 3D features of the feature point set, and the real and pseudo pedestrian detection is achieved by the trained SVM. The proposed real and pseudo pedestrian detection method with CA-YOLOv5s based on stereo image fusion effectively solves the pseudo pedestrian detection problem and efficiently improves the accuracy. Experimental results also show that for the dataset with real and pseudo pedestrians, the proposed method significantly outperforms other existing pedestrian detection algorithms in terms of accuracy and precision.

Pedestrian detection method and Research of Attention Mechanism in Traffic Scene

Aiming at the occlusion problem when pedestrian detection algorithms are applied in traffic scenes, an occlusion-aware pedestrian 2D saliency texture operator TS-LBP detection algorithm combined with dual attention mechanism is proposed. The algorithm first uses the differential feature-aware fusion method to mine the complementary information between multi-modal features to optimize the channel features; then uses the detection framework with an efficient anchor-free mechanism to greatly reduce the computational complexity of the model and improve the detection speed. The experimental results show that the method in this paper is fast and accurate, and has a good detection effect.

PVformer: Pedestrian and Vehicle Detection Algorithm Based on Swin Transformer in Rainy Scenes

Pedestrian and vehicle detection plays a key role in the safe driving of autonomous vehicles. Although transformer-based object detection algorithms have made great progress, the accuracy of detection in rainy scenarios is still challenging. Based on the Swin Transformer, this paper proposes an end-to-end pedestrian and vehicle detection algorithm (PVformer) with deraining module, which improves the image quality and detection accuracy in rainy scenes. Based on Transformer blocks, a four-branch feature mapping model was introduced to achieve deraining from a single image, thereby mitigating the influence of rain streak occlusion on the detector performance. According to the trouble of small object detection only by visual transformer, we designed a local enhancement perception block based on CNN and Transformer. In addition, the deraining module and the detection module were combined to train the PVformer model through transfer learning. The experimental results show that the algorithm performed well on rainy days and significantly improved the accuracy of pedestrian and vehicle detection.

Lightweight Fall Detection Algorithm Based on AlphaPose Optimization Model and ST-GCN

Falls cause great harm to people, and the current, more mature fall detection algorithms cannot be well-migrated to the embedded platform because of the huge amount of calculation. Hence, they do not have a good application. A lightweight fall detection algorithm based on the AlphaPose optimization model and ST-GCN was proposed. Firstly, based on YOLOv4, the structure of GhostNet is used to replace the DSPDarknet53 backbone network of the YOLOv4 network structure, the path convergence network is converted into BiFPN (bidirectional feature pyramid network), and DSC (deep separable convolution) is used to replace the standard volume of spatial pyramid pool, BiFPN, and YOLO head network product. Then, the TensorRt acceleration engine is used to accelerate the improved and optimized YOLO algorithm. In addition, a new type of Mosaic data enhancement algorithm is used to enhance the pedestrian detection algorithm, improving the effect of training. Secondly, use the TensorRt acceleration engine to optimize attitude estimation AlphaPose model, speeding up the inference speed of the attitude joint points. Finally, the spatiotemporal graph convolution (ST-GCN) is applied to detect and recognize actions such as falls, which meets the effective fall in different scenarios. The experimental results show that, on the embedded platform Jeston nano, when the image resolution is 416 × 416, the detection frame rate of this method is stable at about 8.33. At the same time, the accuracy of the algorithm in this paper on the UR dataset and the Le2i dataset has reached 97.28% and 96.86%, respectively. The proposed method has good real-time performance and reliable accuracy. It can be applied in the embedded platform to detect the fall state of people in real time.

Noise-aware deep learning algorithm for one-stage multispectral pedestrian detection

Multispectral pedestrian detection has gained momentum in the research literature with its wide range of applications in car safety, video surveillance, and robotics. We introduce a noise-aware multispectral detector, called NAMPD, that is based on the one-stage YOLOv4 to produce fast and accurate detection. NAMPD merges the color and thermal streams at the decision level by a weighted sum, where the weight is computed using both the noise level and average luminance of a given image pair. We show that the noise level provides a distinction between nighttime and daytime and can be utilized to improve the detection performance. Experiments are carried out on the sanitized KAIST dataset. They demonstrate that NAMPD is able to achieve a log-average miss rate of 4.25%, which is better than that of the conventional IAF R-CNN, cross-modality interactive attention network, MSDS-RCNN, and MCFF. This mechanism can operate at a speed of 15 fps on the general GPU 1080 Ti, and, thus, is a viable option to be deployed in practical pedestrian detection systems.

Illumination and Temperature-Aware Multispectral Networks for Edge-Computing-Enabled Pedestrian Detection

Accurate and efficient pedestrian detection is crucial for the intelligent transportation system regarding pedestrian safety and mobility, e.g., Advanced Driver Assistance Systems, and smart pedestrian crosswalk systems. Among all pedestrian detection methods, the vision-based detection method is demonstrated to be the most effective in previous studies. However, the existing vision-based pedestrian detection algorithms still have two limitations that restrict their implementations, those being real-time performance as well as the resistance to the impacts of environmental factors, e.g., low illumination conditions. To address these issues, this study proposes a lightweight Illumination and Temperature-aware Multispectral Network (IT-MN) for accurate and efficient pedestrian detection. The proposed IT-MN is an efficient one-stage detector. For accommodating the impacts of environmental factors and enhancing the sensing accuracy, thermal image data is fused by the proposed IT-MN with visual images to enrich useful information when visual image quality is limited. In addition, an innovative and effective late fusion strategy is also developed to optimize the image fusion performance. To make the proposed model implementable for edge computing, the model quantization is applied to reduce the model size by 75% while shortening the inference time significantly. The proposed algorithm is evaluated by comparing it with the selected state-of-the-art algorithms using a public dataset collected by in-vehicle cameras. The results show that the proposed algorithm achieves a low miss rate and inference time at 14.19% and 0.03 seconds per image pair on GPU. Besides, the quantized IT-MN achieves an inference time of 0.21 seconds per image pair on the edge device, demonstrating the potentiality of deploying the proposed model on edge devices as a highly efficient pedestrian detection algorithm.

EfficientLiteDet: a real-time pedestrian and vehicle detection algorithm

EfficientLiteDet: a real-time pedestrian and vehicle detection algorithm

AIR-YOLOv3: Aerial Infrared Pedestrian Detection via an Improved YOLOv3 with Network Pruning

Aerial object detection acts a pivotal role in searching and tracking applications. However, the large model, limited memory, and computing power of embedded devices restrict aerial pedestrian detection algorithms’ deployment on the UAV (unmanned aerial vehicle) platform. In this paper, an innovative method of aerial infrared YOLO (AIR-YOLOv3) is proposed, which combines network pruning and the YOLOv3 method. Firstly, to achieve a more appropriate number and size of the prior boxes, the prior boxes are reclustered. Then, to accelerate the inference speed on the premise of ensuring the detection accuracy, we introduced Smooth-L1 regularization on channel scale factors, and we pruned the channels and layers with less feature information to obtain a pruned YOLOv3 model. Meanwhile, we proposed the self-built aerial infrared dataset and designed ablation experiments to perform model evaluation well. Experimental results show that the AP (average precision) of AIR-YOLOv3 is 91.5% and the model size is 10.7 MB (megabyte). Compared to the original YOLOv3, its model volume compressed by 228.7 MB, nearly 95.5 %, while the model AP decreased by only 1.7%. The calculation amount is reduced by about 2/3, and the inference speed on the airborne TX2 has been increased from 3.7 FPS (frames per second) to 8 FPS.

Pedestrian Detection Algorithm Combining Attention Mechanism and Nonmaximum Suppression Method

With the involution of pedestrian detection technology, higher requirements are put forward for the detection accuracy under the conditions of insufficient light, target occlusion, and too small scale. Without information and multiscale pedestrian target, visible light single-mode pedestrian detection algorithm has poor performance. To solve the above problems, a pedestrian detection algorithm combining attention mechanism and nonmaximum suppression method is proposed in this study, aiming to improve the accuracy of pedestrian detection. In addition, residual network ResNet-50 and IoU (intersection over union) loss function are also adopted to improve pedestrian detection accuracy. Attention mechanism was used to optimize and highlight pedestrian area features, and meanwhile, the nonmaximum suppression method was applied to improve the robustness of the algorithm. Experimental results show that the detection accuracy of the proposed algorithm is significantly higher than that of the traditional convolutional neural network algorithm.

High-density pedestrian detection algorithm based on deep information fusion

High-density pedestrian detection algorithm based on deep information fusion

SDWBF Algorithm: A Novel Pedestrian Detection Algorithm in the Aerial Scene

Due to the large amount of video data from UAV aerial photography and the small target size from the aerial perspective, pedestrian detection in drone videos remains a challenge. To detect objects in UAV images quickly and accurately, a small-sized pedestrian detection algorithm based on the weighted fusion of static and dynamic bounding boxes is proposed. First, a weighted filtration algorithm for redundant frames was applied using the inter-frame pixel difference algorithm cascading vision and structural similarity, which solved the redundancy of the UAV video data, thereby reducing the delay. Second, the pre-training and detector learning datasets were scale matched to address the feature representation loss caused by the scale mismatch between datasets. Finally, the static bounding extracted by YOLOv4 and the motion bounding boxes extracted by LiteFlowNet were subject to the weighted fusion algorithm to enhance the semantic information and solve the problem of missing and multiple detections in UAV object detection. The experimental results showed that the small object recognition method proposed in this paper enabled reaching an mAP of 70.91% and an IoU of 57.53%, which were 3.51% and 2.05% higher than the mainstream target detection algorithm.