Augmentation of Severe Weather Impact to Far-Infrared Sensor Images to Improve Pedestrian Detection System

Paulius Tumas,Artūras Serackis,Adam Nowosielski

doi:10.3390/electronics10080934

Paulius Tumas, Artūras Serackis + Show 1 more

Open Access

https://doi.org/10.3390/electronics10080934

Copy DOI

Abstract

Pedestrian detection is an essential task for computer vision and the automotive industry. Complex systems like advanced driver-assistance systems are based on far-infrared data sensors, used to detect pedestrians at nighttime, fog, rain, and direct sun situations. The robust pedestrian detector should work in severe weather conditions. However, only a few datasets include some examples of far-infrared images with distortions caused by atmospheric precipitation and dirt covering sensor optics. This paper proposes the deep learning-based data augmentation technique to enrich far-infrared images collected in good weather conditions by distortions, similar to those caused by bad weather. The six most accurate and fast detectors (TinyV3, TinyL3, You Only Look Once (YOLO)v3, YOLOv4, ResNet50, and ResNext50), performing faster than 15 FPS, were trained on 207,001 annotations and tested on 156,345 annotations, not used for training. The proposed data augmentation technique showed up to a 9.38 mean Average Precision (mAP) increase of pedestrian detection with a maximum of 87.02 mAP (YOLOv4). Proposed in this paper detectors’ Head modifications based on a confidence heat-map gave an additional boost of precision for all six detectors. The most accurate current detector, based on YOLOv4, reached up to 87.20 mAP during our experimental tests.

Highlights

Each year, over 80,000 lives are tragically lost on roads, according to the World Health Organization (WHO) yearly report [1]
Severe weather conditions like rain, snow, fog are visibility affecting factors causing drivers to adapt to the conditions
The confidence heat-map could be extended for advanced driver-assistance system (ADAS) application usage in situations when severe weather conditions occur

Summary

Introduction

Over 80,000 lives are tragically lost on roads, according to the World Health Organization (WHO) yearly report [1]. The main reasons for fatalities are rapid urbanization and motorization, poor safety standards and infrastructure, lack of strong enforcement, drivers being distracted or under the influence of drugs or alcohol, a failure to wear seat belts or helmets, and lack of access to timely post-crash care. Speeding is another critical element causing lack of time to avoid the accident, and early-stage detection of collision could drastically minimize the chance of accident [2,3,4]. A similar study, prepared by Sun et al [6] analyzed rain influence for the diver and, depending on road type, the risk to have an accident increase to 2.61 times

Methods

Findings

Discussion

Conclusion