Abstract
Driving in an adverse rain environment is a crucial challenge for vision-based advanced driver assistance systems (ADAS) in the automotive industry. The vehicle windshield wiper removes adherent raindrops that cause distorted images from in-vehicle frontal view cameras, but, additionally, it causes an occlusion that can hinder visibility at the same time. The wiper-occlusion causes erroneous judgments by vision-based applications and endangers safety. This study proposes behind-the-scenes (BTS) that detects and removes wiper-occlusion in real-time image inputs under rainy weather conditions. The pixel-wise wiper masks are detected by high-pass filtering to predict the optical flow of a sequential image pair. We fine-tuned a deep learning-based optical flow model with a synthesized dataset, which was generated with pseudo-ground truth wiper masks and flows using auto-labeling with acquired real rainy images. A typical optical flow dataset with static synthetic objects is synthesized with real fast-moving objects to enhance data diversity. We annotated wiper masks and scenes as detection ground truths from the collected real images for evaluation. BTS outperforms by achieving a 0.962 SSIM and 91.6% F1 score in wiper mask detection and 88.3% F1 score in wiper image detection. Consequently, BTS enhanced the performance of vision-based image restoration and object detection applications by canceling occlusions and demonstrated it potential role in improving ADAS under rainy weather conditions.
Highlights
The automotive industry has focused on the implementation of autonomous vehicles mounted with only cameras, which, like human drivers, primarily depend on visual perception [1,2], as image data have proven to be the richest source of raw data for advanced driver assistance systems (ADAS) to automate driving tasks through significant advancements in vision-based deep learning methods
We propose a behind-the-scenes (BTS) for windshield wiper-occlusion canceling by leveraging optical flow to maintain clear visibility of images while driving under rainy weather conditions
We achieved the detection of vehicle windshield wipers driving under rainy weather conditions, leveraging a synthesized optical flow dataset with generated pseudo-ground truth wiper data by auto-labeling acquired real datasets
Summary
The automotive industry has focused on the implementation of autonomous vehicles mounted with only cameras, which, like human drivers, primarily depend on visual perception [1,2], as image data have proven to be the richest source of raw data for advanced driver assistance systems (ADAS) to automate driving tasks through significant advancements in vision-based deep learning methods. Vision-based deraining approaches struggle with troublesome objects, vehicles have removed rain with a physical device, a windshield wiper, since 1903 This is one of the reasons why frontal view cameras for ADAS are integrated interior of vehicles to avoid ADAS performance degradation by the debris under rainy weather conditions. We propose a behind-the-scenes (BTS) for windshield wiper-occlusion canceling by leveraging optical flow to maintain clear visibility of images while driving under rainy weather conditions. Acquisition of a real dataset of driving under adverse rainy weather conditions using windshield wipers; Implementation of a fine-tuning optical flow-based model with a synthesized dataset to detect precise windshield wiper-occlusion regions; Conception and realization of wiper-free rain images for autonomous driving datasets. Video demonstration of BTS can be found in the supplementary material
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
