Abstract
A wide range of advanced deep learning algorithms have recently been proposed for image classification and object detection. However, the effectiveness of these methods can be significantly restricted in many real-world scenarios where the visibility or illumination is poor. Compared to RGB cameras, millimeter-wave (mmWave) radars are immune to the above environmental variability and can assist cameras under adverse conditions. To this end, we propose milliEye, a lightweight mmWave radar and camera fusion system for robust object detection on the edge platforms. milliEye has several key advantages over existing sensor fusion approaches. First, while milliEye fuses two sensing modalities in a learning-based fashion, it requires only a small amount of labeled image/radar data of a new scene as it can fully utilize large public image datasets for extensive training. This salient feature enables milliEye to adapt to highly complex real-world environments. Second, based on a novel architecture that decouples the image-based object detector from other modules, milliEye is compatible with different off-the-shelf image-based object detectors. As a result, it can take advantage of the rapid progress of object detection algorithms. Moreover, thanks to the highly compute-efficient fusion approach, milliEye is lightweight and thus suitable for edge-based real-time applications. To evaluate the performance of milliEye, we collect a new radar and camera fusion dataset for object detection, which contains both ordinary-light and low-light illumination conditions. The results show that milliEye can provide substantial performance boosts over state-of-the-art image-based object detectors, including Tiny YOLOv3 and SSD, especially in low-light scenes, while incurring low compute overhead on edge platforms.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.