On Robustness of Robotic and Autonomous Systems Perception

Abstract

We propose an evaluation framework that emulates poor image exposure conditions, low-range image sensors, lossy compression, as well as noise types which are common in robot vision. We present a rigorous evaluation of the robustness of several high-level image recognition models and investigate their performance under distinct image distortions. On one hand, F1 score shows that the majority of CNN models are slightly affected by mild exposure, strong compression, and Poisson Noise. On the other hand, there is a large decrease in precision and accuracy in extreme misexposure, impulse noise, or signal-dependent noise. Using the proposed framework, we obtain a detailed evaluation of a variety of traditional image distortions, typically found in robotics and automated systems pipelines, provides insights and guidance for further development. We propose a pipeline-based approach to mitigate the adverse effects of image distortions by including an image pre-processing step which intends to estimate the proper exposure and reduce noise artifacts. Moreover, we explore the impacts of the image distortions on the segmentation task, a task that plays a primary role in autonomous navigation, obstacle avoidance, object picking and other robotics tasks.