Image quality and computer vision performance: assessing the effects of image distortions and modeling performance relationships using the general image quality equation

Abstract

Substantial research has explored methods to optimize convolutional neural networks (CNNs) for tasks such as image classification and object detection, but research into the image quality drivers of computer vision performance has been limited. Additionally, there are indications that image degradations such as blur and noise affect human visual interpretation and machine interpretation differently. The general image quality equation (GIQE) predicts overhead image quality for human analysis using the National Image Interpretability Rating Scale, but no such model exists to predict image quality for interpretation by CNNs. Here, we assess the relationship between image quality variables and CNN performance. Specifically, we examine the impacts of resolution, blur, and noise on CNN performance for models trained with in-distribution and out-of-distribution distortions. Using two datasets, we observe that while generalization remains a significant challenge for CNNs faced with out-of-distribution image distortions, CNN performance against low visual quality images remains strong with appropriate training, indicating the potential to expand the design trade space for sensors providing data to computer vision systems. Additionally, we find that CNN performance predictions using the functional form of the GIQE can predict CNN performance as a function of image degradation, but we observe that the legacy form of the GIQE (from GIQE versions 3 and 4) does a better job of modeling the impact of blur/relative edge response in our experiments.