Automated Failure Detection in Computer Vision Systems

Abstract

Listen

Translate article

Human validation of computer vision systems increase their operatingcosts and limits their scale. Automated failure detection canmitigate these constraints and is thus of great importance to thecomputer vision industry. Here, we apply a deep neural networkto detect computer vision failures on vehicle detection tasks. Theproposed model is a convolution neural network that estimates theoutput quality of a vehicle detector. We train the network to learnto estimate a pixel-level F1 score between the vehicle detector andhuman annotated data. The model generalizes well to testing data,providing a mechanism for identifying detection failures.