On Development of Reliable Machine Learning Systems Based on Machine Error Tolerance of Input Images

Abstract

With the rapid development of machine learning technologies, more and more practical applications arise. Representative machine learning techniques that receive much attention include object detection and image classification, which can be applied to many applications, such as self-driving cars, traffic flow calculation, and detection of product defects in factories. In this article, we investigate tolerability of errors for input images in machine learning systems and develop a generic reliability enhancement methodology. This work is based on our preliminary studies on image classification, but we put major focuses in object detection applications and the comprehensive comparisons to the prior studies. The first one error-tolerability test method to support reliability enhancement of object detection applications is then proposed based on careful error-tolerability examination of input images. The experimental results show that the test accuracy of this method can achieve 93.06%, which is the state-of-the-art. One special advantage of the proposed method is that unlike the previous error-tolerance methods in the literature, no golden reference data are required for acceptability determination by the proposed method. Hence, on-line testing can be supported. Our method is also implemented and validated in hardware. The results show that the hardware performance is up to 192 frames per second (FPS), which can thus also support real-time operations.