Rethinking compact abating probability modeling for open set recognition problem in Cyber-physical systems

Abstract

Abstract Machine learning (ML) components, especially convolutional neural network (CNN) based components, are becoming increasingly vital in cyber-physical systems (CPS), existing great influence on the dependability of CPS. Different to the popular ML model with close set hypothesis, which limits the classes of testing samples to seen classes in training samples, ML in CPS should face the problem of real-world recognition. Open set recognition (OSR) is a more suitable model for improving the dependability of ML components, which allows the presence of unknown classes at testing time. However, the ability to reject those unknown classes is concurrently required in the model. In this paper, we find the defect in compact abating probability application to most CNN-based OSR model. In order to mitigate the defect, two different methods, OpenSoftMax and OPEB are respectively proposed. Finally, extensive experiments on caltech256 data set have been conducted to verify the effectiveness and efficiency of our proposals. Compared to the well-known CNN-based OSR method OpenMax, our first method OpenSoftMax achieves 15.55% average accuracy performance gain and 28.35% F1-score performance gain, while our second method OPEB achieves 13.17% average accuracy performance gain and 25.70% F1-score performance gain.