Abstract
Most of the existing recognition algorithms are proposed for closed set scenarios, where all categories are known beforehand. However, in practice, recognition is essentially an open set problem. There are categories we know called “knowns”, and there are more we do not know called “unknowns”. Enumerating all categories beforehand is never possible, consequently, it is infeasible to prepare sufficient training samples for those unknowns. Applying closed set recognition methods will naturally lead to unseen-category errors. To address this problem, we propose the prototype-based Open Deep Network (P-ODN) for open set recognition tasks. Specifically, we introduce prototype learning into open set recognition. Prototypes and prototype radiuses are trained jointly to guide a CNN network to derive more discriminative features. Then P-ODN detects the unknowns by applying a multi-class triplet thresholding method based on the distance metric between features and prototypes. Manual labeling the unknowns which are detected in the previous process as new categories. Predictors for new categories are added to the classification layer to “open” the deep neural networks to incorporate new categories dynamically. The weights of new predictors are initialized exquisitely by applying a distances based algorithm to transfer the learned knowledge. Consequently, this initialization method speeds up the fine-tuning process and reduce the samples needed to train new predictors. Extensive experiments show that P-ODN can effectively detect unknowns and needs only few samples with human intervention to recognize a new category. In the real world scenarios, our method achieves state-of-the-art performance on the UCF11, UCF50, UCF101 and HMDB51 datasets.
Highlights
Deep neural networks have demonstrated significant performance on many visual recognition tasks[1,2,3]
We propose prototype-based Open Deep Network (P-Open Deep Network (ODN)) to improve the robustness in detecting unknowns and updating deep neural networks, facilitating open set recognition
This paper proposed a prototype-based Open Deep Network (P-ODN) for open set recognition
Summary
Deep neural networks have demonstrated significant performance on many visual recognition tasks[1,2,3]. In10, a discriminative metric is learned for Nearest Class Mean (NCM) classification on the knowns, and new categories are added according to the mean features This approach, assumes that the number of known categories is relatively large. Abhijit et al.[4] proposed an SVM-based recognition system that could continuously recognize new categories in an open-world model by extending the NCM-like algorithms[15] to a Nearest Non-Outlier (NNO) algorithm. It is not applicable in deep neural networks, and the performance is much worse than deep neural network based algorithms. Relations of categories are defined on the feature scores in emphasis initialization method, which is a simple way to estimate the similarity of categories
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.