Abstract
Aiming at the problem of weakly supervised learning in traditional Chinese painting image classification, a novel multi-instance learning algorithm based on Long and Short-Term Memory neural network with attention mechanism (ALSTM-MIL) is proposed. Firstly, by using the Pyramid Overlapping Grid Division (POGP), a multi-instance modeling scheme is designed to convert Chinese painting images into multi-instance bag, thereby transforming the problem of Chinese painting image classification into a MIL problem. Secondly, an efficient sequence generator is designed. It selects discriminative instances from the positive bags, construct a discriminative instance set (DIS), and convert multi-instance bags into equal-length ordered sequences. Thirdly, an LSTM network model with an attention mechanism is designed to perform semantic analysis on multi-instance bags to obtain their memory coding features, and then combined with the Softmax classifier to achieve semantic classification of traditional Chinese painting images. Experimental results on the Chinese painting (CP) image set show that the LSTM network built on the visual feature set is feasible, and the performance of the proposed MIL algorithm is also superior to other classification algorithms.
Highlights
With the development of the Internet and high-fidelity imaging technology, many art galleries and museums have provided online digital art work viewing services [1], [2]
Xu et al [25] used deep learning method to obtain the convolutional neural networks (CNN) features of images, and trained MIL classifiers based on these CNN features for medical image analysis; Wu et al [26] focused on the image classification and image annotation, a pre-trained deep learning model was used to predict the label of instance in the MIL framework, and the labels of all the instances in the bag were synthesized to predict the final label of the bag
IMAGE SET AND EXPERTMENTAL SETUP In order to verify the performance of the proposed ALSTM-MIL algorithm in Chinese painting image classification, we applied it on the CP image set
Summary
With the development of the Internet and high-fidelity imaging technology, many art galleries and museums have provided online digital art work viewing services [1], [2]. Xu et al [25] used deep learning method to obtain the CNN features of images, and trained MIL classifiers based on these CNN features for medical image analysis; Wu et al [26] focused on the image classification and image annotation, a pre-trained deep learning model was used to predict the label of instance in the MIL framework, and the labels of all the instances in the bag were synthesized to predict the final label of the bag He et al [27] in order to address the problem of medical image classification, based on prototype learning and bag feature transformation function, a multiinstance convolutional neural network algorithm is designed. In the modeling of multiple instances, in order to capture the global and local characteristics of Chinese painting images, that is, the description of the overall style and the details of local brush strokes, this article designed a block method of ‘‘Pyramid Overlapping Grid Division (POGP)’’ to achieve multi-bag multi-instance modeling. Each instance is represented by 163-dimensional features
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
