Pre-trained Convolutional Network Research Articles

A DROPOUT TECHNIQUE STUDY FOR THE FASTER R-CNN DETECTORS WITH PRETRAINED CONVOLUTIONAL NEURAL NETWORKS FOR DETECTING VERY SIMPLE OBJECTS THAT CAN BE MASKED

One of the best object detection methods, the Faster R-CNN, uses a pretrained convolutional neural network allowing to train the detector on small training sets typical in the object detection practice. Convolutional networks are prevented from overfitting by inserting DropOut layers. An open question is whether the DropOut technique improves much the object Faster R-CNN detector accuracy. Therefore, the goal is to show how the DropOut technique influences on the object detector performance. An original image classification dataset for pretraining a convolutional neural network is CIFAR-10. An appropriate convolutional network architecture for classifying CIFAR-10 images has a 50 % DropOut layer inserted in-between two fully-connected layers. Object detection tasks used for training and testing the Faster R-CNN detector are of monochrome images wherein small black rectangles are to be detected. Despite such objects are very simple, they can be masked around some dark localities so that detection would not be easy. One detection task is to detect the black rectangles in suburb house frontal views. Another one is to detect the rectangles in office room views. The suburb view dataset is divided into a training set of 120 images and a testing set of 121 images, every entry with a black rectangle. The office view dataset is divided likewise into a training set of 115 images and a testing set of 100 images, every entry with a black rectangle. Performance of the detector is studied against three training parameters: bounding box overlap ratio for positive training samples, minimum anchor box size, and anchor box pyramid scale factor. The performance is meant by the number of detected objects along with the intersection-over-union. However, neither graphs for the summed intersection-over-unions, nor graphs for the number of detected objects show that the DropOut technique influences on the Faster R-CNN object detector performance. Even for letting miss a few objects and decreasing an accuracy threshold, this influence is not significant. Therefore, a pretrained convolutional neural network to be included into the Faster R-CNN object detector should not contain a DropOut layer, especially if the network is trained much longer with the DropOut layer. Key words: object detection, Faster R-CNN object detector, pretrained convolutional network, DropOut, monochrome image, training set, intersection-over-union, number of detected objects.

Pre-trained convolutional neural networks as feature extractors for tuberculosis detection

It is estimated that in 2015, approximately 1.8 million people infected by tuberculosis died, most of them in developing countries. Many of those deaths could have been prevented if the disease had been detected at an earlier stage, but the most advanced diagnosis methods are still cost prohibitive for mass adoption. One of the most popular tuberculosis diagnosis methods is the analysis of frontal thoracic radiographs; however, the impact of this method is diminished by the need for individual analysis of each radiography by properly trained radiologists. Significant research can be found on automating diagnosis by applying computational techniques to medical images, thereby eliminating the need for individual image analysis and greatly diminishing overall costs. In addition, recent improvements on deep learning accomplished excellent results classifying images on diverse domains, but its application for tuberculosis diagnosis remains limited. Thus, the focus of this work is to produce an investigation that will advance the research in the area, presenting three proposals to the application of pre-trained convolutional neural networks as feature extractors to detect the disease. The proposals presented in this work are implemented and compared to the current literature. The obtained results are competitive with published works demonstrating the potential of pre-trained convolutional networks as medical image feature extractors.

Multi-Level and Multi-Scale Feature Aggregation Using Pretrained Convolutional Neural Networks for Music Auto-Tagging

Music auto-tagging is often handled in a similar manner to image classification by regarding the 2D audio spectrogram as image data. However, music auto-tagging is distinguished from image classification in that the tags are highly diverse and have different levels of abstractions. Considering this issue, we propose a convolutional neural networks (CNN)-based architecture that embraces multi-level and multi-scaled features. The architecture is trained in three steps. First, we conduct supervised feature learning to capture local audio features using a set of CNNs with different input sizes. Second, we extract audio features from each layer of the pre-trained convolutional networks separately and aggregate them altogether given a long audio clip. Finally, we put them into fully-connected networks and make final predictions of the tags. Our experiments show that using the combination of multi-level and multi-scale features is highly effective in music auto-tagging and the proposed method outperforms previous state-of-the-arts on the MagnaTagATune dataset and the Million Song Dataset. We further show that the proposed architecture is useful in transfer learning.

MO-DE-207B-06: Computer-Aided Diagnosis of Breast Ultrasound Images Using Transfer Learning From Deep Convolutional Neural Networks

Purpose: To assess the performance of using transferred features from pre-trained deep convolutional networks (CNNs) in the task of classifying cancer in breast ultrasound images, and to compare this method of transfer learning with previous methods involving human-designed features. Methods: A breast ultrasound dataset consisting of 1125 cases and 2393 regions of interest (ROIs) was used. Each ROI was labeled as cystic, benign, or malignant. Features were extracted from each ROI using pre-trained CNNs and used to train support vector machine (SVM) classifiers in the tasks of distinguishing non-malignant (benign+cystic) vs malignant lesions and benign vs malignant lesions. For a baseline comparison, classifiers were also trained on prior analytically-extracted tumor features. Five-fold cross-validation (by case) was conducted with the area under the receiver operating characteristic curve (AUC) as the performance metric. Results: Classifiers trained on CNN-extracted features were comparable to classifiers trained on human-designed features. In the non-malignant vs malignant task, both the SVM trained on CNN-extracted features and the SVM trained on human-designed features obtained an AUC of 0.90. In the task of determining benign vs malignant, the SVM trained on CNN-extracted features obtained an AUC of 0.88, compared to the AUC of 0.85 obtained by the SVM trained on human-designed features. Conclusion: We obtained strong results using transfer learning to characterize ultrasound breast cancer images. This method allows us to directly classify a small dataset of lesions in a computationally inexpensive fashion without any manual input. Modern deep learning methods in computer vision are contingent on large datasets and vast computational resources, which are often inaccessible for clinical applications. Consequently, we believe transfer learning methods will be important for computer-aided diagnosis schemes in order to utilize advancements in deep learning and computer vision without the associated costs. This work was partially funded by NIH grant U01 CA195564 and the University of Chicago Metcalf program. M.L.G. is a stockholder in R2/Hologic, co-founder and equity holder in Quantitative Insights, and receives royalties from Hologic, GE Medical Systems, MEDIAN Technologies, Riverain Medical, Mitsubishi, and Toshiba. K.D. received royalties from Hologic.

Weakly Supervised Large Scale Object Localization with Multiple Instance Learning and Bag Splitting.

Localizing objects of interest in images when provided with only image-level labels is a challenging visual recognition task. Previous efforts have required carefully designed features and have difficulty in handling images with cluttered backgrounds. Up-scaling to large datasets also poses a challenge to applying these methods to real applications. In this paper, we propose an efficient and effective learning framework called MILinear, which is able to learn an object localization model from large-scale data without using bounding box annotations. We integrate rich general prior knowledge into a learning model using a large pre-trained convolutional network. Moreover, to reduce ambiguity in positive images, we present a bag-splitting algorithm that iteratively generates new negative bags from positive ones. We evaluate the proposed approach on the challenging Pascal VOC 2007 dataset, and our method outperforms other state-of-the-art methods by a large margin; some results are even comparable to fully supervised models trained with bounding box annotations. To further demonstrate scalability, we also present detection results on the ILSVRC 2013 detection dataset, and our method outperforms supervised deformable part-based model without using box annotations.