Pre-trained Deep Convolutional Network Research Articles

An Effective Ensemble Convolutional Learning Model with Fine-Tuning for Medicinal Plant Leaf Identification

Accurate and efficient medicinal plant image classification is of utmost importance as these plants produce a wide variety of bioactive compounds that offer therapeutic benefits. With a long history of medicinal plant usage, different parts of plants, such as flowers, leaves, and roots, have been recognized for their medicinal properties and are used for plant identification. However, leaf images are extensively used due to their convenient accessibility and are a major source of information. In recent years, transfer learning and fine-tuning, which use pre-trained deep convolutional networks to extract pertinent features, have emerged as an extremely effective approach for image-identification problems. This study leveraged the power by three-component deep convolutional neural networks, namely VGG16, VGG19, and DenseNet201, to derive features from the input images of the medicinal plant dataset, containing leaf images of 30 classes. The models were compared and ensembled to make four hybrid models to enhance the predictive performance by utilizing the averaging and weighted averaging strategies. Quantitative experiments were carried out to evaluate the models on the Mendeley Medicinal Leaf Dataset. The resultant ensemble of VGG19+DensNet201 with fine-tuning showcased an enhanced capability in identifying medicinal plant images with an improvement of 7.43% and 5.8% compared with VGG19 and VGG16. Furthermore, VGG19+DensNet201 can outperform its standalone counterparts by achieving an accuracy of 99.12% on the test set. A thorough assessment with metrics such as accuracy, recall, precision, and the F1-score firmly established the effectiveness of the ensemble strategy.

RADAM: Texture recognition through randomized aggregated encoding of deep activation maps

Texture analysis is a classical yet challenging task in computer vision for which deep neural networks are actively being applied. Most approaches are based on building feature aggregation modules around a pre-trained backbone and then fine-tuning the new architecture on specific texture recognition tasks. Here we propose a new method named Random encoding of Aggregated Deep Activation Maps (RADAM) which extracts rich texture representations without ever changing the backbone. The technique consists of encoding the output at different depths of a pre-trained deep convolutional network using a Randomized Autoencoder (RAE). The RAE is trained locally to each image using a closed-form solution, and its decoder weights are used to compose a 1-dimensional texture representation that is fed into a linear SVM. This means that no fine-tuning or backpropagation is needed for the backbone. We explore RADAM on several texture benchmarks and achieve state-of-the-art results with different computational budgets. Our results suggest that pre-trained backbones may not require additional fine-tuning for texture recognition if their learned representations are better encoded.

Deep Learning Cascaded Feature Selection Framework for Breast Cancer Classification: Hybrid CNN with Univariate-Based Approach

With the help of machine learning, many of the problems that have plagued mammography in the past have been solved. Effective prediction models need many normal and tumor samples. For medical applications such as breast cancer diagnosis framework, it is difficult to gather labeled training data and construct effective learning frameworks. Transfer learning is an emerging strategy that has recently been used to tackle the scarcity of medical data by transferring pre-trained convolutional network knowledge into the medical domain. Despite the well reputation of the transfer learning based on the pre-trained Convolutional Neural Networks (CNN) for medical imaging, several hurdles still exist to achieve a prominent breast cancer classification performance. In this paper, we attempt to solve the Feature Dimensionality Curse (FDC) problem of the deep features that are derived from the transfer learning pre-trained CNNs. Such a problem is raised due to the high space dimensionality of the extracted deep features with respect to the small size of the available medical data samples. Therefore, a novel deep learning cascaded feature selection framework is proposed based on the pre-trained deep convolutional networks as well as the univariate-based paradigm. Deep learning models of AlexNet, VGG, and GoogleNet are randomly selected and used to extract the shallow and deep features from the INbreast mammograms, whereas the univariate strategy helps to overcome the dimensionality curse and multicollinearity issues for the extracted features. The optimized key features via the univariate approach are statistically significant (p-value ≤ 0.05) and have good capability to efficiently train the classification models. Using such optimal features, the proposed framework could achieve a promising evaluation performance in terms of 98.50% accuracy, 98.06% sensitivity, 98.99% specificity, and 98.98% precision. Such performance seems to be beneficial to develop a practical and reliable computer-aided diagnosis (CAD) framework for breast cancer classification.

Driver drowsiness detection in video sequences using hybrid selection of deep features

Monitoring driver’s drowsiness is a complex problem that involves many indicators whether behavioral or physiological. Drowsiness is a challenging problem that can lead to road disasters. Sleeping driver is more dangerous on the road than a speeding driver. Many statistics showed that one-fifth of road accidents in the world were due to driver fatigue, hence safety modules that can alert drowsy drivers in the hopes of reducing the risk of accidents are very important. This paper proposes a framework for driver drowsiness detection based on a computer vision solution. The proposed framework’s first task is to detect the driver’s face. A transfer learning is then performed for extracting the deep features from the driver’s face image using a pre-trained deep convolutional network model trained on a facial recognition dataset. The previous tasks are applied in a sliding temporal window (less than a second) in which the frames are sampled. In this work, 9 frames were the best choice. The extracted features of these frames represent the observation matrix. Then temporal feature aggregation is applied to construct the raw feature vector. To obtain the final feature vector, a proposed feature selection is applied to omit possible irrelevant features. The final feature vector is finally fed to a binary classifier to decide whether there is drowsiness or not. Extensive experiments are applied to NTHU Drowsy Driver Detection (NTHU-DDD) video dataset. The outcomes show the outperformance of the proposed approach compared with the state-of-the-art approaches.

Deep neural network for diagnostics of railway track components

The problem of increasing the speed of railway transportation and ensuring reliability is associated with constant monitoring of the condition of the railway tracks. The modern track measuring cars are equipped with video cameras and computer equipment for processing the received information. However, manual processing of data by operators in real-time is not possible. The article proposes a deep convolutional neural network for automatically recognizing and classifying defects in rail joints on rail track images. The rail video observation forms the image array during the passage of the track recording car. The formation of classes of rail joints is described. Regular rail joints with connectors, insulating joints, and welded joints are considered. Additional classes are identified, corresponding to various anomalous configurations of rail joints in the images. A modified structure of a pre-trained deep convolutional network is constructed. When preparing training samples, the actual images of rail joints were supplemented with artificial images obtained by affine transformations. The process of training and testing the classifier based on the developed convolutional network is described. The Transfer Learning is used to train the neural network. As a result of the experiments, the accuracy of classifying rail joints and detecting defects was at least 96%.

Unsupervised anomaly segmentation via deep feature reconstruction

Automatic detecting anomalous regions in images of objects or textures without priors of the anomalies is challenging, especially when the anomalies appear in very small areas of the images, making difficult-to-detect visual variations, such as defects on manufacturing products. This paper proposes an effective unsupervised anomaly segmentation approach that can detect and segment out the anomalies in small and confined regions of images. Concretely, we develop a multi-scale regional feature generator which can generate multiple spatial context-aware representations from pre-trained deep convolutional networks for every subregion of an image. The regional representations not only describe the local characteristics of corresponding regions but also encode their multiple spatial context information, making them discriminative and very beneficial for anomaly detection. Leveraging these descriptive regional features, we then design a deep yet efficient convolutional autoencoder and detect anomalous regions within images via fast feature reconstruction. Our method is simple yet effective and efficient. It advances the state-of-the-art performances on several benchmark datasets and shows great potential for real applications.

IMPROVING DEEP LEARNING BASED SEMANTIC SEGMENTATION WITH MULTI VIEW OUTLIER CORRECTION

Abstract. The goal of this paper is to use transfer learning for semi supervised semantic segmentation in 2D images: given a pretrained deep convolutional network (DCNN), our aim is to adapt it to a new camera-sensor system by enforcing predictions to be consistent for the same object in space. This is enabled by projecting 3D object points into multi-view 2D images. Since every 3D object point is usually mapped to a number of 2D images, each of which undergoes a pixelwise classification using the pretrained DCNN, we obtain a number of predictions (labels) for the same object point. This makes it possible to detect and correct outlier predictions. Ultimately, we retrain the DCNN on the corrected dataset in order to adapt the network to the new input data. We demonstrate the effectiveness of our approach on a mobile mapping dataset containing over 10’000 images and more than 1 billion 3D points. Moreover, we manually annotated a subset of the mobile mapping images and show that we were able to rise the mean intersection over union (mIoU) by approximately 10% with Deeplabv3+, using our approach.

What is damaged: a benchmark dataset for abnormal traffic object classification

Traffic-related multimedia analysis has become increasingly important in both research community and industry. In this paper, we study the problem of image-based classification of abnormal traffic objects. Different from previous works that focusing on only the normal object categories, our work aims to classify both the category and the working status of a traffic object. We construct a new dataset, namely Abnormal Traffic Object Classification (ATOC), for the study of the above problem. ATOC contains 6 kinds of traffic objects and for each main category there are also two sub-categories covering the normal and abnormal status of the objects. We propose a novel deep-learning based framework to solve our problem and provide a strong baseline for future studies. Specifically, we adopt a pre-trained deep convolutional network for feature extraction and use support vector machine for classification. We also utilize random sample pairing to augment the dataset and introduce attention mechanism to further refine the feature representation. Experimental results demonstrate that the proposed method achieves superior performance than the state-of-art deep learning approaches for the recognition of objects’ categories and the corresponding working status in traffic scenarios.

A Superpixel-Based Relational Auto-Encoder for Feature Extraction of Hyperspectral Images

Filter banks transferred from a pre-trained deep convolutional network exhibit significant performance in heightening the inter-class separability for hyperspectral image feature extraction, but weakening the intra-class consistency simultaneously. In this paper, we propose a new superpixel-based relational auto-encoder for cohesive spectral–spatial feature learning. Firstly, multiscale local spatial information and global semantic features of hyperspectral images are extracted by filter banks transferred from the pre-trained VGG-16. Meanwhile, we utilize superpixel segmentation to construct the low-dimensional manifold embedded in the spectral domain. Then, representational consistency constraint among each superpixel is added in the objective function of sparse auto-encoder, which iteratively assist and supervisedly learn hidden representation of deep spatial feature with greater cohesiveness. Superpixel-based local consistency constraint in this work not only reduces the computational complexity, but builds the neighborhood relationships adaptively. The final feature extraction is accomplished by collaborative encoder of spectral–spatial feature and weighting fusion of multiscale features. A large number of experimental results demonstrate that our proposed method achieves expected results in discriminant feature extraction and has certain advantages over some existing methods, especially on extremely limited sample conditions.

Democratized image analytics by visual programming through integration of deep models and small-scale machine learning

Analysis of biomedical images requires computational expertize that are uncommon among biomedical scientists. Deep learning approaches for image analysis provide an opportunity to develop user-friendly tools for exploratory data analysis. Here, we use the visual programming toolbox Orange (http://orange.biolab.si) to simplify image analysis by integrating deep-learning embedding, machine learning procedures, and data visualization. Orange supports the construction of data analysis workflows by assembling components for data preprocessing, visualization, and modeling. We equipped Orange with components that use pre-trained deep convolutional networks to profile images with vectors of features. These vectors are used in image clustering and classification in a framework that enables mining of image sets for both novel and experienced users. We demonstrate the utility of the tool in image analysis of progenitor cells in mouse bone healing, identification of developmental competence in mouse oocytes, subcellular protein localization in yeast, and developmental morphology of social amoebae.

How Deep Should be the Depth of Convolutional Neural Networks: a Backyard Dog Case Study

The work concerns the problem of reducing a pre-trained deep neuronal network to a smaller network, with just few layers, whilst retaining the network’s functionality on a given task. In this particular case study, we are focusing on the networks developed for the purposes of face recognition. The proposed approach is motivated by the observation that the aim to deliver the highest accuracy possible in the broadest range of operational conditions, which many deep neural networks models strive to achieve, may not necessarily be always needed, desired or even achievable due to the lack of data or technical constraints. In relation to the face recognition problem, we formulated an example of such a use case, the ‘backyard dog’ problem. The ‘backyard dog’, implemented by a lean network, should correctly identify members from a limited group of individuals, a ‘family’, and should distinguish between them. At the same time, the network must produce an alarm to an image of an individual who is not in a member of the family, i.e. a ‘stranger’. To produce such a lean network, we propose a network shallowing algorithm. The algorithm takes an existing deep learning model on its input and outputs a shallowed version of the model. The algorithm is non-iterative and is based on the advanced supervised principal component analysis. Performance of the algorithm is assessed in exhaustive numerical experiments. Our experiments revealed that in the above use case, the ‘backyard dog’ problem, the method is capable of drastically reducing the depth of deep learning neural networks, albeit at the cost of mild performance deterioration. In this work, we proposed a simple non-iterative method for shallowing down pre-trained deep convolutional networks. The method is generic in the sense that it applies to a broad class of feed-forward networks, and is based on the advanced supervise principal component analysis. The method enables generation of families of smaller-size shallower specialized networks tuned for specific operational conditions and tasks from a single larger and more universal legacy network.

MO-DE-207B-06: Computer-Aided Diagnosis of Breast Ultrasound Images Using Transfer Learning From Deep Convolutional Neural Networks

Purpose: To assess the performance of using transferred features from pre-trained deep convolutional networks (CNNs) in the task of classifying cancer in breast ultrasound images, and to compare this method of transfer learning with previous methods involving human-designed features. Methods: A breast ultrasound dataset consisting of 1125 cases and 2393 regions of interest (ROIs) was used. Each ROI was labeled as cystic, benign, or malignant. Features were extracted from each ROI using pre-trained CNNs and used to train support vector machine (SVM) classifiers in the tasks of distinguishing non-malignant (benign+cystic) vs malignant lesions and benign vs malignant lesions. For a baseline comparison, classifiers were also trained on prior analytically-extracted tumor features. Five-fold cross-validation (by case) was conducted with the area under the receiver operating characteristic curve (AUC) as the performance metric. Results: Classifiers trained on CNN-extracted features were comparable to classifiers trained on human-designed features. In the non-malignant vs malignant task, both the SVM trained on CNN-extracted features and the SVM trained on human-designed features obtained an AUC of 0.90. In the task of determining benign vs malignant, the SVM trained on CNN-extracted features obtained an AUC of 0.88, compared to the AUC of 0.85 obtained by the SVM trained on human-designed features. Conclusion: We obtained strong results using transfer learning to characterize ultrasound breast cancer images. This method allows us to directly classify a small dataset of lesions in a computationally inexpensive fashion without any manual input. Modern deep learning methods in computer vision are contingent on large datasets and vast computational resources, which are often inaccessible for clinical applications. Consequently, we believe transfer learning methods will be important for computer-aided diagnosis schemes in order to utilize advancements in deep learning and computer vision without the associated costs. This work was partially funded by NIH grant U01 CA195564 and the University of Chicago Metcalf program. M.L.G. is a stockholder in R2/Hologic, co-founder and equity holder in Quantitative Insights, and receives royalties from Hologic, GE Medical Systems, MEDIAN Technologies, Riverain Medical, Mitsubishi, and Toshiba. K.D. received royalties from Hologic.