Deep Classification Research Articles

Monitoring Vegetation Change in the Presence of High Cloud Cover with Sentinel-2 in a Lowland Tropical Forest Region in Brazil

Forests play major roles in climate regulation, ecosystem services, carbon storage, biodiversity, terrain stabilization, and water retention, as well as in the economy of numerous countries. Nevertheless, deforestation and forest degradation are rampant in many parts of the world. In particular, the Amazonian rainforest faces the constant threats posed by logging, mining, and burning for agricultural expansion. In Brazil, the “Sete de Setembro Indigenous Land”, a protected area located in a lowland tropical forest region at the border between the Mato Grosso and Rondônia states, is subject to illegal deforestation and therefore necessitates effective vegetation monitoring tools. Optical satellite imagery, while extensively used for landcover assessment and monitoring, is vulnerable to high cloud cover percentages, as these can preclude analysis and strongly limit the temporal resolution. We propose a cloud computing-based coupled detection strategy using (i) cloud and cloud shadow/vegetation detection systems with Sentinel-2 data analyzed on the Google Earth Engine with deep neural network classification models, with (ii) a classification error correction and vegetation loss and gain analysis tool that dynamically compares and updates the classification in a time series. The initial results demonstrate that such a detection system can constitute a powerful monitoring tool to assist in the prevention, early warning, and assessment of deforestation and forest degradation in cloudy tropical regions. Owing to the integrated cloud detection system, the temporal resolution is significantly improved. The limitations of the model in its present state include classification issues during the forest fire period, and a lack of distinction between natural vegetation loss and anthropogenic deforestation. Two possible solutions to the latter problem are proposed, namely, the mapping of known agricultural and bare areas and its subsequent removal from the analyzed data, or the inclusion of radar data, which would allow a large amount of finetuning of the detection processes.

Synthesis of fracture radiographs with deep neural networks.

We describe a machine learning system for converting diagrams of fractures into realistic X-ray images. We further present a method for iterative, human-guided refinement of the generated images and show that the resulting synthetic images can be used during training to increase the accuracy of deep classifiers on clinically meaningful subsets of fracture X-rays. A neural network was trained to reconstruct images from programmatically created line drawings of those images. The images were then further refined with an optimization-based technique. Ten physicians were recruited into a study to assess the realism of synthetic radiographs created by the neural network. They were presented with mixed sets of real and synthetic images and asked to identify which images were synthetic. Two classifiers were trained to detect humeral shaft fractures: one only on true fracture images, and one on both true and synthetic images. Physicians were 49.63% accurate in identifying whether images were synthetic or real. This is close to what would be expected by pure chance (i.e.random guessing). A classifier trained only on real images detected fractures with 67.21% sensitivity when no fracture fixation hardware was present. A classifier trained on both real images and synthetic images was 75.54% sensitive. Our method generates X-rays realistic enough to be indistinguishable from real X-rays. We also show that synthetic images generated using this method can be used to increase the accuracy of deep classifiers on clinically meaningful subsets of fracture X-rays.

Structured prediction models for argumentative claim parsing from text

The internet abounds with opinions expressed in text. While a number of natural language processing techniques have been proposed for opinion analysis from text, most offer only a shallow analysis without providing any insights into reasons supporting the opinions. In online discussions, however, opinions are typically expressed as arguments, consisting of a set of claims endowed with internal semantic structure amenable to deeper analysis. In this article, we introduce the task of argumentative claim parsing (ACP), which aims at extracting semantic structures of claims from argumentative text. The task is split into two subtasks: claim segmentation and claim structuring. We present a new dataset on two discussion topics with claims manually annotated for both subtasks. Inspired by structured prediction approaches, we propose a number of supervised machine learning models for the ACP task, including deep learning, chain classifier, and joint learning models. Our experiments reveal that claim segmentation is a relatively feasible task, with the best-performing model achieving up to 0.37 and 0.79 exact and lenient macro-averaged F1-score, respectively. Claim structuring, however, proved to be a more challenging task, with the best-performing models achieving at most 0.08 macro-averaged F1-score.

Development of a real-time indoor location system using bluetooth low energy technology and deep learning to facilitate clinical applications.

An indoor, real-time location system (RTLS) can benefit both hospitals and patients by improving clinical efficiency through data-driven optimization of procedures. Bluetooth-based RTLS systems are cost-effective but lack accuracy because Bluetooth signal is subject to significant fluctuation. We aim to improve the accuracy of RTLS using the deep learning technique. We installed a Bluetooth sensor network in a three-floor clinic building to track patients, staff, and devices. The Bluetooth sensors measured the strength of the signal broadcasted from Bluetooth tags, which was fed into a deep neural network to calculate the location of the tags. The proposed deep neural network consists of a long short-term memory (LSTM) network and a deep classifier for tracking moving objects. Additionally, a spatial-temporal constraint algorithm was implemented to further increase the accuracy and stability of the results. To train the neural network, we divided the building into 115 zones and collected training data in each zone. We further augmented the training data to generate cross-zone trajectories, mimicking the real-world scenarios. We tuned the parameters for the proposed neural network to achieve relatively good accuracy. The proposed deep neural network achieved an overall accuracy of about 97% for tracking objects in each individual zone in the whole three-floor building, 1.5% higher than the baseline neural network that was proposed in an earlier paper, when using 10s of signals. The accuracy increased with the density of Bluetooth sensors. For tracking moving objects, the proposed neural network achieved stable and accurate results. When latency is less of a concern, we eliminated the effect of latency from the accuracy and gained an accuracy of 100% for our testing trajectories, significantly improved from the baseline method. The proposed deep neural network composed of a LSTM, a deep classifier and a posterior constraint algorithm significantly improved the accuracy and stability of RTLS for tracking moving objects.

Deep Binary Classification via Multi-Resolution Network and Stochastic Orthogonality for Subcompact Vehicle Recognition.

To encourage people to save energy, subcompact cars have several benefits of discount on parking or toll road charge. However, manual classification of the subcompact car is highly labor intensive. To solve this problem, automatic vehicle classification systems are good candidates. Since a general pattern-based classification technique can not successfully recognize the ambiguous features of a vehicle, we present a new multi-resolution convolutional neural network (CNN) and stochastic orthogonal learning method to train the network. We first extract the region of a bonnet in the vehicle image. Next, both extracted and input image are engaged to low and high resolution layers in the CNN model. The proposed network is then optimized based on stochastic orthogonality. We also built a novel subcompact vehicle dataset that will be open for a public use. Experimental results show that the proposed model outperforms state-of-the-art approaches in term of accuracy, which means that the proposed method can efficiently classify the ambiguous features between subcompact and non-subcompact vehicles.

SaaS software performance issue diagnosis using independent component analysis and restricted Boltzmann machine

SummarySaaS software performance issue diagnosis aims to classify the type of the performance records. Deep classification method has gained much attention as a way to construct hierarchical representations from a small amount of labeled data. However, there are few researches on how to solve the classification problem of performance issues by using the deep classification method. In addition, shallow classification methods exist some problems, such as the training sample is large and the ability to fit complex functions is weak. In this article, we proposed a deep performance issue classification method based on Independent Component Analysis (ICA) and Restricted Boltzmann Machine (RBM). ICA is used to extract the features, after this process, the classification feature is obtained as RBM input, and the extracted information about performance issue is transformed into identifiable information for the classifier via visible structure of input; Hidden layer for RBM is built to realize the data transmission between hidden structure, keeping the key information; And the classification algorithm is implemented to solve our performance issue diagnosis problem of SaaS software. Experiments show that the performance of our approach is superior to the classical shallow classification algorithm, and it also meet the efficiency requirement.

A Distributed Gaussian Weight Function based Deep Spectral Cluster Learning and Enhanced Deep Patch Level Classifier for Detection of Diabetic Retinopathy

A Distributed Gaussian Weight Function based Deep Spectral Cluster Learning and Enhanced Deep Patch Level Classifier for Detection of Diabetic Retinopathy

Coupled-View Deep Classifier Learning from Multiple Noisy Annotators

Typically, learning a deep classifier from massive cleanly annotated instances is effective but impractical in many real-world scenarios. An alternative is collecting and aggregating multiple noisy annotations for each instance to train the classifier. Inspired by that, this paper proposes to learn deep classifier from multiple noisy annotators via a coupled-view learning approach, where the learning view from data is represented by deep neural networks for data classification and the learning view from labels is described by a Naive Bayes classifier for label aggregation. Such coupled-view learning is converted to a supervised learning problem under the mutual supervision of the aggregated and predicted labels, and can be solved via alternate optimization to update labels and refine the classifiers. To alleviate the propagation of incorrect labels, small-loss metric is proposed to select reliable instances in both views. A co-teaching strategy with class-weighted loss is further leveraged in the deep classifier learning, which uses two networks with different learning abilities to teach each other, and the diverse errors introduced by noisy labels can be filtered out by peer networks. By these strategies, our approach can finally learn a robust data classifier which less overfits to label noise. Experimental results on synthetic and real data demonstrate the effectiveness and robustness of the proposed approach.

ActiveThief: Model Extraction Using Active Learning and Unannotated Public Data

Machine learning models are increasingly being deployed in practice. Machine Learning as a Service (MLaaS) providers expose such models to queries by third-party developers through application programming interfaces (APIs). Prior work has developed model extraction attacks, in which an attacker extracts an approximation of an MLaaS model by making black-box queries to it. We design ActiveThief – a model extraction framework for deep neural networks that makes use of active learning techniques and unannotated public datasets to perform model extraction. It does not expect strong domain knowledge or access to annotated data on the part of the attacker. We demonstrate that (1) it is possible to use ActiveThief to extract deep classifiers trained on a variety of datasets from image and text domains, while querying the model with as few as 10-30% of samples from public datasets, (2) the resulting model exhibits a higher transferability success rate of adversarial examples than prior work, and (3) the attack evades detection by the state-of-the-art model extraction detection method, PRADA.

Nonlinear Mixup: Out-Of-Manifold Data Augmentation for Text Classification

Data augmentation with Mixup (Zhang et al. 2018) has shown to be an effective model regularizer for current art deep classification networks. It generates out-of-manifold samples through linearly interpolating inputs and their corresponding labels of random sample pairs. Despite its great successes, Mixup requires convex combination of the inputs as well as the modeling targets of a sample pair, thus significantly limits the space of its synthetic samples and consequently its regularization effect. To cope with this limitation, we propose “nonlinear Mixup”. Unlike Mixup where the input and label pairs share the same, linear, scalar mixing policy, our approach embraces nonlinear interpolation policy for both the input and label pairs, where the mixing policy for the labels is adaptively learned based on the mixed input. Experiments on benchmark sentence classification datasets indicate that our approach significantly improves upon Mixup. Our empirical studies also show that the out-of-manifold samples generated by our strategy encourage training samples in each class to form a tight representation cluster that is far from others.

Context propagation embedding network for weakly supervised semantic segmentation

Weakly-supervised semantic segmentation with image-level labels has much significance as it facilitates related practical applications under lightweight manual annotations. Recent researches first infer the visual cues, referred as discriminative regions corresponding to each object category on images using deep convolutional classification networks. Then they expand visual cues to generate initial segmentation masks. Despite the remarkable progress, the segmentation performance still remains unsatisfactory due to the absence of complete visual cues. Using these low-quality visual cues as prior will have the limitation on improving segmentation performance. To overcome this problem, we propose a novel context propagation embedding network, i.e., the CPENet to generate high-quality visual cues, which focuses on learning semantic relationship between each region and its surrounding neighbors and selectively propagates discriminative information to non-discriminative, object related regions. Our methods can provide reliable initial segmentation masks for training subsequent segmentation network to generate final segmentation results. In addition, we refined convolutional block attention module (CBAM) [30] to hierarchically extract more category-aware features by capturing global context related information and further promote the propagation process. Experiments on benchmark demonstrate that our proposed method obtains superior performance over the state-of-the-arts.

Generative Adversarial Capsule Network With ConvLSTM for Hyperspectral Image Classification

Recently, deep learning has been widely applied in hyperspectral image (HSI) classification since it can extract high-level spatial–spectral features. However, deep learning methods are restricted due to the lack of sufficient annotated samples. To address this problem, this letter proposes a novel generative adversarial network (GAN) for HSI classification that can generate artificial samples for data augmentation to improve the HSI classification performance with few training samples. In the proposed network, a new discriminator is designed by exploiting capsule network (CapsNet) and convolutional long short-term memory (ConvLSTM), which extracts the low-level features and combines them together with local space sequence information to form the high-level contextual features. In addition, a structured sparse $L_{2,1}$ constraint is imposed on sample generation to control the modes of data being generated and achieve more stable training. The experimental results on two real HSI data sets show that the proposed method can obtain better classification performance than the several state-of-the-art deep classification methods.

Deep Relation Network for Hyperspectral Image Few-Shot Classification

Deep learning has achieved great success in hyperspectral image classification. However, when processing new hyperspectral images, the existing deep learning models must be retrained from scratch with sufficient samples, which is inefficient and undesirable in practical tasks. This paper aims to explore how to accurately classify new hyperspectral images with only a few labeled samples, i.e., the hyperspectral images few-shot classification. Specifically, we design a new deep classification model based on relational network and train it with the idea of meta-learning. Firstly, the feature learning module and the relation learning module of the model can make full use of the spatial–spectral information in hyperspectral images and carry out relation learning by comparing the similarity between samples. Secondly, the task-based learning strategy can enable the model to continuously enhance its ability to learn how to learn with a large number of tasks randomly generated from different data sets. Benefitting from the above two points, the proposed method has excellent generalization ability and can obtain satisfactory classification results with only a few labeled samples. In order to verify the performance of the proposed method, experiments were carried out on three public data sets. The results indicate that the proposed method can achieve better classification results than the traditional semisupervised support vector machine and semisupervised deep learning models.

Soft labeling with quasi-Gaussian structure for training samples of deep classification trackers

Deep classification tracking aims at classifying the candidate samples into target or background by a classifier generally trained with a binary label. However, the binary label merely distinguishes samples of different classes, while inadvertently ignoring the distinction among the samples belonging to the same class, which weakens the classification and locating ability. To cope with this problem, this article proposes a soft labeling with quasi-Gaussian structure instead of the binary labeling, which distinguishes the samples belonging to different classes and the same class simultaneously. Like as the binary label, the signs of labels for target and background samples are set to be plus and minus respectively to distinguish samples of different classes. Further, to exploit the difference among samples in the same class, the label values of samples in the same class are designed as a monotonically decreasing quasi-Gaussian function about Intersection over Union. Therefore, the corresponding response function is a two-piecewise monotonically increasing quasi-Gaussian combination function about Intersection over Union. Due to such response function, deep classification tracking trained with this proposed soft labeling achieves better classification and location performance. To validate this, the proposed soft labeling is integrated into the pipeline of the deep classification tracker SiamFC. Experimental results on OTB-2015 and VOT benchmark show that our variant achieves significant improvement to the baseline tracker while maintaining real-time tracking speed and acquires comparable accuracy as recent state-of-the-art trackers.

Deep Learning based Emotion Classification using Multi Modal Bio-signals

Deep Learning based Emotion Classification using Multi Modal Bio-signals

Abstract P5-02-02: Artificial intelligence powered predictive analysis of atypical ductal hyperplasia from digitized pathology images

Abstract Background: Approximately 15-25% of patients with atypical ductal hyperplasia (ADH) diagnosed on breast core needle biopsy (CNB) are upgraded to ductal carcinoma in situ (DCIS) or invasive carcinoma (IC) on surgical excision. The reproducible identification of patients with ADH on CNB who are more likely to have upgrades at excision remains elusive. We hypothesized that a machine learning approach could be utilized to train models to recognize ADH on digitized pathology images and to identify cases of ADH more likely to be upgraded to DCIS or IC at excision. The purpose of this study was to determine the accuracy of the machine learning approach to identify ADH. Methods: 726 digitized images of CNB slides derived from 306 cases with a diagnosis of ADH between 11/2004-3/2018 were included in this study. Independent histologic review by two breast pathologists identified slides with and without ADH from each case. 39 board certified pathologists with experience in evaluation of breast biopsies were employed for tissue region annotation on the PathAI research platform (not intended for diagnostic purposes), yielding 14,118 tissue region annotations. Region annotations included ADH, ADH stroma, flat epithelial atypia (FEA), lobular neoplasia (LN), calcifications (Ca), columnar cell change/hyperplasia, sclerosing adenosis, papilloma, normal terminal duct lobular units and other non-atypical breast tissue regions. These annotations were used to train a convolutional neural network (CNN) with 35 layers and approximately 9 million parameters to identify ADH. The data were split into training and testing sets, representing 61.1% and 38.9% of the data respectively. The distribution of cases, images with ADH and cases with upgrade were balanced between the training and testing sets. Results: CNB specimens were assigned labels of “ADH” or “No ADH” based on histologic assessment. AI models were able to predict the diagnosis of ADH with 85% sensitivity (144 of 168 images within the test set) and 69% specificity (78 of 113 images within the test set). The slide-level area under the receiver operator curve (ROC) for this model was 0.84. Conclusions: A deep learning-based classifier showed strong performance for the identification of ADH from whole slide images of H&E stained breast CNBs. With further development, this approach may improve the reproducibility and standardization of the diagnosis of ADH. Future analyses will focus on determining if morphologic features of ADH extracted by the deep learning system can be used to predict upgrade to DCIS and IC. This approach may help stratify patients with ADH on CNB into those who require surgical excision and those who can be followed with active surveillance. Citation Format: Jennifer K. Kerner, Allison Cleary, Suyog Jain, Harsha Pokkalla, Benjamin Glass, Sam Grossmith, Maya Harary, Elizabeth Mittendorf, Andrew H. Beck, Aditya Khosla, Stuart J. Schnitt, Ilan Wapinski, Tari King. Artificial intelligence powered predictive analysis of atypical ductal hyperplasia from digitized pathology images [abstract]. In: Proceedings of the 2019 San Antonio Breast Cancer Symposium; 2019 Dec 10-14; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2020;80(4 Suppl):Abstract nr P5-02-02.

Interpreting deep convolutional neural network classification results indirectly through the preprocessing feature fusion method in ship image classification

In spite of widespread adoption, deep convolutional neural networks (DCNNs) have a reputation for remaining “black boxes.” Due to the feature extraction ability of DCNN itself, good predicted results can be achieved without any processing of the raw input images, but it is hard to explain what features of the raw images the results are based on. We propose a preprocessing feature fusion method, based on the Sobel and Canny edge operators, which we treat as the interpretation of the raw images. This method obtains the interpretable edge feature by fusing features in raw ship images, which helps to explain that the DCNN ship classification result is based on the ship edge feature. That is to say, through the interpretation of the raw ship images, we indirectly interpret the classification results of the DCNN to some degree. In subsequent experiments, the highest average accuracy of 85.31% was achieved from the fusion-1 dataset, whereas the highest average accuracy of 74.4% was achieved from the balanced fusion-1 dataset, and the best mean average accuracy of 74.3% is achieved from the cross-validated experiments, which helps to verify the validity of our preprocessing method.

CINET: A Brain-Inspired Deep Learning Context-Integrating Neural Network Model for Resolving Ambiguous Stimuli.

The brain uses contextual information to uniquely resolve the interpretation of ambiguous stimuli. This paper introduces a deep learning neural network classification model that emulates this ability by integrating weighted bidirectional context into the classification process. The model, referred to as the CINET, is implemented using a convolution neural network (CNN), which is shown to be ideal for combining target and context stimuli and for extracting coupled target-context features. The CINET parameters can be manipulated to simulate congruent and incongruent context environments and to manipulate target-context stimuli relationships. The formulation of the CINET is quite general; consequently, it is not restricted to stimuli in any particular sensory modality nor to the dimensionality of the stimuli. A broad range of experiments is designed to demonstrate the effectiveness of the CINET in resolving ambiguous visual stimuli and in improving the classification of non-ambiguous visual stimuli in various contextual environments. The fact that the performance improves through the inclusion of context can be exploited to design robust brain-inspired machine learning algorithms. It is interesting to note that the CINET is a classification model that is inspired by a combination of brain’s ability to integrate contextual information and the CNN, which is inspired by the hierarchical processing of information in the visual cortex.

PolSAR image classification via a novel semi-supervised recurrent complex-valued convolution neural network

Due to that polarimetric synthetic aperture radar (PolSAR) data suffers from missing labeled samples and complex-valued data, this article presents a novel semi-supervised PolSAR terrain classification method named recurrent complex-valued convolution neural network (RCV-CNN) which combines semi-supervised learning and complex-valued convolution neural network (CV-CNN). The proposed method only needs a small number of labeled samples to achieve good classification results. First, a Wishart classifier is used to select some reliable PolSAR samples. Then, two new semi-supervised deep classification model RCV-CNN1 and RCV-CNN2 have been proposed to improve PolSAR image classification accuracy. Moreover, our proposed methods could solve the problem of network overfitting phenomenon to some extend when the number of training samples is too small. Finally, three real PolSAR dataset are applied to verify the effectiveness of our algorithms. Compared with the other five state-of-the-art methods, the proposed RCV-CNN1 and RCV-CNN2 classification models show good performance in accuracy and generalization.

Interpolative Leishman-Stained transformation invariant deep pattern classification for white blood cells

Blood analysis is regarded as the one of the most predominant examinations in medicine field to obtain patient physiological state. A significant process in the classification of white blood cells (WBC) is blood sample analysis. An automatic system that is potential of identifying WBC aids the physicians in early disease diagnosis. In contrast to previous methods, thus resulting in trade-off among computational time (CT) and performance efficiency, an interpolative Leishman-stained multi-directional transformation invariant deep classification (LSM-TIDC) for WBC is presented. LSM-TIDC method discovers possibilities of interpolation and Leishman-stained function, because they require no explicit segmentation, and yet they eliminated false regions for several input images. Next, with the preprocessed images, optimal and relevant features are extracted by applying multi-directional feature extraction. To identify and classify blood cells, a system is developed via the implementation of transformation invariant model for extraction of nucleus and subsequently performs classification through convolutional and pooling characteristics. The proposed method is evaluated by extensive experiments on benchmark database like blood cell images from Kaggle. Experimental results confirm that LSM-TIDC method significantly captures optimal and relevant features and improves the classification accuracy without compromising CT and computational overhead.