Boost Classification Accuracy Research Articles

WetNet: A Spatial–Temporal Ensemble Deep Learning Model for Wetland Classification Using Sentinel-1 and Sentinel-2

While deep learning models have been extensively applied to land-use land-cover (LULC) problems, it is still a relatively new and emerging topic for separating and classifying wetland types. On the other hand, ensemble learning has demonstrated promising results in improving and boosting classification accuracy. Accordingly, this study aims to develop a classification system for mapping complex wetland areas by incorporating deep ensemble learning and satellite datasets. To this end, time series of Sentinel-1 dual-polarized Synthetic Aperture Radar (SAR) dataset, alongside Sentinel-2 multispectral imagery (MSI), are used as input data to the model. In order to increase the diversity of the extracted features, the proposed model, herein called WetNet, consists of three different submodels, comprising several recurrent and convolutional layers. Furthermore, multiple ensembling sections are added to different stages of the model to increase the transferability of the model (to other areas) and the reliability of the final results. WetNet is evaluated in a complex wetland area located in Newfoundland, Canada. Experimental results indicate that WetNet outperforms the state-of-the-art deep models (e.g., InceptionResnetV2, InceptionV3, and DenseNet121) in terms of both the classification accuracy and processing time. This makes WetNet an efficient model for large-scale wetland mapping application. The python code of the proposed WetNet model is available at the following link for the sake of reproducibility: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://colab.research.google.com/drive/1pvMOd3_tFYaMYGyHNfxqDxOiwF78lKgN?usp=sharing</uri>

Aggregating Features From Dual Paths for Remote Sensing Image Scene Classification

Scene classification is an important and challenging task employed toward understanding remote sensing images. Convolutional neural networks have been widely applied in remote sensing scene classification in recent years, boosting classification accuracy. However, with improvements in resolution, the categories of remote sensing images have become ever more fine-grained. The high intraclass diversity and interclass similarity are the main characteristics that differentiate remote scene image classification from natural image classification. To extract discriminative representation from images, we propose an end-to-end feature fusion method that aggregates features from dual paths (AFDP). First, lightweight convolutional neural networks with fewer parameters and calculations are used to construct a feature extractor with dual branches. Then, in the feature fusion stage, a novel feature fusion method that integrates the concepts of bilinear pooling and feature connection is adopted to learn discriminative features from images. The AFDP method was evaluated on three public remote sensing image benchmarks. The experimental results indicate that the AFDP method outperforms current state-of-the-art methods, with advantages of simple form, strong versatility, fewer parameters, and less calculation.

Toward an Automatic Power Quality Measurement System: An Effective Classifier of Power Signal Alterations

This paper presents an innovative and accurate automatic classifier of Power Signal (PS) alterations. Effective detection and an accurate classification are an important step toward the design of an automatic power quality measurement system that is becoming a must in current scenarios. The aim of the paper is to boost classification accuracy. This goal is achieved by merging the Hilbert Huang transform (HHT) and Convolutional Neural Network. Indeed, the first is used to extract the features of the PSs and is robust to the non-stationarity introduced by the alterations. The second is suitable to extract information from the bi-dimensional information of PS features and is robust to noise. Numerical tests comparing several classifiers and the proposed one show increased classification accuracy. Experimental tests based on emulated PSs confirm the numerical ones.

Enhancing argumentation component classification using contextual language model

Arguments facilitate humans to deliver their ideas. The outcome of the discussion heavily relies on the validity of the argument. If an argument is well-composed, it is more effective to grasp the core idea behind the argument. To grade the argument, machines can be utilized by decomposing into semantic label components. In natural language processing, multiple language models are available to perform this task. It is divided into context-free and contextual models. The majority of previous studies used hand-crafted features to perform argument component classification, while state of the art language models utilize machine learning. The majority of these language models ignore the context in an argument. This research paper aims to analyze whether by including the context in the classification process may improve the accuracy of the language model which will enhance the argumentation mining process as well. The same document corpus is fed into several language models. Word2Vec and GLoVe represent the context free models, while BERT and ELMo as context sensitive language models. Accuracy and time from each model are then compared to determine the importance of context. The result shows that contextual language models are proven to be able to boost classification accuracy by approximately 20%. However, time comes as a cost where contextual models require longer training and prediction time. The benefit from the increase in accuracy outweighs the burden of time. Thus, as a contextual task, argumentation mining is suggested to use contextual model where context must be included to achieve promising results.

Machine Learning Methods for Fear Classification Based on Physiological Features.

This paper focuses on the binary classification of the emotion of fear, based on the physiological data and subjective responses stored in the DEAP dataset. We performed a mapping between the discrete and dimensional emotional information considering the participants’ ratings and extracted a substantial set of 40 types of features from the physiological data, which represented the input to various machine learning algorithms—Decision Trees, k-Nearest Neighbors, Support Vector Machine and artificial networks—accompanied by dimensionality reduction, feature selection and the tuning of the most relevant hyperparameters, boosting classification accuracy. The methodology we approached included tackling different situations, such as resolving the problem of having an imbalanced dataset through data augmentation, reducing overfitting, computing various metrics in order to obtain the most reliable classification scores and applying the Local Interpretable Model-Agnostic Explanations method for interpretation and for explaining predictions in a human-understandable manner. The results show that fear can be predicted very well (accuracies ranging from 91.7% using Gradient Boosting Trees to 93.5% using dimensionality reduction and Support Vector Machine) by extracting the most relevant features from the physiological data and by searching for the best parameters which maximize the machine learning algorithms’ classification scores.

A New Convolutional Kernel Classifier for Hyperspectral Image Classification

Multiple Kernel Learning (MKL) algorithms are among the most successful classification methods for hyperspectral data. Nevertheless, these algorithms suffer from two main drawbacks of computational complexity and debility to admit to the end-to-end learning paradigm. This paper proposed a Convolutional Kernel Classifier (CKC) for hyperspectral remote sensing images to address these issues. The CKC uses the Nystrm approximation method to estimate a low-rank approximation of the basis kernels, thus solves the issues associated with the high dimensionality of the basis kernels. The CKC uses deep architecture to learn the optimal combination of the basis kernels and the classification task to enable end-to-end learning. The proposed CKCs architecture is based on a 1D-Convolutional Neural Network (CNN-1D), and it uses kernel dropout to prevent overfitting. It is the first instance of deep-kernel algorithms in the field of remote sensing. The proposed method was compared with several well-known hyperspectral image analysis MKL algorithms, including a multi-kernel variant of the deep kernel machine optimization (M-DKMO), MKL-average, Simple-MKL, and Generalize MKL (GMKL), and state-of-the-art deep learning models, including Vanilla Recurrent Neural Network (VanillaRNN) and CNN-1D in classifying four benchmark hyperspectral datasets. The experimental results show that the CKC consistently outperforms all the competitor methods, and its runtime is lower than its MKL algorithm counterparts on four benchmark hyperspectral datasets. Moreover, the Nystrm approximation solves the high dimensionality of the basis kernels and boosts classification accuracy. The source codes of CKC are available from: https://github.com/MohsenAnsari1373/A-New-Convolutional-Kernel-Classifier-for-Hyperspectral-Image-Classification .

Input imagery, classifiers, and cloud computing: Insights from multi-temporal LULC mapping in the Cambodian Mekong Delta

ABSTRACT The increased open-access availability of radar and optical satellite imagery has engendered numerous land use and land cover (LULC) analyses combining these data sources. In parallel, cloud computing platforms have enabled a wider community to perform LULC classifications over long periods and large areas. However, an assessment of how the performance of classifiers available on these cloud platforms can be optimized for the use of multi-imagery data has been lacking for multi-temporal LULC approaches. This study provides such an assessment for the supervised classifiers available on the open-access Google Earth Engine platform: Naïve Bayes (NB), Classification and Regression Trees (CART), Random Forest (RF), Gradient Tree Boosting (GTB), and Support Vector Machines (SVM). A multi-temporal LULC analysis using Sentinel-1 and 2 is implemented for a study area in the Mekong Delta. Classifier performance is compared for different combinations of input imagery, band sets, and training datasets. The results show that GTB and RF yield the highest overall accuracies, at 94% and 93%. Combining optical and radar imagery boosts classification accuracy for CART, RF, GTB, and SVM by 10–15 percentage points. Furthermore, it reduces the impact of limited training dataset quality for RF, GTB, and SVM.

Pulmonary nodule risk classification in adenocarcinoma from CT images using deep CNN with scale transfer module

Pulmonary nodules risk classification in adenocarcinoma is essential for early detection of lung cancer and clinical treatment decision. Improving the level of early diagnosis and the identification of small lung adenocarcinoma has been always an important topic for imaging studies. In this study, the authors propose a deep convolutional neural network (CNN) with scale-transfer module (STM) and incorporate multi-feature fusion operation, named STM-Net. This network can amplify small targets and adapt to different resolution images. The evaluation data were obtained from the computed tomography (CT) database provided by Zhongshan Hospital Fudan University (ZSDB). All data have a pathological label and their lung adenocarcinomas risk are classified into four categories: atypical adenomatous hyperplasia, adenocarcinoma in situ, minimally invasive adenocarcinoma, and invasive adenocarcinoma. The authors' deep learning network STM-Net was trained and tested for the risk stage prediction. The accuracy and the average area under the receiver operating characteristic curve achieved by their method are 95.455% and 0.987 for the ZSDB dataset. The experimental results show that STM-Net largely boosts classification accuracy on the pulmonary nodules classification compared with state-of-the-art approaches. The proposed method will be an effective auxiliary to help physicians diagnosis pulmonary nodules risk classification in adenocarcinoma in early-stage.

Using case-level context to classify cancer pathology reports.

Individual electronic health records (EHRs) and clinical reports are often part of a larger sequence—for example, a single patient may generate multiple reports over the trajectory of a disease. In applications such as cancer pathology reports, it is necessary not only to extract information from individual reports, but also to capture aggregate information regarding the entire cancer case based off case-level context from all reports in the sequence. In this paper, we introduce a simple modular add-on for capturing case-level context that is designed to be compatible with most existing deep learning architectures for text classification on individual reports. We test our approach on a corpus of 431,433 cancer pathology reports, and we show that incorporating case-level context significantly boosts classification accuracy across six classification tasks—site, subsite, laterality, histology, behavior, and grade. We expect that with minimal modifications, our add-on can be applied towards a wide range of other clinical text-based tasks.

Temporally Constrained Sparse Group Spatial Patterns for Motor Imagery BCI.

Common spatial pattern (CSP)-based spatial filtering has been most popularly applied to electroencephalogram (EEG) feature extraction for motor imagery (MI) classification in brain-computer interface (BCI) application. The effectiveness of CSP is highly affected by the frequency band and time window of EEG segments. Although numerous algorithms have been designed to optimize the spectral bands of CSP, most of them selected the time window in a heuristic way. This is likely to result in a suboptimal feature extraction since the time period when the brain responses to the mental tasks occurs may not be accurately detected. In this paper, we propose a novel algorithm, namely temporally constrained sparse group spatial pattern (TSGSP), for the simultaneous optimization of filter bands and time window within CSP to further boost classification accuracy of MI EEG. Specifically, spectrum-specific signals are first derived by bandpass filtering from raw EEG data at a set of overlapping filter bands. Each of the spectrum-specific signals is further segmented into multiple subseries using sliding window approach. We then devise a joint sparse optimization of filter bands and time windows with temporal smoothness constraint to extract robust CSP features under a multitask learning framework. A linear support vector machine classifier is trained on the optimized EEG features to accurately identify the MI tasks. An experimental study is implemented on three public EEG datasets (BCI Competition III dataset IIIa, BCI Competition IV datasets IIa, and BCI Competition IV dataset IIb) to validate the effectiveness of TSGSP in comparison to several other competing methods. Superior classification performance (averaged accuracies are 88.5%, 83.3%, and 84.3% for the three datasets, respectively) based on the experimental results confirms that the proposed algorithm is a promising candidate for performance improvement of MI-based BCIs.

Content-Based Classification of Sensitive Tweets

Online Social Networks (OSNs), such as Facebook and Twitter, provide open platforms for users to easily share their statuses, opinions, and ideas, ranging from personal experiences/activities to breaking news. With the increasing popularity of online social networks and the explosion of blog and microblog messages, we have observed large amounts of potentially sensitive or private messages being published to OSNs inadvertently or voluntarily. The owners of these messages may become vulnerable to online stalkers or adversaries, especially considering that many online social network platforms (e.g. Twitter) provide open access to the public, including unregistered users and search engine bots. Studies show that users often regret posting sensitive or private messages. However, it is very difficult to completely erase such messages from the Internet, especially when the messages have been indexed by the search engines or forwarded (e.g. re-tweet in Twitter) by other users. Therefore, it is critical to identify messages that reveal private/sensitive information, and warn users before they post the messages to the public. However, the definition of sensitive information is subjective and different from user to user. For example, some users may feel comfortable sharing political opinions, while others do not. To develop a privacy protection mechanism that is customizable to fit the needs of diverse audiences, it is essential to accurately and automatically classify potentially sensitive messages into topic categories, such as health, politics, family, relationship, religion, etc. In this paper, we make the first attempt to classify sensitive tweets into 13 pre-defined topic categories. In particular, we model the semantic content of tweets with term distribution features as well as users’ topic-preferences based on personal tweet history. We also add domain-specific features, i.e. domain knowledge, to improve classification performance. Experiments show that our method can boost classification accuracy compared with the well-known Bag-of-Words and TF-IDF methods. a

Fast Fine-Grained Image Classification via Weakly Supervised Discriminative Localization

Fine-grained image classification is to recognize hundreds of subcategories in each basic-level category. Existing methods employ discriminative localization to find the key distinctions among subcategories. However, they generally have two limitations: (1) Discriminative localization relies on region proposal methods to hypothesize the locations of discriminative regions, which are time-consuming. (2) The training of discriminative localization depends on object or part annotations, which are heavily labor-consuming. It is highly challenging to address the two key limitations simultaneously, and existing methods only focus on one of them. Therefore, we propose a weakly supervised discriminative localization approach (WSDL) for fast fine-grained image classification to address the two limitations at the same time, and its main advantages are: (1) n-pathway end-to-end discriminative localization network is designed to improve classification speed, which simultaneously localizes multiple different discriminative regions for one image to boost classification accuracy, and shares full-image convolutional features generated by region proposal network to accelerate the process of generating region proposals as well as reduce the computation of convolutional operation. (2) Multi-level attention guided localization learning is proposed to localize discriminative regions with different focuses automatically, without using object and part annotations, avoiding the labor consumption. Different level attentions focus on different characteristics of the image, which are complementary and boost the classification accuracy. Both are jointly employed to simultaneously improve classification speed and eliminate dependence on object and part annotations. Compared with state-of-the-art methods on 2 widely-used fine-grained image classification datasets, our WSDL approach achieves the best performance.

Land-Use Classification via Extreme Learning Classifier Based on Deep Convolutional Features

One of the challenging issues in high-resolution remote sensing images is classifying land-use scenes with high quality and accuracy. An effective feature extractor and classifier can boost classification accuracy in scene classification. This letter proposes a deep-learning-based classification method, which combines convolutional neural networks (CNNs) and extreme learning machine (ELM) to improve classification performance. A pretrained CNN is initially used to learn deep and robust features. However, the generalization ability is finite and suboptimal, because the traditional CNN adopts fully connected layers as classifier. We use an ELM classifier with the CNN-learned features instead of the fully connected layers of CNN to obtain excellent results. The effectiveness of the proposed method is tested on the UC-Merced data set that has 2100 remotely sensed land-use-scene images with 21 categories. Experimental results show that the proposed CNN-ELM classification method achieves satisfactory results.

Robust geometric ℓp-norm feature pooling for image classification and action recognition

Feature pooling is a key component in modern visual classification system. However, the conventional two prevailing pooling techniques, namely average and max poolings, are not theoretically optimal, due to the unrecoverable loss of the spatial information during the statistical summarization and the underlying over-simplified assumption about the feature distribution. Addressing these issues, this paper proposes to generalize previous pooling methods toward a weighted ℓp-norm spatial pooling function tailored for class-specific feature spatial distribution. Optimizing such a pooling function toward discriminative class separability that is subject to a spatial smoothness constraint yields a so-called geometric ℓp-norm pooling (GLP) method. Furthermore, to handle the variation of object scale/position, which would affect not only the learning of discriminative pooling weights but also the applicability of the learned weights, we propose a simple yet effective self-alignment step during both learning and testing to adaptively adjust the pooling weights for individual images. Image segmentation and visual saliency map are utilized to construct a directed pixel adjacency graph. The discriminative pooling weights are diffused using random walk on the constructed graph and therefore the discriminative pooling weights are propagated onto the salient and foreground region. This leads to a robust version of GLP (RGLP) which can cope with the misalignment of object position and scale in images. Comprehensive experiments validate the effectiveness of the proposed GLP feature pooling framework. The proposed random walk based self-alignment step can effectively alleviate the image misalignment issue and further boost classification accuracy. State-of-the-art image classification and action recognition performances are attained on several benchmarks.

Response to Dr Fried & Dr Kievit, and Dr Malhi et al.

Response to Dr Fried & Dr Kievit, and Dr Malhi et al.

Boosting Classification Accuracy of Diffusion MRI Derived Brain Networks for the Subtypes of Mild Cognitive Impairment Using Higher Order Singular Value Decomposition.

Mild cognitive impairment (MCI) is an intermediate stage between normal aging and Alzheimer's disease (AD), and around 10-15% of people with MCI develop AD each year. More recently, MCI has been further subdivided into early and late stages, and there is interest in identifying sensitive brain imaging biomarkers that help to differentiate stages of MCI. Here, we focused on anatomical brain networks computed from diffusion MRI and proposed a new feature extraction and classification framework based on higher order singular value decomposition and sparse logistic regression. In tests on publicly available data from the Alzheimer's Disease Neuroimaging Initiative, our proposed framework showed promise in detecting brain network differences that help in classifying early versus late MCI.

Large-scale multi-task image labeling with adaptive relevance discovery and feature hashing

It remains challenging to train an effective classifier for the new image classification tasks provided with only a few or even no labeled samples. Although multi-task learning approaches have been introduced into this field to exploit available label information to boost classification accuracy, these approaches discover intrinsic task relationships only at task level, which will lead to limited useful labels being exploited and shared. Motivated by clustered multi-task learning, this paper proposes a robust multi-task feature hashing learning algorithm for image classification. Specifically, the original input samples are first projected into a low-dimensional hash feature subspace, upon which not only the inherent relatedness but also the fine-grained clustering among samples can be revealed well. Then, the task relationships are captured by interacting at task level as well as at feature level, and finally the auxiliary labels can be shared across different tasks. We conduct extensive experiments on three large-scale multi-label image classification datasets, and results demonstrate the superiorities of the proposed formulation in comparison with several state-of-the-arts.

Invariant time-series factorization

Time-series analysis is an important domain of machine learning and a plethora of methods have been developed for the task. This paper proposes a new representation of time series, which in contrast to existing approaches, decomposes a time-series dataset into latent patterns and membership weights of local segments to those patterns. The process is formalized as a constrained objective function and a tailored stochastic coordinate descent optimization is applied. The time-series are projected to a new feature representation consisting of the sums of the membership weights, which captures frequencies of local patterns. Features from various sliding window sizes are concatenated in order to encapsulate the interaction of patterns from different sizes. The derived representation offers a set of features that boosts classification accuracy. Finally, a large-scale experimental comparison against 11 baselines over 43 real life datasets, indicates that the proposed method achieves state-of-the-art prediction accuracy results.

IMPROVED ADAPTIVE SPLITTING AND SELECTION: THE HYBRID TRAINING METHOD OF A CLASSIFIER BASED ON A FEATURE SPACE PARTITIONING

Currently, methods of combined classification are the focus of intense research. A properly designed group of combined classifiers exploiting knowledge gathered in a pool of elementary classifiers can successfully outperform a single classifier. There are two essential issues to consider when creating combined classifiers: how to establish the most comprehensive pool and how to design a fusion model that allows for taking full advantage of the collected knowledge. In this work, we address the issues and propose an AdaSS+, training algorithm dedicated for the compound classifier system that effectively exploits local specialization of the elementary classifiers. An effective training procedure consists of two phases. The first phase detects the classifier competencies and adjusts the respective fusion parameters. The second phase boosts classification accuracy by elevating the degree of local specialization. The quality of the proposed algorithms are evaluated on the basis of a wide range of computer experiments that show that AdaSS+ can outperform the original method and several reference classifiers.

ROC curve, lift chart and calibration plot

This paper presents ROC curve, lift chart and calibration plot, three well known graphical techniques that are useful for evaluating the quality of classification models used in data mining and machine learning. Each technique, normally used and studied separately, defines its own measure of classification quality and its visualization. Here, we give a brief survey of the methods and establish a common mathematical framework which adds some new aspects, explanations and interrelations between these techniques. We conclude with an empirical evaluation and a few examples on how to use the presented techniques to boost classification accuracy.