Loss In Classification Accuracy Research Articles

Combining high spatial resolution multi-temporal satellite data with leaf-on LiDAR to enhance tree species discrimination at the crown level

ABSTRACTThe long-standing goal of discriminating tree species at the crown-level from high spatial resolution imagery remains challenging. The aim of this study is to evaluate whether combining (a) high spatial resolution multi-temporal images from different phenological periods (spring, summer and autumn), and (b) leaf-on LiDAR height and intensity data can enhance the ability to discriminate the species of individual tree crowns of red oak (Quercus rubra), sugar maple (Acer saccharum), tulip poplar (Liriodendron tulipifera), and black cherry (Prunus serotina) in the Fernow Experimental Forest, West Virginia, USA. We used RandomForest models to measure a loss of classification accuracy caused by iteratively removing from the classification one or more groups from six groups of variables: spectral reflectance from all multispectral bands in the (1) spring, (2) summer, and (3) autumn images, (4) vegetation indices derived from the three multispectral datasets, (5) canopy height and intensity from the LiDAR imagery, and (6) texture related variables from the panchromatic and LiDAR datasets. We also used ANOVA and decision tree analyses to elucidate how the multispectral and LiDAR datasets combine to help discriminate tree species based on their unique phenological, spectral, textural, and crown architectural traits. From these results, we conclude that combing high spatial resolution multi-temporal satellite data with LiDAR datasets can enhance the ability to discriminate tree species at the crown level.

Multiview gait-based gender classification through pose-based voting

The incorporation of soft biometrics can significantly increase the performance of hard biometric systems. Gender is found to be the most popular soft biometric that can be derived from human gait. The state of the art produces accuracies up to 98% but are constrained by the necessity of requiring a complete gait cycle to function properly. We propose to remove this requirement using pose-based voting (PBV) – a method which treats every frame as a labeled instance. Linear discriminant analysis (LDA) is used in conjunction with the Bayes’ rule for classification as an alternative to the popular support vector machine (SVM). The robust design of this technique facilitates the system to cope with partially occluded gait cycles with minimal loss in classification accuracy. Furthermore, when multiple cycles are taken to account, the error becomes negligibly small. We also investigate the applicability of elliptic Fourier descriptors and and depth gait histograms for the gait-based gender classification problem. The efficiency of our approach is evaluated using all view angles of the CASIA-B gait database and the TUM-GAID database under prescribed test conditions.

Attribute reduction based on max-decision neighborhood rough set model

The neighborhood rough set model only focuses on the consistent samples whose neighborhoods are completely contained in some decision classes, and ignores the divisibility of the boundary samples whose neighborhoods can not be contained in any decision classes. In this paper, we pay close attention to the boundary samples, and enlarge the positive region by adding the samples whose neighborhoods have maximal intersection with some decision classes. Applying the mentioned idea, we introduce a new neighborhood rough set model, named max-decision neighborhood rough set model. An attribute reduction algorithm is designed based on the model. Both theoretical analysis and experimental results show that the proposed algorithm is effective for removing most redundant attributes without loss of classification accuracy.

Asthmatic Wheeze Detection From Compressively Sensed Respiratory Sound Spectra.

Quantification of wheezing by a sensor system consisting of a wearable wireless acoustic sensor and smartphone performing respiratory sound classification may contribute to the diagnosis, long-term control, and lowering treatment costs of asthma. In such battery-powered sensor system, compressive sensing (CS) was verified as a method for simultaneously cutting down power cost of signal acquisition, compression, and communication on the wearable sensor. Matching real-time CS reconstruction algorithms, such as orthogonal matching pursuit (OMP), have been demonstrated on the smartphone. However, their lossy performance limits the accuracy of wheeze detection from CS-recovered short-term Fourier spectra (STFT), when using existing respiratory sound classification algorithms. Thus, here we present a novel, robust algorithm tailored specifically for wheeze detection from the CS-recovered STFT. The proposed algorithm identifies occurrence and tracks multiple individual wheeze frequency lines using hidden Markov model. The algorithm yields 89.34% of sensitivity, 96.28% of specificity, and 94.91% of accuracy on Nyquist-rate sampled respiratory sounds STFT. It enables for less than 2% loss of classification accuracy when operating over STFT reconstructed by OMP, at the signal compression ratio of up to 4 $\times$ (classification from only 25% signal samples). It features execution speed comparable to referent algorithms, and offers good prospects for parallelism.

Fuzzy qualitative deep compression network

Recently, Convolutional Neural Networks (CNNs) have become a popular choice to tackle image classification tasks. Despite that, it is almost infeasible to embed the CNNs into resource limited hardware (e.g. mobile devices) due to its extremely high memory requirement. To address this problem, several methods were proposed to reduce the CNN memory requirement with a minimum compensation on the classification accuracy. In this paper, we propose a novel one-shot deep compression method based on the fuzzy quantity space to remove redundant CNN weights. Experiments in three public datasets (i.e. MNIST, CIFAR-10 and ImageNet) showed that our proposed approach is able to compress the CNN up to 14 × with a minimal loss of classification accuracy. Also, we present the first attempt to train an end-to-end fuzzy qualitative deep compression model in the fine-art paintings classification problem. We argue that the classification of fine-art collections is a more challenging problem in comparison to objects classification. This is because some of the artworks are neither non-representational nor figurative, and might even require imagination to recognize them. Hence, a question may arise as to whether a machine is able to capture imagination in paintings. One way to find out is by training a deep model and then visualize the low-level to high-level features learnt. Extensive experiments have been conducted on the recently publicly available Wikiart paintings dataset that consists of more than 80,000 paintings and our solution achieve state-of-the-art results (68%) in overall performance. The source code and models are available at: https://github.com/cs-chan/fuzzyDCN.

Memory-Efficient NBNN Image Classification

Naive Bayes nearest neighbor (NBNN) is a simple image classifier based on identifying nearest neighbors. NBNN uses original image descriptors (e.g., SIFTs) without vector quantization for preserving the discriminative power of descriptors and has a powerful generalization characteristic. However, it has a distinct disadvantage. Its memory requirement can be prohibitively high while processing a large amount of data. To deal with this problem, we apply a spherical hashing binary code embedding technique, to compactly encode data without significantly losing classification accuracy. We also propose using an inverted index to identify nearest neighbors among binarized image descriptors. To demonstrate the benefits of our method, we apply our method to two existing NBNN techniques with an image dataset. By using 64 bit length, we are able to reduce memory 16 times with higher runtime performance and no significant loss of classification accuracy. This result is achieved by our compact encoding scheme for image descriptors without losing much information from original image descriptors.

DiP-SVM : Distribution Preserving Kernel Support Vector Machine for Big Data

In literature, the task of learning a support vector machine for large datasets has been performed by splitting the dataset into manageable sized “partitions” and training a sequential support vector machine on each of these partitions separately to obtain local support vectors. However, this process invariably leads to the loss in classification accuracy as global support vectors may not have been chosen as local support vectors in their respective partitions. We hypothesize that retaining the original distribution of the dataset in each of the partitions can help solve this issue. Hence, we present DiP-SVM, a distribution preserving kernel support vector machine where the first and second order statistics of the entire dataset are retained in each of the partitions. This helps in obtaining local decision boundaries which are in agreement with the global decision boundary, thereby reducing the chance of missing important global support vectors. We show that DiP-SVM achieves a minimal loss in classification accuracy among other distributed support vector machine techniques on several benchmark datasets. We further demonstrate that our approach reduces communication overhead between partitions leading to faster execution on large datasets and making it suitable for implementation in cloud environments.

Randomly selected decision tree for test-cost sensitive learning

In many real-world applications, decision trees that take account of the cost of acquiring attributes for decision making have been the research focuses. The decision-making process must learn which sequence to perform, and how to build an inexpensive and reliable inductive learning model to accomplish its task. Many previous works in the area of test-cost sensitive decision tree learning have successfully reduced the total test cost, unfortunately also degraded the classification accuracy simultaneously. This paper works on a new idea, i.e., it does not has to reduce the total test cost at the cost of the loss of classification accuracy. For that, we propose a multi-target adaptive attribute selection measure and a simple but effective method for building and testing decision trees. Instead of using a greedy attribute selection measure like many other decision tree learning algorithms, our algorithm uses a random attribute selection measure to find an appropriate attribute to test at each node in the tree. Specifically, we conduct a random search through the whole space of attributes in tree building, and we call the resulting model randomly selected decision tree (RSDT). By this way, RSDT significantly reduces the total test cost, yet at the same time maintains the higher classification accuracy compared to its competitors. The experimental results on 36 UCI datasets validate the effectiveness of our proposed RSDT.

A novel approach to pre-extracting support vectors based on the theory of belief functions

Applications of the support vector machine (SVM) in the large scale datasets are seriously hampered by its high computational cost for training. In SVM training, the classification hyperplane is determined by support vectors (SVs). If those samples likely to be SVs can be pre-extracted and used for training, the computational cost can be reduced without the loss of classification accuracy. An approach to pre-extracting SVs is proposed where the training samples’ uncertainty in terms of classification is modeled using belief functions. Those samples with a higher degree of uncertainty are more likely to be SVs. Our approach can also detect outliers and noisy samples. Experimental results based on benchmark datasets show that the proposed approach performs better compared with traditional approaches, where the training time is significantly reduced (approximate to one or two orders of magnitude), meanwhile it can obtain good classification accuracies.

Wrapper based Feature Selection using Binary Flower Pollination Algorithm

Pattern recognition involves identifying a new instance as a member of a class based on the features extracted from problem space. Not all features contribute to increase in accuracy of classification. Wrapper based methodology selects an optimal subset of reduced features with no much loss in classification accuracy. Inspired by the bio evolutionary behaviour of flowers in optimal reproduction, a wrapper based optimal feature selection methodology is devised. In this paper, a Binary Flower Pollination Algorithm (BFPA) is proposed to address the combinatorial problem of feature selection. Support Vector machine (SVM) is used as a subset evaluator to calculate the fitness function to be optimized. Proposed algorithm is evaluated using 5 datasets from UCI repository. Switching probability is varied to achieve a fair tradeoff between exploration and exploitation. Feature subset chosen by the proposed method serves as input to 6 traditional classifiers to analyse the success of the algorithm. Experimental results are also compared with existing literature and observed that the proposed methodology identifies feature subset with reduced features and improved accuracy.

Evaluation of data discretization methods to derive platform independent isoform expression signatures for multi-class tumor subtyping.

BackgroundMany supervised learning algorithms have been applied in deriving gene signatures for patient stratification from gene expression data. However, transferring the multi-gene signatures from one analytical platform to another without loss of classification accuracy is a major challenge. Here, we compared three unsupervised data discretization methods--Equal-width binning, Equal-frequency binning, and k-means clustering--in accurately classifying the four known subtypes of glioblastoma multiforme (GBM) when the classification algorithms were trained on the isoform-level gene expression profiles from exon-array platform and tested on the corresponding profiles from RNA-seq data.ResultsWe applied an integrated machine learning framework that involves three sequential steps; feature selection, data discretization, and classification. For models trained and tested on exon-array data, the addition of data discretization step led to robust and accurate predictive models with fewer number of variables in the final models. For models trained on exon-array data and tested on RNA-seq data, the addition of data discretization step dramatically improved the classification accuracies with Equal-frequency binning showing the highest improvement with more than 90% accuracies for all the models with features chosen by Random Forest based feature selection. Overall, SVM classifier coupled with Equal-frequency binning achieved the best accuracy (> 95%). Without data discretization, however, only 73.6% accuracy was achieved at most.ConclusionsThe classification algorithms, trained and tested on data from the same platform, yielded similar accuracies in predicting the four GBM subgroups. However, when dealing with cross-platform data, from exon-array to RNA-seq, the classifiers yielded stable models with highest classification accuracies on data transformed by Equal frequency binning. The approach presented here is generally applicable to other cancer types for classification and identification of molecular subgroups by integrating data across different gene expression platforms.

Remote Sensing Image Classification: No Features, No Clustering

In this paper, we consider the problem of remote sensing image classification, in which feature extraction and feature coding are critical steps. Various feature extraction methods aim at an abstract and discriminative image representation. Most of them are either theoretically too complex or practically infeasible to compute for large datasets. Motivated by this observation, we propose a simple yet efficient feature extraction method within the bag-of-words (BoW) framework. It has two main innovations. First and most interestingly, this method does not need any complex local feature extraction; instead, it uses directly the pixel values from a local window as low level features. Second, in contrast to many unsupervised feature learning methods, a random dictionary is applied to feature space quantization. The advantage of a random dictionary is that it does not need the time-consuming process of dictionary learning yet without a significant loss of classification accuracy. These two novel improvements over state-of-the-art methods significantly reduce the computational time and enable it scalable to a large data volume. An extensive experimental evaluation has been performed and compared with other feature extraction methods. It is demonstrated that our feature extraction method is quite competitive and can achieve rather promising performance figures for both optical and SAR satellite images.

Neurons vs Weights Pruning in Artificial Neural Networks

<p class="R-AbstractKeywords">Artificial neural networks (ANN) are well known for their good classification abilities. Recent advances in deep learning imposed second ANN renaissance. But neural networks possesses some problems like choosing hyper parameters such as neuron layers count and sizes which can greatly influence classification rate. Thus pruning techniques were developed that can reduce network sizes, increase its generalization abilities and overcome overfitting. Pruning approaches, in contrast to growing neural networks approach, assume that sufficiently large ANN is already trained and can be simplified with acceptable classification accuracy loss.</p><p class="R-AbstractKeywords">Current paper compares nodes vs weights pruning algorithms and gives experimental results for pruned networks accuracy rates versus their non-pruned counterparts. We conclude that nodes pruning is more preferable solution, with some sidenotes.</p>

Understanding angular effects in VHR imagery and their significance for urban land-cover model portability: A study of two multi-angle in-track image sequences

This paper investigates the angular effects causing spectral distortions in multi-angle remote sensing imagery. We study two WorldView-2 multispectral in-track sequences acquired over the cities of Atlanta, USA, and Rio de Janeiro, Brazil, consisting of 13 and 20 co-located images, respectively. The sequences possess off-nadir acquisition angles up to 47.5° and bear markedly different sun-satellite configurations with respect to each other. Both scenes comprise classic urban structures such as buildings of different size, road networks, and parks. First, we quantify the degree of distortion affecting the sequences by means of a non-linear measure of distance between probability distributions, the Maximum Mean Discrepancy. Second, we assess the ability of a classification model trained on an image acquired at a certain view angle to predict the land-cover of all the other images in the sequence. The portability across the sequence is investigated for supervised classifiers of different nature by analyzing the evolution of the classification accuracy with respect to the off-nadir look angle. For both datasets, the effectiveness of physically- and statistically-based normalization methods in obtaining angle-invariant data spaces is compared and synergies are discussed. The empirical results indicate that, after a suitable normalization (histogram matching, atmospheric compensation), the loss in classification accuracy when using a model trained on the near-nadir image to classify the most off-nadir acquisitions can be reduced to as little as 0.06 (Atlanta) or 0.03 (Rio de Janeiro) Kappa points when using a SVM classifier.

Optimal Combination of Classification Algorithms and Feature Ranking Methods for Object-Based Classification of Submeter Resolution Z/I-Imaging DMC Imagery

Object-based image analysis allows several different features to be calculated for the resulting objects. However, a large number of features means longer computing times and might even result in a loss of classification accuracy. In this study, we use four feature ranking methods (maximum correlation, average correlation, Jeffries–Matusita distance and mean decrease in the Gini index) and five classification algorithms (linear discriminant analysis, naive Bayes, weighted k-nearest neighbors, support vector machines and random forest). The objective is to discover the optimal algorithm and feature subset to maximize accuracy when classifying a set of 1,076,937 objects, produced by the prior segmentation of a 0.45-m resolution multispectral image, with 356 features calculated on each object. The study area is both large (9070 ha) and diverse, which increases the possibility to generalize the results. The mean decrease in the Gini index was found to be the feature ranking method that provided highest accuracy for all of the classification algorithms. In addition, support vector machines and random forest obtained the highest accuracy in the classification, both using their default parameters. This is a useful result that could be taken into account in the processing of high-resolution images in large and diverse areas to obtain a land cover classification.

The Sequential Probability Ratio Test and Binary Item Response Models

The sequential probability ratio test (SPRT) is a common method for terminating item response theory (IRT)-based adaptive classification tests. To decide whether a classification test should stop, the SPRT compares a simple log-likelihood ratio, based on the classification bound separating two categories, to prespecified critical values. As has been previously noted (Spray & Reckase, 1994), the SPRT test statistic is not necessarily monotonic with respect to the classification bound when item response functions have nonzero lower asymptotes. Because of nonmonotonicity, several researchers (including Spray & Reckase, 1994) have recommended selecting items at the classification bound rather than the current ability estimate when terminating SPRT-based classification tests. Unfortunately, this well-worn advice is a bit too simplistic. Items yielding optimal evidence for classification depend on the IRT model, item parameters, and location of an examinee with respect to the classification bound. The current study illustrates, in depth, the relationship between the SPRT test statistic and classification evidence in binary IRT models. Unlike earlier studies, we examine the form of the SPRT-based log-likelihood ratio while altering the classification bound and item difficulty. These investigations motivate a novel item selection algorithm based on optimizing the expected SPRT criterion given the current ability estimate. The new expected log-likelihood ratio algorithm results in test lengths noticeably shorter than current, commonly used algorithms, and with no loss in classification accuracy.

Improving hyperspectral band selection by constructing an estimated reference map

We investigate band selection for hyperspectral image classification. Mutual information (MI) measures the statistical dependence between two random variables. By modeling the reference map as one of the two random variables, MI can, therefore, be used to select the bands that are more useful for image classification. A new method is proposed to estimate the MI using an optimally constructed reference map, reducing reliance on ground-truth information. To reduce the interferences from noise and clutters, the reference map is constructed by averaging a subset of spectral bands that are chosen with the best capability to approximate the ground truth. To automatically find these bands, we develop a searching strategy consisting of differentiable MI, gradient ascending algorithm, and random-start optimization. Experiments on AVIRIS 92AV3C dataset and Pavia University scene dataset show that the proposed method outperformed the benchmark methods. In AVIRIS 92AV3C dataset, up to 55% of bands can be removed without significant loss of classification accuracy, compared to the 40% from that using the reference map accompanied with the dataset. Meanwhile, its performance is much more robust to accuracy degradation when bands are cut off beyond 60%, revealing a better agreement in the MI calculation. In Pavia University scene dataset, using 45 bands achieved 86.18% classification accuracy, which is only 1.5% lower than that using all the 103 bands.

Artificial Neural Network Generalization and Simplification via Pruning/ Mākslīgo neironu tīklu vispārināšana un vienkāršošana, izmantojot samazināšanu/ Обобщение и упрощение искусственных нейронных сетей через урезание

Artificial neural networks (ANNs) are well known for their classification abilities. Although choosing hyper- parameters such as neuron layer count and size can be a quite tedious task. Pruning approaches assume that a sufficiently large ANN has already been trained and can be simplified with acceptable classification accuracy loss. The current paper presents a node pruning algorithm and gives experimental results for pruned network accuracy rates versus their non-pruned counterparts.

Analysis the effect of PCA for feature reduction in non-stationary EEG based motor imagery of BCI system

Many brain computer–interface (BCI) systems depend on imagined movement to control external devices. But how to extract the imagination feature and classify them to control systems is an important problem. To simplify the complexity of the classification, the power band and a small number of electrodes have been used, but there is still a loss in classification accuracy in the state-of-art approaches. The critical problem is the machine learning art that when the signal into source has property of non-stationary causing the estimation of the population parameter to change over time. In this paper, we analyzed the performance of feature extraction method using several spatial filter such as common average reference (CAR), Laplacian (LAP), common spatial pattern analysis (CSP) and no-spatial filter techniques and feature reduction method using principle component analysis (PCA) based 90% rule variance and leave-one-out correct classification accuracy selection method; where support vector machine is the classifier. The simulation with non-stationary data set from BCI competition III-Iva shows that CAR best performance CSP method in non-stationary data and PCA with leave-one-out CCA could maintain CCA performance and reduced the trading off between training and testing 13.96% compared to not using PCA and 0.46% compared PCA with 90% variance.

Agricultural Case Studies of Classification Accuracy, Spectral Resolution, and Model Over-Fitting

This paper describes the relationship between spectral resolution and classification accuracy in analyses of hyperspectral imaging data acquired from crop leaves. The main scope is to discuss and reduce the risk of model over-fitting. Over-fitting of a classification model occurs when too many and/or irrelevant model terms are included (i.e., a large number of spectral bands), and it may lead to low robustness/repeatability when the classification model is applied to independent validation data. We outline a simple way to quantify the level of model over-fitting by comparing the observed classification accuracies with those obtained from explanatory random data. Hyperspectral imaging data were acquired from two crop-insect pest systems: (1) potato psyllid (Bactericera cockerelli) infestations of individual bell pepper plants (Capsicum annuum) with the acquisition of hyperspectral imaging data under controlled-light conditions (data set 1), and (2) sugarcane borer (Diatraea saccharalis) infestations of individual maize plants (Zea mays) with the acquisition of hyperspectral imaging data from the same plants under two markedly different image-acquisition conditions (data sets 2a and b). For each data set, reflectance data were analyzed based on seven spectral resolutions by dividing 160 spectral bands from 405 to 907 nm into 4, 16, 32, 40, 53, 80, or 160 bands. In the two data sets, similar classification results were obtained with spectral resolutions ranging from 3.1 to 12.6 nm. Thus, the size of the initial input data could be reduced fourfold with only a negligible loss of classification accuracy. In the analysis of data set 1, several validation approaches all demonstrated consistently that insect-induced stress could be accurately detected and that therefore there was little indication of model over-fitting. In the analyses of data set 2, inconsistent validation results were obtained and the observed classification accuracy (81.06%) was only a few percentage points above that obtained using random data (66.7-77.4%). Thus, our analysis highlights a potential risk of model over-fitting and emphasizes the importance of testing for this important aspect as part of developing reliable and robust classification models.