High Inter-class Similarity Research Articles

Fine-Grained Visual Categorization by Localizing Object Parts With Single Image

Fine-grained visual categorization (FGVC) refers to assigning fine-grained labels to images which belong to the same base category. Due to the high inter-class similarity, it is challenging to distinguish fine-grained images under different subcategories. Recently, researchers have proposed to firstly localize key object parts within images and then find discriminative clues on object parts. To localize object parts, existing methods train detectors for different kinds of object parts. However, due to the fact that the same kind of object part in different images often changes intensely in appearance, the existing methods face two shortages: 1) Training part detector for object parts with diverse appearance is laborious; 2) Discriminative parts with unusual appearance may be neglected by the trained part detectors. To localize the key object parts efficiently and accurately, a novel FGVC method is proposed in the paper. The main novelty is that the proposed method localizes the key object parts within each image only depending on a single image and hence avoid the influence of diversity between parts in different images. The proposed FGVC method consists of two key steps. Firstly, the proposed method localizes the key parts in each image independently. To this end, potential object parts in each image are identified and then these potential parts are merged to generate the final representative object parts. Secondly, two kinds of features are extracted for simultaneously describing the discriminative clues within each part and the relationship between object parts. In addition, a part based dropout learning technique is adopted to boost the classification performance further in the paper. The proposed method is evaluated in comparison experiments and the experiment results show that the proposed method can achieve comparable or better performance than state-of-the-art methods.

Facial Expression Recognition by Jointly Partial Image and Deep Metric Learning

The performance of facial expression recognition (FER) tends to deteriorate due to high intraclass variations and high interclass similarities. To address this problem, an expression recognition model based on a joint partial image and deep metric learning method (PI&DML) is proposed. First, we propose cropping the active units (AU) that are most closely related to the expression to generate a partial image for feature extraction, which is conducive to mitigating the negative impact of the abovementioned problems to some extent. Second, a novel expression metric loss function (EMLF) is suggested to enhance the intraclass similarities and interclass variations. Finally, superior performance is achieved by jointly optimizing the expression metric loss and classification loss. As demonstrated by the visualization results, the proposed EMLF is effective at increasing the distance between various expressions and reducing the distance between the same expressions. The evaluations on three public expression databases have demonstrated that our method is capable of achieving better results than the state-of-the-art methods.

Hierarchical Attention and Bilinear Fusion for Remote Sensing Image Scene Classification

Remote sensing image scene classification is an important means for the understanding of remote sensing images. Convolutional neural networks (CNNs) have been successfully applied to remote sensing image scene classification and have demonstrated remarkable performance. However, with improvements in image resolution, remote sensing image categories are becoming increasingly diverse, and problems such as high intraclass diversity and high interclass similarity have arisen. The performance of ordinary CNNs at distinguishing increasingly complex remote sensing images is still limited. Therefore, we propose a feature fusion framework based on hierarchical attention and bilinear pooling called HABFNet for the scene classification of remote sensing images. First, the deep CNN ResNet50 is used to extract the deep features from different layers of the image, and these features are fused to boost their robustness and effectiveness. Second, we design an improved channel attention scheme to enhance the features from different layers. Finally, the enhanced features are cross-layer bilinearly pooled and fused, and the fused features are used for classification. Extensive experiments were conducted on three publicly available remote sensing image benchmarks. Comparisons with the state-of-the-art methods demonstrated that the proposed HABFNet achieved competitive classification performance.

Object detection in optical remote sensing images: A survey and a new benchmark

Substantial efforts have been devoted more recently to presenting various methods for object detection in optical remote sensing images. However, the current survey of datasets and deep learning based methods for object detection in optical remote sensing images is not adequate. Moreover, most of the existing datasets have some shortcomings, for example, the numbers of images and object categories are small scale, and the image diversity and variations are insufficient. These limitations greatly affect the development of deep learning based object detection methods. In the paper, we provide a comprehensive review of the recent deep learning based object detection progress in both the computer vision and earth observation communities. Then, we propose a large-scale, publicly available benchmark for object DetectIon in Optical Remote sensing images, which we name as DIOR. The dataset contains 23,463 images and 192,472 instances, covering 20 object classes. The proposed DIOR dataset (1) is large-scale on the object categories, on the object instance number, and on the total image number; (2) has a large range of object size variations, not only in terms of spatial resolutions, but also in the aspect of inter- and intra-class size variability across objects; (3) holds big variations as the images are obtained with different imaging conditions, weathers, seasons, and image quality; and (4) has high inter-class similarity and intra-class diversity. The proposed benchmark can help the researchers to develop and validate their data-driven methods. Finally, we evaluate several state-of-the-art approaches on our DIOR dataset to establish a baseline for future research.

Automated Leaf Alignment and Partial Shape Feature Extraction for Plant Leaf Classification

The last few decades have witnessed various approaches to automate the process of plant classification using the characteristics of the leaf. Several approaches have been proposed, and the majority focused on global shape features. However, one challenge that faces this task is the high interclass similarity amongst the leaves of different species in terms of the global shape. Furthermore, there always has been an obstacle against full automation as several approaches require user intervention to align the leaf. Therefore, a new set of Quartile Features (QF) is proposed in this paper to describe the partial shape of the leaf, in addition to an automated alignment approach to automate the system. The QF are extracted from the horizontal and vertical leaf quartiles to describe the partial shape of the leaf and the relations among its parts. The well-known Flavia dataset has been selected for the evaluation of the proposed system. The experimental results indicate the ability of the proposed alignment algorithm to align leaves with different shapes and maintain a correct classification accuracy regardless of the orientation of the input leaf samples. Furthermore, the proposed QF indicated promising results by increasing the accuracy of the classification by a range of approximately 26% to 30% when combined with Hu’s Moment Invariants, using k-fold cross-validation technique.

Multi-Weighted Co-Occurrence Descriptor Encoding for Vein Recognition

Despite being highly secure, vein recognition suffers from the high inter-class similarity and intra-class variation resulting from the uncontrolled image capture, making the design of discriminative and robust representation very important. The recent success of convolutional neural network (CNN) for various image understanding tasks makes it a promising method for feature extraction. However, limited variability in small-scale datasets leads to systems derived from the direct training or fine-tuning not transferable and unreliable for practical biometric applications. This motivates the design of a multi-weighted co-occurrence descriptor encoding (MWCDE) model for vein recognition. Instead of directly conducting a feed-forward operation with a pre-trained CNN for obtaining the semantic features from the fully connected layers, co-occurrence features among convolutional filters are modeled first in MWCDE by a simple convolution between an indicator filter in a higher layer with a to-be-reweighted filter in a lower layer, and a redundancy-driven indicator filter selection algorithm is designed for filtering out some ambiguous representations. Second, another hard feature weighting strategy with a binary masking scheme is proposed for discarding noisy background and feature redundancy. The selected high-order descriptors are then embedded and aggregated into the compact feature vectors with a saliency driven spatial weighted Fisher vector algorithm, followed by the introduction of a generalized support vector machine for recognition. Extensive experiments with three benchmark vein datasets demonstrate that the proposed framework can achieve state-of-the-art results, and an additional experiment with the PolyU multispectral palmprint database illustrates its generalization ability. Code is available at (https://github.com/RobinCSIRO/MWCDE-for-Vein-Recognition).

Fusing Object Semantics and Deep Appearance Features for Scene Recognition

Scene images generally show the characteristics of large intra-class variety and high inter-class similarity because of complicated appearances, subtle differences, and ambiguous categorization. Hence, it is difficult to achieve satisfactory accuracy by using a single representation. For solving this issue, we present a comprehensive representation for scene recognition by fusing deep features extracted from three discriminative views, including the information of object semantics, global appearance, and contextual appearance. These views show diversity and complementarity of features. The object semantics representation of the scene image, denoted by spatial-layout-maintained object semantics features, is extracted from the output of a deep-learning-based multi-classes detector by using spatial fisher vectors, which can simultaneously encode the category and layout information of objects. A multi-direction long short-term memory-based model is built to represent contextual information of the scene image, and the activation of the fully connected layer of a convolutional neural network is used to represent the global appearance of scene image. These three kinds of deep features are then fused to draw a final conclusion for scene recognition. Extensive experiments are conducted to evaluate the proposed comprehensive representation on three benchmarks scene image database. The results show that the three deep features complement to each other strongly and are effective in improving recognition performance after fusion. The proposed method can achieve scene recognition accuracy of 89.51% on the MIT67 database, 78.93% on the SUN397 database, and 57.27% on the Places365 databases, respectively, which are better percentages than the accuracies obtained by the latest reported deep-learning-based scene recognition methods.

L-Seg: An end-to-end unified framework for multi-lesion segmentation of fundus images

Diabetic retinopathy and diabetic macular edema are the two leading causes for blindness in working-age people, and the quantitative and qualitative diagnosis of these two diseases usually depends on the presence and areas of lesions in fundus images. The main related lesions include soft exudates, hard exudates, microaneurysms, and haemorrhages. However, segmentation of these four kinds of lesions is difficult due to their uncertainty in size, contrast, and high interclass similarity. Therefore, we aim to design a multi-lesion segmentation model. We have designed the first small object segmentation network (L-Seg) that can segment the four kinds of lesions simultaneously. Taking into account that small lesion regions could not response at high level of network, we propose a multi-scale feature fusion method to handle this problem. In addition, when considering the cases of both class-imbalance and loss-imbalance problems, we propose a multi-channel bin loss. We have evaluated L-Seg on three fundus datasets including two publicly available datasets - IDRiD and e-ophtha and one private dataset - DDR. Extensive experiments have demonstrated that L-Seg achieves better performance in small lesion segmentation than other deep learning models and traditional methods. Specially, the mAUC score of L-Seg is over 16.8%, 1.51% and 3.11% higher than that of DeepLab v3+ on IDRiD, e-ophtha and DDR datasets, respectively. Moreover, our framework shows competitive performance compared with top-3 teams in IDRiD challenge. The source code of L-Seg is available at: https://github.com/guomugong/L-Seg.

HANDWRITTEN AMHARIC CHARACTER RECOGNITION SYSTEM USING CONVOLUTIONAL NEURAL NETWORKS

Amharic language is an official language of the federal government of the Federal Democratic Republic of Ethiopia. Accordingly, there is a bulk of handwritten Amharic documents available in libraries, information centres, museums, and offices. Digitization of these documents enables to harness already available language technologies to local information needs and developments. Converting these documents will have a lot of advantages including (i) to preserve and transfer history of the country (ii) to save storage space (ii) proper handling of documents (iv) enhance retrieval of information through internet and other applications. Handwritten Amharic character recognition system becomes a challenging task due to inconsistency of a writer, variability in writing styles of different writers, relatively large number of characters of the script, high interclass similarity, structural complexity and degradation of documents due to different reasons. In order to recognize handwritten Amharic character a novel method based on deep neural networks is used which has recently shown exceptional performance in various pattern recognition and machine learning applications, but has not been endeavoured for Ethiopic script. The Convolutional neural network model is evaluated for its performance using our database that contains 132,500 datasets of handwritten Amharic characters. Common handwritten recognition systems using machine learning use a combination of both feature extractors and classifiers. Currently the use of deep learning techniques shows promising improvements for machine learning based classification tasks. Our proposed CNN model gives an accuracy of 91.83% on training data and 90.47% on validation data.

Enhancing the performance of Multiple Endmember Spectral Mixture Analysis (MESMA) for urban land cover mapping using airborne lidar data and band selection

Multiple Endmember Spectral Mixture Analysis (MESMA) is a widely applied tool to retrieve spatially explicit information on urban land cover from both hyperspectral and multispectral data, but is still prone to misclassification errors when faced with high inter-class similarity, typical of the complex urban environment. In this study we assessed multiple ways to minimize spectral confusion using airborne lidar data as an additional data source and spectral feature selection. Several approaches were tested using simulated hyperspectral data and two case studies in the city of Brussels, Belgium, one based on hyperspectral (APEX) data and one on multispectral (Sentinel-2) data. We found that the implementation of height distribution information (1) as an endmember model selection tool and (2) as a basis for additional fraction constraints at the individual pixel scale, significantly reduced spectral confusion between spectrally similar, but structurally different land cover classes (on average by 80% for the APEX case). This had a net positive effect on subpixel fraction estimations (average R2 increased from 0.34 to 0.80 and from 0.23 to 0.63 for APEX and Sentinel-2, respectively) and pixel classification accuracies (kappa increased from 0.38 to 0.6 for the APEX case). When applied to fine spatial resolution data containing many single-class pixels, endmember model selection based on height information resulted in the additional benefit of lowering computation times by 85%. Spectral feature selection successfully discarded redundant spectral information (on average retaining only 19 out of 218 bands), thereby further lowering processing times by 50%, without affecting accuracies. Despite these significant improvements, spectral confusion remained an issue between classes showing no distinction in height information, particularly pavement and soil. Future research should therefore focus on integrating the proposed approach with advanced endmember detection and selection algorithms, along with exploring innovative ways of highlighting small spectral differences using spectral transformations. The algorithm we propose constitutes a viable approach for mapping of structurally diverse ecosystems, such as urban environments, at multiple spatial scales and with varying level of thematic detail.

American Sign Language alphabet recognition using Convolutional Neural Networks with multiview augmentation and inference fusion

American Sign Language (ASL) alphabet recognition by computer vision is a challenging task due to the complexity in ASL signs, high interclass similarities, large intraclass variations, and constant occlusions. This paper describes a method for ASL alphabet recognition using Convolutional Neural Networks (CNN) with multiview augmentation and inference fusion, from depth images captured by Microsoft Kinect. Our approach augments the original data by generating more perspective views, which makes the training more effective and reduces the potential overfitting. During the inference step, our approach comprehends information from multiple views for the final prediction to address the confusing cases caused by orientational variations and partial occlusions. On two public benchmark datasets, our method outperforms the state-of-the-arts.

ERN: Edge Loss Reinforced Semantic Segmentation Network for Remote Sensing Images

The semantic segmentation of remote sensing images faces two major challenges: high inter-class similarity and interference from ubiquitous shadows. In order to address these issues, we develop a novel edge loss reinforced semantic segmentation network (ERN) that leverages the spatial boundary context to reduce the semantic ambiguity. The main contributions of this paper are as follows: (1) we propose a novel end-to-end semantic segmentation network for remote sensing, which involves multiple weighted edge supervisions to retain spatial boundary information; (2) the main representations of the network are shared between the edge loss reinforced structures and semantic segmentation, which means that the ERN simultaneously achieves semantic segmentation and edge detection without significantly increasing the model complexity; and (3) we explore and discuss different ERN schemes to guide the design of future networks. Extensive experimental results on two remote sensing datasets demonstrate the effectiveness of our approach both in quantitative and qualitative evaluation. Specifically, the semantic segmentation performance in shadow-affected regions is significantly improved.

Surface Defects Classification of Hot-Rolled Steel Strips Using Multi-directional Shearlet Features

In this paper, a method combining the use of discrete shearlet transform (DST) and the gray-level co-occurrence matrix (GLCM) is presented to classify surface defects of hot-rolled steel strips into the six classes of rolled-in scale, patches, crazing, pitted surface, inclusion and scratches. Feature extraction involves the extraction of multi-directional shearlet features from each input image followed by GLCM calculations from all extracted sub-bands, from which a set of statistical features is extracted. The resultant high-dimensional feature vectors are then reduced using principal component analysis. A supervised support vector machine classifier is finally trained to classify the surface defects. The proposed feature set is compared against the Gabor, wavelets and the original GLCM in order to evaluate and validate its robustness. Experiments were conducted on a database of hot-rolled steel strips consisting of 1800 grayscale images whose defects exhibit high inter-class similarity as well as high intra-class appearance variations. Results indicate that the proposed DST–GLCM method is superior to other methods and achieves classification rates of 96.00%.

Exudate-based diabetic macular edema recognition in retinal images using cascaded deep residual networks

Abstract Diabetic macular edema (DME), one of the leading causes of visual impairment and blindness, is usually diagnosed by the presence of exudates. However, exudate detection is challenging due to the large intraclass variation and high interclass similarity. To overcome these challenges, we propose the cascaded deep residual networks to recognize DME. Specifically, we first design a fully convolutional residual network that fuses multi-level hierarchical information to segment exudates accurately with a fast speed. Compared with previous methods, our approach avoids a wide range of preprocessing or postprocessing steps, reducing the impact of subjective factors. Then based on the segmentation results, the region centered on the pixel with the maximum probability is cropped and fed into the other deep residual network (for classification) to distinguish DME from its hard mimics. This makes the classification network to extract more representative features based on the segmentation results instead of the original images, further reducing the influence of complicated background. We evaluate the proposed method on two publicly available databases, the HEI-MED and e-ophtha EX databases. Extensive experiments demonstrate that our approach achieves better performance than the state-of-the-art methods with a fast processing speed, making it suitable for real-world clinical applications.

Deep Fully Convolutional Network-Based Spatial Distribution Prediction for Hyperspectral Image Classification

Most of the existing spatial-spectral-based hyperspectral image classification (HSIC) methods mainly extract the spatial-spectral information by combining the pixels in a small neighborhood or aggregating the statistical and morphological characteristics. However, those strategies can only generate shallow appearance features with limited representative ability for classes with high interclass similarity and spatial diversity and therefore reduce the classification accuracy. To this end, we present a novel HSIC framework, named deep multiscale spatial-spectral feature extraction algorithm, which focuses on learning effective discriminant features for HSIC. First, the well pretrained deep fully convolutional network based on VGG-verydeep-16 is introduced to excavate the potential deep multiscale spatial structural information in the proposed hyperspectral imaging framework. Then, the spectral feature and the deep multiscale spatial feature are fused by adopting the weighted fusion method. Finally, the fusion feature is put into a generic classifier to obtain the pixelwise classification. Compared with the existing spectral-spatial-based classification techniques, the proposed method provides the state-of-the-art performance and is much more effective, especially for images with high nonlinear distribution and spatial diversity.

Classification of Complex Urban Fringe Land Cover Using Evidential Reasoning Based on Fuzzy Rough Set: A Case Study of Wuhan City

Urban fringe is the transition zone fine grained with urban and non-urban land cover types. The complex landscape mosaic in this area challenges the land cover classification based on the remote-sensing data. Spectral signatures are not efficient to discriminate all pixels into classes. To improve the recognition and handle the uncertainty, this paper provides a novel integrated approach, based on a fuzzy rough set and evidential reasoning (FRSER), for land cover classification in an urban fringe area. The approach is implemented on Landsat Operation Land Imager data covering the urban fringe area of Wuhan city, China. A fuzzy rough set is first used to define a decision table from multispectral imagery and ground reference data. Then the fuzzy rough information system is interpreted using the Dempster–Shafer theory, based on an evidential reasoning system. A final land cover classification with uncertainty is achieved by evidential reasoning. The results are compared with the traditional maximum likelihood classifier (MLC) and some rough set-based classifiers including classical rough set classifier (RS), fuzzy rough set classifier (FRS), and variable precision fuzzy rough set classifier (VPFRS). The better overall accuracy, user’s and producer’s accuracies, and the kappa coefficient, in comparison with the other classifiers, suggest that the proposed approach can effectively discriminate land cover types in urban fringe areas with high inter-class similarities and intra-class heterogeneity. It is also capable of handling the uncertainty in data processing, and the final land cover map comes with a degree of uncertainty. The proposed approach that can efficiently integrate the merits of both the fuzzy rough set and DS theory provides an efficient method for urban fringe land cover classification.

Scrutiny of large datasets using clustering techniques

Clustering technique is critically important step in data mining process. Clustering is a significant task in data analysis and data mining applications. It is the task of arrangement a set of objects so that objects in the identical group are more related to each other than to those in other groups (clusters). Data mining can do by passing through various phases. Mining can be done by using supervised and unsupervised learning. The clustering is unsupervised learning. A good clustering method will produce high superiority clusters with high intra-class similarity and low inter-class similarity. Its main distinctiveness is the fastest processing time. In this paper, an analysis of clustering and its different techniques in data mining is done. Results were quite encouraging and had shown high accuracy.

BENCHMARK EVALUATION OF HOG DESCRIPTORS AS FEATURES FOR CLASSIFICATION OF TRAFFIC SIGNS

The purpose of this paper is to analyze the performance of the Histograms of Oriented Gradients (HOG) as descriptors for traffic signs recognition. The test dataset consists of speed limit traffic signs because of their high inter-class similarities. HOG features of speed limit signs, which were extracted from different traffic scenes, were computed and a Gentle AdaBoost classifier was invoked to evaluate the different features. The performance of HOG was tested with a dataset consisting of 1727 Swedish speed signs images. Different numbers of HOG features per descriptor, ranging from 36 features up 396 features, were computed for each traffic sign in the benchmark testing. The results show that HOG features perform high classification rate as the Gentle AdaBoost classification rate was 99.42%, and they are suitable to real time traffic sign recognition. However, it is found that changing the number of orientation bins has insignificant effect on the classification rate. In addition to this, HOG descriptors are not robust with respect to sign orientation.