Fine-grained Classification Research Articles

Locally-Enriched Cross-Reconstruction for Few-Shot Fine-Grained Image Classification

Few-shot fine-grained image classification has attracted considerable attention in recent years for its realistic setting to imitate how humans conduct recognition tasks. Metric-based few-shot classifiers have achieved high accuracies. However, their metric function usually requires two arguments of vectors, while transforming or reshaping three-dimensional feature maps to vectors can result in loss of spatial information. Image reconstruction is thus involved to retain more appearance details: the test images are reconstructed by different classes and then classified to the one with the smallest reconstruction error. However, discriminative local information, vital to distinguish sub-categories in fine-grained images with high similarities, is not well elaborated when only the base features from a usual embedding module are adopted for reconstruction. Hence, we propose the novel local content-enriched cross-reconstruction network (LCCRN) for few-shot fine-grained classification. In LCCRN, we design two new modules: the local content-enriched module (LCEM) to learn the discriminative local features, and the cross-reconstruction module (CRM) to fully engage the local features with the appearance details obtained from a separate embedding module. The classification score is calculated based on the weighted sum of reconstruction errors of the cross-reconstruction tasks, with weights learnt from the training process. Extensive experiments on four fine-grained datasets showcase the superior classification performance of LCCRN compared with the state-of-the-art few-shot classification methods. Codes are available at: https://github.com/lutsong/LCCRN.

Reconstruction Guided Meta-Learning for Few Shot Open Set Recognition.

In many applications, we are constrained to learn classifiers from very limited data (few-shot classification). The task becomes even more challenging if it is also required to identify samples from unknown categories (open-set classification). Learning a good abstraction for a class with very few samples is extremely difficult, especially under open-set settings. As a result, open-set recognition has received limited attention in the few-shot setting. However, it is a critical task in many applications like environmental monitoring, where the number of labeled examples for each class is limited. Existing few-shot open-set recognition (FSOSR) methods rely on thresholding schemes, with some considering uniform probability for open-class samples. However, this approach is often inaccurate, especially for fine-grained categorization, and makes them highly sensitive to the choice of a threshold. To address these concerns, we propose Reconstructing Exemplar-based Few-shot Open-set ClaSsifier (ReFOCS). By using a novel exemplar reconstruction-based meta-learning strategy ReFOCS streamlines FSOSR eliminating the need for a carefully tuned threshold by learning to be self-aware of the openness of a sample. The exemplars, act as class representatives and can be either provided in the training dataset or estimated in the feature domain. By testing on a wide variety of datasets, we show ReFOCS to outperform multiple state-of-the-art methods.

Chinese Fine-Grained Named Entity Recognition Based on BILTAR and GlobalPointer Modules

The task of fine-grained named entity recognition is to locate entities in text and classify them into predefined fine-grained categories. At present, Chinese fine-grained NER only uses the pretrained language model to encode the characters in the sentence and lacks the ability to extract the deep semantic, sequence, and position information. The sequence annotation method is character-based and lacks the processing of entity boundaries. Fine-grained entity categories have a high degree of similarity, which makes it difficult to distinguish similar categories. To solve the above problems, this paper constructs the BILTAR deep semantic extraction module and adds the GlobalPointer module to improve the accuracy of Chinese fine-grained named entity recognition. The BILTAR module is used to extract deep semantic features from the coding information of pretrained language models and use higher-quality features to improve the model performance. In the GlobalPointer module, the model first adds the rotation position encoding information to the feature vector, using the position information to achieve data enhancement. Finally, the model considers all possible entity boundaries through the GlobalPointer module and calculates the scores for all possible entity boundaries in each category. In this paper, all possible entity boundaries in the text are considered by the above method, and the accuracy of entity recognition is improved. In this paper, the corresponding experiments were carried out on CLUENER 2020 and the micro Chinese fine-grained NER dataset, and the F1 scores of the model in this paper reached 80.848% and 75.751%, respectively. In ablation experiments, the proposed method outperforms the most advanced baseline model and improves the performance of the basic model.

Research on a High-Performance Rock Image Classification Method

Efficient and convenient rock image classification methods are important for geological research. They help in identifying and categorizing rocks based on their physical and chemical properties, which can provide insights into their geological history, origin, and potential uses in various applications. The classification and identification of rocks often rely on experienced and knowledgeable professionals and are less efficient. Fine-grained rock image classification is a challenging task because of the inherent subtle differences between highly confusing categories, which require a large number of data samples and computational resources, resulting in low recognition accuracy, and are difficult to apply in mobile scenarios, requiring the design of a high-performance image processing classification architecture. In this paper we design a knowledge distillation and high-accuracy feature localization comparison network (FPCN)-based learning architecture for generating small high-performance rock image classification models. Specifically, for a pair of images, we interact with the feature vectors generated from the localized feature maps to capture common and unique features, let the network focus on more complementary information according to the different scales of the objects, and then the important features of the images learned in this way are made available for the micro-model to learn the critical information for discrimination via model distillation. The proposed method improves the accuracy of the micro-model by 3%.

Re-abstraction and perturbing support pair network for few-shot fine-grained image classification

The goal of few-shot fine-grained image classification (FSFGIC) is to distinguish subordinate-level categories with subtle visual differences such as the species of bird and models of car with only a few samples. In this work, we argue that a designed network that has the ability to better distinguish feature descriptors of different categories will effectively improve the performance of FSFGIC. We propose a re-abstraction and perturbing support pair network (RaPSPNet) for FSFGIC. Specifically, we first design a feature re-abstraction embedding (FRaE) module which can not only effectively amplify the difference between the feature information from different categories but also better extract the feature information from images. Furthermore, a novel perturbing support pair (PSP) based similarity measure module is designed which evaluates the relationships of feature information among a query image and two different categories of support images (a support pair) at the same time for guiding the designed FRaE module to find salient feature information from the same category of query and support images and find distinguishable feature information from the different categories of query and support images. Extensive experiments on FSFGIC tasks demonstrate the superiority of the proposed methods over state-of-the-art benchmarks.

Fine-grained wetland classification for national wetland reserves using multi-source remote sensing data and Pixel Information Expert Engine (PIE-Engine)

ABSTRACT Timely and accurate wetland information is necessary for wetland resource management. Recent advances in machine learning and remote sensing have facilitated cost-effective monitoring of wetlands. However, reliable methods for fine-grained and rapid wetland mapping are still lacking. To address the issue, a wetland sample set with 20 categories for China was collected based on a sampling strategy that combines automatic sample generation and visual interpretation. Simultaneously, a novel multi-stage method for fine-grained wetland classification was proposed, which integrates pixel-based and object-based strategies using ensemble learning algorithms and multi-source remote sensing data. First, a pixel-based ensemble learning algorithm was implemented to classify five rough wetland categories and six non-wetland categories. Second, an object-based ensemble learning approach was designed to separate the water cover in the pixel-based classification results into eight detailed categories. Third, the merged pixel-based and object-based classification results were refined with knowledge-based post-processing procedures to identify 14 fine-grained wetland categories. Results using the Pixel Information Expert Engine (PIE-Engine) cloud platform proved the effectiveness of the proposed wetland classification method. The overall accuracy, kappa, and weighted F1 reached 87.39%, 82.80%, and 86.02%, respectively. The adopted ensemble learning algorithm yielded better performance than classifiers such as CatBoost, random forest, and XGBoost. The incorporation of spectral, texture, shape, topographic, and geographic features from multi-source data contributed to differentiating wetland categories. According to the relative contribution, spectral indexes (NDVI and NDWI), texture features (sum average and contrast), and topographic features (slope and elevation) were identified as important leading predictors for the first-stage pixel-based classification. Shape features (shape index and compactness) and auxiliary features (geographic location) were crucial predictors for the second-stage object-based classification. Compared with other products, our 10-m wetland mapping results for national wetland reserves were rich in detail and fine in categories. Overall, the constructed sample set and developed classification method show promise in laying a foundation for large-scale wetland mapping. The derived wetland maps can provide support for wetland protection and restoration.

Uncertainty-aware network for fine-grained and imbalanced reflux esophagitis grading

Computer-aided diagnosis (CAD) assists endoscopists in analyzing endoscopic images, reducing misdiagnosis rates and enabling timely treatment. A few studies have focused on CAD for gastroesophageal reflux disease, but CAD studies on reflux esophagitis (RE) are still inadequate. This paper presents a CAD study on RE using a dataset collected from hospital, comprising over 3000 images. We propose an uncertainty-aware network with handcrafted features, utilizing representation and classifier decoupling with metric learning to address class imbalance and achieve fine-grained RE classification. To enhance interpretability, the network estimates uncertainty through test time augmentation. The experimental results demonstrate that the proposed network surpasses previous methods, achieving an accuracy of 90.2% and an F1 score of 90.1%.

Fast online classification of network traffic using new feature-embedded hierarchical structure

The fast online classification (FOC) of network traffic plays a critical role in the network resource management and quality of service support. However, traditional network flow features result in poor performance in FOC (with fewer packets). To tackle the issue, this study proposes two new features: (1) The conditional frequency of packet size (PSize), for which the PSize is quantized into several equal bins and the PSize-level conditional frequency of two consecutive packets is calculated; (2) The statistical feature of rate sequence that is obtained by dividing the inter-arrival time into the PSize sequences. Due to the real-time requirement of online classification, we analyze the time complexity of flow feature calculation and attempt to balance the classification speed and the accuracy in feature selection by reducing the feature dimensionality. In addition, a new feature-embedded hierarchical classification structure is developed for the scenario in which the network video traffic accounts for a relatively large proportion. Fewer packets are used in the early stage of binary classification of non-video vs. video, and then, the subsequent data packets are employed for the fine-grained classification of their respective flows. The effectiveness of the proposed method is evaluated on two real-world network datasets, and our method is compared with the state-of-the-art methods in terms of time performance, resource usage, and classification accuracy. The experimental results confirm the superiority of our approach in fast online classification.

Feature fusion network based on few-shot fine-grained classification.

The objective of few-shot fine-grained learning is to identify subclasses within a primary class using a limited number of labeled samples. However, many current methodologies rely on the metric of singular feature, which is either global or local. In fine-grained image classification tasks, where the inter-class distance is small and the intra-class distance is big, relying on a singular similarity measurement can lead to the omission of either inter-class or intra-class information. We delve into inter-class information through global measures and tap into intra-class information via local measures. In this study, we introduce the Feature Fusion Similarity Network (FFSNet). This model employs global measures to accentuate the differences between classes, while utilizing local measures to consolidate intra-class data. Such an approach enables the model to learn features characterized by enlarge inter-class distances and reduce intra-class distances, even with a limited dataset of fine-grained images. Consequently, this greatly enhances the model's generalization capabilities. Our experimental results demonstrated that the proposed paradigm stands its ground against state-of-the-art models across multiple established fine-grained image benchmark datasets.

2D Shape Detection for Solving Geometry Word Problems

The increasing scale of education has decreased the teacher–student ratio, making it difficult to keep track of student's progress individually. Therefore, teachers are leveraging artificial intelligence tools such as smart tutoring digital tools like learning management systems, and automated solvers for various subjects. Mathematics is one of the most fundamental subjects in school education. This work deals with facilitating the automatic solving of geometry math word problems (MWPs) at the elementary school level. To solve such problems, the information from the text as well as the accompanying figure needs to be interpreted. The preliminary task in solving such problems is to first identify which geometric shape is given in the question. Most existing systems or geometry solvers give the shape as one of the inputs to their systems to solve it further. In this work, the focus is on detecting the shape automatedly and proposing a technique for the same using traditional image processing methods. In addition, further, the triangles and quadrilaterals are classified into fine-grained classifications by taking into account their geometrical properties. Performance metrics like precision, recall, accuracy, and F1-score are used to present the results of this approach. The testing is done on various datasets that are readily available. The results show that on Geometry3K, GeoQA, and GeoS datasets the proposed technique attains an accuracy of 77.8%, 93.38%, and 95.02% and an F1 score of 60%, 71%, and 91.97% respectively. Like all other techniques, our system also has a limitation in that it does not detect all inscribed shapes.

Neighbor self-knowledge distillation

Self-Knowledge Distillation (Self-KD), a technique that enables neural networks to learn from themselves, often relies on auxiliary modules or networks to generate supervisory signals for training. However, this approach incurs significant additional resource costs. Moreover, incorporating auxiliary classifiers within the network architecture creates a capacity mismatch when distilling knowledge from deep to shallow classifiers. This paper proposes a concise and efficient Self-KD method called Neighbor Self-Knowledge Distillation (NSKD), which introduces teacher assistants into the Self-KD by adding auxiliary classifiers to the shallow part of the network to construct distillations of multiple neighboring student-teacher assistant combinations to reduce the mismatch between the students' and teachers' abilities. During distillation, NSKD utilizes only the soft labels generated by each classifier and corresponding ground truth labels as supervisory signals, minimizing resource consumption. NSKD enables neighboring modules to learn from each other through neighboring distillation, enhancing overall network performance. Experimental results on five network models and seven popular datasets demonstrate the superiority of NSKD over other state-of-the-art Self-KD methods. Notably, NSKD achieves average accuracy improvements of 2.26%, 2.32%, and 2.4% on the CIFAR100, TinyImageNet, and fine-grained visual classification datasets.

A dataset for benchmarking Neotropical anuran calls identification in passive acoustic monitoring

Global change is predicted to induce shifts in anuran acoustic behavior, which can be studied through passive acoustic monitoring (PAM). Understanding changes in calling behavior requires automatic identification of anuran species, which is challenging due to the particular characteristics of neotropical soundscapes. In this paper, we introduce a large-scale multi-species dataset of anuran amphibians calls recorded by PAM, that comprises 27 hours of expert annotations for 42 different species from two Brazilian biomes. We provide open access to the dataset, including the raw recordings, experimental setup code, and a benchmark with a baseline model of the fine-grained categorization problem. Additionally, we highlight the challenges of the dataset to encourage machine learning researchers to solve the problem of anuran call identification towards conservation policy. All our experiments and resources have been made available at https://soundclim.github.io/anuraweb/.

Automating Intended Target Identification for Paraphasias in Discourse Using a Large Language Model.

To date, there are no automated tools for the identification and fine-grained classification of paraphasias within discourse, the production of which is the hallmark characteristic of most people with aphasia (PWA). In this work, we fine-tune a large language model (LLM) to automatically predict paraphasia targets in Cinderella story retellings. Data consisted of 332 Cinderella story retellings containing 2,489 paraphasias from PWA, for which research assistants identified their intended targets. We supplemented these training data with 256 sessions from control participants, to which we added 2,415 synthetic paraphasias. We conducted four experiments using different training data configurations to fine-tune the LLM to automatically "fill in the blank" of the paraphasia with a predicted target, given the context of the rest of the story retelling. We tested the experiments' predictions against our human-identified targets and stratified our results by ambiguity of the targets and clinical factors. The model trained on controls and PWA achieved 50.7% accuracy at exactly matching the human-identified target. Fine-tuning on PWA data, with or without controls, led to comparable performance. The model performed better on targets with less human ambiguity and on paraphasias from participants with fluent or less severe aphasia. We were able to automatically identify the intended target of paraphasias in discourse using just the surrounding language about half of the time. These findings take us a step closer to automatic aphasic discourse analysis. In future work, we will incorporate phonological information from the paraphasia to further improve predictive utility. https://doi.org/10.23641/asha.24463543.

A hierarchical fine-grained classification approach for COVID-19 severity assessment based on CT images

The Coronavirus Disease 2019 (COVID-19) is still an ongoing health issue. Appropriate treatment is important for the recovery of COVID-19 patients. At present, numerous deep learning (DL) methods have been applied to the classification of COVID-19 based on computerized tomography (CT) images. However, most of these methods directly extract visual features from the entire CT image, leading to the inclusion of irrelevant background information and making it hard to extract critical features about lesions. Thus, we propose a hierarchical fine-grained classification approach for automatic severity assessment of COVID-19 patients. The CT images are analyzed from coarse to fine in two stages, to focus on the critical features gradually. In the first stage, we detect the lesion regions from the CT image utilizing deep learning based model. In the second stage, we extract critical features from lesion regions and make a severity grading by fine-grained classification model. The experimental results show that the proposed method achieves the classification results of 95.04%, 93.98%, 93.46% and 93.72% in the metrics of accuracy, precision, recall and F1-Score, respectively.

A Lightweight Multimodal Footprint Recognition Network Based on Progressive Multi-Granularity Feature Fusion

The main differences in images of footprints are the proportion of the parts of foot and the distribution of pressure, which can be considered as fine-grained image classification. Moreover, the deviation of human body weight and muscle strength increases the difficulty of identifying the left and right feet. While using a fine-grained image classification network to solve the footprint image classification problem is certainly a feasible approach, the number of parameters in a fine-grained image classification network is generally large, and therefore we would like to build a lightweight classification network that is suitable for several small footprint datasets. In this paper, a multimodal footprint recognition algorithm based on progressive multi-granularity feature fusion is proposed. First, the shallow dense connection network is used to extract features. The feature extraction ability of the model is improved with the help of channel splicing and feature multiplexing. Second, to learn footprint images of different granularities, the progressive training strategy and puzzle scrambler are applied to the model. Finally, factorized bilinear coding can aggregate local features to obtain more discriminative global representation features. Experiments show that our network achieves comparable classification accuracy to some fine-grained image classification models (PMG, MSEC) on the complete pressure footprint dataset, but the number of parameters in our network is greatly reduced. Meanwhile, our network also achieves good classification results on several other footprint datasets, which demonstrates the effectiveness of our network. At the same time, an ablation experiment was carried out to verify the effectiveness of the progressive strategy and the factorized bilinear coding.

SA-RPN: A Spacial Aware Region Proposal Network for Acne Detection.

Automated detection of skin lesions offers excellent potential for interpretative diagnosis and precise treatment of acne vulgar. However, the blurry boundary and small size of lesions make it challenging to detect acne lesions with traditional object detection methods. To better understand the acne detection task, we construct a new benchmark dataset named AcneSCU, consisting of 276 facial images with 31777 instance-level annotations from clinical dermatology. To the best of our knowledge, AcneSCU is the first acne dataset with high-resolution imageries, precise annotations, and fine-grained lesion categories, which enables the comprehensive study of acne detection. More importantly, we propose a novel method called Spatial Aware Region Proposal Network(SA-RPN) to improve the proposal quality of two-stage detection methods. Specifically, the representation learning for the classification and localization task is disentangled with a double head component to promote the proposals for hard samples. Then, Normalized Wasserstein Distance of each proposal is predicted to improve the correlation between the classification scores and the proposals' intersection-over-unions(IoUs). SA-RPN can serve as a plug-and-play module to enhance standard two-stage detectors. Extensive experiments are conducted on both AcneSCU and the public dataset ACNE04, and the results show that the proposed method can consistently outperform state-of-the-art methods. Code and the collected dataset are made available at https://github.com/pingguokiller/acnedetection to stimulate the future research in the acne healthcare community.

Bridge the gap between supervised and unsupervised learning for fine-grained classification

Bridge the gap between supervised and unsupervised learning for fine-grained classification

Novel utilization of a paper-level classification system for the evaluation of journal impact: An update of the CAS Journal Ranking

Abstract Since its first release in 2004, the CAS Journal Ranking, a ranking system of journals based on a citation impact indicator, has been widely used both in selecting journals when submitting manuscripts and in conducting research evaluation in China. This paper introduces an upgraded version of the CAS Journal Ranking released in 2020 and the corresponding improvements. We will discuss the following improvements: a) the CWTS paper-level classification system, a fine-grained classification system utilized for field normalization; b) the Field Normalized Citation Success Index (FNCSI), an indicator that is robust against not only extremely highly cited publications but also wrongly assigned document types; and c) document type difference. In addition, this paper will present part of the ranking results and an interpretation of the features of the FNCSI indicator.

Emotion, Language and Aesthetic Expression: On Motherwell and His Art

Robert Motherwell is regarded as one of the great American abstract expressionists. He was highly intelligent and articulate about his art. In this essay, I explore the thesis that the ability to make fine category discriminations, which can be indexed by language, is necessary to produce great art. I argue that Motherwell might not have been as great an artist if he were not so articulate. Relying on a constructivist view, I argue that fine-grained categories of human emotions can be represented in language; language carves out affective space in a way that makes these states explicit and easier to communicate. Ineffability in art implies exhausting the effable. Being articulate about emotions allows one to reach for higher states of ineffability and aspire to great art.

In-vehicle network intrusion detection systems: a systematic survey of deep learning-based approaches

Developments in connected and autonomous vehicle technologies provide drivers with many convenience and safety benefits. Unfortunately, as connectivity and complexity within vehicles increase, more entry points or interfaces that may directly or indirectly access in-vehicle networks (IVNs) have been introduced, causing a massive rise in security risks. An intrusion detection system (IDS) is a practical method for controlling malicious attacks while guaranteeing real-time communication. Regarding the ever-evolving security attacks on IVNs, researchers have paid more attention to employing deep learning-based techniques to deal with privacy concerns and security threats in the IDS domain. Therefore, this article comprehensively reviews all existing deep IDS approaches on in-vehicle networks and conducts fine-grained classification based on applied deep network architecture. It investigates how deep-learning techniques are utilized to implement different IDS models for better performance and describe their possible contributions and limitations. Further compares and discusses the studied schemes concerning different facets, including input data strategy, benchmark datasets, classification technique, and evaluation criteria. Furthermore, the usage preferences of deep learning in IDS, the influence of the dataset, and the selection of feature segments are discussed to illuminate the main potential properties for designing. Finally, possible research directions for follow-up studies are provided.