Image Classification Research Articles

Assessing interpreter’s disagreements in land cover reference data collection from historical Landsat time series in Amazon

ABSTRACT Land cover information, derived from the classification of Remote Sensing images, is only useful if accompanied by a rigorous accuracy assessment, usually dependent on reference samples. As field data can be very rare for multi-temporal analysis, many studies use samples visually interpreted from Remote Sensing images. This subjective process is prone to errors in interpretation, leading to uncertainties not fully quantified in the reference sample collection of historical time series. We evaluated the agreement rate among independent interpreters in the identification of land cover classes assigned to pixels for yearly Landsat observations from 1984 to 2020, in the Lower Tapajós Basin region (within the Brazilian Amazon). Considering 10 classes, we found that sets of three interpreters completely disagreed on the label of 14.8% of the pixels for each year, and completely agreed in less than half (39.6%). These values change to 2.7% and 68.8% when we consider only five classes in the analysis. We observed time-dependent variations in agreement for specific land cover classes: whereas forest areas tend to present an increase in the disagreement rate over time, land cover types that happen only in previously deforested areas tend to present a higher proportion of pixels with better agreement rates. Disagreements tend to be higher in pixels identified as borders or near borders between targets. However, there is a non-negligible amount of disagreements in pixels identified as not borders caused by limitations derived from the unit of analysis, image composition techniques, and intrinsic similarities among classes. We highlight problems with the identification of classes that happen in complex spatiotemporal patterns (Secondary Vegetation, Small-scale Agriculture, and Degraded Forest). These results alert to possible inconsistencies in reference samples collected for the accuracy assessment of land cover time series and the correspondent values of the calculated accuracy indexes.

Image classification based on few-shot learning algorithms: a review

Image classification is a critical task in the field of computer vision, and its importance has significantly increased over the past few years. Machine learning and deep learning techniques have demonstrated immense potential in this field. However, traditional image classification models require a vast amount of training data, which can be challenging and expensive to obtain. To overcome this limitation, researchers are turning to few-shot learning, which aims to classify images with limited training samples. This paper presents a detailed analysis of the field of image classification using few-shot learning. First, it investigates the use of data augmentation, transfer learning, and meta-learning methods in this field. Then, it introduces several commonly used datasets and evaluation metrics in few-shot classification, compares several classical few-shot classification methods, and summarizes the experimental results obtained from public datasets. Finally, this paper analyzes the current challenges in few-shot image classification and suggests potential future directions.

A Multiobjective Genetic Algorithm to Evolving Local Interpretable Model-Agnostic Explanations for Deep Neural Networks in Image Classification

Deep convolutional neural networks have become a dominant solution for numerous image classification tasks. However, a main criticism is the poor explainability due to the black-box characteristic, which hurdles the extensive usage of deep convolutional neural networks. To address this issue, this paper proposes a new evolutionary multi-objective based method, which aims to explain the behaviours of deep convolutional neural networks by evolving local explanations on specific images. To the best of our knowledge, this is the first evolutionary multi-objective method to evolve local explanations. The proposed method is model-agnostic, i.e. it is applicable to explain any deep convolutional neural networks. ImageNet is used to examine the effectiveness of the proposed method. Three well-known deep convolutional neural networks -VGGNet, ResNet, and MobileNet, are chosen to demonstrate the modelagnostic characteristic. Based on the experimental results, it can be observed that the local explanations are understandable to end-users, who need to check the sensibility of the evolved explanations to decide whether to trust the predictions made by the deep convolutional neural networks. Furthermore, the local explanations evolved by the proposed method improves the confidence of deep convolutional neural networks making the predictions. Lastly, the pareto front and convergence analyses indicate that the proposed method can form a good set of nondominated solutions.

A mutual reconstruction network model for few-shot classification of histological images: addressing interclass similarity and intraclass diversity

A mutual reconstruction network model for few-shot classification of histological images: addressing interclass similarity and intraclass diversity

Tissue Classification After Bone-Anchored Hearing Implant Surgery: A Machine Learning Approach to Monitoring Skin Response.

Here, we aim to 1) expand the available evidence for the use of machine learning techniques for soft tissue classification after BCD surgery and 2) discuss the implications of such approaches toward the development of classification applications to aid in tissue monitoring. The application of machine learning techniques in the soft tissue literature has become a large field of study. One of the most commonly reported outcomes after percutaneous bone-conduction device (BCD) surgery is soft tissue health. Unfortunately, the classification of tissue around the abutment as healthy versus not healthy is a subjective process, even though such decisions can have implications for treatment (i.e., topical steroid versus surgical revision) and resources (e.g., clinician time). We built and tested a convolutional neural network (CNN) model for the classification of tissues that were rated as "green" (i.e., healthy), "yellow" (i.e., unhealthy minor), and "red" (i.e., unhealthy severe). Representative image samples were gathered from a regional bone-conduction amplification site (N = 398; 181 samples of green; 144 samples of yellow; 73 samples of red). The image samples were cropped, zoomed, and normalized. Feature extraction was then implemented and used as the input to train an advanced CNN model. Accuracy of image classification for the healthy ("green") versus not healthy ("yellow" and "red") model was approximately 87%. Accuracy of image classification for the unhealthy ("yellow") versus unhealthy ("red") model was approximately 94%. Monitoring tissue health is an ongoing challenge for BCD users and their clinicians not trained in soft tissue management (e.g., audiologists). If machine learning can aid in the classification of tissue health, this would have significant implications for stakeholders. Here we discuss how machine learning can be applied to tissue classification as a potential technological aid in the coming years.

Hypersensitivity Pneumonitis on Thin-Section Chest CT Scans: Diagnostic Performance of the ATS/JRS/ALAT versus ACCP Imaging Guidelines.

Purpose To compare the diagnostic performance of the American Thoracic Society, Japanese Respiratory Society, and Asociación Latinoamericana del Tórax (ATS/JRS/ALAT) versus the American College of Chest Physicians (ACCP) imaging classifications for hypersensitivity pneumonitis (HP). Materials and Methods Patients in the institutional review board-approved Interstitial Lung Disease (ILD) registry referred for multidisciplinary discussion (MDD) at the authors' institution (January 1, 2006-April 1, 2021) were included in this retrospective study when ILD was diagnosed at MDD. MDD diagnoses included HP, connective tissue disease-ILD, and idiopathic pulmonary fibrosis. Retrospective review of thin-section CT images was performed in consensus by two cardiothoracic radiologists blinded to the diagnosis. Diagnostic patterns were determined for thin-section CT images using both classifications. Discordance rates were determined. Sensitivity, specificity, positive predictive value, negative predictive value, and accuracy were assessed using MDD diagnosis as the reference standard. Results A total of 297 patients were included in the study: 200 (67%) with HP, 49 (16%) with connective tissue disease-ILD, and 48 (16%) with idiopathic pulmonary fibrosis at MDD. The discordance rate between the two classifications was 21%. Assuming low HP prevalence (10%), ATS/JRS/ALAT classification outperformed ACCP classification, with greater accuracy (92.3% vs 87.6%) and greater positive predictive value (60.7% vs 42.9%). Assuming high prevalence (50%), accuracy and negative predictive value were superior using ACCP classification (81.7% vs 79.7% and 77.7% vs 72.6%, respectively), and positive predictive value was superior using ATS/JRS/ALAT classification (93.3% vs 87.1%). Conclusion Accuracy of the ATS/JRS/ALAT and ACCP HP classifications was greater in settings with low and high HP prevalence, respectively. Diagnostic performance of both classifications was discordant in a minority of cases. Keywords: CT, Thorax, Hypersensitivity Pneumonitis, Interstitial Lung Disease Supplemental material is available for this article. © RSNA, 2024.

Deep Feature Segmentation Model Driven by Hybrid Convolution Network for Hyper Spectral Image Classification

Deep Feature Segmentation Model Driven by Hybrid Convolution Network for Hyper Spectral Image Classification

SI-GAT: Enhancing Side-Scan Sonar Image Classification Based on Graph Structure

SI-GAT: Enhancing Side-Scan Sonar Image Classification Based on Graph Structure

Evaluating the efficacy of few-shot learning for GPT-4Vision in neurodegenerative disease histopathology: A comparative analysis with convolutional neural network model.

Recent advances in artificial intelligence, particularly with large language models like GPT-4Vision (GPT-4V)-a derivative feature of ChatGPT-have expanded the potential for medical image interpretation. This study evaluates the accuracy of GPT-4V in image classification tasks of histopathological images and compares its performance with a traditional convolutional neural network (CNN). We utilised 1520 images, including haematoxylin and eosin staining and tau immunohistochemistry, from patients with various neurodegenerative diseases, such as Alzheimer's disease (AD), progressive supranuclear palsy (PSP) and corticobasal degeneration (CBD). We assessed GPT-4V's performance using multi-step prompts to determine how textual context influences image interpretation. We also employed few-shot learning to enhance improvements in GPT-4V's diagnostic performance in classifying three specific tau lesions-astrocytic plaques, neuritic plaques and tufted astrocytes-and compared the outcomes with the CNN model YOLOv8. GPT-4V accurately recognised staining techniques and tissue origin but struggled with specific lesion identification. The interpretation of images was notably influenced by the provided textual context, which sometimes led to diagnostic inaccuracies. For instance, when presented with images of the motor cortex, the diagnosis shifted inappropriately from AD to CBD or PSP. However, few-shot learning markedly improved GPT-4V's diagnostic capabilities, enhancing accuracy from 40% in zero-shot learning to 90% with 20-shot learning, matching the performance of YOLOv8, which required 100-shot learning to achieve the same accuracy. Although GPT-4V faces challenges in independently interpreting histopathological images, few-shot learning significantly improves its performance. This approach is especially promising for neuropathology, where acquiring extensive labelled datasets is often challenging.

A review of federated learning algorithms in image classification

Image classification is one of the most popular applications of machine learning. It has shown its potential in fields like healthcare, auto-driving and face recognition. Federated learning (FL) emerged in 2017, creating a major innovation to the field. The new structure brings new possibilities, but create new challenges such as data heterogeneity, privacy leakage and communication burdens in parameter updating. The paper solves the problem that there are relatively few papers providing a complete analysis relating to the application of FL in image classification. The paper contributes to describe the new challenges of FL in image classification and show the related reasons behind respectively, then analyze the current state-of-the-art algorithms designed for solving the challenges and improving image model performance by discussing the basic ideas and steps of algorithms and showing their pros and cons. The paper further more contributes to compare the performance of each model in accuracy and communication speed, and outline several possible directions for future advancement.

Classification of Microorganism Using Convolutional Neural Network and H2O AutoML

The study advances microorganism image classification through a hybrid approach that integrates a Convolutional Neural Network (CNN), modified from the VGG19 architecture, with an ensemble model powered by H2O AutoML. Employing data augmentation and feature extraction, the approach enhances performance on a dataset encapsulating a broad spectrum of microorganism classes. The CNN model shows significant accuracy enhancements in complex bacteria classes, as depicted by the confusion matrix. Concurrently, the AutoML ensemble delivers comparable accuracy, notably in some classes where CNNs struggles. This research highlights the complementary strengths of deep learning and AutoML, demonstrating their impact in achieving high-precision microorganism recognition. Such advancements promise to significantly benefit bioinformatics and diagnostic applications, addressing the complexity of multi-class image classification tasks. The results indicate a successful combination of CNN and AutoML methodologies, setting a benchmark in automated microorganism classification, and also showcase the unique contributions and nuances of each method.

Optimization of semi-supervised generative adversarial network models: a survey

PurposeWith the development of intelligent technology, deep learning has made significant progress and has been widely used in various fields. Deep learning is data-driven, and its training process requires a large amount of data to improve model performance. However, labeled data is expensive and not readily available.Design/methodology/approachTo address the above problem, researchers have integrated semi-supervised and deep learning, using a limited number of labeled data and many unlabeled data to train models. In this paper, Generative Adversarial Networks (GANs) are analyzed as an entry point. Firstly, we discuss the current research on GANs in image super-resolution applications, including supervised, unsupervised, and semi-supervised learning approaches. Secondly, based on semi-supervised learning, different optimization methods are introduced as an example of image classification. Eventually, experimental comparisons and analyses of existing semi-supervised optimization methods based on GANs will be performed.FindingsFollowing the analysis of the selected studies, we summarize the problems that existed during the research process and propose future research directions.Originality/valueThis paper reviews and analyzes research on generative adversarial networks for image super-resolution and classification from various learning approaches. The comparative analysis of experimental results on current semi-supervised GAN optimizations is performed to provide a reference for further research.

Effective type label-based synergistic representation learning for biomedical event trigger detection

BackgroundDetecting event triggers in biomedical texts, which contain domain knowledge and context-dependent terms, is more challenging than in general-domain texts. Most state-of-the-art models rely mainly on external resources such as linguistic tools and knowledge bases to improve system performance. However, they lack effective mechanisms to obtain semantic clues from label specification and sentence context. Given its success in image classification, label representation learning is a promising approach to enhancing biomedical event trigger detection models by leveraging the rich semantics of pre-defined event type labels.ResultsIn this paper, we propose the Biomedical Label-based Synergistic representation Learning (BioLSL) model, which effectively utilizes event type labels by learning their correlation with trigger words and enriches the representation contextually. The BioLSL model consists of three modules. Firstly, the Domain-specific Joint Encoding module employs a transformer-based, domain-specific pre-trained architecture to jointly encode input sentences and pre-defined event type labels. Secondly, the Label-based Synergistic Representation Learning module learns the semantic relationships between input texts and event type labels, and generates a Label-Trigger Aware Representation (LTAR) and a Label-Context Aware Representation (LCAR) for enhanced semantic representations. Finally, the Trigger Classification module makes structured predictions, where each label is predicted with respect to its neighbours. We conduct experiments on three benchmark BioNLP datasets, namely MLEE, GE09, and GE11, to evaluate our proposed BioLSL model. Results show that BioLSL has achieved state-of-the-art performance, outperforming the baseline models.ConclusionsThe proposed BioLSL model demonstrates good performance for biomedical event trigger detection without using any external resources. This suggests that label representation learning and context-aware enhancement are promising directions for improving the task. The key enhancement is that BioLSL effectively learns to construct semantic linkages between the event mentions and type labels, which provide the latent information of label-trigger and label-context relationships in biomedical texts. Moreover, additional experiments on BioLSL show that it performs exceptionally well with limited training data under the data-scarce scenarios.

Dune Morphology Classification and Dataset Construction Method Based on Unmanned Aerial Vehicle Orthoimagery

Dunes are the primary geomorphological type in deserts, and the distribution of dune morphologies is of significant importance for studying regional characteristics, formation mechanisms, and evolutionary processes. Traditional dune morphology classification methods rely on visual interpretation by humans, which is not only time-consuming and inefficient but also subjective in classification judgment. These issues have impeded the intelligent development of dune morphology classification. However, convolutional neural network (CNN) models exhibit robust feature representation capabilities for images and have achieved excellent results in image classification, providing a new method for studying dune morphology classification. Therefore, this paper summarizes five typical dune morphologies in the deserts of western Inner Mongolia, which can be used to define and describe most of the dune types in Chinese deserts. Subsequently, field surveys and the experimental collection of unmanned aerial vehicle (UAV) orthoimages for different dune types were conducted. Five different types of dune morphology datasets were constructed through manual segmentation, automatic rule segmentation, random screening, and data augmentation. Finally, the classification of dune morphologies and the exploration of dataset construction methods were conducted using the VGG16 and VGG19 CNN models. The classification results of dune morphologies were comprehensively analyzed using different evaluation metrics. The experimental results indicate that when the regular segmentation scale of UAV orthoimages is 1024 × 1024 pixels with an overlap of 100 pixels, the classification accuracy, precision, recall, and F1-Score of the VGG16 model reached 97.05%, 96.91%, 96.76%, and 96.82%, respectively. The method for constructing a dune morphology dataset from automatically segmented UAV orthoimages provides a reference value for the study of large-scale dune morphology classification.

Research on the performance of hybrid vision models based on ViT

ViT is a model proposed by the Google team in 2020 to apply a transformer in image classification, although it is not the first paper to apply a transformer in visual tasks, because of its model is simple and effective, and scalable, it has become a milestone work in the application of transformer in CV field, and has also triggered the subsequent related research. The core conclusion of the original ViT paper is that when there is enough data for pre-training, ViT outperforms CNN, breaks through the limitation of the transformer's lack of inductive bias, and can achieve better migration results in downstream tasks. However, when the training dataset is not large enough, ViT usually performs worse than ResNets of the same size, because Transformer lacks inductive bias, a kind of a priori knowledge, assumptions made in advance, compared with CNN. Through its innovative architecture and powerful performance, the visual representation transform (ViT) model continues to advance the field of computer vision, while facing some challenges and room for improvement. With the deepening of research and the continuous development of technology, ViT is expected to play a greater role in more practical applications. The article aims to explore the advantages and applicability of the ViT model and tries to construct a hybrid visual model to improve its generalization ability for different types of datasets, demonstrating the hybrid model's significance in improving the performance of the ViT model.

UniMiSS+: Universal Medical Self-Supervised Learning From Cross-Dimensional Unpaired Data.

Self-supervised learning (SSL) opens up huge opportunities for medical image analysis that is well known for its lack of annotations. However, aggregating massive (unlabeled) 3D medical images like computerized tomography (CT) remains challenging due to its high imaging cost and privacy restrictions. In our pilot study, we advocated bringing a wealth of 2D images like chest X-rays as compensation for the lack of 3D data, aiming to build a universal medical self-supervised representation learning framework, called UniMiSS. Especially, we designed a pyramid U- like medical Transformer (MiT) as the backbone to make UniMiSS possible to perform SSL with both 2D and 3D images. Consequently, the predecessor UniMiSS has two obvious merits compared to current 3D-specific SSL: (1) more effective - superior to learning strong representations, benefiting from more and diverse data; and (2) more versatile - suitable for various downstream tasks without the restriction on the dimensionality barrier. Unfortunately, UniMiSS did not dig deeply into the intrinsic anatomy correlation between 2D medical images and 3D volumes due to the lack of paired multi-modal/dimension patient data. In this extension paper, we propose the UniMiSS+, in which we introduce the digitally reconstructed radiographs (DRR) technology to simulate X-ray images from a CT volume to access paired CT and X-ray data. Benefiting from the paired group, we introduce an extra pair- wise constraint to boost the cross-modality correlation learning, which also can be adopted as a cross-dimension regularization to further improve the representations. We conduct expensive experiments on multiple 3D/2D medical image analysis tasks, including segmentation and classification. The results show that the proposed UniMiSS+ achieves promising performance on various downstream tasks, not only outperforming the ImageNet pre-training and other advanced SSL counterparts substantially but also improving the predecessor UniMiSS pre-training. Code is available at: https://github.com/YtongXie/UniMiSS-code.

CLIB: Contrastive learning of ignoring background for underwater fish image classification

Aiming at the problem that the existing methods are insufficient in dealing with the background noise anti-interference of underwater fish images, a contrastive learning method of ignoring background called CLIB for underwater fish image classification is proposed to improve the accuracy and robustness of underwater fish image classification. First, CLIB effectively separates the subject from the background in the image through the extraction module and applies it to contrastive learning by composing three complementary views with the original image. To further improve the adaptive ability of CLIB in complex underwater images, we propose a multi-view-based contrastive loss function, whose core idea is to enhance the similarity between the original image and the subject and maximize the difference between the subject and the background, making CLIB focus more on learning the core features of the subject during the training process, and effectively ignoring the interference of background noise. Experiments on the Fish4Knowledge, Fish-gres, WildFish-30, and QUTFish-89 public datasets show that our method performs well, with improvements of 1.43–6.75%, 8.16–8.95%, 13.1–14.82%, and 3.92–6.19%, respectively, compared with the baseline model, further validating the effectiveness of CLIB.

Border-Enhanced Triple Attention Mechanism for High-Resolution Remote Sensing Images and Application to Land Cover Classification

With the continuous development and popularization of remote sensing technology, remote sensing images have been widely used in the field of land cover classification. Since remote sensing images have complex spatial structure and texture features, it is becoming a challenging problem to accurately categorize them. Land cover classification has practical application value in various fields, such as environmental monitoring and protection, urban and rural planning and management, and climate change research. In recent years, remote sensing image classification methods based on deep learning have been rapidly developed, in which semantic segmentation technology has become one of the mainstream methods for land cover classification using remote sensing image. Traditional semantic segmentation algorithms tend to ignore the edge information, resulting in poor classification of the edge part in land cover classification, and there are numerous attention mechanisms to make improvements for these problems. In this paper, a triple attention mechanism, BETAM (Border-Enhanced Triple Attention Mechanism), for edge feature enhancement of high-resolution remote sensing images is proposed. Furthermore, a new model on the basis of the semantic segmentation network model DeeplabV3+ is also introduced, which is called DeepBETAM. The triple attention mechanism BETAM is able to capture feature dependencies in three dimensions: position, space, and channel, respectively. Through feature importance weighting, modeling of spatial relationships, and adaptive learning capabilities, the model BETAM pays more attention to edge features, thus improving the accuracy of edge detection. A remote sensing image dataset SMCD (Subject Meticulous Categorization Dataset) is constructed to verify the robustness of the attention mechanism BETAM and the model DeepBETAM. Extensive experiments were conducted on the two self-built datasets FRSID and SMCD. Experimental results showed that the mean Intersection over Union (mIoU), mean Pixel Accuracy (mPA), and mean Recall (mRecall) of DeepBETAM are 63.64%, 71.27%, and 71.31%, respectively. These metrics are superior to DeeplabV3+, DeeplabV3+(SENet), DeeplabV3+(CBAM), DeeplabV3+(SAM), DeeplabV3+(ECANet), and DeeplabV3+(CAM), which are network models that incorporate different attention mechanisms. The reason is that BETAM has better edge segmentation results and segmentation accuracy. Meanwhile, on the basis of the self-built dataset, the four main classifications of buildings, cultivated land, water bodies and vegetation were subdivided and detected, and good experimental results were obtained, which verified the robustness of the attention mechanism BETAM and the model DeepBETAM. The method has broad application prospects and can provide favorable support for research and application in the field of surface classification.

Breast cancer histopathology image classification using an ensemble of optimized pretrained models with a trainable ensemble strategy classifier

Breast cancer histopathology image classification using an ensemble of optimized pretrained models with a trainable ensemble strategy classifier

Exploring Distributed Scatterers Interferometric Synthetic Aperture Radar Attributes for Synthetic Aperture Radar Image Classification

Land cover classification of Synthetic Aperture Radar (SAR) imagery is a significant research direction in SAR image interpretation. However, due to the unique imaging methodology of SAR, interpreting SAR images presents numerous challenges, and land cover classification using SAR imagery often lacks innovative features. Distributed scatterers interferometric synthetic aperture radar (DS-InSAR), a common technique for deformation extraction, generates several intermediate parameters during its processing, which have a close relationship with land features. Therefore, this paper utilizes the coherence matrix, the number of statistically homogeneous pixels (SHPs), and ensemble coherence, which are involved in DS-InSAR as classification features, combined with the backscatter intensity of multi-temporal SAR imagery, to explore the impact of these features on the discernibility of land objects in SAR images. The results indicate that the adopted features improve the accuracy of land cover classification. SHPs and ensemble coherence demonstrate significant importance in distinguishing land features, proving that these proposed features can serve as new attributes for land cover classification in SAR imagery.