Image Classification Framework Research Articles

Introspective Deep Metric Learning.

This paper proposes an introspective deep metric learning (IDML) framework for uncertainty-aware comparisons of images. Conventional deep metric learning methods focus on learning a discriminative embedding to describe the semantic features of images, which ignore the existence of uncertainty in each image resulting from noise or semantic ambiguity. Training without awareness of these uncertainties causes the model to overfit the annotated labels during training and produce overconfident judgments during inference. Motivated by this, we argue that a good similarity model should consider the semantic discrepancies with awareness of the uncertainty to better deal with ambiguous images for more robust training. To achieve this, we propose to represent an image using not only a semantic embedding but also an accompanying uncertainty embedding, which describes the semantic characteristics and ambiguity of an image, respectively. We further propose an introspective similarity metric to make similarity judgments between images considering both their semantic differences and ambiguities. The gradient analysis of the proposed metric shows that it enables the model to learn at an adaptive and slower pace to deal with the uncertainty during training. Our framework attains state-of-the-art performance on the widely used CUB-200-2011, Cars196, and Stanford Online Products datasets for image retrieval. We further evaluate our framework for image classification on the ImageNet-1 K, CIFAR-10, and CIFAR-100 datasets, which shows that equipping existing data mixing methods with the proposed introspective metric consistently achieves better results (e.g., +0.44% for CutMix on ImageNet-1 K).

Transformer Driven Matching Selection Mechanism for Multi-Label Image Classification

Graph Matching has recently emerged as an attractive technique applied to various computer vision tasks. Graph Matching based multi-label image classification, in particular, treats each image as a bag of instances and reformulates the classification task as an instance-label matching selection problem, achieving state-of-the-art results on diverse benchmarks. However, the generalization and scalability of such learned model cannot be well guaranteed due to its manually predetermined graph structure and high-dimension embedding of dense connections between instances and labels. To address these limitations, in this work, we propose a novel Transformer Driven Matching Selection framework for Multi-Label Image Classification (C-TMS), where instance structural relationships, class-wise global dependencies, and the co-occurrence possibility of varying instance-label assignments are simultaneously taken into consideration in a unified and adaptive manner. Moreover, the parallelization capability of the Transformer enables efficient computation, making our model scalable to large-scale datasets. Specifically, we first represent instances and labels as nodes in the visual space and label space respectively, and then compute the hidden representation of each node in its individual space, by attending a self-attention strategy over its entire neighborhood. Subsequently, the cross-attention is adopted to excavate the correct assignments between instances and labels, and further interprets how classifying each label depends on the instances within an image and its interaction with other labels. Finally, an asymmetric focal loss is designed to optimize the instance-label correspondence, and read out image-level category confidences. Extensive experiments conducted on various multi-label image datasets demonstrate the superiority of our proposed method.

Automatic Identification and Severity Classification of Retinal Biomarkers in SD-OCT Using Dilated Depthwise Separable Convolution ResNet with SVM Classifier

Purpose Diagnosis of Uveitic Macular Edema (UME) using Spectral Domain OCT (SD-OCT) is a promising method for early detection and monitoring of sight-threatening visual impairment. Viewing multiple B-scans and identifying biomarkers is challenging and time-consuming for clinical practitioners. To overcome these challenges, this paper proposes an image classification hybrid framework for predicting the presence of biomarkers such as intraretinal cysts (IRC), hyperreflective foci (HRF), hard exudates (HE) and neurosensory detachment (NSD) in OCT B-scans along with their severity. Methods A dataset of 10880 B-scans from 85 Uveitic patients is collected and graded by two board-certified ophthalmologists for the presence of biomarkers. A novel image classification framework, Dilated Depthwise Separable Convolution ResNet (DDSC-RN) with SVM classifier, is developed to achieve network compression with a larger receptive field that captures both low and high-level features of the biomarkers without loss of classification accuracy. The severity level of each biomarker is predicted from the feature map, extracted by the proposed DDSC-RN network. Results The proposed hybrid model is evaluated using ground truth labels from the hospital. The deep learning model initially, identified the presence of biomarkers in B-scans. It achieved an overall accuracy of 98.64%, which is comparable to the performance of other state-of-the-art models, such as DRN-C-42 and ResNet-34. The SVM classifier then predicted the severity of each biomarker, achieving an overall accuracy of 89.3%. Conclusions A new hybrid model accurately identifies four retinal biomarkers on a tissue map and predicts their severity. The model outperforms other methods for identifying multiple biomarkers in complex OCT B-scans. This helps clinicians to screen multiple B-scans of UME more effectively, leading to better treatment outcomes.

CMAF: Cross-Modal Augmentation via Fusion for Underwater Acoustic Image Recognition

Underwater image recognition is crucial for underwater detection applications. Fish classification has been one of the emerging research areas in recent years. Existing image classification models usually classify data collected from terrestrial environments. However, existing image classification models trained with terrestrial data are unsuitable for underwater images, as identifying underwater data is challenging due to their incomplete and noisy features. To address this, we propose a cross-modal augmentation via fusion ( CMAF ) framework for acoustic-based fish image classification. Our approach involves separating the process into two branches: visual modality and sonar signal modality, where the latter provides a complementary character feature. We augment the visual modality, design an attention-based fusion module, and adopt a masking-based training strategy with a mask-based focal loss to improve the learning of local features and address the class imbalance problem. Our proposed method outperforms the state-of-the-art methods. Our source code is available at https://github.com/WilkinsYang/CMAF .

DDCNN-F: double decker convolutional neural network 'F' feature fusion as a medical image classification framework

Melanoma is a severe skin cancer that involves abnormal cell development. This study aims to provide a new feature fusion framework for melanoma classification that includes a novel ‘F’ Flag feature for early detection. This novel ‘F’ indicator efficiently distinguishes benign skin lesions from malignant ones known as melanoma. The article proposes an architecture that is built in a Double Decker Convolutional Neural Network called DDCNN future fusion. The network's deck one, known as a Convolutional Neural Network (CNN), finds difficult-to-classify hairy images using a confidence factor termed the intra-class variance score. These hirsute image samples are combined to form a Baseline Separated Channel (BSC). By eliminating hair and using data augmentation techniques, the BSC is ready for analysis. The network's second deck trains the pre-processed BSC and generates bottleneck features. The bottleneck features are merged with features generated from the ABCDE clinical bio indicators to promote classification accuracy. Different types of classifiers are fed to the resulting hybrid fused features with the novel 'F' Flag feature. The proposed system was trained using the ISIC 2019 and ISIC 2020 datasets to assess its performance. The empirical findings expose that the DDCNN feature fusion strategy for exposing malignant melanoma achieved a specificity of 98.4%, accuracy of 93.75%, precision of 98.56%, and Area Under Curve (AUC) value of 0.98. This study proposes a novel approach that can accurately identify and diagnose fatal skin cancer and outperform other state-of-the-art techniques, which is attributed to the DDCNN ‘F’ Feature fusion framework. Also, this research ascertained improvements in several classifiers when utilising the ‘F’ indicator, resulting in the highest specificity of + 7.34%.

Attention Feature Fusion Network via Knowledge Propagation for Automated Respiratory Sound Classification.

Goal: In light of the COVID-19 pandemic, the early diagnosis of respiratory diseases has become increasingly crucial. Traditional diagnostic methods such as computed tomography (CT) and magnetic resonance imaging (MRI), while accurate, often face accessibility challenges. Lung auscultation, a simpler alternative, is subjective and highly dependent on the clinician's expertise. The pandemic has further exacerbated these challenges by restricting face-to-face consultations. This study aims to overcome these limitations by developing an automated respiratory sound classification system using deep learning, facilitating remote and accurate diagnoses. Methods: We developed a deep convolutional neural network (CNN) model that utilizes spectrographic representations of respiratory sounds within an image classification framework. Our model is enhanced with attention feature fusion of low-to-high-level information based on a knowledge propagation mechanism to increase classification effectiveness. This novel approach was evaluated using the ICBHI benchmark dataset and a larger, self-collected Pediatric dataset comprising outpatient children aged 1 to 6 years. Results: The proposed CNN model with knowledge propagation demonstrated superior performance compared to existing state-of-the-art models. Specifically, our model showed higher sensitivity in detecting abnormalities in the Pediatric dataset, indicating its potential for improving the accuracy of respiratory disease diagnosis. Conclusions: The integration of a knowledge propagation mechanism into a CNN model marks a significant advancement in the field of automated diagnosis of respiratory disease. This study paves the way for more accessible and precise healthcare solutions, which is especially crucial in pandemic scenarios.

S2HM2: A Spectral–Spatial Hierarchical Masked Modeling Framework for Self-Supervised Feature Learning and Classification of Large-Scale Hyperspectral Images

S²HM²: A Spectral–Spatial Hierarchical Masked Modeling Framework for Self-Supervised Feature Learning and Classification of Large-Scale Hyperspectral Images

CoD-MIL: Chain-of-Diagnosis Prompting Multiple Instance Learning for Whole Slide Image Classification.

Multiple instance learning (MIL) has emerged as a prominent paradigm for processing the whole slide image with pyramid structure and giga-pixel size in digital pathology. However, existing attention-based MIL methods are primarily trained on the image modality and a pre-defined label set, leading to limited generalization and interpretability. Recently, vision language models (VLM) have achieved promising performance and transferability, offering potential solutions to the limitations of MIL-based methods. Pathological diagnosis is an intricate process that requires pathologists to examine the WSI step-by-step. In the field of natural language process, the chain-of-thought (CoT) prompting method is widely utilized to imitate the human reasoning process. Inspired by the CoT prompt and pathologists' clinic knowledge, we propose a chain-of-diagnosis prompting multiple instance learning (CoD-MIL) framework for whole slide image classification. Specifically, the chain-of-diagnosis text prompt decomposes the complex diagnostic process in WSI into progressive sub-processes from low to high magnification. Additionally, we propose a text-guided contrastive masking module to accurately localize the tumor region by masking the most discriminative instances and introducing the guidance of normal tissue texts in a contrastive way. Extensive experiments conducted on three real-world subtyping datasets demonstrate the effectiveness and superiority of CoD-MIL.

CMAAC: Combining Multiattention and Asymmetric Convolution Global Learning Framework for Hyperspectral Image Classification

CMAAC: Combining Multiattention and Asymmetric Convolution Global Learning Framework for Hyperspectral Image Classification

Prompt-driven Latent Domain Generalization for Medical Image Classification.

Deep learning models for medical image analysis easily suffer from distribution shifts caused by dataset artifact bias, camera variations, differences in the imaging station, etc., leading to unreliable diagnoses in real-world clinical settings. Domain generalization (DG) methods, which aim to train models on multiple domains to perform well on unseen domains, offer a promising direction to solve the problem. However, existing DG methods assume domain labels of each image are available and accurate, which is typically feasible for only a limited number of medical datasets. To address these challenges, we propose a unified DG framework for medical image classification without relying on domain labels, called Prompt-driven Latent Domain Generalization (PLDG). PLDG consists of unsupervised domain discovery and prompt learning. This framework first discovers pseudo domain labels by clustering the bias-associated style features, then leverages collaborative domain prompts to guide a Vision Transformer to learn knowledge from discovered diverse domains. To facilitate cross-domain knowledge learning between different prompts, we introduce a domain prompt generator that enables knowledge sharing between domain prompts and a shared prompt. A domain mixup strategy is additionally employed for more flexible decision margins and mitigates the risk of incorrect domain assignments. Extensive experiments on three medical image classification tasks and one debiasing task demonstrate that our method can achieve comparable or even superior performance than conventional DG algorithms without relying on domain labels. Our code is publicly available at https://github.com/SiyuanYan1/PLDG/tree/main.

RSSL: A Unified Reliability-Aware Semi-Supervised Learning Framework for Medical Image Classification

RSSL: A Unified Reliability-Aware Semi-Supervised Learning Framework for Medical Image Classification

PolSAR Image Classification Framework With POA Align and Cyclic Channel Attention

PolSAR Image Classification Framework With POA Align and Cyclic Channel Attention

Hyperspectral Image Classification Framework Based on Multichannel Graph Convolutional Networks and Class-Guided Attention Mechanism

Hyperspectral Image Classification Framework Based on Multichannel Graph Convolutional Networks and Class-Guided Attention Mechanism

Dual Discriminative Low-Rank Projection Learning for Robust Image Classification

Numerous methods have exploited projection learning to extract low-dimensional features for image classification. Some projection learning methods integrate low-rank matrix recovery into classification models to equip the projection subspace with discrimination and robustness against corruption. However, these methods cannot directly recover “clean” components from the new datum in a low-dimensional subspace. Additionally, they are sensitive to the selection of projection dimensions. To overcome these shortcomings, we propose a dual discriminative low-rank projection learning framework for robust image classification. Specifically, the proposed method learns a low-rank projection and a semi-orthogonal projection to recover “clean” components from the original data and simultaneously obtain a low-dimensional subspace. Thereafter, to preserve discriminative information in the low-dimensional subspace, an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">L</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2,1</sub> -norm term is constructed by concentrating the projected intra-class samples around their adaptive class centroids. Regression-based terms are appended using the low-dimensional features extracted from the recovered clean data and the class centroids for more accurate classification. Experiments on five public databases with various corruptions demonstrate that the proposed method can robustly classify image data despite a small training sample sizes and gross corruption. The superiority of the proposed method is further verified on the large-scale PubFig83 database, on which it achieves an 87.58% classification accuracy.

A Dual-Attention Deep Discriminative Domain Generalization Model for Hyperspectral Image Classification

Recently, hyperspectral image classification has made great progress with the development of convolutional neural networks. However, due to the challenges of distribution shifts and data redundancies, the classification accuracy is low. Some existing domain adaptation methods try to mitigate the distribution shifts by training source samples and some labeled target samples. However, in practice, labeled target domain samples are difficult or even impossible to obtain. To solve the above challenges, we propose a novel dual-attention deep discriminative domain generalization framework (DAD3GM) for cross-scene hyperspectral image classification without training the labeled target samples. In DAD3GM, we mainly design two blocks: dual-attention feature learning (DAFL) and deep discriminative feature learning (DDFL). DAFL is designed to extract spatial features by multi-scale self-attention and extract spectral features by multi-head external attention. DDFL is further designed to extract deep discriminative features by contrastive regularization and class discrimination regularization. The combination of DAFL and DDFL can effectively reduce the computational time and improve the generalization performance of DAD3GM. The proposed model achieves 84.25%, 83.53%, and 80.63% overall accuracy on the public Houston, Pavia, and GID benchmarks, respectively. Compared with some classical and state-of-the-art methods, the proposed model achieves optimal results, which reveals its effectiveness and feasibility.

Correction to: A trustworthy neural architecture search framework for pneumonia image classification utilizing blockchain technology

Correction to: A trustworthy neural architecture search framework for pneumonia image classification utilizing blockchain technology

Cross Modality Feature Fusion Framework for Diabetic Retinopathy Image Classification

Diabetic retinopathy (DR) is a prevalent ocular condition and a prominent contributor to vision impairment in individuals with diabetes. Consistent monitoring through fundus photography and prompt intervention represents the most efficient strategy for controlling this ailment. Given the substantial diabetic patient population and their extensive screening needs, there is a growing inclination towards harnessing computer-aided and entirely automated methods for diagnosing DR. Over the past years, deep neural networks have achieved remarkable progress across a wide range of applications. Consequently, automating the diagnosis of DR and delivering tailored recommendations to DR patients underscores the significance of accurate and intricate DR classification. In this work, we have present a cross modality feature fusion based framework for diabetic retinopathy (DR) image classification. Here, cross modality means RGB image and it’s green channel. Initially, we have present multi-scale multi- receptive feature extraction block to learn the local and global features from both the modalities. Further, the learned features at various scale are fused effectively with present multi-level feature fusion block for image classification task. We evaluated our present framework on MESSIDOR and IDRID database by comparing it to state-of-the-art (SOTA) deep learning frame- works for DR image classification. The result analysis clearly demonstrate that the present cross-modality feature fusion based classification framework outperforms existing SOTA frameworks in terms of various evaluation parameters.

Solution for sports image classification using modified MobileNetV3 optimized by modified battle royal optimization algorithm

Sports image classification using image processing and machine vision is a growing area of research that involves the use of algorithms and techniques to identify and analyze objects in sports images and videos. This technology has a wide range of applications, including detecting illegal plays, analyzing team performance, and creating highlight reels. Additionally, it can provide valuable visual feedback during training and competition. In this paper, we propose a novel deep learning and optimization hybrid framework for sports image classification. Specifically, we use a modified version of the Battle Royal optimization algorithm as a feature selector to reduce the dimensionality of the images and achieve higher accuracy with only the essential features. We evaluate the proposed framework using sports images and demonstrate that our WOA-based framework outperforms other methods in terms of both classification accuracy and dimensionality reduction. Our results highlight the effectiveness of the proposed approach and its potential to improve sports image classification.

An efficient PDE framework for satellite image classification

In this paper, we present a method based on coupling between shock filter and diffusion process for image restoration. This method smoothes homogeneous part of the image, while it enhances edges efficiently. This selective smoothing propriety is used as a pretreatment step for an efficient classification of infrared satellite images, in order to cluster the solar field.

A novel self‐supervised ensemble learning framework for land use and land cover classification of polarimetric synthetic aperture radar images

AbstractClassification with a few samples of training set has been a longstanding issue in the field of polarimetric synthetic aperture radar (PolSAR) image analysis and processing. Aiming at the small number of training samples of the PolSAR image classification task, a novel Self‐Supervised Ensemble Learning Framework (SSELF) is designed. The designed SSELF can automatically extract PolSAR features conducive to PolSAR image classification with a small number of training samples. In addition, it can significantly decrease the dependence of neural network algorithms on large labelled samples of training set. First, utilise the spatial–polarimetric features of PolSAR data perfectly, the EfficientNet‐B0 is presented and utilised as the main section of the Deep Learning (DL) model to extract DL features of PolSAR data. Then, using an optimisation function that constrains the cross‐correlation matrix of various distortions of each sample to the identity matrix, the designed DL model can obtain the effective features of homogeneous samples gathering and heterogeneous samples separating from each other in a self‐supervised manner. Moreover, following the great success of curriculum learning in the area of machine learning, a novel deep curriculum‐learning model is proposed, entitled Deep Curriculum Learning (DCL), to train the DL network in our self‐supervised model. The proposed DCL utilises the entropy‐alpha target decomposition to estimate the degree of complexity of each PolSAR image patch before applying it to the EfficientNet‐B0. Also, an accumulative mini‐batch pacing function is used to introduce more difficult patches to EfficientNet‐B0 in the training process of the designed self‐supervised model. Furthermore, two EL models, feature‐level and view‐level ensemble, are proposed to increase the feature extraction capability and classification result of PolSAR data by jointly using spatial features at different scales and polarimetric information at different bands. In fact, in the proposed feature‐level ensemble strategy, to improve the classification result, the extracted features of the different scales are used as the input of the designed feature fusion algorithm. Therefore, the extracted features are mapped to the new space with lower dimension. In general, in the proposed view‐level ensemble strategy, first, the Mutual Information (MI) between each feature and the other features calculates, and then based‐on the calculated MI, different group of features are selected as various views. Next, the final classification result of the PolSAR image classification is obtained by using the designed majority vote of the result in each view. It should be noted that unlike existing traditional methods, both amplitude and phase information of SAR images to devolve the training process of the proposed model are used. In addition, all of the parameters of the proposed network are developed to a complex domain. In addition, a complex backpropagation algorithm by using the gradient‐based model is used for training the network. Finally, experimental results on three well‐known PolSAR data sets illustrate that the designed SSELF can extract more discriminant features using the designed DL model and can achieve better classification results than the other six novel DL models in terms of small training samples and adequate labelled samples.