Image Classification Performance Research Articles

CGUN-2A: Deep Graph Convolutional Network via Contrastive Learning for Large-Scale Zero-Shot Image Classification

Taxonomy illustrates that natural creatures can be classified with a hierarchy. The connections between species are explicit and objective and can be organized into a knowledge graph (KG). It is a challenging task to mine features of known categories from KG and to reason on unknown categories. Graph Convolutional Network (GCN) has recently been viewed as a potential approach to zero-shot learning. GCN enables knowledge transfer by sharing the statistical strength of nodes in the graph. More layers of graph convolution are stacked in order to aggregate the hierarchical information in the KG. However, the Laplacian over-smoothing problem will be severe as the number of GCN layers deepens, which leads the features between nodes toward a tendency to be similar and degrade the performance of zero-shot image classification tasks. We consider two parts to mitigate the Laplacian over-smoothing problem, namely reducing the invalid node aggregation and improving the discriminability among nodes in the deep graph network. We propose a top-k graph pooling method based on the self-attention mechanism to control specific node aggregation, and we introduce a dual structural symmetric knowledge graph additionally to enhance the representation of nodes in the latent space. Finally, we apply these new concepts to the recently widely used contrastive learning framework and propose a novel Contrastive Graph U-Net with two Attention-based graph pooling (Att-gPool) layers, CGUN-2A, which explicitly alleviates the Laplacian over-smoothing problem. To evaluate the performance of the method on complex real-world scenes, we test it on the large-scale zero-shot image classification dataset. Extensive experiments show the positive effect of allowing nodes to perform specific aggregation, as well as homogeneous graph comparison, in our deep graph network. We show how it significantly boosts zero-shot image classification performance. The Hit@1 accuracy is 17.5% relatively higher than the baseline model on the ImageNet21K dataset.

A Filtering Method for LiDAR Point Cloud Based on Multi-Scale CNN with Attention Mechanism

Point cloud filtering is an important prerequisite for three-dimensional surface modeling with high precision based on LiDAR data. To cope with the issues of low filtering accuracy or excessive model complexity in traditional filtering algorithms, this paper proposes a filtering method for LiDAR point cloud based on a multi-scale convolutional neural network incorporated with the attention mechanism. Firstly, a regular image patch centering on each point is constructed based on the elevation information of point clouds. As thus, the point cloud filtering problem is transformed into the image classification problem. Then, considering the ability of multi-scale convolution to extract features at different scales and the potential of the attention mechanism to capture key information in images, a multi-scale convolutional neural network framework is constructed, and the attention mechanism is incorporated to coordinate multi-scale convolution kernel with channel and spatial attention modules. After this, the feature maps of the LiDAR point clouds can be acquired at different scales. For these feature maps, the weights of each channel layer and different spatial regions can be further tuned adaptively, which makes the network training more targeted, thereby improving the model performance for image classification and eventually separating of ground points and non-ground points preferably. Finally, the proposed method is compared with the cloth simulation filtering method (CSF), deep neural network method (DNN), k-nearest neighbor method (KNN), deep convolutional neural network method (DCNN) and scale-irrelevant and terrain-adaptive method (SITA) for the standard ISPRS dataset of point cloud filtering and the filter dataset of Qinghai. The experimental results show that the proposed method can obtain lower classification errors, which proves the superiority of this method in point cloud filtering.

Research on Deep Compression Method of Expressway Video Based on Content Value

Aiming at the problem that the storage space and network bandwidth of expressway surveillance video are occupied largely due to data redundancy and sparse information, this paper proposes a deep compression method of expressway video depth based on content value. Firstly, the YOLOv4 algorithm is used to analyze the content value of the original video, extract video frames with vehicle information, and eliminate unintentional frames. An improved CNN is then designed by adding Feature Pyramids and the Inception module to accelerate the extraction and fusion of features at all levels and improve the performance of image classification and prediction. Finally, the whole model is integrated into HEVC encoder for compressing the preprocessed video. The experimental results show that at the expense of only a 5.96% increase of BD-BR, and only a 0.19 dB loss of BD-PSNR, the proposed method achieves a 64% compression ratio and can save 62.82% coding time compared with other classic data compression methods based on deep learning.

Comparison of pre-trained deep learning model classification performance of COVID-19 and normal chest X-ray images

Comparison of pre-trained deep learning model classification performance of COVID-19 and normal chest X-ray images

Deep Learning Approaches for Classification of Breast Cancer in Ultrasound (US) Images

Breast cancer is one of the deadliest cancer types affecting women worldwide. As with all types of cancer, early detection of breast cancer is of vital importance. Early diagnosis plays an important role in reducing deaths and fighting cancer. Ultrasound (US) imaging is a painless and common technique used in the early detection of breast cancer. In this article, deep learning-based approaches for the classification of breast US images have been extensively reviewed. Classification performance of breast US images of architectures such as AlexNet, VGG, ResNet, GoogleNet and EfficientNet, which are among the most basic CNN architectures, has been compared. Then, transformer models, which are one of the most popular deep learning architectures these days and show similar performance to the performance of CNN' architectures in medical images, are examined. BUSI, the only publicly available dataset, was used in experimental studies. Experimental studies have shown that the transformer and CNN models successfully classify US images of the breast. It has been observed that vision transformer model outperforms other models with 88.6% accuracy, 90.1% precison, 87.4% recall and 88.7% F1-score. This study shows that deep learning architectures are successful in classification of US images and can be used in the clinic experiments in the near future.

2C-Net: integrate image compression and classification via deep neural network

Providing effective support for intelligent vision tasks without image reconstruction can save numerous computational costs in the era of big data. With the help of the Deep Neural Network (DNN), integrating image compression and intelligent vision tasks at a feature representation level becomes a new promising approach. But how to perform non-linear transformation for image compression and extract image patterns for intelligent vision tasks simultaneously within a shared DNN remains an open problem. In this paper, a versatile framework is studied to explore the common feature representations for both image compression and classification. A fully shared latent representation is extracted in a more compact way to support compression and classification task. The General Feature Extraction and Feature-Analytic Classifier are proposed to generate and utilize shared latent representation. Then, the whole framework is joint optimized by considering multiple factors (i.e., rate, quality, and accuracy). Extensive experiments are carried out to validate that the proposals can improve the performance of both learning-based image compression and classification. The results show that the proposed method outperforms the conventional codecs like BPG and JPEG2000 in compression efficiency, while achieving acceptable accuracy on different image classification datasets without image reconstruction.

Holographic Microwave Image Classification Using a Convolutional Neural Network.

Holographic microwave imaging (HMI) has been proposed for early breast cancer diagnosis. Automatically classifying benign and malignant tumors in microwave images is challenging. Convolutional neural networks (CNN) have demonstrated excellent image classification and tumor detection performance. This study investigates the feasibility of using the CNN architecture to identify and classify HMI images. A modified AlexNet with transfer learning was investigated to automatically identify, classify, and quantify four and five different HMI breast images. Various pre-trained networks, including ResNet18, GoogLeNet, ResNet101, VGG19, ResNet50, DenseNet201, SqueezeNet, Inception v3, AlexNet, and Inception-ResNet-v2, were investigated to evaluate the proposed network. The proposed network achieved high classification accuracy using small training datasets (966 images) and fast training times.

Cassava Leaf Disease Classification Based On CNN

Cassava, a key food security crop in Africa, can provide a large amount of carbohydrates, which is widely grown in Africa. This kind of crop has been disturbed by viral diseases for a long time, which is the main reason for low production. The symptoms of cassava diseases can be reflected in their leaves but artificial visual diagnosis of diseases is inefficient. In order to get rid of this inefficient method, it is important to establish a model to automatically diagnose cassava by using the captured crop images to help farmers save their crops. Convolutional neural network has a good performance in image classification. Based on the research of Kaiming He et al. on residual convolution network, this paper trains the image dataset of cassava leaves from Kaggle by residual networks and then compares the classification effects by using different layers. Moreover, data augmentation and Focalloss function are used to optimize the model, which achieve better performance.

The Incoherence of Deep Isotropic Neural Networks Increases Their Performance in Image Classification

Although neural-network architectures are critical for their performance, how the structural characteristics of a neural network affect its performance has still not been fully explored. Here, we map architectures of neural networks to directed acyclic graphs (DAGs), and find that incoherence, a structural characteristic to measure the order of DAGs, is a good indicator for the performance of corresponding neural networks. Therefore, we propose a deep isotropic neural-network architecture by folding a chain of the same blocks and then connecting the blocks with skip connections at different distances. Our model, named FoldNet, has two distinguishing features compared with traditional residual neural networks. First, the distances between block pairs connected by skip connections increase from always equal to one to specially selected different values, which lead to more incoherent graphs and let the neural network explore larger receptive fields and, thus, enhance its multi-scale representation ability. Second, the number of direct paths increases from one to multiple, which leads to a larger proportion of shorter paths and, thus, improves the direct propagation of information throughout the entire network. Image-classification results on CIFAR-10 and Tiny ImageNet benchmarks suggested that our new network architecture performs better than traditional residual neural networks. FoldNet with 25.4M parameters can achieve 72.67% top-1 accuracy on the Tiny ImageNet after 100 epochs, which is competitive compared with the-state-of-art results on the Tiny ImageNet.

Supervised Contrastive Learning-Based Classification for Hyperspectral Image

Recently, deep learning methods, especially convolutional neural networks (CNNs), have achieved good performance for hyperspectral image (HSI) classification. However, due to limited training samples of HSIs and the high volume of trainable parameters in deep models, training deep CNN-based models is still a challenge. To address this issue, this study investigates contrastive learning (CL) as a pre-training strategy for HSI classification. Specifically, a supervised contrastive learning (SCL) framework, which pre-trains a feature encoder using an arbitrary number of positive and negative samples in a pair-wise optimization perspective, is proposed. Additionally, three techniques for better generalization in the case of limited training samples are explored in the proposed SCL framework. First, a spatial–spectral HSI data augmentation method, which is composed of multiscale and 3D random occlusion, is designed to generate diverse views for each HSI sample. Second, the features of the augmented views are stored in a queue during training, which enriches the positives and negatives in a mini-batch and thus leads to better convergence. Third, a multi-level similarity regularization method (MSR) combined with SCL (SCL–MSR) is proposed to regularize the similarities of the data pairs. After pre-training, a fully connected layer is combined with the pre-trained encoder to form a new network, which is then fine-tuned for final classification. The proposed methods (SCL and SCL–MSR) are evaluated on four widely used hyperspectral datasets: Indian Pines, Pavia University, Houston, and Chikusei. The experiment results show that the proposed SCL-based methods provide competitive classification accuracy compared to the state-of-the-art methods.

Adaptive Edge Offloading for Image Classification Under Rate Limit

This article considers a setting where embedded devices are used to acquire and classify images. Because of limited computing capacity, embedded devices rely on a parsimonious classification model with uneven accuracy. When local classification is deemed inaccurate, devices can decide to offload the image to an edge server with a more accurate but resource-intensive model. Resource constraints, e.g., network bandwidth, however, require regulating such transmissions to avoid congestion and high latency. This article investigates this offloading problem when transmissions regulation is through a token bucket, a mechanism commonly used for such purposes. The goal is to devise a lightweight, online offloading policy that optimizes an application-specific metric (e.g., classification accuracy) under the constraints of the token bucket. This article develops a policy based on a deep <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Q$ </tex-math></inline-formula> -network (DQN), and demonstrates both its efficacy and the feasibility of its deployment on embedded devices. Of note is the fact that the policy can handle complex input patterns, including correlation in image arrivals and classification accuracy. The evaluation is carried out by performing image classification over a local testbed using synthetic traces generated from the ImageNet image classification benchmark. Implementation of this work is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/qiujiaming315/edgeml-dqn</uri> .

ITERATIVE COLLABORATIVE ROUTING AMONG EQUIVARIANT CAPSULES FOR TRANSFORMATION-ROBUST CAPSULE NETWORKS

Transformation-robustness is an important feature for machine learning models that perform image classification. Many methods aim to bestow this property to models by the use of data augmentation strategies, while more formal guarantees are obtained via the use of equivariant models. We recognise that compositional, or part-whole structure is also an important aspect of images that has to be considered for building transformation-robust models. Thus, we propose a capsule network model that is, at once, equivariant and compositionality aware. Equivariance of our capsule network model comes from the use of equivariant convolutions in a carefully-chosen novel architecture. The awareness of compositionality comes from the use of our proposed novel, iterative, graph-based routing algorithm, termed Iterative collaborative routing (ICR). ICR, the core of our contribution, weights the predictions made for capsules based on an iteratively averaged score of the degree-centralities of its nearest neighbours. Experiments on transformed image classification on FashionMNIST, CIFAR-10, and CIFAR-100 show that our model that uses ICR outperforms convolutional and capsule baselines to achieve state-of-the-art performance.

Eyebirds: Enabling the Public to Recognize Water Birds at Hand.

Enabling the public to easily recognize water birds has a positive effect on wetland bird conservation. However, classifying water birds requires advanced ornithological knowledge, which makes it very difficult for the public to recognize water bird species in daily life. To break the knowledge barrier of water bird recognition for the public, we construct a water bird recognition system (Eyebirds) by using deep learning, which is implemented as a smartphone app. Eyebirds consists of three main modules: (1) a water bird image dataset; (2) an attention mechanism-based deep convolution neural network for water bird recognition (AM-CNN); (3) an app for smartphone users. The waterbird image dataset currently covers 48 families, 203 genera and 548 species of water birds worldwide, which is used to train our water bird recognition model. The AM-CNN model employs attention mechanism to enhance the shallow features of bird images for boosting image classification performance. Experimental results on the North American bird dataset (CUB200-2011) show that the AM-CNN model achieves an average classification accuracy of 85%. On our self-built water bird image dataset, the AM-CNN model also works well with classification accuracies of 94.0%, 93.6% and 86.4% at three levels: family, genus and species, respectively. The user-side app is a WeChat applet deployed in smartphones. With the app, users can easily recognize water birds in expeditions, camping, sightseeing, or even daily life. In summary, our system can bring not only fun, but also water bird knowledge to the public, thus inspiring their interests and further promoting their participation in bird ecological conservation.

Investigating the impact of CNN Layers on Dysgraphia Handwriting Image Classification Performance

Investigating the impact of CNN Layers on Dysgraphia Handwriting Image Classification Performance

Evaluation and Optimization of Biomedical Image-Based Deep Convolutional Neural Network Model for COVID-19 Status Classification

Accurate detection of an individual’s coronavirus disease 2019 (COVID-19) status has become critical as the COVID-19 pandemic has led to over 615 million cases and over 6.454 million deaths since its outbreak in 2019. Our proposed research work aims to present a deep convolutional neural network-based framework for the detection of COVID-19 status from chest X-ray and CT scan imaging data acquired from three benchmark imagery datasets. VGG-19, ResNet-50 and Inception-V3 models are employed in this research study to perform image classification. A variety of evaluation metrics including kappa statistic, Root-Mean-Square Error (RMSE), accuracy, True Positive Rate (TPR), False Positive Rate (FPR), Recall, precision, and F-measure are used to ensure adequate performance of the proposed framework. Our findings indicate that the Inception-V3 model has the best performance in terms of COVID-19 status detection.

Band selection technique based on binary modified equilibrium optimizer for hyperspectral image classification

The performance of hyperspectral image (HSI) classification can be enhanced by utilizing a metaheuristic technique with high exploration and exploitation capabilities for band selection. We design a framework for the improved classification of HSIs by utilizing a binary modified equilibrium optimizer (BMEO) for band selection. First, the proposed framework preprocesses the HSI data using the Gaussian filter to remove the undesired variations. Band selection is performed on the preprocessed HSI data using a BMEO, which exhibits balanced and high exploration and exploitation capabilities. The proposed V-shape transfer function is used for enhancing the exploration capabilities of the BMEO. An analysis for band selection is performed by comparing the reflectance spectra, correlation of each band with ground truth, and correlation between the bands before and after the band selection. The classification of HSI data by utilizing selected bands was done using the classifier selected by classifier analysis. The proposed framework outperforms 11 state-of-the-art techniques, including filter, wrapper, and deep learning-based techniques. The 95.85%, 98.13%, and 98.31% overall accuracies on the Indian pine, Pavia University, and Salinas datasets, respectively, show the significance of the proposed framework.

Breast cancer detection and classification in mammogram using a three-stage deep learning framework based on PAA algorithm

In recent years, deep learning has been used to develop an automatic breast cancer detection and classification tool to assist doctors. In this paper, we proposed a three-stage deep learning framework based on an anchor-free object detection algorithm, named the Probabilistic Anchor Assignment (PAA) to improve diagnosis performance by automatically detecting breast lesions (i.e., mass and calcification) and further classifying mammograms into benign or malignant. Firstly, a single-stage PAA-based detector roundly finds suspicious breast lesions in mammogram. Secondly, we designed a two-branch ROI detector to further classify and regress these lesions that aim to reduce the number of false positives. Besides, in this stage, we introduced a threshold-adaptive post-processing algorithm with dense breast information. Finally, the benign or malignant lesions would be classified by an ROI classifier which combines local-ROI features and global-image features. In addition, considering the strong correlation between the task of detection head of PAA and the task of whole mammogram classification, we added an image classifier that utilizes the same global-image features to perform image classification. The image classifier and the ROI classifier jointly guide to enhance the feature extraction ability and further improve the performance of classification. We integrated three public datasets of mammograms (CBIS-DDSM, INbreast, MIAS) to train and test our model and compared our framework with recent state-of-the-art methods. The results show that our proposed method can improve the diagnostic efficiency of radiologists by automatically detecting and classifying breast lesions and classifying benign and malignant mammograms.

In-Memory Computation With Improved Linearity Using Adaptive Sparsity-Based Compact Thermometric Code

The article presents an efficient static random access memory (SRAM)-based in-memory computation (IMC) architecture which is capable of performing image classification with improved linearity. In this work, we proposed a thermometric code-based IMC (TC-IMC) to perform multibit multiply-and-accumulate (MAC) operations with improved linearity. An input sparsity-aware compact thermometric code approach is proposed to reduce the number of SRAM bitcells compared to the thermometric code-based encoding without loss of accuracy in the MAC operation. We proposed an optimal sampling time to improve the linearity of the TC-IMC MAC operation with maximum signal margin (SM) based on the detailed nonlinearity analysis of 8T SRAM-based TC-IMC architecture. The test chip measurement results in 180 nm process show that the proposed TC-IMC has 72% better linearity than the traditional IMC. The measured results on 100 Modified National Institute of Standards and Technology (MNIST) and CIFAR-10 test images show an accuracy of 97% and 87%, respectively. In addition, the proposed TC-IMC architecture achieves MAC compute latency of 25 ns, GOPS/kb (normalized to 1 b <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times $ </tex-math></inline-formula> 1 b) of 29.8, and energy efficiency of 2.3 TOPS/W. The simulation result of the entire 10000 MNIST and CIFAR-10 images considering process variation effects shows an accuracy of 98.7% and 90.4%, respectively.

Toward Adversary-aware Non-iterative Model Pruning through D ynamic N etwork R ewiring of DNNs

We present a dynamic network rewiring (DNR) method to generate pruned deep neural network (DNN) models that both are robust against adversarially generated images and maintain high accuracy on clean images. In particular, the disclosed DNR training method is based on a unified constrained optimization formulation using a novel hybrid loss function that merges sparse learning with robust adversarial training. This training strategy dynamically adjusts inter-layer connectivity based on per-layer normalized momentum computed from the hybrid loss function. To further improve the robustness of the pruned models, we propose DNR++, an extension of the DNR method where we introduce the idea of sparse parametric Gaussian noise tensor that is added to the weight tensors to yield robust regularization. In contrast to existing robust pruning frameworks that require multiple training iterations, the proposed DNR and DNR++ achieve an overall target pruning ratio with only a single training iteration and can be tuned to support both irregular and structured channel pruning. To demonstrate the efficacy of the proposed method under the no-increased-training-time “free” adversarial training scenario, we finally present FDNR++, a simple yet effective training modification that can yield robust yet compressed models requiring training time comparable to that of an unpruned non-adversarial training. To evaluate the merits of our disclosed training methods, experiments were performed with two widely accepted models, namely VGG16 and ResNet18, on CIFAR-10 and CIFAR-100 as well as with VGG16 on Tiny-ImageNet. Compared to the baseline uncompressed models, our methods provide over 20× compression on all the datasets without any significant drop of either clean or adversarial classification performance. Moreover, extensive experiments show that our methods consistently find compressed models with better clean and adversarial image classification performance than what is achievable through state-of-the-art alternatives. We provide insightful observations to help make various model, parameter density, and prune-type selection choices and have open-sourced our saved models and test codes to ensure reproducibility of our results.

Deep Learning Multi-label Tongue Image Analysis and Its Application in a Population Undergoing Routine Medical Checkup.

Background Research on intelligent tongue diagnosis is a main direction in the modernization of tongue diagnosis technology. Identification of tongue shape and texture features is a difficult task for tongue diagnosis in traditional Chinese medicine (TCM). This study aimed to explore the application of deep learning techniques in tongue image analyses. Methods A total of 8676 tongue images were annotated by clinical experts, into seven categories, including the fissured tongue, tooth-marked tongue, stasis tongue, spotted tongue, greasy coating, peeled coating, and rotten coating. Based on the labeled tongue images, the deep learning model faster region-based convolutional neural networks (Faster R-CNN) was utilized to classify tongue images. Four performance indices, i.e., accuracy, recall, precision, and F1-score, were selected to evaluate the model. Also, we applied it to analyze tongue image features of 3601 medical checkup participants in order to explore gender and age factors and the correlations among tongue features in diseases through complex networks. Results The average accuracy, recall, precision, and F1-score of our model achieved 90.67%, 91.25%, 99.28%, and 95.00%, respectively. Over the tongue images from the medical checkup population, the model Faster R-CNN detected 41.49% fissured tongue images, 37.16% tooth-marked tongue images, 29.66% greasy coating images, 18.66% spotted tongue images, 9.97% stasis tongue images, 3.97% peeled coating images, and 1.22% rotten coating images. There were significant differences in the incidence of the fissured tongue, tooth-marked tongue, spotted tongue, and greasy coating among age and gender. Complex networks revealed that fissured tongue and tooth-marked were closely related to hypertension, dyslipidemia, overweight and nonalcoholic fatty liver disease (NAFLD), and a greasy coating tongue was associated with hypertension and overweight. Conclusion The model Faster R-CNN shows good performance in the tongue image classification. And we have preliminarily revealed the relationship between tongue features and gender, age, and metabolic diseases in a medical checkup population.