CNN Features Research Articles

Few-Shot Learning Meets Transformer: Unified Query-Support Transformers for Few-Shot Classification

The goal of Few-shot classification (FSL) is to identify unseen classes with very limited samples has attracted more and more attention. Usually, it is formulated as a metric learning problem. The core issue of few-shot classification is how to learn (1) consistent representations for images in both support and query sets and (2) effective metric learning for images between support and query sets. In this paper, we show that the two challenges can be well modeled simultaneously via a unified Query-Support TransFormer (QSFormer) model. To be specific, the proposed QSFormer involves <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">global</i> query-support sample Transformer (sampleFormer) branch and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">local</i> patch Transformer (patchFormer) learning branch. sampleFormer aims to capture the dependence of samples in support and query sets for image representation. It adopts the Encoder, QS-Decoder and Cross-Attention to respectively model the Support, Query (image) representation and Metric learning for few-shot classification task. Also, as a complementary to global learning branch, we adopt a local patch Transformer to extract structural representation for each image sample by capturing the long-range dependence of local image patches. In addition, we introduce a novel Cross-scale Interactive Feature Extractor (CIFE) to extract and fuse different scale CNN features as an effective backbone module for the proposed few-shot learning method. We integrate these into a unified framework and train it in an end-to-end way. A large number of experiments are conducted on four popular datasets to validate the superiority and effectiveness of the proposed QSFormer.

A novel occluded face detection approach using Enhanced ORB and optimized GAN

Objectives:This research presents an approach to detect occluded face images. In order to achieve this, the research presents a novel technique that involves feature extraction and occluded face recognition. Methods: Feature extraction is performed by the enhanced ORB algorithm, which is proposed by the modification of the Oriented Fast and Rotated Brief (ORB) algorithm, by adding a phase for contrast adjustment, together with CNN features. For occluded face recognition, a Generative Adversarial Network (GAN) optimized by the proposed SR-SSA is designed. SR-SSA is proposed by the integration of Search and Rescue Optimization (SAR) in the Sparrow Search Algorithm (SSA). Results: The experimental results demonstrate that the SR-SSA-based GAN algorithm outperforms existing methods in terms of accuracy of 0.956, FAR of 0.045 and FRR of 0.021.

Image super-resolution method based on the interactive fusion of transformer and CNN features

Image super-resolution method based on the interactive fusion of transformer and CNN features

Multi-Layer Perceptron Classifier with the Proposed Combined Feature Vector of 3D CNN Features and Lung Radiomics Features for COPD Stage Classification.

Computed tomography (CT) has been regarded as the most effective modality for characterizing and quantifying chronic obstructive pulmonary disease (COPD). Therefore, chest CT images should provide more information for COPD diagnosis, such as COPD stage classification. This paper proposes a features combination strategy by concatenating three-dimension (3D) CNN features and lung radiomics features for COPD stage classification based on the multi-layer perceptron (MLP) classifier. First, 465 sets of chest HRCT images are automatically segmented by a trained ResU-Net, obtaining the lung images with the Hounsfield unit. Second, the 3D CNN features are extracted from the lung region images based on a truncated transfer learning strategy. Then, the lung radiomics features are extracted from the lung region images by PyRadiomics. Third, the MLP classifier with the best classification performance is determined by the 3D CNN features and the lung radiomics features. Finally, the proposed combined feature vector is used to improve the MLP classifier's performance. The results show that compared with CNN models and other ML classifiers, the MLP classifier with the best classification performance is determined. The MLP classifier with the proposed combined feature vector has achieved accuracy, mean precision, mean recall, mean F1-score, and AUC of 0.879, 0.879, 0.879, 0.875, and 0.971, respectively. Compared to the MLP classifier with the 3D CNN features selected by Lasso, our method based on the MLP classifier has improved the classification performance by 5.8% (accuracy), 5.3% (mean precision), 5.8% (mean recall), 5.4% (mean F1-score), and 2.5% (AUC). Compared to the MLP classifier with lung radiomics features selected by Lasso, our method based on the MLP classifier has improved the classification performance by 5.0% (accuracy), 5.1% (mean precision), 5.0% (mean recall), 5.1% (mean F1-score), and 2.1% (AUC). Therefore, it is concluded that our method is effective in improving the classification performance for COPD stage classification.

CNN feature and classifier fusion on novel transformed image dataset for dysgraphia diagnosis in children

CNN feature and classifier fusion on novel transformed image dataset for dysgraphia diagnosis in children

Graph-Based Progressive Fusion Network for Multi-Modality Vehicle Re-Identification

Vehicle re-identification (Re-ID) is a critical task in intelligent transportation, aiming to match vehicle images of the same identity captured by non-overlapping cameras. However, it is difficult to achieve satisfactory results based on RGB images alone in darkness. Therefore, it is of great importance to consider multi-modality vehicle re-identification. Currently, the proposed works deal with different modality features through direct summation and fusion based on heat map, which however ignores the relationship between them. Meanwhile, there is a huge gap between the different modalities, which needs to be reduced. In this paper, to solve the above two problems, we propose a Graph-based Progressive Fusion Network (GPFNet) using a graph convolutional network to adaptively fuse multi-modality features in an end-to-end learning framework. GPFNet consists of a CNN feature extraction module (FEM), a GCN feature fusion module (FFM), and a loss function module (LFM). Firstly, in FEM, we employ a multi-stream network architecture to extract single-modality features and common-modality features and employ a random modality substitution module to extract mixed-modality features. Secondly, in FFM, we design an efficient graph structure to associate the features of different modalities and adopt a progressive two-stage strategy to fuse them. Finally, in LFM, we use GCN-aware multi-modality loss to constrain the features. For reducing modality differences and contributing better initial mixed-modality features to FFM, we propose random modality substitution as a data enhancement method for multi-modality datasets. Extensive experiments on multi-modality vehicle Re-ID datasets RGBN300 and RGBNT100 show that our model achieves state-of-the-art performance.

Efficient Detection of Covid-19 from Chest X-ray Images using CNN Feature Extraction and an Ensemble of Machine Learning Classifiers

Efficient Detection of Covid-19 from Chest X-ray Images using CNN Feature Extraction and an Ensemble of Machine Learning Classifiers

Broken stitch detection method for sewing operation using CNN feature map and image processing technique .

Quality in industrial processes has become increasingly important and cost reduction and process optimization are becoming increasingly necessary Quality control brings increased production and even increased profits for a process. It can be said, then, that it is the most important metric when it comes to production. It is extremely difficult to have a 100% defect-free manufacturing process. One of the industrial processes that has received such attention regarding defects is the weaving process. The present work will make a global study on Machine Learning techniques and also on Wavelets. This study may serve as a basis for future academic work. The application built in the present work will also serve as an example of how a computer vision system can vary from the classifier algorithm used to the feature extraction technique, which in this case, will use the Wavelet Transform. In this work we Survey the state of the art in methods of recognizing defects in fabrics. We will also Create the database, as well as the set of images to be used. Extract information from the image with the Wavelet Transform. Test different classification algorithms in order to find the best answer for this problem. Improve the performance of the classifier algorithms through the CNN algorithm. Validate the system using the k-fold cross validation technique.

Hybrid generative model for grading the severity of diabetic retinopathy images

ABSTRACT One of the common eye conditions affecting patients with diabetes is diabetic retinopathy (DR). It is characterised by the progressive impairment to the blood vessels with the increase of glucose level in the blood. The grading efficiency still finds challenging because of the existence of intra-class variations and imbalanced data distributions on the retinal images. Traditional machine learning techniques utilise hand-engineered features for classification of the affected retinal images. As convolutional neural network produces better image classification accuracy in many medical images, this work utilises the CNN-based feature extraction method. This feature has been used to build Gaussian mixture model (GMM) for each class that maps the CNN features to log-likelihood dimensional vector spaces. Since the Gaussian mixture model can be realised as a mixture of both parametric and nonparametric density models and has their flexibility in capturing different data distributions, probabilistic outputs, interpretability, efficient parameter estimation, and robustness to outliers, the proposed model aimed to obtain and provide a smooth approximation of the underlying distribution of features for training the model. Then these vector spaces are trained by the SVM classifier. Experimental results illustrate the efficacy of the proposed model with accuracy 86.3% and 89.1%, respectively.

XCNN-SC: Explainable CNN for SARS-CoV-2 variants classification and mutation detection

The COVID-19 pandemic spread rapidly all over the world in 2019, causing the deaths of millions of people. One of the main challenges in controlling the pandemic was the high rate of virus mutation, which evaded the immune system and reduced the vaccine's effectiveness. Due to the differences in symptoms, transmission and mortality rate, and effective prevention strategies of each variant, identifying and classifying different variants is a great necessity. In the present study, 82802 whole genomes of Alpha, Beta, Delta, Eta, Epsilon, Lota, and Omicron variants of SARS-CoV-2 were obtained from the NCBI database. The label encoding method was applied to convert the genomic sequences to numeric sequences. Then, a single-layer 1D-CNN model was used to classify seven variants of SARS-CoV-2 and achieved 99.78% accuracy, 97.98% precision, 97.66% sensitivity, 99.95% specificity, and 98.68% F1-score. The max pool layer feature maps of this network were investigated to discover the sequence discriminative regions in SARS-CoV-2 variants and their effect on biological functional differences. The feature map examination led to the detection of three mutations; one of them was a missense mutation, which has been reported previously. The rest of the detected mutations were silent mutations. In conclusion, the achieved results indicated that CNN models can extract effective features to discriminate several SARS-CoV-2 variants. Also, investigation of CNN feature maps led to an explainable CNN revealing the learned knowledge of the network from the training database. This knowledge helped us to detect mutations and their functional effects in different variants.

VHXLA: A post-earthquake damage prediction method for high-speed railway track-bridge system using VMD and hybrid neural network

The timely and accurate prediction of post-earthquake damage is critically important for ensuring the safety of high-speed railway track-bridge systems. The study introduces a novel method known as the VHXLA model. This innovative model blends variational mode decomposition (VMD) with a hybrid neural network for predicting non-linear multi-component post-earthquake damage in high-speed railways. The VHXLA model comprises a signal processing module and a deep learning module. The signal processing module utilises VMD decomposition, and the Hilbert transform (HT) to transform two-dimensional seismic signals into four-dimensional complex time–frequency signals, thus simplifying the task of identifying seismic time–frequency characteristics via the neural network. The deep learning module integrates diverse types of neural network components, such as the Xception CNN feature extraction submodule, the LSTM RNN temporal learning submodule, and the multi-head attention mechanism submodule. A Bayesian self-optimisation method is implemented to determine the number of decomposition layers in VMD and select essential hyperparameters. The model's effectiveness is evaluated by predicting the damage of an experimentally validated finite element model, subjecting it to seismic loads, and thus gauging its performance. The results indicated that the signal processing module, based on VMD decomposition, significantly improves the neural network's signal processing capability. In addition, the synergistic integration of different modules in the VHXLA model provides superior prediction accuracy compared to pre-existing damage prediction models. Notably, the prediction accuracy is consistent across different positions of the same predicted component in the high-speed railway track-bridge system.

Cow detection and tracking system utilizing multi-feature tracking algorithm

In modern cattle farm management systems, video-based monitoring has become important in analyzing the high-level behavior of cattle for monitoring their health and predicting calving for providing timely assistance. Conventionally, sensors have been used for detecting and tracking their activities. As the body-attached sensors cause stress, video cameras can be used as an alternative. However, identifying and tracking individual cattle can be difficult, especially for black and brown varieties that are so similar in appearance. Therefore, we propose a new method of using video cameras for recognizing cattle and tracking their whereabouts. In our approach, we applied a combination of deep learning and image processing techniques to build a robust system. The proposed system processes images in separate stages, namely data pre-processing, cow detection, and cow tracking. Cow detection is performed using a popular instance segmentation network. In the cow tracking stage, for successively associating each cow with the corresponding one in the next frame, we employed the following three features: cow location, appearance features, as well as recent features of the cow region. In doing so, we simply exploited the distance between two gravity center locations of the cow regions. As color and texture suitably define the appearance of an object, we analyze the most appropriate color space to extract color moment features and use a Co-occurrence Matrix (CM) for textural representation. Deep features are extracted from recent cow images using a Convolutional Neural Network (CNN features) and are also jointly applied in the tracking process to boost system performance. We also proposed a robust Multiple Object Tracking (MOT) algorithm for cow tracking by employing multiple features from the cow region. The experimental results proved that our proposed system could handle the problems of MOT and produce reliable performance.

Semantic Knowledge Guided Class-Incremental Learning

Driven by practical needs, research on Class-Incremental Learning (CIL) has received more and more attentions in recent years. A technical challenge to be conquered by CIL methods is the catastrophic forgetting problem, where the model’s performance improves rapidly on new classes while deteriorates drastically on old ones. The main causes behind catastrophic forgetting include network drifts, inter-class confusions, etc. In this paper, we propose a novel CIL method that solves the catastrophic forgetting problem from two aspects. First, to solve the inter-class confusion problem, we propose a novel Semantic knOwledge gUided ciL framework (SOUL) that consists of a CNN feature extractor and a Bi-GCN (Graph Convolutional Network) classifier. In each CIL session, we use the semantic knowledge extracted from the class labels to build two inter-class relation graphs among all the encountered old and new classes. Using these two relation graphs, we develop a Bi-GCN classifier to fuse two kinds of semantic relations in a balanced way, and then to transfer the inter-class relations from semantic modality to image classification weights. The entire SOUL framework is trained end-to-end by the standard BP algorithm, which optimizes the Bi-GCN classifier and the CNN feature extractor jointly. Second, to prevent the network drift, we develop the local topology preserving strategy that divides the global topological structure of the learned feature space into a set of local topological relations, and maintains these local relations at CIL session. Experimental evaluations demonstrate the state-of-the-art performance accuracies on benchmark image classification datasets.

Utilization of information from CNN feature maps for offline word-level writer identification

This paper proposes a text-independent method for authorship identification using handwritten word images. Our method is text-independent and imposes no limitations on the size of the input word images being analyzed. To begin with, the SIFT algorithm is utilized to extract key-point regions (fragments) across different levels of abstraction, encompassing allographs, characters, or character combinations. These fragments are subsequently processed through a CNN network, yielding feature maps corresponding to convolution layers. The information in these maps is then transformed into a fixed-dimension representation using a modified version of the HOG feature descriptor.The noteworthy contribution of our proposal lies in harnessing additional cues from CNN’s feature maps for writer identification. We propose a measure to gauge the importance or ‘saliency’ of feature maps within a CNN layer during training. This measure originates from applying Sparse Principal Component Analysis (SPCA) to Histogram Of Gradient (HOG) features. Once saliency values are obtained, we combine them with HOG representations to create writer descriptors tailored to a CNN layer. The derived descriptors are then passed through a set of SVM classifiers that are scored at two levels to determine the identity of the handwritten word image.The efficacy of our system has been demonstrated on the word-level data of the CVL, IAM and CERUG-EN datasets with an accuracy of 82.10%, 87.68%, and 75.80% respectively. The results obtained are shown to be promising when compared with previous works.

Transformer-based descriptors with fine-grained region supervisions for visual place recognition

Visual place recognition is a fundamental component in autonomous systems and robotics, which is easily limited in the real world with different viewpoints and changes in appearance. Existing approaches to tackle this problem mainly rely on dominant CNN-based architectures, which are difficult to model global correlation. Most recently, there has emerged little work that focuses on the effectiveness of the Transformer in modeling long-range dependencies, but this strategy ignores local interactions thus fails to localize the really important regions. To address the above issue, this paper proposes an effective Transformer-based architecture that takes full advantages of the strengths of Transformer related to global context modeling and local specific region capturing. We first design a dual-level Transformer descriptor encoder to successively perform self-attention within local windows and global extent of the CNN feature map to obtain multi-scale spatial context, which combines local interaction and global information. Specifically, multi-layer classification tokens from the Transformer encoder are integrated to form the global image representation. Moreover, a Transformer-guided geometric verification module is introduced to leverage the strengths of the hierarchical Transformer’s inherent self-attention mechanism for fusing multi-level attention, which is employed to filter the output token to obtain key patches and associated attention weights for achieving spatial matching. Finally, we propose a descriptor refinement strategy that employs fine-grained region-level supervisions to further enhance the capability of the network to learn local discriminative features, which effectively alleviates the confusion caused from weak image-level labels. Extensive experiments on benchmark datasets show that our approach outperforms state-of-the-art methods with promising performance.

Dual Swin-transformer based mutual interactive network for RGB-D salient object detection

Depth information for RGB-D Salient Object Detection(SOD) is important and conventional deep models are usually relied on the CNN feature extractors. The long-range contextual dependencies, dense modeling on the saliency decoder, and multi-task learning assistance are usually ignored. In this work, we propose a Dual Swin-Transformer-based Mutual Interactive Network (DTMINet), aiming to learn contextualized, dense, and edge-aware features for RGB-D SOD. We adopt the Swin-Transformer as the visual backbone to extract contextualized features. A self-attention-based Cross-Modality Interaction module is proposed to strengthen the visual backbone for cross-modal interaction. In addition, a Gated Modality Attention module is designed for cross-modal fusion. At different decoding stages, enhanced with dense connections and progressively merge the multi-level encoding features with the proposed Dense Saliency Decoder. Considering the depth quality issue, a Skip Convolution module is introduced to provide guidance to the RGB modality for the saliency prediction. In addition, we add the edge prediction to the saliency predictor to regularize the learning process. Comprehensive experiments on five standard RGB-D SOD benchmark datasets over four evaluation metrics demonstrate the superiority of the proposed method.

Automatic prediction of acute coronary syndrome based on pericoronary adipose tissue and atherosclerotic plaques.

Cardiovascular disease is the leading cause of human death worldwide, and acute coronary syndrome (ACS) is a common first manifestation of this. Studies have shown that pericoronary adipose tissue (PCAT) computed tomography (CT) attenuation and atherosclerotic plaque characteristics can be used to predict future adverse ACS events. However, radiomics-based methods have limitations in extracting features of PCAT and atherosclerotic plaques. Therefore, we propose a hybrid deep learning framework capable of extracting coronary CT angiography (CCTA) imaging features of both PCAT and atherosclerotic plaques for ACS prediction. The framework designs a two-stream CNN feature extraction (TSCFE) module to extract the features of PCAT and atherosclerotic plaques, respectively, and a channel feature fusion (CFF) to explore feature correlations between their features. Specifically, a trilinear-based fully-connected (FC) prediction module stepwise maps high-dimensional representations to low-dimensional label spaces. The framework was validated in retrospectively collected suspected coronary artery disease cases examined by CCTA. The prediction accuracy, sensitivity, specificity, and area under curve (AUC) are all higher than the classical image classification networks and state-of-the-art medical image classification methods. The experimental results show that the proposed method can effectively and accurately extract CCTA imaging features of PCAT and atherosclerotic plaques and explore the feature correlations to produce impressive performance. Thus, it has the potential value to be applied in clinical applications for accurate ACS prediction.

Express parcel detection based on improved faster regions with CNN features

Machine vision-based semi-automatic sorting in parcel sorting relies on specific sensors to read form information and synchronize it to the control system to complete a sort. The cost of traditional Faster RCNN parameter calculation is high, and the requirements for hardware equipment are high. In order to reduce the consumption of hardware resources and improve efficiency, we redesigned the traditional Faster RCNN to reduce the hardware cost requirements. The number of categories in package data sets varies greatly, and category imbalance is also one of the problems. To solve the express parcel category imbalance problem, an adaptive Mosaic method is proposed to improve the recognition accuracy of fine-grained similar parcels. To be deployed on edge devices with limited computational resources, a new lightweight network, Reparameterization Large Depthwise conv Normalization-based Attention (ReLDWNAM), is proposed. The experimental results show that compared with MobileNetV2, the number of parameters is reduced by 3.07M, and the computing resources are reduced by more than twice, 10 times faster time for feature extraction network, and more than double the overall detection speed of Faster RCNN with little difference in accuracy.

RGB-T salient object detection via excavating and enhancing CNN features

RGB-T salient object detection via excavating and enhancing CNN features

KPCA-WRF-prediction of heart rate using deep feature fusion and machine learning classification with tuned weighted hyper-parameter

ABSTRACT The rapid advancement of technology such as stream processing technologies, deep-learning approaches, and artificial intelligence plays a prominent and vital role, to detect heart rate using a prediction model. However, the existing methods could not handle high -dimensional datasets, and deep feature learning to improvise the performance. Therefore, this work proposed a real-time heart rate prediction model, using K-nearest neighbour (KNN) adhered to the principle component analysis algorithm (PCA) and weighted random forest algorithm for feature fusion (KPCA-WRF) approach and deep CNN feature learning framework. The feature selection, from the fused features, was optimized by ant colony optimization (ACO) and particle swarm optimization (PSO) algorithm to enhance the selected fused features from deep CNN. The optimized features were reduced to low dimensions using the PCA algorithm. The significant straight heart rate features are plotted by capturing out nearest similar data point values using the algorithm. The fused features were then classified for aiding the training process. The weighted values are assigned to those tuned hyper parameters (feature matrix forms). The optimal path and continuity of the weighted feature representations are moved using the random forest algorithm, in K-fold validation iterations.