Feature Learning Network Research Articles

ITIMCA: Image-Text Information and Cross-Attention for Multi-Modal Cassava Leaf Disease Classification Based on a Novel Multi-Modal Dataset in Natural Environments

With artificial intelligence and deep learning development, crop disease recognition methods leverage deep networks for automatic feature learning but rely on the volume of training data. Addressing the scarcity of data in agriculture, few-shot learning (FSL) and multi-modal learning have become focal points. However, existing methods are confined to a single modality or insufficiently exploit cross-modal features. To address this, we propose a multi-modal contrastive learning approach integrating images and text to tackle the problem of small-sample recognition. This method combines CLIP multi-modal pre-training with cross-attention, termed ITIMCA. Experimental validation demonstrates the effectiveness of our approach in cassava leaf disease recognition tasks under natural conditions. Experimental results show that the proposed model achieved an accuracy of 78.00%, a precision of 88.48%, a recall rate of 80.00%, and an F1-Score of 79.00% on the cassava leaf disease identification and classification dataset. These results suggest that the proposed network effectively identifies cassava leaf diseases.

Class‐wised domain decoupling‐guided adversarial feature learning for cross‐age face recognition

AbstractHow to extract age‐independent identity features from face images has long been the major challenge in cross‐age face recognition task. In this letter, a class‐wised domain decoupling‐guided adversarial feature learning network to extract age‐independent face features is proposed. In this model, an adversarial feature learning module is designed to decompose identity features and age features. Considering that artificially designed age interference suppression strategies are difficult to fit the complex relationship between identity features and age features, an adversarial training strategy incorporated with an attention mechanism in the adversarial feature learning module is introduced to suppress age interference adaptively. Besides, under the idea of maximizing the distribution difference of identity feature domain and age feature domain, a class‐wised domain decoupling module is further designed to guide the model to extract age‐independent identity features. The proposed model has been validated on the well‐known public datasets CALFW and CACD‐VS, where it achieved remarkable recognition accuracy rates of 94.5% and 99.5%, respectively. This represents improvements of 4.2% and 0.1% over the state‐of‐the‐art single‐task‐based models, clearly showcasing the effectiveness of the proposed class‐wised domain decoupling‐guided adversarial feature learning model.

Efficient skin lesion segmentation with boundary distillation.

Medical image segmentation models are commonly known for their complex structures, which often render them impractical for use on edge computing devices and compromising efficiency in the segmentation process. In light of this, the industry has proposed the adoption of knowledge distillation techniques. Nevertheless, the vast majority of existing knowledge distillation methods are focused on the classification tasks of skin diseases. Specifically, for the segmentation tasks of dermoscopy lesion images, these knowledge distillation methods fail to fully recognize the importance of features in the boundary regions of lesions within medical images, lacking boundary awareness for skin lesions. This paper introduces pioneering medical image knowledge distillation architecture. The aim of this method is to facilitate the efficient transfer of knowledge from existing complex medical image segmentation networks to a more simplified student network. Initially, a masked boundary feature (MBF) distillation module is designed. By applying random masking to the periphery of skin lesions, the MBF distillation module obliges the student network to reproduce the comprehensive features of the teacher network. This process, in turn, augments the representational capabilities of the student network. Building on the MBF distillation module, this paper employs a cascaded combination approach to integrate the MBF distillation module into a multi-head boundary feature (M2BF) distillation module, further strengthening the student network's feature learning ability and enhancing the overall image segmentation performance of the distillation model. This method has been experimentally validated on the public datasets ISIC-2016 and PH2, with results showing significant performance improvements in the student network. Our findings highlight the practical utility of the lightweight network distilled using our approach, particularly in scenarios demanding high operational speed and minimal storage usage. This research offers promising prospects for practical applications in the realm of medical image segmentation.

Multi-Branch Feature Learning Network via Global-Local Self-Distillation for Vehicle Re-Identification

Multi-Branch Feature Learning Network via Global-Local Self-Distillation for Vehicle Re-Identification

Comparing Mask R-CNN backbone architectures for human detection using thermal imaging

We introduce a method for detecting humans in thermal imaging using an end-to-end deep learning model. Our objective is to optimize the human detection process in thermal imaging by investigating the mask region-based convolutional neural network (Mask R-CNN). The model, an advancement of the faster region-based convolutional neural network (Faster R-CNN), not only captures bounding boxes encompassing human subjects but also delineates segmentation masks around them. Our investigation extends to the evaluation and comparison of various convolutional neural networks for feature learning, like residual network (ResNet) and Inception ResNet, all integrated into the Mask R-CNN framework. Furthermore, the experimental results show that our proposed technique achieves high accuracy. Specifically, the Mask R-CNN model using ResNet50-V1 achieved the best results, with an F-value of 87.85%, a recall of 79.33%, and a precision of 98.41%.

Supervertex Sampling Network: A Geodesic Differential SLIC Approach for 3D Mesh.

The analysis of 3D meshes with deep learning has become prevalent in computer graphics. As an essential structure, hierarchical representation is critical for mesh pooling in multiscale analysis. Existing clustering-based mesh hierarchy construction methods involve nonlinear discretization optimization operations, making them nondifferential and challenging to embed in other trainable networks for learning. Inspired by deep superpixel learning methods in image processing, we extend them from 2D images to 3D meshes by proposing a novel differentiable chart-based segmentation method named geodesic differential supervertex (GDSV). The key to the GDSV method is to ensure that the geodesic position updates are differentiable while satisfying the constraint that the renewed supervertices lie on the manifold surface. To this end, in addition to using the differential SLIC clustering algorithm to update the nonpositional features of the supervertices, a reparameterization trick, the Gumbel-Softmax trick, is employed to renew the geodesic positions of the supervertices. Therefore, the geodesic position update problem is converted into a linear matrix multiplication issue. The GDSV method can be an independent module for chart-based segmentation tasks. Meanwhile, it can be combined with the front-end feature learning network and the back-end task-specific network as a plug-in-plug-out module for training; and be applied to tasks such as shape classification, part segmentation, and 3D scene understanding. Experimental results show the excellent performance of our proposed algorithm on a range of datasets.

A Joint Network of Edge-Aware and Spectral-Spatial Feature Learning for Hyperspectral Image Classification.

Hyperspectral image (HSI) classification is a vital part of the HSI application field. Since HSIs contain rich spectral information, it is a major challenge to effectively extract deep representation features. In existing methods, although edge data augmentation is used to strengthen the edge representation, a large amount of high-frequency noise is also introduced at the edges. In addition, the importance of different spectra for classification decisions has not been emphasized. Responding to the above challenges, we propose an edge-aware and spectral-spatial feature learning network (ESSN). ESSN contains an edge feature augment block and a spectral-spatial feature extraction block. Firstly, in the edge feature augment block, the edges of the image are sensed, and the edge features of different spectral bands are adaptively strengthened. Then, in the spectral-spatial feature extraction block, the weights of different spectra are adaptively adjusted, and more comprehensive depth representation features are extracted on this basis. Extensive experiments on three publicly available hyperspectral datasets have been conducted, and the experimental results indicate that the proposed method has higher accuracy and immunity to interference compared to state-of-the-art (SOTA) method.

Heterogeneous feature learning network for multimodal remote sensing image collaborative classification

ABSTRACT While multimodal remote sensing data processing has garnered growing interests in the geoscience field, how to effectively extract, represent, and fuse heterogeneous features remains challenging. In this article, a novel heterogeneous feature learning network (HFLN) architecture is investigated for multimodal remote sensing collaborative classification, which takes both advantages of long-range feature modelling capability of transformer as well as local feature extraction of convolutional neural network (CNN). First, the global spectral dependencies are extracted from hyperspectral images using the spectral transformer structure. Then, local spatial invariant characteristics are extracted from multimodal remote sensing images by convolutional operation. Next, heterogeneous spectral and spatial characteristics, having significant structural and semantic differences, are dynamically integrated through the feature coupling module in an interactive manner. Finally, the multi-stage network architecture is utilized for extracting hierarchical characteristics, where the numbers of feature maps in CNN and heads in transformer are gradually increased to represent more complex features. Four benchmark remote sensing datasets (i.e. hyperspectral images and multimodal remote sensing datasets) are utilized for experimental analysis, and extensive experimental results have certified the progressiveness of the investigated approach.

A Machine Learning-based Method for COVID-19 and Pneumonia Detection

Pneumonia is described as an acute infection of lung tissue produced by one or more bacteria, and Coronavirus Disease (COVID-19) is a deadly virus that affects the lungs of the human body. The symptoms of COVID-19 disease are closely related to pneumonia. In this work, we identify the patients of pneumonia and coronavirus from chest X-ray images. We used a convolutional neural network for spatial feature learning from X-ray images. We experimented with pneumonia and coronavirus X-ray images in the Kaggle dataset. Pneumonia and corona patients are classified using a feed-forward neural network and hybrid models (CNN+SVM, CNN+RF, and CNN+Xgboost). The experimental findings on the Pneumonia dataset demonstrate that CNN detects Pneumonia patients with 99.47% recall. The overall experiments on COVID-19 x-ray images show that CNN detected the COVID-19 and pneumonia with 95.45% accuracy.

High-Speed Motion Target Real-Time Detection Based on Lightweight Deep Feature Learning Network

High-Speed Motion Target Real-Time Detection Based on Lightweight Deep Feature Learning Network

Reservoir properties inversion using attention-based parallel hybrid network integrating feature selection and transfer learning

The inversion of subsurface reservoir properties is of profound significance to the oil and gas energy development and utilization. The strong heterogeneity and complex pore structure of underground reservoirs pose a challenge to efficient oil and gas energy development and utilization across the world, which increases the necessity of developing an efficient reservoir properties inversion method. However, traditional model-driven methods are confronted with the challenges of strong nonlinearity and geological heterogeneity. Moreover, previous studies rarely emphasized the importance of nonlinear feature selection and transfer learning (TL). Aiming to address the research gaps, a reservoir properties inversion method was proposed by combining random forest (RF) feature selection, bidirectional temporal convolutional network (BiTCN), bidirectional gated recurrent units (BiGRU) network, multi-head attention (MHA) mechanism and TL strategy. First, the RF was used to screen the features with significant correlation with the target reservoir properties. Thereafter, combining BiTCN and BiGRU network to leverage their complementary strengths, a parallel dual branch feature learning network was constructed to learn richer geological information from logging data. Meanwhile, MHA was introduced to fuse the output features of the dual network structure. Finally, the fused features were passed through the fully connected module to output the inversion results. TL was used to associate the correlation between reservoir properties and model inversion to improve the inversion performance. The application results with actual field data showed that the proposed method was accurate and robust in reservoir properties inversion. This study can provide a new way for reliable reservoir properties inversion and promote the application of artificial intelligence in data-driven energy science.

Unified feature learning network for few-shot fault diagnosis

Few-shot fault diagnosis aims at diagnosing the state of mechanical signals with only a few training samples. Numerous contemporary approaches incorporate Time–Frequency Images (TFIs) derived from vibration signals to provide a thorough understanding of both the time and frequency domains. Current approaches often neglect the exploration of sample-level features, which hinders the enrichment and refinement of sample features. To this end, we propose a Unified Feature Learning Network (UFLN) designed to comprehensively model TFIs at two levels. We first present a Texture Enhancement Module (TEM) to amplify intricate details, aiding in the extraction of category-level features. Subsequently, we devise a dynamic Feature Selection Module (DFSM), tailored for the extraction of fault-related features. Finally, we develop an Intra-Diversity (ID) loss function to promote intra-class diversity, enriching the representation of each sample. Extensive experiments on three cases have demonstrated the effectiveness of the proposed UFLN approach.

Dual-branch interactive cross-frequency attention network for deep feature learning

As random noises contained in the high-frequency data could interfere with the feature learning of deep networks, low-pass filtering or wavelet transform have been integrated with deep networks to exclude the high-frequency component of input image. However, useful image details like contour and texture are also lost in such a process. In this paper, we propose Dual-branch interactive Cross-frequency attention Network (DiCaN) to separately process low-frequency and high-frequency components of input image, such that useful information is extracted from high-frequency data and included in deep learning. Our DiCaN first decomposes input image into low-frequency and high-frequency components using wavelet decomposition, and then applies two parallel residual-style branches to extract features from the two components. We further design an interactive cross-frequency attention mechanism, to highlight the useful information in the high-frequency data and interactively fuse them with the features in low-frequency branch. The features learned by our framework are then applied for both image classification and object detection and tested using ImageNet-1K and COCO datasets. The results suggest that DiCaN achieves better classification performance than different ResNet variants. Both one-stage and two-stage detectors with our DiCaN backbone also achieve better detection performance than that with ResNet backbone. The code of DiCaN will be released.

Mus4mCPred: Accurate Identification of DNA N4-Methylcytosine Sites in Mouse Genome Using Multi-View Feature Learning and Deep Hybrid Network

Mus4mCPred: Accurate Identification of DNA N4-Methylcytosine Sites in Mouse Genome Using Multi-View Feature Learning and Deep Hybrid Network

SensorNet: An Adaptive Attention Convolutional Neural Network for Sensor Feature Learning.

This work develops a generalizable neural network, SENSORNET, for sensor feature learning across various applications. The primary challenge addressed is the poor portability of pretrained neural networks to new applications with limited sensor data. To solve this challenge, we design SensorNet, which integrates the flexibility of self-attention with multi-scale feature locality of convolution. Moreover, we invent patch-wise self-attention with stacked multi-heads to enrich the sensor feature representation. SensorNet is generalizable to pervasive applications with any number of sensor inputs, and is much smaller than the state-of-the-art self-attention and convolution hybrid baseline (0.83 M vs. 3.87 M parameters) with similar performance. The experimental results show that SensorNet is able to achieve state-of-the-art performance compared with the top five models on a competition activity recognition dataset (SHL'18). Moreover, pretrained SensorNet in a large inertial measurement unit (IMU) dataset can be fine-tuned to achieve the best accuracy on a much smaller IMU dataset (up to 5% improvement in WISDM) and to achieve the state-of-the-art performance on an EEG dataset (SLEEP-EDF-20), showing the strong generalizability of our approach.

PMA-Net: Progressive multi-stage adaptive feature learning for two-view correspondence

Establishing high-quality correspondences is a fundamental step for many computer vision tasks. Leveraging the local consistency of correct correspondences (i.e., inliers) has been a prevalent approach to distinguish them from incorrect correspondences (i.e., outliers). However, the random distribution of a significant proportion of outliers complicates the local neighborhood construction of inliers, making it inevitably contain many outliers, which compromises the reliability of the consistency information. In this paper, we propose a Progressive Multi-Stage Adaptive Feature Learning Network (PMA-Net) to solve this problem, which progressively and adaptively learns over multiple stages to remove random outliers embedded in the initial consistency. Specifically, we propose an Adaptive Dynamic Graph Construction (ADGC) module to construct a global topological graph with a high inlier ratio by calculating the affinity between the corresponding neighbors. Additionally, we also design a Feature Mapping Processing (FMP) block and Multi-Stage Prediction (MSP) block to improve the accuracy of subsequent local neighborhood information aggregation, as well as information loss due to channel dimension compression in the prediction phase. Experimental results in camera pose estimation, remote sensing image registration, and homography estimation demonstrate that the proposed PMA-Net performs better than other state-of-the-art methods on various public datasets. Code: https://github.com/XiaojieLi11/PMA-Net

Multi-task contrastive learning for change detection in remote sensing images

ABSTRACT This letter proposes a novel multi-task contrastive learning (MTCL) approach for change detection of high-resolution remote sensing images. Current self-supervised learning approaches have limitations in utilizing multiview information of remote sensing images, and they cannot directly train the change detection backbone network for feature learning. The investigated method exploits multiview information contained in remote sensing images to directly train the change detection backbone network. Specifically, multiview views are constructed from high-resolution remote sensing images by handcrafted feature extraction approach, which contain complementary information from different domains. Then each constructed view is utilized for contrastive learning, and the multi-task learning strategy of parameter sharing is used to learn more rich and robust information. Furthermore, instead of learning feature representations from single remote sensing image, the proposed approach directly utilizes image pairs to train the change detection backbone, further improving the performance of change detection network. Two benchmark datasets are employed for comparative experiments, and experimental results certify the effectiveness and superiority of the investigated approach.

Multiscale and multiperception feature learning for pancreatic lesion detection based on noncontrast CT

Background. Pancreatic cancer is one of the most malignant tumours, demonstrating a poor prognosis and nearly identically high mortality and morbidity, mainly because of the difficulty of early diagnosis and timely treatment for localized stages. Objective. To develop a noncontrast CT (NCCT)-based pancreatic lesion detection model that could serve as an intelligent tool for diagnosing pancreatic cancer early, overcoming the challenges associated with low contrast intensities and complex anatomical structures present in NCCT images. Approach. We design a multiscale and multiperception (MSMP) feature learning network with ResNet50 coupled with a feature pyramid network as the backbone for strengthening feature expressions. We added multiscale atrous convolutions to expand different receptive fields, contextual attention to perceive contextual information, and channel and spatial attention to focus on important channels and spatial regions, respectively. The MSMP network then acts as a feature extractor for proposing an NCCT-based pancreatic lesion detection model with image patches covering the pancreas as its input; Faster R-CNN is employed as the detection method for accurately detecting pancreatic lesions. Main results. By using the new MSMP network as a feature extractor, our model outperforms the conventional object detection algorithms in terms of the recall (75.40% and 90.95%), precision (40.84% and 68.21%), F1 score (52.98% and 77.96%), F2 score (64.48% and 85.26%) and Ap50 metrics (53.53% and 70.14%) at the image and patient levels, respectively. Significance.The good performance of our new model implies that MSMP can mine NCCT imaging features for detecting pancreatic lesions from complex backgrounds well. The proposed detection model is expected to be further developed as an intelligent method for the early detection of pancreatic cancer.

Many birds, one stone: Medical image segmentation with multiple partially labeled datasets

Medical image segmentation is fundamental in the field of medical image analysis and has wide clinical applications in disease diagnosis and surgical planning etc. Current prevalent solution is to train a deep network in a fully supervised way with a large-scale fully labeled dataset. However, due to the high labor cost and requirement on medical expertise, such dataset is always absent. Instead, there are multiple partially labeled datasets which are originally established for specific purposes. To make full use of these partially labeled datasets, we propose a novel partially supervised segmentation network, named PSSNet, which consists of a task-specific feature learning network followed by a cross-task attention module (xTA) to exploit task dependencies to enhance task-specific features. To solve the challenges raised by unlabeled classes and domain shift across datasets, we propose an adversarial self-training strategy. We conduct experiments on two medical image segmentation tasks. One is the fine-grained fundus image segmentation aiming to simultaneously segment four-class lesions, OD and OC, and vessels. Validation on seven datasets demonstrates that our PSSNet performs the best among three baselines and three state-of-the-arts. The other is the multiple abdominal organ segmentation in CT images. Our PSSNet is trained on three partially labeled datasets, i.e., LiTS, KiTS and Spleen. Validation on one fully labeled dataset, i.e., BTCV, demonstrates that our PSSNet achieves better performances than four state-of-the-arts. The code is publicly available at https://github.com/CVIU-CSU/PSSNet.

PKG-DTSFLN: Process Knowledge-guided Deep Temporal–spatial Feature Learning Network for anode effects identification

In the aluminum electrolysis process, the accurate identification of anode effect (AE) can improve production efficiency. However, the existing methods fail to effectively capture the features of the anode current signal (ACS) due to its complex dynamic characteristics and temporal–spatial dependence. To address this issue, we propose a Process Knowledge-guided Deep Temporal–spatial Feature Learning Network (PKG-DTSFLN). We believe that knowledge and production data are complementary. Knowledge has potential to deduce beyond observational conditions. Data can be used to detect unexpected patterns. The combination of data and knowledge is potential to improve the performance. Specifically, knowledge is utilized to construct the adjacency matrix to represent the spatial structure of ACS. Then, a deep learning model is constructed by integrating the 1D-CNN and GAT, which is used to capture the temporal–spatial features of ACS. The experimental results on ACS dataset show that the accuracy is more than 99% with low computational cost.