Feature Learning Network Research Articles

ChromTR: chromosome detection in raw metaphase cell images via deformable transformers.

Chromosome karyotyping is a critical way to diagnose various hematological malignancies and genetic diseases, of which chromosome detection in raw metaphase cell images is the most critical and challenging step. In this work, focusing on the joint optimization of chromosome localization and classification, we propose ChromTR to accurately detect and classify 24 classes of chromosomes in raw metaphase cell images. ChromTR incorporates semantic feature learning and class distribution learning into a unified DETR-based detection framework. Specifically, we first propose a Semantic Feature Learning Network (SFLN) for semantic feature extraction and chromosome foreground region segmentation with object-wise supervision. Next, we construct a Semantic-Aware Transformer (SAT) with two parallel encoders and a Semantic-Aware decoder to integrate global visual and semantic features. To provide a prediction with a precise chromosome number and category distribution, a Category Distribution Reasoning Module (CDRM) is built for foreground-background objects and chromosome class distribution reasoning. We evaluate ChromTR on 1404 newly collected R-band metaphase images and the public G-band dataset AutoKary2022. Our proposed ChromTR outperforms all previous chromosome detection methods with an average precision of 92.56% in R-band chromosome detection, surpassing the baseline method by 3.02%. In a clinical test, ChromTR is also confident in tackling normal and numerically abnormal karyotypes. When extended to the chromosome enumeration task, ChromTR also demonstrates state-of-the-art performances on R-band and G-band two metaphase image datasets. Given these superior performances to other methods, our proposed method has been applied to assist clinical karyotype diagnosis.

FLTrans-Net: Transformer-based feature learning network for wheat head detection

The accurate identification of wheat heads is critical for assessing production and managing agricultural fields effectively. One major challenge lies in detecting overlapping and small wheat spikes amidst complex backgrounds. Recent advancements in deep learning, specifically Convolutional Neural Networks (CNNs), have been employed to address this challenge. However, conventional CNNs often struggle with capturing global interdependencies due to their focus on local features and limited scale invariance. To overcome these limitations, this study proposes a Transformer-based Feature Learning Network (FLTrans-Net). FLTrans-Net aims to learn discriminative features while reducing background noise. The methodology comprises three key components: a Multi-Scale Fusion Block, a Spatial Attention Block, and a Lightweight RetinaNet Detection Block. The Multi-Scale Fusion Block integrates high-scale and low-scale features to extract multi-scale representations, essential for detecting small-sized wheat heads in complex backgrounds. The Spatial Attention Block utilizes transformer encoders to highlight salient object characteristics, enhancing the model’s performance. Additionally, a Lightweight RetinaNet Detection Block is incorporated to capture potentially valuable features extracted by preceding blocks. Experimental results demonstrate FLTrans-Net’s effectiveness in wheat head detection in challenging field environments, indicating its potential for real-time deployment on resource-constrained devices. Overall, FLTrans-Net represents a significant advancement in wheat head detection, offering both efficacy and efficiency in handling complex agricultural landscapes.

Time-segment-wise feature fusion transformer for multi-modal fault diagnosis

Mechanical fault diagnosis is crucial to ensure the safe operations of equipment in intelligent manufacturing systems. Recently, deep learning based fault diagnosis methods have achieved remarkable advancements with monitored data from a single sensor. However, obtaining satisfactory diagnostic results based on a single sensor is often difficult because the complementary information between different sensors is ignored. Extracting comprehensive fault features from multi-modal data is a problem that remains to be solved. To address these challenges, a time-segment-wise feature fusion Transformer (FFTR) is proposed in this paper. First, the signals from various modalities as multiple channels are normalized channel-by-channel and form a multi-modal sample. Second, a time-segment-wise feature learning network is designed to transform a multi-modal sample into several fusion features through the sequential processes of sample segmentation, segment-level feature extraction and time-aligned feature fusion. Finally, a Transformer network is employed for comprehensive multi-modal feature analysis and fault classification. In addition, a joint loss function is designed to comprehensively train the end-to-end FFTR. The comparison experiment with other baseline methods is conducted on two multi-modal datasets. The experimental results show that FFTR achieves 3.69% and 3.93% higher diagnostic accuracy than baseline on two datasets respectively and can address real-world problems effectively.

A Dual Transformer-Based Deep Learning Model for Passenger Anomaly Behavior Detection in Elevator Cabs

Effective detection of abnormal behaviors within elevator cabins is critical to ensure elevator safety. While existing deep learning based anomaly detection methods mainly focus on convolutional neural networks for spatial feature extraction and recurrent networks for temporal feature learning, recent advancements in the Transformer architecture have demonstrated its power in time series predictions, and extended its capabilities to vision detection tasks. In this study, we present a duel transformer-based framework that can proficiently detect falling and fighting events in elevator cabs. The proposed solution leverages the vision transformer (ViT) to extract frame-level spatial features, followed by a temporal Transformer to identify abnormalities in surveillance videos. A comprehensive comparison between the proposed transformer-based method and other traditional recurrent neural network variants is carried out to validate the effectiveness of the method.

Class‐wised domain decoupling‐guided adversarial feature learning for cross‐age face recognition

AbstractHow to extract age‐independent identity features from face images has long been the major challenge in cross‐age face recognition task. In this letter, a class‐wised domain decoupling‐guided adversarial feature learning network to extract age‐independent face features is proposed. In this model, an adversarial feature learning module is designed to decompose identity features and age features. Considering that artificially designed age interference suppression strategies are difficult to fit the complex relationship between identity features and age features, an adversarial training strategy incorporated with an attention mechanism in the adversarial feature learning module is introduced to suppress age interference adaptively. Besides, under the idea of maximizing the distribution difference of identity feature domain and age feature domain, a class‐wised domain decoupling module is further designed to guide the model to extract age‐independent identity features. The proposed model has been validated on the well‐known public datasets CALFW and CACD‐VS, where it achieved remarkable recognition accuracy rates of 94.5% and 99.5%, respectively. This represents improvements of 4.2% and 0.1% over the state‐of‐the‐art single‐task‐based models, clearly showcasing the effectiveness of the proposed class‐wised domain decoupling‐guided adversarial feature learning model.

Efficient skin lesion segmentation with boundary distillation.

Medical image segmentation models are commonly known for their complex structures, which often render them impractical for use on edge computing devices and compromising efficiency in the segmentation process. In light of this, the industry has proposed the adoption of knowledge distillation techniques. Nevertheless, the vast majority of existing knowledge distillation methods are focused on the classification tasks of skin diseases. Specifically, for the segmentation tasks of dermoscopy lesion images, these knowledge distillation methods fail to fully recognize the importance of features in the boundary regions of lesions within medical images, lacking boundary awareness for skin lesions. This paper introduces pioneering medical image knowledge distillation architecture. The aim of this method is to facilitate the efficient transfer of knowledge from existing complex medical image segmentation networks to a more simplified student network. Initially, a masked boundary feature (MBF) distillation module is designed. By applying random masking to the periphery of skin lesions, the MBF distillation module obliges the student network to reproduce the comprehensive features of the teacher network. This process, in turn, augments the representational capabilities of the student network. Building on the MBF distillation module, this paper employs a cascaded combination approach to integrate the MBF distillation module into a multi-head boundary feature (M2BF) distillation module, further strengthening the student network's feature learning ability and enhancing the overall image segmentation performance of the distillation model. This method has been experimentally validated on the public datasets ISIC-2016 and PH2, with results showing significant performance improvements in the student network. Our findings highlight the practical utility of the lightweight network distilled using our approach, particularly in scenarios demanding high operational speed and minimal storage usage. This research offers promising prospects for practical applications in the realm of medical image segmentation.

Multi-Branch Feature Learning Network via Global-Local Self-Distillation for Vehicle Re-Identification

Multi-Branch Feature Learning Network via Global-Local Self-Distillation for Vehicle Re-Identification

Coronary artery segmentation in CCTA images based on multi-scale feature learning.

Coronary artery segmentation is a prerequisite in computer-aided diagnosis of Coronary Artery Disease (CAD). However, segmentation of coronary arteries in Coronary Computed Tomography Angiography (CCTA) images faces several challenges. The current segmentation approaches are unable to effectively address these challenges and existing problems such as the need for manual interaction or low segmentation accuracy. A Multi-scale Feature Learning and Rectification (MFLR) network is proposed to tackle the challenges and achieve automatic and accurate segmentation of coronary arteries. The MFLR network introduces a multi-scale feature extraction module in the encoder to effectively capture contextual information under different receptive fields. In the decoder, a feature correction and fusion module is proposed, which employs high-level features containing multi-scale information to correct and guide low-level features, achieving fusion between the two-level features to further improve segmentation performance. The MFLR network achieved the best performance on the dice similarity coefficient, Jaccard index, Recall, F1-score, and 95% Hausdorff distance, for both in-house and public datasets. Experimental results demonstrate the superiority and good generalization ability of the MFLR approach. This study contributes to the accurate diagnosis and treatment of CAD, and it also informs other segmentation applications in medicine.

Comparing Mask R-CNN backbone architectures for human detection using thermal imaging

We introduce a method for detecting humans in thermal imaging using an end-to-end deep learning model. Our objective is to optimize the human detection process in thermal imaging by investigating the mask region-based convolutional neural network (Mask R-CNN). The model, an advancement of the faster region-based convolutional neural network (Faster R-CNN), not only captures bounding boxes encompassing human subjects but also delineates segmentation masks around them. Our investigation extends to the evaluation and comparison of various convolutional neural networks for feature learning, like residual network (ResNet) and Inception ResNet, all integrated into the Mask R-CNN framework. Furthermore, the experimental results show that our proposed technique achieves high accuracy. Specifically, the Mask R-CNN model using ResNet50-V1 achieved the best results, with an F-value of 87.85%, a recall of 79.33%, and a precision of 98.41%.

Supervertex Sampling Network: A Geodesic Differential SLIC Approach for 3D Mesh.

The analysis of 3D meshes with deep learning has become prevalent in computer graphics. As an essential structure, hierarchical representation is critical for mesh pooling in multiscale analysis. Existing clustering-based mesh hierarchy construction methods involve nonlinear discretization optimization operations, making them nondifferential and challenging to embed in other trainable networks for learning. Inspired by deep superpixel learning methods in image processing, we extend them from 2D images to 3D meshes by proposing a novel differentiable chart-based segmentation method named geodesic differential supervertex (GDSV). The key to the GDSV method is to ensure that the geodesic position updates are differentiable while satisfying the constraint that the renewed supervertices lie on the manifold surface. To this end, in addition to using the differential SLIC clustering algorithm to update the nonpositional features of the supervertices, a reparameterization trick, the Gumbel-Softmax trick, is employed to renew the geodesic positions of the supervertices. Therefore, the geodesic position update problem is converted into a linear matrix multiplication issue. The GDSV method can be an independent module for chart-based segmentation tasks. Meanwhile, it can be combined with the front-end feature learning network and the back-end task-specific network as a plug-in-plug-out module for training; and be applied to tasks such as shape classification, part segmentation, and 3D scene understanding. Experimental results show the excellent performance of our proposed algorithm on a range of datasets.

A Joint Network of Edge-Aware and Spectral-Spatial Feature Learning for Hyperspectral Image Classification.

Hyperspectral image (HSI) classification is a vital part of the HSI application field. Since HSIs contain rich spectral information, it is a major challenge to effectively extract deep representation features. In existing methods, although edge data augmentation is used to strengthen the edge representation, a large amount of high-frequency noise is also introduced at the edges. In addition, the importance of different spectra for classification decisions has not been emphasized. Responding to the above challenges, we propose an edge-aware and spectral-spatial feature learning network (ESSN). ESSN contains an edge feature augment block and a spectral-spatial feature extraction block. Firstly, in the edge feature augment block, the edges of the image are sensed, and the edge features of different spectral bands are adaptively strengthened. Then, in the spectral-spatial feature extraction block, the weights of different spectra are adaptively adjusted, and more comprehensive depth representation features are extracted on this basis. Extensive experiments on three publicly available hyperspectral datasets have been conducted, and the experimental results indicate that the proposed method has higher accuracy and immunity to interference compared to state-of-the-art (SOTA) method.

Progressive Discriminative Feature Learning for Visible-Infrared Person Re-Identification

The visible-infrared person re-identification (VI-ReID) task aims to retrieve the same pedestrian between visible and infrared images. VI-ReID is a challenging task due to the huge modality discrepancy and complex intra-modality variations. Existing works mainly complete the modality alignment at one stage. However, aligning modalities at different stages has positive effects on the intra-class and inter-class distances of cross-modality features, which are often ignored. Moreover, discriminative features with identity information may be corrupted in the processing of modality alignment, further degrading the performance of person re-identification. In this paper, we propose a progressive discriminative feature learning (PDFL) network that adopts different alignment strategies at different stages to alleviate the discrepancy and learn discriminative features progressively. Specifically, we first design an adaptive cross fusion module (ACFM) to learn the identity-relevant features via modality alignment with channel-level attention. For well preserving identity information, we propose a dual-attention-guided instance normalization module (DINM), which can well guide instance normalization to align two modalities into a unified feature space through channel and spatial information embedding. Finally, we generate multiple part features of a person to mine subtle differences. Multi-loss optimization is imposed during the training process for more effective learning supervision. Extensive experiments on the public datasets of SYSU-MM01 and RegDB validate that our proposed method performs favorably against most state-of-the-art methods.

Heterogeneous feature learning network for multimodal remote sensing image collaborative classification

ABSTRACT While multimodal remote sensing data processing has garnered growing interests in the geoscience field, how to effectively extract, represent, and fuse heterogeneous features remains challenging. In this article, a novel heterogeneous feature learning network (HFLN) architecture is investigated for multimodal remote sensing collaborative classification, which takes both advantages of long-range feature modelling capability of transformer as well as local feature extraction of convolutional neural network (CNN). First, the global spectral dependencies are extracted from hyperspectral images using the spectral transformer structure. Then, local spatial invariant characteristics are extracted from multimodal remote sensing images by convolutional operation. Next, heterogeneous spectral and spatial characteristics, having significant structural and semantic differences, are dynamically integrated through the feature coupling module in an interactive manner. Finally, the multi-stage network architecture is utilized for extracting hierarchical characteristics, where the numbers of feature maps in CNN and heads in transformer are gradually increased to represent more complex features. Four benchmark remote sensing datasets (i.e. hyperspectral images and multimodal remote sensing datasets) are utilized for experimental analysis, and extensive experimental results have certified the progressiveness of the investigated approach.

A Machine Learning-based Method for COVID-19 and Pneumonia Detection

Pneumonia is described as an acute infection of lung tissue produced by one or more bacteria, and Coronavirus Disease (COVID-19) is a deadly virus that affects the lungs of the human body. The symptoms of COVID-19 disease are closely related to pneumonia. In this work, we identify the patients of pneumonia and coronavirus from chest X-ray images. We used a convolutional neural network for spatial feature learning from X-ray images. We experimented with pneumonia and coronavirus X-ray images in the Kaggle dataset. Pneumonia and corona patients are classified using a feed-forward neural network and hybrid models (CNN+SVM, CNN+RF, and CNN+Xgboost). The experimental findings on the Pneumonia dataset demonstrate that CNN detects Pneumonia patients with 99.47% recall. The overall experiments on COVID-19 x-ray images show that CNN detected the COVID-19 and pneumonia with 95.45% accuracy.

High-Speed Motion Target Real-Time Detection Based on Lightweight Deep Feature Learning Network

High-Speed Motion Target Real-Time Detection Based on Lightweight Deep Feature Learning Network

Reservoir properties inversion using attention-based parallel hybrid network integrating feature selection and transfer learning

The inversion of subsurface reservoir properties is of profound significance to the oil and gas energy development and utilization. The strong heterogeneity and complex pore structure of underground reservoirs pose a challenge to efficient oil and gas energy development and utilization across the world, which increases the necessity of developing an efficient reservoir properties inversion method. However, traditional model-driven methods are confronted with the challenges of strong nonlinearity and geological heterogeneity. Moreover, previous studies rarely emphasized the importance of nonlinear feature selection and transfer learning (TL). Aiming to address the research gaps, a reservoir properties inversion method was proposed by combining random forest (RF) feature selection, bidirectional temporal convolutional network (BiTCN), bidirectional gated recurrent units (BiGRU) network, multi-head attention (MHA) mechanism and TL strategy. First, the RF was used to screen the features with significant correlation with the target reservoir properties. Thereafter, combining BiTCN and BiGRU network to leverage their complementary strengths, a parallel dual branch feature learning network was constructed to learn richer geological information from logging data. Meanwhile, MHA was introduced to fuse the output features of the dual network structure. Finally, the fused features were passed through the fully connected module to output the inversion results. TL was used to associate the correlation between reservoir properties and model inversion to improve the inversion performance. The application results with actual field data showed that the proposed method was accurate and robust in reservoir properties inversion. This study can provide a new way for reliable reservoir properties inversion and promote the application of artificial intelligence in data-driven energy science.

Unified feature learning network for few-shot fault diagnosis

Few-shot fault diagnosis aims at diagnosing the state of mechanical signals with only a few training samples. Numerous contemporary approaches incorporate Time–Frequency Images (TFIs) derived from vibration signals to provide a thorough understanding of both the time and frequency domains. Current approaches often neglect the exploration of sample-level features, which hinders the enrichment and refinement of sample features. To this end, we propose a Unified Feature Learning Network (UFLN) designed to comprehensively model TFIs at two levels. We first present a Texture Enhancement Module (TEM) to amplify intricate details, aiding in the extraction of category-level features. Subsequently, we devise a dynamic Feature Selection Module (DFSM), tailored for the extraction of fault-related features. Finally, we develop an Intra-Diversity (ID) loss function to promote intra-class diversity, enriching the representation of each sample. Extensive experiments on three cases have demonstrated the effectiveness of the proposed UFLN approach.

Dual-branch interactive cross-frequency attention network for deep feature learning

As random noises contained in the high-frequency data could interfere with the feature learning of deep networks, low-pass filtering or wavelet transform have been integrated with deep networks to exclude the high-frequency component of input image. However, useful image details like contour and texture are also lost in such a process. In this paper, we propose Dual-branch interactive Cross-frequency attention Network (DiCaN) to separately process low-frequency and high-frequency components of input image, such that useful information is extracted from high-frequency data and included in deep learning. Our DiCaN first decomposes input image into low-frequency and high-frequency components using wavelet decomposition, and then applies two parallel residual-style branches to extract features from the two components. We further design an interactive cross-frequency attention mechanism, to highlight the useful information in the high-frequency data and interactively fuse them with the features in low-frequency branch. The features learned by our framework are then applied for both image classification and object detection and tested using ImageNet-1K and COCO datasets. The results suggest that DiCaN achieves better classification performance than different ResNet variants. Both one-stage and two-stage detectors with our DiCaN backbone also achieve better detection performance than that with ResNet backbone. The code of DiCaN will be released.

Mus4mCPred: Accurate Identification of DNA N4-Methylcytosine Sites in Mouse Genome Using Multi-View Feature Learning and Deep Hybrid Network

Mus4mCPred: Accurate Identification of DNA N4-Methylcytosine Sites in Mouse Genome Using Multi-View Feature Learning and Deep Hybrid Network

SensorNet: An Adaptive Attention Convolutional Neural Network for Sensor Feature Learning.

This work develops a generalizable neural network, SENSORNET, for sensor feature learning across various applications. The primary challenge addressed is the poor portability of pretrained neural networks to new applications with limited sensor data. To solve this challenge, we design SensorNet, which integrates the flexibility of self-attention with multi-scale feature locality of convolution. Moreover, we invent patch-wise self-attention with stacked multi-heads to enrich the sensor feature representation. SensorNet is generalizable to pervasive applications with any number of sensor inputs, and is much smaller than the state-of-the-art self-attention and convolution hybrid baseline (0.83 M vs. 3.87 M parameters) with similar performance. The experimental results show that SensorNet is able to achieve state-of-the-art performance compared with the top five models on a competition activity recognition dataset (SHL'18). Moreover, pretrained SensorNet in a large inertial measurement unit (IMU) dataset can be fine-tuned to achieve the best accuracy on a much smaller IMU dataset (up to 5% improvement in WISDM) and to achieve the state-of-the-art performance on an EEG dataset (SLEEP-EDF-20), showing the strong generalizability of our approach.