Local Feature Learning Research Articles

Patch-Based Auxiliary Node Classification for Domain Adaptive Object Detection

Domain adaptive object detection (DAOD) aims to leverage labeled source domain data to train object detection models that can generalize well to unlabeled target domains. Recently, many researchers have considered implementing fine-grained pixel-level domain adaptation using graph representations. Existing methods construct semantically complete graphs and align them across domains via graph matching. This work introduced an auxiliary node classification task before domain alignment through graph matching, which utilizes the inherent information of graph nodes to classify them, in order to avoid suboptimal graph matching results caused by node class confusion. However, previous methods neglected the contextual information of graph nodes, leading to biased node classification and suboptimal graph matching. To solve this issue, we propose a novel patch-based auxiliary node classification method for DAOD. Unlike existing methods that use only the inherent information of nodes for node classification, our method exploits the local region information of nodes and employs multi-layer convolutional neural networks to learn the local region feature representation of nodes, enriching the node context information. Thus, accurate and robust node classification results are produced and the risk of class confusion is reduced. Moreover, we propose a progressive strategy to fuse the inherent features and the learned local region features of nodes, which ensures that the network can stably and reliably utilize local region features for accurate node classification. In this paper, we conduct abundant experiments on various DAOD scenarios and demonstrate that our proposed model outperforms existing works.

Rolling-Unet: Revitalizing MLP’s Ability to Efficiently Extract Long-Distance Dependencies for Medical Image Segmentation

Medical image segmentation methods based on deep learning network are mainly divided into CNN and Transformer. However, CNN struggles to capture long-distance dependencies, while Transformer suffers from high computational complexity and poor local feature learning. To efficiently extract and fuse local features and long-range dependencies, this paper proposes Rolling-Unet, which is a CNN model combined with MLP. Specifically, we propose the core R-MLP module, which is responsible for learning the long-distance dependency in a single direction of the whole image. By controlling and combining R-MLP modules in different directions, OR-MLP and DOR-MLP modules are formed to capture long-distance dependencies in multiple directions. Further, Lo2 block is proposed to encode both local context information and long-distance dependencies without excessive computational burden. Lo2 block has the same parameter size and computational complexity as a 3×3 convolution. The experimental results on four public datasets show that Rolling-Unet achieves superior performance compared to the state-of-the-art methods.

Dynamic clustering transformer network for point cloud segmentation

Point cloud segmentation is one of the most important tasks in LiDAR remote sensing with widespread scientific, industrial, and commercial applications. The research thereof has resulted in many breakthroughs in 3D object and scene understanding. Existing methods typically utilize hierarchical architectures for feature representation. However, the commonly used sampling and grouping methods in hierarchical networks are not only time-consuming but also limited to point-wise 3D coordinates, ignoring the local semantic homogeneity of point clusters. To address these issues, we propose a novel 3D point cloud representation network, called Dynamic Clustering Transformer Network (DCTNet). It has an encoder–decoder architecture, allowing for both local and global feature learning. Specifically, the encoder consists of a series of dynamic clustering-based Local Feature Aggregating (LFA) blocks and Transformer-based Global Feature Learning (GFL) blocks. In the LFA block, we propose novel semantic feature-based dynamic sampling and clustering methods, which enable the model to be aware of local semantic homogeneity for local feature aggregation. Furthermore, instead of traditional interpolation approaches, we propose a new semantic feature-guided upsampling method in the decoder for dense prediction. To our knowledge, DCTNet is the first work to introduce semantic information-based dynamic clustering into 3D Transformers. Extensive experiments on an object-based dataset (ShapeNet), and an airborne multispectral LiDAR dataset demonstrate the State-of-the-Art (SOTA) segmentation performance of DCTNet in terms of both accuracy and efficiency. Our code will be made publicly available.

Attention-Based DenseNet for Lung Cancer Classification Using CT Scan and Histopathological Images

Lung cancer is identified by the uncontrolled proliferation of cells in lung tissues. The timely detection of malignant cells in the lungs, crucial for processes such as oxygen provision and carbon dioxide elimination in the human body, is imperative. The application of deep learning for discerning lymph node involvement in CT scans and histopathological images has garnered widespread attention due to its potential impact on patient diagnosis and treatment. This paper suggests employing DenseNet for lung cancer detection, leveraging its ability to transmit learned features backward through each layer continuously. This characteristic not only reduces model parameters but also enhances the learning of local features, facilitating a better comprehension of the structural complexity and uneven distribution in CT scans and histopathological cancer images. Furthermore, DenseNet accompanied by an attention mechanism (ATT-DenseNet) allows the model to focus on specific parts of an image, giving more weight to relevant regions. Compared to existing algorithms, the ATT-DenseNet demonstrates a remarkable enhancement in accuracy, precision, recall, and the F1-Score. It achieves an average improvement of 20% in accuracy, 19.66% in precision, 24.33% in recall, and 22.33% in the F1-Score across these metrics. The motivation behind the research is to leverage deep learning technologies to enhance the precision and reliability of lung cancer diagnostics, thus addressing the gap in early detection and treatment. This pursuit is driven by the potential of deep learning models, like DenseNet, to provide significant improvements in analyzing complex medical images for better clinical outcomes.

Global–local feature learning for fine-grained food classification based on Swin Transformer

Separable object parts, such as the head and tail in a bird, are vital for fine-grained visual classifications. For those objects without separable parts, the classification task relies only on local and global textural image features. Although the Swin Transformer architecture was proposed to efficiently capture both local and global visual features, it still exhibits a bias towards global features. Therefore, our goal is to enhance the local feature learning capability of the Swin Transformer by adding four new modules of the Local Feature Extraction Network (L-FEN), Convolution Patch-Merging (CP), Multi-Path (MP), and Multi-View (MV). The L-FEN enhances the Swin transformer with the improved local feature capture. The CP is a localized and hierarchical adaptation of the Swin’s Patch Merging technique. The MP method integrates features across various Swin stages to accentuate local details. Meanwhile, the MV Swin transformer block supersedes traditional Swin blocks with those incorporating varied receptive fields, ensuring a broader scope of local feature capture. Our enhanced architecture, named Global–Local Swin Transformer (GL-Swin), is applied to solve a fine-grained food classification task. On three major food datasets: ISIA Food-500 UEC Food-256, and Food-101, our GL-Swin achieved accuracies of 66.75%, 85.78%, and 92.93% respectively, consistently outperforming other leading methods.

Privacy-Preserving Face Recognition Method Based on Randomization and Local Feature Learning.

Personal privacy protection has been extensively investigated. The privacy protection of face recognition applications combines face privacy protection with face recognition. Traditional face privacy-protection methods encrypt or perturb facial images for protection. However, the original facial images or parameters need to be restored during recognition. In this paper, it is found that faces can still be recognized correctly when only some of the high-order and local feature information from faces is retained, while the rest of the information is fuzzed. Based on this, a privacy-preserving face recognition method combining random convolution and self-learning batch normalization is proposed. This method generates a privacy-preserved scrambled facial image and an image fuzzy degree that is close to an encryption of the image. The server directly recognizes the scrambled facial image, and the recognition accuracy is equivalent to that of the normal facial image. The method ensures the revocability and irreversibility of the privacy preserving of faces at the same time. In this experiment, the proposed method is tested on the LFW, Celeba, and self-collected face datasets. On the three datasets, the proposed method outperforms the existing face privacy-preserving recognition methods in terms of face visual information elimination and recognition accuracy. The recognition accuracy is >99%, and the visual information elimination is close to an encryption effect.

Convolutional transformer-driven robust electrocardiogram signal denoising framework with adaptive parametric ReLU.

The electrocardiogram (ECG) is a widely used diagnostic tool for cardiovascular diseases. However, ECG recording is often subject to various noises, which can limit its clinical evaluation. To address this issue, we propose a novel Transformer-based convolutional neural network framework with adaptively parametric ReLU (APtrans-CNN) for ECG signal denoising. The proposed APtrans-CNN architecture combines the strengths of transformers in global feature learning and CNNs in local feature learning to address the inadequacy of learning with long sequence time-series features. By fully exploiting the global features of ECG signals, our framework can effectively extract critical information that is necessary for signal denoising. We also introduce an adaptively parametric ReLU that can assign a value to the negative information contained in the ECG signal, thereby overcoming the limitation of ReLU to retain negative information. Additionally, we introduce a dynamic feature aggregation module that enables automatic learning and retention of valuable features while discarding useless noise information. Results obtained from two datasets demonstrate that our proposed APtrans-CNN can accurately extract pure ECG signals from noisy datasets and is adaptable to various applications. Specifically, when the input consists of ECG signals with a signal-to-noise ratio (SNR) of -4 dB, APtrans-CNN successfully increases the SNR to more than 6 dB, resulting in the diagnostic model's accuracy exceeding 96%.

SCGFormer: Semantic Chebyshev Graph Convolution Transformer for 3D Human Pose Estimation

With the rapid advancement of deep learning, 3D human pose estimation has largely freed itself from reliance on manually annotated methods. The effective utilization of joint features has become significant. Utilizing 2D human joint information to predict 3D human skeletons is of paramount importance. Effectively leveraging 2D joint data can improve the accuracy of 3D human skeleton prediction. In this paper, we propose the SCGFormer model to reduce the error in predicting human skeletal poses in three-dimensional space. The network architecture of SCGFormer encompasses Transformer and two distinct types of graph convolution, organized into two interconnected modules: SGraAttention and AcChebGconv. SGraAttention extracts global feature information from each 2D human joint, thereby augmenting local feature learning by integrating prior knowledge of human joint relationships. Simultaneously, AcChebGconv broadens the receptive field for graph structure information and constructs implicit joint relationships to aggregate more valuable adjacent features. SCGraFormer is tested on widely recognized benchmark datasets such as Human3.6M and MPI-INF-3DHP and achieves excellent results. In particular, on Human3.6M, our method achieves the best results in 9 actions (out of a total of 15 actions), with an overall average error reduction of about 1.5 points compared to state-of-the-art methods, demonstrating the excellent performance of SCGFormer.

Dual-stream pyramid and attention network for stereo matching

In order to accurately estimate the disparity of ill-posed regions, such as weak texture and occlusion regions, we propose DSPANet, a stereo matching network that incorporates a dual-stream pyramid module and a channel and spatial attention module. The dual-stream pyramid module captures numerous complementary features from different layers by utilizing multi-resolution inputs and feature extraction blocks. This approach enables the learning of local detailed features at various scales. These features at various scales are then combined to calculate the stereo matching cost. By incorporating channel and spatial attention module into the feature extraction process to obtain richer and more concise contextual information, the matching cost can be constructed more accurately, which provides powerful conditions for subsequent cost aggregation. In the cost aggregation stage, we utilize the stacked hourglass module for both encoding and decoding. Additionally, we incorporate 3D global attention upsampling during the decoding stage, which enables high-level features to provide guidance information to low-level features in a simple way. We evaluate our proposed method on the Scene Flow dataset, as well as the KITTI2012 and KITTI2015 datasets. The experimental results demonstrate that our DSPANet achieves superior performance and effectively enhances the matching results in ill-posed regions. Our code has been implemented using PyTorch and will be released after paper publication at https://github.com/Shi-LiQing/DSPANet.

Delineating Sight Occlusions of Head-On Traffic Signboards under Varying Available Sight Distances Using LiDAR Point Clouds

It is essential to delineate visual occlusions of head-on traffic signboards under varying available sight distances (ASDs) to improve road safety and guide maintenance. However, the status of sight occlusions in actual road scenarios is unclear because of the lack of an automatic and quantitative evaluation approach. This study presents a new cluster attention traffic-sign occlusion (CASO) method with a cluster attention traffic-sign network (CASNet) to automatically classify road infrastructures, and an occlusion delineating module to dynamically describe the occlusions of head-on traffic signboards using point clouds. CASNet consists of a cluster module to alleviate the interference of redundant object features and an attention module to focus on learning local features of small samples on road scenes. The occlusion delineating module is investigated to rapidly construct a signboard-related oblique cone and dynamically assess occlusions. The sight occlusions change with varying ASDs and are delineated by two indices: the degree of shaded signboard area, and the degree of occlusion volume from the driver’s dynamic perspective to the head-on signboard. Compared with the state-of-the-art networks, the experimental extraction results achieved for signboards show an overall improved performance. The degree of occlusion volumes toward head-on signboards fluctuates between 26.75% and 36.70%, and the degree of shaded areas on signboards increases from 22.75% to 59.63%, with ASDs varying from 20 to 75 m. This research contributes to evaluating road safety in intelligent transportation systems and accurately guiding the allocation of maintenance budgets to the heavily occluded road sections.

CMAF: Cross-Modal Augmentation via Fusion for Underwater Acoustic Image Recognition

Underwater image recognition is crucial for underwater detection applications. Fish classification has been one of the emerging research areas in recent years. Existing image classification models usually classify data collected from terrestrial environments. However, existing image classification models trained with terrestrial data are unsuitable for underwater images, as identifying underwater data is challenging due to their incomplete and noisy features. To address this, we propose a cross-modal augmentation via fusion ( CMAF ) framework for acoustic-based fish image classification. Our approach involves separating the process into two branches: visual modality and sonar signal modality, where the latter provides a complementary character feature. We augment the visual modality, design an attention-based fusion module, and adopt a masking-based training strategy with a mask-based focal loss to improve the learning of local features and address the class imbalance problem. Our proposed method outperforms the state-of-the-art methods. Our source code is available at https://github.com/WilkinsYang/CMAF .

DomainFeat: Learning Local Features With Domain Adaptation

Accurate and efficient keypoint detection and description is a fundamental step in various computer vision tasks. In this paper, we extract robust descriptors and detect accurate keypoints by learning local Features with Domain adaptation (DomainFeat). Specifically, our Domainfeat includes image-level domain invariance supervision, pixel-level domain consistency supervision, Pixel-Adaptive keypoint Detection(PA-Det), and cross-domain dataset with domain stable point supervision. First, we introduce the image-level domain invariance supervision to make the high-level feature distributions from different domains close by fusing domain-invariant representations in the decoder. Furthermore, to compensate for the inconsistency between descriptors corresponding to the keypoints at the pixel level, we propose the pixel-level domain consistency supervision. Then we present the Pixel-Adaptive keypoint Detection to efficiently detect accurate keypoints, which can improve accuracy by enhancing the local consistency of heatmaps. Finally, we propose an efficient approach to construct data and supervision labels in diverse domains, which can tackle complex application scenarios. With these novel modules and supervision methods, our DomainFeat can make feature detectors more accurate and descriptors more robust. Extensive experiments confirm that Domainfeat achieves state-of-the-art performance on benchmarks such as Aachen-Day-Night localization, HPatches image matching, and the challenging DNIM dataset.

Learning Local Features by Jointly Semantic-guided and Task Rewards

Learning Local Features by Jointly Semantic-guided and Task Rewards

Unsupervised learning of local features for person re-identication with loss funciton

Unsupervised learning of local features for person re-identication with loss funciton

Attention-Based Multimodal tCNN for Classification of Steady-State Visual Evoked Potentials and Its Application to Gripper Control.

The classification problem for short time-window steady-state visual evoked potentials (SSVEPs) is important in practical applications because shorter time-window often means faster response speed. By combining the advantages of the local feature learning ability of convolutional neural network (CNN) and the feature importance distinguishing ability of attention mechanism, a novel network called AttentCNN is proposed to further improve the classification performance for short time-window SSVEP. Considering the frequency-domain features extracted from short time-window signals are not obvious, this network starts with the time-domain feature extraction module based on the filter bank (FB). The FB consists of four sixth-order Butterworth filters with different bandpass ranges. Then extracted multimodal features are aggregated together. The second major module is a set of residual squeeze and excitation blocks (RSEs) that has the ability to improve the quality of extracted features by learning the interdependence between features. The final major module is time-domain CNN (tCNN) that consists of four CNNs for further feature extraction and followed by a fully connected (FC) layer for output. Our designed networks are validated over two large public datasets, and necessary comparisons are given to verify the effectiveness and superiority of the proposed network. In the end, in order to demonstrate the application potential of the proposed strategy in the medical rehabilitation field, we design a novel five-finger bionic hand and connect it to our trained network to achieve the control of bionic hand by human brain signals directly. Our source codes are available on Github: https://github.com/JiannanChen/AggtCNN.git.

Enhanced Local Feature Learning With Simple Offset Attention for Semantic Segmentation of Large-Scale Point Clouds

Enhanced Local Feature Learning With Simple Offset Attention for Semantic Segmentation of Large-Scale Point Clouds

Shared Coupling-Bridge Scheme for Weakly Supervised Local Feature Learning

Shared Coupling-Bridge Scheme for Weakly Supervised Local Feature Learning

JLONMFSC: Clustering scRNA-seq data based on joint learning of non-negative matrix factorization and subspace clustering

The development of single cell RNA sequencing (scRNA-seq) has provided new perspectives to study biological problems at the single cell level. One of the key issues in scRNA-seq data analysis is to divide cells into several clusters for discovering the heterogeneity and diversity of cells. However, the existing scRNA-seq data are high-dimensional, sparse, and noisy, which challenges the existing single-cell clustering methods. In this study, we propose a joint learning framework (JLONMFSC) for clustering scRNA-seq data. In our method, the dimension of the original data is reduced to minimize the effect of noise. In addition, the graph regularized matrix factorization is used to learn the local features. Further, the Low-Rank Representation (LRR) subspace clustering is utilized to learn the global features. Finally, the joint learning of local features and global features is performed to obtain the results of clustering. We compare the proposed algorithm with eight state-of-the-art algorithms for clustering performance on six datasets, and the experimental results demonstrate that the JLONMFSC achieves better performance in all datasets. The code is avalable at https://github.com/lanbiolab/JLONMFSC.

Estimating chronological age through learning local and global features of panoramic radiographs in the Korean population

This study suggests a hybrid method based on ResNet50 and vision transformer (ViT) in an age estimation model. To this end, panoramic radiographs are used for learning by considering both local features and global information, which is important in estimating age. Transverse and longitudinal panoramic images of 9663 patients were selected (4774 males and 4889 females with a mean age of 39 years and 3 months). To compare ResNet50, ViT, and the hybrid model, the mean absolute error, mean square error, root mean square error, and coefficient of determination (R2) were used as metrics. The results confirmed that the age estimation model designed using the hybrid method performed better than those using only ResNet50 or ViT. The estimation is highly accurate for young people at an age with distinct growth characteristics. When examining the basis for age estimation in the hybrid model through attention rollout, the proposed model used logical and important factors rather than relying on unclear elements as the basis for age estimation.

Bio-inspired affordance learning for 6-DoF robotic grasping: A transformer-based global feature encoding approach

The 6-Degree-of-Freedom (6-DoF) robotic grasping is a fundamental task in robot manipulation, aimed at detecting graspable points and corresponding parameters in a 3D space, i.e affordance learning, and then a robot executes grasp actions with the detected affordances. Existing research works on affordance learning predominantly focus on learning local features directly for each grid in a voxel scene or each point in a point cloud scene, subsequently filtering the most promising candidate for execution. Contrarily, cognitive models of grasping highlight the significance of global descriptors, such as size, shape, and orientation, in grasping. These global descriptors indicate a grasp path closely tied to actions. Inspired by this, we propose a novel bio-inspired neural network that explicitly incorporates global feature encoding. In particular, our method utilizes a Truncated Signed Distance Function (TSDF) as input, and employs the recently proposed Transformer model to encode the global features of a scene directly. With the effective global representation, we then use deconvolution modules to decode multiple local features to generate graspable candidates. In addition, to integrate global and local features, we propose using a skip-connection module to merge lower-layer global features with higher-layer local features. Our approach, when tested on a recently proposed pile and packed grasping dataset for a decluttering task, surpassed state-of-the-art local feature learning methods by approximately 5% in terms of success and declutter rates. We also evaluated its running time and generalization ability, further demonstrating its superiority. We deployed our model on a Franka Panda robot arm, with real-world results aligning well with simulation data. This underscores our approach’s effectiveness for generalization and real-world applications.