Local Feature Learning Research Articles

Global induced local network for infrared: dim small target detection

Abstract It is challenging to detect infrared dim targets submerged in complicated backgrounds due to their small size and faint intensity. The previous attention-based detection networks frequently require global long-range dependence. Significant calculations are required to determine the target's sparse but meaningful position. To prevent wasting calculations on the background, this paper offers a detection network guided by global context for local feature learning, named GILNet (Global Induced Local Network). It designs a global location module (GLM) and a local feature interaction module (LFIM) to capture the global position and features of targets, respectively. More specifically, using global context interaction, the GLM finds the region that might contain dim small targets, that is, the coarse location. In the coarsely located regions, the LFIM further acquires feature information about targets. Next, we also design an eight-directional attention operation to obtain the contour information of targets in the low feature map. It is fused with the high feature map in the multi-directional feature fusion module (MFFM), which retains more semantic and spatial information about targets. Finally, quantitative and qualitative analysis show that the GILNet performs better than eight comparison methods on two public datasets.

Improving Human Pose Estimation with Integrated Dual Self-Attention Mechanism in High-Resolution Network

Human Pose Estimation (HPE) in computer vision (CV) has garnered significant attention due to its diverse applications. Deep convolutional neural networks (CNNs) may be solutions for addressing this challenge, but still face several critical issues. Many existing models employ serial convolution with pooling, leading to low-resolution outputs that are suboptimal for the precise localisation required in HPE. They often prioritise local feature learning, overlooking crucial contextual relationships between key-points. This work addresses these challenges by proposing a novel approach for enhancing HPE. Firstly, the paper evaluates the high-resolution network (HRNet) and its comparative advantages over other CNN architectures. Secondly, it introduces a dual self-attention (DSA) mechanism designed to enhance the model’s global awareness, thereby enriching feature maps with contextual information. Thirdly, it integrates the DSA mechanism into HRNet, crafting DSA-HRNet. The model performance was tested on the COCO Val 2017 validation dataset, showing improvements of 2.3% in mean average precision (mAP), 3% in AP at 50 (AP50), and 2.7% in AP at 75 (AP75). Finally, the work includes an investigation into the effectiveness of the DSA mechanism within the HRNet framework, through a series of experiments, showing this work offers a streamlined and effective solution for improving HPE.

Unsupervised anomaly detection of multivariate time series based on multi-standard fusion

With the wide application of the Internet of Things, industrial systems generate a large amount of multivariate time series data every day. Industrial data often contains a wide variety of anomalies. The existing anomaly detection methods have some shortcomings in dealing with large-scale, high-dimensional, and real-time industrial data. Particularly in the absence of prior knowledge, unsupervised learning methods are easily influenced by noise. Therefore, we propose a new unsupervised anomaly detection algorithm of multivariate time series based on multi-standard fusion (MSAD). Specifically, MSAD first classifies the data by analyzing the data density and sample spacing, thereby converting unsupervised anomaly detection into weakly supervised anomaly detection. Then, based on the characteristics of the data and deep learning technology, the degree of abnormality in the sample is comprehensively measured from four perspectives: holistic, local, time, and deep feature learning. At this stage, MSAD 's ability to learn edge features of data has been improved. Finally, based on MSAD, we design an anomaly detection method for industrial data streams in combination with data filtering techniques. The experimental results show that MSAD can detect anomalies more accurately and show stronger robustness to the influence of noise.

SpemNet: A Cotton Disease and Pest Identification Method Based on Efficient Multi-Scale Attention and Stacking Patch Embedding.

We propose a cotton pest and disease recognition method, SpemNet, based on efficient multi-scale attention and stacking patch embedding. By introducing the SPE module and the EMA module, we successfully solve the problems of local feature learning difficulty and insufficient multi-scale feature integration in the traditional Vision Transformer model, which significantly improve the performance and efficiency of the model. In our experiments, we comprehensively validate the SpemNet model on the CottonInsect dataset, and the results show that SpemNet performs well in the cotton pest recognition task, with significant effectiveness and superiority. The SpemNet model excels in key metrics such as precision and F1 score, demonstrating significant potential and superiority in the cotton pest and disease recognition task. This study provides an efficient and reliable solution in the field of cotton pest and disease identification, which is of great theoretical and applied significance.

A high-quality self-supervised image denoising method based on SDDW-GAN and CHRNet

Image denoising remains a classic and crucial issue in the field of image processing, significantly impacting the outcomes of subsequent image processing tasks. For instance, the denoising network depends on “noise-clean” image pairs to train network effectively. However, it is often hampered by issues such as useful information loss, low training efficiency, and poor blind denoising. To address these challenges, this study proposes a novel image denoising network that integrates the complementary strengths of model-based and learning-based approaches, specifically leveraging the capabilities of both SDDW-GAN and CHRNet. Firstly, SDDW-GAN is designed to estimate the noise distribution on the input noisy images, and a fast-smoothing noisy block sampling algorithm is proposed to extract the noise blocks in noisy images in SDDW-GAN. Secondly, a network with dual generators and dual discriminators based on W-GAN is designed to estimate the noise distribution on the input noisy images and generate noise sample pairs with the same noise distribution, which solves the problem of relying on “noise-clean” image pairs. Thirdly, CHRNet is designed to compute the mapping relationship between the double-noise samples and the single-noise samples. In order to further improve the denoising effect, the dual-channel residual attention module is proposed for fusion learning of global and local features. Experimental results show that the proposed method has a better denoising effect in complex environments and outperforms existing denoising methods. Specifically, in comparison with the stand-alone denoising methods BM3D, DnCNN, Noise2Noise, and Blind2Unblind, the proposed method improves the average peak signal-to-noise ratio (PSNR) by 0.23 dB to 0.78 dB on two benchmarking datasets crossing different noise levels. Its denoising effect is also greater than other competitive stand-alone and combination methods. The proposed method can also extend to low-light image enhancement, deblurring, and super-resolution.

EAAC-Net: An Efficient Adaptive Attention and Convolution Fusion Network for Skin Lesion Segmentation.

Accurate segmentation of skin lesions in dermoscopic images is of key importance for quantitative analysis of melanoma. Although existing medical image segmentation methods significantly improve skin lesion segmentation, they still have limitations in extracting local features with global information, do not handle challenging lesions well, and usually have a large number of parameters and high computational complexity. To address these issues, this paper proposes an efficient adaptive attention and convolutional fusion network for skin lesion segmentation (EAAC-Net). We designed two parallel encoders, where the efficient adaptive attention feature extraction module (EAAM) adaptively establishes global spatial dependence and global channel dependence by constructing the adjacency matrix of the directed graph and can adaptively filter out the least relevant tokens at the coarse-grained region level, thus reducing the computational complexity of the self-attention mechanism. The efficient multiscale attention-based convolution module (EMA⋅C) utilizes multiscale attention for cross-space learning of local features extracted from the convolutional layer to enhance the representation of richly detailed local features. In addition, we designed a reverse attention feature fusion module (RAFM) to enhance the effective boundary information gradually. To validate the performance of our proposed network, we compared it with other methods on ISIC 2016, ISIC 2018, and PH2 public datasets, and the experimental results show that EAAC-Net has superior segmentation performance under commonly used evaluation metrics.

Liver vessel MRI image segmentation based on dual-path diffusion model

Accurate segmentation of liver blood vessels in magnetic resonance imaging (MRI) images is a challenging task due to the complex tree-like structure and anisotropic diffusion properties of blood vessels. To solve this problem, we propose a new Dual-Path Diffusion Model (DPDM) framework. The framework consists of two collaborative diffusion paths: a local feature learning path based on convolution operations and a global context modeling path based on transform blocks. Local path encodes rich shape priors and preserve spatial details, while global path captures long-distance dependencies and enhance representation. In the decoding phase, the boundary features from the boundary path are fused with the features of the ordinary path decoding, which further enhances the shape sensitivity. In addition, we leverage a multi-task learning scheme to jointly optimize vascular segmentation and boundary prediction tasks in an end-to-end manner. Experiments on retrospective clinical dataset demonstrate that the proposed DPDM framework achieves excellent performance on the liver vessel segmentation task. Compared with state-of-the-art methods, our approach achieved a 6.0% and 7.3% performance improvement in Dice coefficient and IoU index, respectively. Our approach offers a promising solution for automated blood vessel segmentation in precision medicine.

Intelligent skin lesion segmentation using deformable attention Transformer U-Net with bidirectional attention mechanism in skin cancer images.

In recent years, the increasing prevalence of skin cancers, particularly malignant melanoma, has become a major concern for public health. The development of accurate automated segmentation techniques for skin lesions holds immense potential in alleviating the burden on medical professionals. It is of substantial clinical importance for the early identification and intervention of skin cancer. Nevertheless, the irregular shape, uneven color, and noise interference of the skin lesions have presented significant challenges to the precise segmentation. Therefore, it is crucial to develop a high-precision and intelligent skin lesion segmentation framework for clinical treatment. A precision-driven segmentation model for skin cancer images is proposed based on the Transformer U-Net, called BiADATU-Net, which integrates the deformable attention Transformer and bidirectional attention blocks into the U-Net. The encoder part utilizes deformable attention Transformer with dual attention block, allowing adaptive learning of global and local features. The decoder part incorporates specifically tailored scSE attention modules within skip connection layers to capture image-specific context information for strong feature fusion. Additionally, deformable convolution is aggregated into two different attention blocks to learn irregular lesion features for high-precision prediction. A series of experiments are conducted on four skin cancer image datasets (i.e., ISIC2016, ISIC2017, ISIC2018, and PH2). The findings show that our model exhibits satisfactory segmentation performance, all achieving an accuracy rate of over 96%. Our experiment results validate the proposed BiADATU-Net achieves competitive performance supremacy compared to some state-of-the-art methods. It is potential and valuable in the field of skin lesion segmentation.

Meta-Path Semantic and Global-Local Representation Learning Enhanced Graph Convolutional Model for Disease-Related miRNA Prediction.

Dysregulation of miRNAs is closely related to the progression of various diseases, so identifying disease-related miRNAs is crucial. Most recently proposed methods are based on graph reasoning, while they did not completely exploit the topological structure composed of the higher-order neighbor nodes and the global and local features of miRNA and disease nodes. We proposed a prediction method, MDAP, to learn semantic features of miRNA and disease nodes based on various meta-paths, as well as node features from the entire heterogeneous network perspective, and node pair attributes. Firstly, for both the miRNA and disease nodes, node category-wise meta-paths were constructed to integrate the similarity and association connection relationships. Each target node has its specific neighbor nodes for each meta-path, and the neighbors of longer meta-paths constitute its higher-order neighbor topological structure. Secondly, we constructed a meta-path specific graph convolutional network module to integrate the features of higher-order neighbors and their topology, and then learned the semantic representations of nodes. Thirdly, for the entire miRNA-disease heterogeneous network, a global-aware graph convolutional autoencoder was built to learn the network-view feature representations of nodes. We also designed semantic-level and representation-level attentions to obtain informative semantic features and node representations. Finally, the strategy based on the parallel convolutional-deconvolutional neural networks were designed to enhance the local feature learning for a pair of miRNA and disease nodes. The experiment results showed that MDAP outperformed other state-of-the-art methods, and the ablation experiments demonstrated the effectiveness of MDAP's major innovations. MDAP's ability in discovering potential disease-related miRNAs was further analyzed by the case studies over three diseases.

A universal multiple instance learning framework for whole slide image analysis

BackgroundThe emergence of digital whole slide image (WSI) has driven the development of computational pathology. However, obtaining patch-level annotations is challenging and time-consuming due to the high resolution of WSI, which limits the applicability of fully supervised methods. We aim to address the challenges related to patch-level annotations. MethodsWe propose a universal framework for weakly supervised WSI analysis based on Multiple Instance Learning (MIL). To achieve effective aggregation of instance features, we design a feature aggregation module from multiple dimensions by considering feature distribution, instances correlation and instance-level evaluation. First, we implement instance-level standardization layer and deep projection unit to improve the separation of instances in the feature space. Then, a self-attention mechanism is employed to explore dependencies between instances. Additionally, an instance-level pseudo-label evaluation method is introduced to enhance the available information during the weak supervision process. Finally, a bag-level classifier is used to obtain preliminary WSI classification results. To achieve even more accurate WSI label predictions, we have designed a key instance selection module that strengthens the learning of local features for instances. Combining the results from both modules leads to an improvement in WSI prediction accuracy. ResultsExperiments conducted on Camelyon16, TCGA-NSCLC, SICAPv2, PANDA and classical MIL benchmark datasets demonstrate that our proposed method achieves a competitive performance compared to some recent methods, with maximum improvement of 14.6 % in terms of classification accuracy. ConclusionOur method can improve the classification accuracy of whole slide images in a weakly supervised way, and more accurately detect lesion areas.

A vulnerability detection framework with enhanced graph feature learning

Vulnerability detection in smart contracts is critical to secure blockchain systems. Existing methods represent the bytecode as a graph structure and leverage graph neural networks to learn graph features for vulnerability detection. However, these methods are limited to handling the long-range dependencies between nodes. This means that they might focus on learning local node feature while ignoring global node information. In this paper, we propose a novel vulnerability detection framework with Enhanced Graph Feature Learning (EGFL), which aims to extract the global node information and utilize it to improve vulnerability detection in smart contracts. Specifically, we first represent the bytecode as a Control Flow Graph (CFG). To extract global node information, EGFL constructs a linear node feature matrix from CFG, and uses the feature-aware and relationship-aware modules to handle long-range dependencies between nodes. Meanwhile, a graph neural network is adopted to extract the local node feature from CFG. Subsequently, we fuse the global node information and local node feature to generate an enhanced graph feature for capturing more vulnerability features. We evaluate EGFL on the benchmark dataset with six types of smart contract vulnerabilities. Results show that EGFL outperforms fourteen state-of-the-art vulnerability detection methods by 10.83%–60.28% in F1 score.

TriClsNet: Surface Reconstruction via Graph-based Triangle Classification

In this paper, we introduce TriClsNet, a novel learning-based network that reconstructs surfaces by reframing the triangle classification problem as a graph node classification problem. An improved graph-based triangle classification module is employed to aggregate information from neighboring triangles, effectively leveraging local neighborhood information and enhancing triangle classification accuracy. Additionally, a self-supervised learning branch is incorporated to predict point cloud normals, aiding our network in better learning local point cloud features. Furthermore, a new loss function is designed to guide our network in effective multi-task learning, encompassing both graph node classification and normal prediction. Comparative experimental results on ShapeNet demonstrate that our method can efficiently perform surface reconstruction, outperforming existing methods in the aspects of preserving surface details, reducing holes, and generalization.

JGULF: Joint global and unilateral local feature network for micro-expression recognition

Micro-expression is a subtle facial movement that is fleeting and manifest in localized areas, making it difficult for the human eye to detect and recognize it. Although algorithms that extract facial features from specific regions or the entire face have shown potential, the classification of micro-expressions using features from symmetrical left and right regions can be challenging in the presence of unilateral movements. This can ultimately affect the performance of micro-expression recognition. To address this issue, we propose a network called Joint Global and Unilateral Local Features (JGULF) for micro-expression recognition. Initially, we employ a Convolutional Neural Network (CNN) and an adjusted Vision Transformer (ViT) model to extract global features from micro-expressions. The local feature extraction module is designed based on global features. The facial features are divided into multiple local regions with varying scales. After that, local feature learning and selection are performed to filter out unilateral local features related to micro-expression movements efficiently. Finally, global and local features are combined to classify micro-expressions. Through comprehensive experimental validation, our algorithm achieves state-of-the-art classification performance on the SMIC, CASMEII, and SAMM micro-expression datasets, demonstrating the effectiveness of combining global features and selecting local features.

A Multi-Level Relation-Aware Transformer model for occluded person re-identification

Occluded person re-identification (Re-ID) is a challenging task, as pedestrians are often obstructed by various occlusions, such as non-pedestrian objects or non-target pedestrians. Previous methods have heavily relied on auxiliary models to obtain information in unoccluded regions, such as human pose estimation. However, these auxiliary models fall short in accounting for pedestrian occlusions, thereby leading to potential misrepresentations. In addition, some previous works learned feature representations from single images, ignoring the potential relations among samples. To address these issues, this paper introduces a Multi-Level Relation-Aware Transformer (MLRAT) model for occluded person Re-ID. This model mainly encompasses two novel modules: Patch-Level Relation-Aware (PLRA) and Sample-Level Relation-Aware (SLRA). PLRA learns fine-grained local features by modeling the structural relations between key patches, bypassing the dependency on auxiliary models. It adopts a model-free method to select key patches that have high semantic correlation with the final pedestrian representation. In particular, to alleviate the interference of occlusion, PLRA captures the structural relations among key patches via a two-layer Graph Convolution Network (GCN), effectively guiding the local feature fusion and learning. SLRA is designed to facilitate the model to learn discriminative features by modeling the relations among samples. Specifically, to mitigate noisy relations of irrelevant samples, we present a Relation-Aware Transformer (RAT) block to capture the relations among neighbors. Furthermore, to bridge the gap between training and testing phases, a self-distillation method is employed to transfer the sample-level relations captured by SLRA to the backbone. Extensive experiments are conducted on four occluded datasets, two partial datasets and two holistic datasets. The results show that the proposed MLRAT model significantly outperforms existing baselines on four occluded datasets, while maintains top performance on two partial datasets and two holistic datasets.

A Novel Improved Variational Mode Decomposition-Temporal Convolutional Network-Gated Recurrent Unit with Multi-Head Attention Mechanism for Enhanced Photovoltaic Power Forecasting

Photovoltaic (PV) power forecasting plays a crucial role in optimizing renewable energy integration into the grid, necessitating accurate predictions to mitigate the inherent variability of solar energy generation. We propose a novel forecasting model that combines improved variational mode decomposition (IVMD) with the temporal convolutional network-gated recurrent unit (TCN-GRU) architecture, enriched with a multi-head attention mechanism. By focusing on four key environmental factors influencing PV output, the proposed IVMD-TCN-GRU framework targets a significant research gap in renewable energy forecasting methodologies. Initially, leveraging the sparrow search algorithm (SSA), we optimize the parameters of VMD, including the mode component K-value and penalty factor, based on the minimum envelope entropy principle. The optimized VMD then decomposes PV power, while the TCN-GRU model harnesses TCN’s proficiency in learning local temporal features and GRU’s capability in rapidly modeling sequence data, while leveraging multi-head attention to better utilize the global correlation information within sequence data. Through this design, the model adeptly captures the correlations within time series data, demonstrating superior performance in prediction tasks. Subsequently, the SSA is employed to optimize GRU parameters, and the decomposed PV power mode components and environmental feature attributes are inputted into the TCN-GRU neural network. This facilitates dynamic temporal modeling of multivariate feature sequences. Finally, the predicted values of each component are summed to realize PV power forecasting. Validation using real data from a PV station corroborates that the novel model demonstrates a substantial reduction in RMSE and MAE of up to 55.1% and 54.5%, respectively, particularly evident in instances of pronounced photovoltaic power fluctuations during inclement weather conditions. The proposed method exhibits marked improvements in accuracy compared to traditional PV power prediction methods, underscoring its significance in enhancing forecasting precision and ensuring the secure scheduling and stable operation of power systems.

G2ViT: Graph Neural Network-Guided Vision Transformer Enhanced Network for retinal vessel and coronary angiograph segmentation

Blood vessel segmentation is a crucial stage in extracting morphological characteristics of vessels for the clinical diagnosis of fundus and coronary artery disease. However, traditional convolutional neural networks (CNNs) are confined to learning local vessel features, making it challenging to capture the graph structural information and fail to perceive the global context of vessels. Therefore, we propose a novel graph neural network-guided vision transformer enhanced network (G2ViT) for vessel segmentation. G2ViT skillfully orchestrates the Convolutional Neural Network, Graph Neural Network, and Vision Transformer to enhance comprehension of the entire graphical structure of blood vessels. To achieve deeper insights into the global graph structure and higher-level global context cognizance, we investigate a graph neural network-guided vision transformer module. This module constructs graph-structured representation in an unprecedented manner using the high-level features extracted by CNNs for graph reasoning. To increase the receptive field while ensuring minimal loss of edge information, G2ViT introduces a multi-scale edge feature attention module (MEFA), leveraging dilated convolutions with different dilation rates and the Sobel edge detection algorithm to obtain multi-scale edge information of vessels. To avoid critical information loss during upsampling and downsampling, we design a multi-level feature fusion module (MLF2) to fuse complementary information between coarse and fine features. Experiments on retinal vessel datasets (DRIVE, STARE, CHASE_DB1, and HRF) and coronary angiography datasets (DCA1 and CHUAC) indicate that the G2ViT excels in robustness, generality, and applicability. Furthermore, it has acceptable inference time and computational complexity and presents a new solution for blood vessel segmentation.

UViT-Seg: An Efficient ViT and U-Net-Based Framework for Accurate Colorectal Polyp Segmentation in Colonoscopy and WCE Images.

Colorectal cancer (CRC) stands out as one of the most prevalent global cancers. The accurate localization of colorectal polyps in endoscopy images is pivotal for timely detection and removal, contributing significantly to CRC prevention. The manual analysis of images generated by gastrointestinal screening technologies poses a tedious task for doctors. Therefore, computer vision-assisted cancer detection could serve as an efficient tool for polyp segmentation. Numerous efforts have been dedicated to automating polyp localization, with the majority of studies relying on convolutional neural networks (CNNs) to learn features from polyp images. Despite their success in polyp segmentation tasks, CNNs exhibit significant limitations in precisely determining polyp location and shape due to their sole reliance on learning local features from images. While gastrointestinal images manifest significant variation in their features, encompassing both high- and low-level ones, a framework that combines the ability to learn both features of polyps is desired. This paper introduces UViT-Seg, a framework designed for polyp segmentation in gastrointestinal images. Operating on an encoder-decoder architecture, UViT-Seg employs two distinct feature extraction methods. A vision transformer in the encoder section captures long-range semantic information, while a CNN module, integrating squeeze-excitation and dual attention mechanisms, captures low-level features, focusing on critical image regions. Experimental evaluations conducted on five public datasets, including CVC clinic, ColonDB, Kvasir-SEG, ETIS LaribDB, and Kvasir Capsule-SEG, demonstrate UViT-Seg's effectiveness in polyp localization. To confirm its generalization performance, the model is tested on datasets not used in training. Benchmarking against common segmentation methods and state-of-the-art polyp segmentation approaches, the proposed model yields promising results. For instance, it achieves a mean Dice coefficient of 0.915 and a mean intersection over union of 0.902 on the CVC Colon dataset. Furthermore, UViT-Seg has the advantage of being efficient, requiring fewer computational resources for both training and testing. This feature positions it as an optimal choice for real-world deployment scenarios.

SADGFeat: Learning local features with layer spatial attention and domain generalization

Local feature plays a pivotal role in various robotic tasks, including 3D reconstruction and visual localization. Although deep learning-based local features have proven superior to their traditional counterparts, they still face challenges in practical applications due to matching failures. These challenges primarily stem from inaccuracies in keypoint localization and the limited robustness of descriptors particularly with significant appearance changes. In this study, we introduce a novel method for local feature learning based on layer spatial attention and domain generalization. Firstly, a keypoint extraction strategy driven by layer spatial attention from high-level feature is proposed to enhance the accuracy of keypoint localization, progressing from coarse to fine granularity. Secondly, a new learning paradigm based on domain generalization is developed to extract local features with resilience against variations in illumination. To enrich the domain diversity within the training dataset, a real-time and lossless Fourier transform-based domain augmentation method is introduced. This method seamlessly integrates into the training process, enhancing the model's adaptability to varying domains. Additionally, explicit feature alignment-based representation learning is performed, further reinforcing the extraction of domain-invariant local features. Experimental results on public datasets demonstrate that the proposed method achieves state-of-the-art performance across various downstream tasks reliant on local feature matching, such as image matching, 3D reconstruction, and long-term visual localization.

Multi-Level interaction network for temporal sentence grounding in videos

Temporal sentence grounding in videos (TSGV), which aims to retrieve video segments from an untrimmed videos that semantically match a given query. Most previous methods focused on learning either local or global query features and then performed cross-modal interaction, but ignore the complementarity between local and global features. In this paper, we propose a novel Multi-Level Interaction Network for Temporal Sentence Grounding in Videos. This network explores the semantics of queries at both phrase and sentence levels, interacting phrase-level features with video features to highlight video segments relevant to the query phrase and sentence-level features with video features to learn more about global localization information. A stacked fusion gate module is designed, which effectively captures the temporal relationships and semantic information among video segments. This module also introduces a gating mechanism to enable the model to adaptively regulate the fusion degree of video features and query features, further improving the accuracy of predicting the target segments. Extensive experiments on the ActivityNet Captions and Charades-STA benchmark datasets demonstrate that the proposed method outperforms the state-of-the-art methods.

Patch-Based Auxiliary Node Classification for Domain Adaptive Object Detection

Domain adaptive object detection (DAOD) aims to leverage labeled source domain data to train object detection models that can generalize well to unlabeled target domains. Recently, many researchers have considered implementing fine-grained pixel-level domain adaptation using graph representations. Existing methods construct semantically complete graphs and align them across domains via graph matching. This work introduced an auxiliary node classification task before domain alignment through graph matching, which utilizes the inherent information of graph nodes to classify them, in order to avoid suboptimal graph matching results caused by node class confusion. However, previous methods neglected the contextual information of graph nodes, leading to biased node classification and suboptimal graph matching. To solve this issue, we propose a novel patch-based auxiliary node classification method for DAOD. Unlike existing methods that use only the inherent information of nodes for node classification, our method exploits the local region information of nodes and employs multi-layer convolutional neural networks to learn the local region feature representation of nodes, enriching the node context information. Thus, accurate and robust node classification results are produced and the risk of class confusion is reduced. Moreover, we propose a progressive strategy to fuse the inherent features and the learned local region features of nodes, which ensures that the network can stably and reliably utilize local region features for accurate node classification. In this paper, we conduct abundant experiments on various DAOD scenarios and demonstrate that our proposed model outperforms existing works.