Modulation Of Aggregation Research Articles

Graph Multi-Convolution and Attention Pooling for Graph Classification.

Many studies have achieved excellent performance in analyzing graph-structured data. However, learning graph-level representations for graph classification is still a challenging task. Existing graph classification methods usually pay less attention to the fusion of node features and ignore the effects of different-hop neighborhoods on nodes in the graph convolution process. Moreover, they discard some nodes directly during the graph pooling process, resulting in the loss of graph information. To tackle these issues, we propose a new Graph Multi-Convolution and Attention Pooling based graph classification method (GMCAP). Specifically, the designed Graph Multi-Convolution (GMConv) layer explicitly fuses node features learned from different perspectives. The proposed weight-based aggregation module combines the outputs of all GMConv layers, for adaptively exploiting the information over different-hop neighborhoods to generate informative node representations. Furthermore, the designed Local information and Global Attention based Pooling (LGAPool) utilizes the local information of a graph to select several important nodes and aggregates the information of unselected nodes to the selected ones by a global attention mechanism when reconstructing a pooled graph, thus effectively reducing the loss of graph information. Extensive experiments show that GMCAP outperforms the state-of-the-art methods on graph classification tasks, demonstrating that GMCAP can learn graph-level representations effectively.

MaskDGNets: Masked-attention guided dynamic graph aggregation network for event extraction.

Considering that the traditional deep learning event extraction method ignores the correlation between word features and sequence information, it cannot fully explore the hidden associations between events and events and between events and primary attributes. To solve these problems, we developed a new framework for event extraction called the masked attention-guided dynamic graph aggregation network. On the one hand, to obtain effective word representation and sequence representation, an interaction and complementary relationship are established between word vectors and character vectors. At the same time, a squeeze layer is introduced in the bidirectional independent recurrent unit to model the sentence sequence from both positive and negative directions while retaining the local spatial details to the maximum extent and establishing practical long-term dependencies and rich global context representations. On the other hand, the designed masked attention mechanism can effectively balance the word vector features and sequence semantics and refine these features. The designed dynamic graph aggregation module establishes effective connections between events and events, and between events and essential attributes, strengthens the interactivity and association between them, and realizes feature transfer and aggregation on graph nodes in the neighborhood through dynamic strategies to improve the performance of event extraction. We designed a reconstructed weighted loss function to supervise and adjust each module individually to ensure the optimal feature representation. Finally, the proposed MaskDGNets framework is evaluated on two baseline datasets, DuEE and CCKS2020. It demonstrates its robustness and event extraction performance, with F1 of 81.443% and 87.382%, respectively.

Retinal artery/vein vessel segmentation and measurement for hypertension based on multi-level edge-guided network

AbstractHypertension is a primary risk factor for the onset of cardiocerebrovascular diseases, leading to increased mortality. In the early stages of hypertension, changes in the diameter of the retinal arteries and veins are closely observed. Therefore, the automatic segmentation of these arteries and veins in retinal fundus images is crucial for hypertension monitoring. However, current deep learning based methods still fail to generate semantically consistent segmentation result, especially for narrow blood vessel segments. Moreover, the pixels near the vessel edge are prone to be missclassified by current methods, struggling to obtain edge-preserving segmentation results. These critical issues severely hinder downstream applications, which require accurate measurement of artery/vein diameter. First, to alleviate the issue of lacking semantic consistency for vessel segments, we propose a long-range spatial dependency modeling module to learn to model the long-range spatial dependency. On this basis, a multi-level edge guided spatial aggregation module is further presented to enhance the ability to accurately classify the edge pixels, generating edge-preserving results. Extensive experimental results on the widely used DRIVE dataset and our constructed dataset (HFC-50) show the superiority of the proposed model over state-of-the-art methods. Our method achieves balanced accuracy of 95.7% on DRIVE dataset, outperforming all state-of-art methods. Finally, in order to directly demonstrate the benefit of more accurate retinal artery/vein vessel segmentation for the measurement of the ratio of artery and vein, we quantitatively evaluate the computed artery/vein ratio for hypertension patients, with the artery/vein segmentation results generated by our proposed method.

HDRSA-Net: Hybrid dynamic residual self-attention network for SAR-assisted optical image cloud and shadow removal

Clouds and shadows often contaminate optical remote sensing images, resulting in missing information. Consequently, continuous spatiotemporal monitoring of the Earth’s surface requires the efficient removal of clouds and shadows. Unlike optical satellites, synthetic aperture radar (SAR) has active imaging capabilities in all weather conditions, supplying valuable supplementary information for reconstructing missing regions. Nevertheless, the reconstruction of high-fidelity cloud-free images based on SAR-optical data fusion remains challenging due to differences in imaging mechanisms and the considerable contamination from speckle noise inherent in SAR imagery. To solve the aforementioned challenges, this paper presents a novel hybrid dynamic residual self-attention network (HDRSA-Net), aiming to fully exploit the potential of SAR images in reconstructing missing regions. The proposed HDRSA-Net comprises multiple dynamic interaction residual (DIR) groups organized into an end-to-end trainable deep hierarchical stacked architecture. Specifically, the omni-dimensional dynamic local exploration (ODDLE) module and the sparse global context aggregation (SGCA) module are used to form a local–global feature adaptive extraction and implicit enhancement. A multi-task cooperative optimization loss function is designed to ensure that the results exhibit high spectral fidelity and coherent spatial structures. Additionally, this paper releases a large dataset that can comprehensively evaluate the reconstruction quality under different cloud coverages and various types of ground cover, providing a solid foundation for restoring satisfactory sensory effects and reliable semantic application value. In comparison to the current representative algorithms, the presented approach exhibits effectiveness and advancement in reconstructing missing regions with stability. The project is accessible at: https://github.com/RSIIPAC/LuojiaSET-OSFCR.

CMPNet: A cross-modal multi-scale perception network for RGB-T crowd counting

The cross-modal crowd counting method demonstrates better scene adaptability under complex conditions by introducing independent supplementary information. However, existing methods still face problems such as insufficient fusion of modal features, underutilization of crowd structure, and the neglect of scale information. In response to the above issues, this paper proposes a cross-modal multi-scale perception network (CMPNet). Specifically, CMPNet mainly consists of a cross-modal perception fusion module and a multi-scale feature aggregation module. The cross-modal perception fusion module effectively suppresses noise features while sharing features between different modalities, thereby significantly improving the robustness of the crowd counting process. The multi-scale feature aggregation module obtains rich crowd structure information through a spatial context aware graph convolution unit, and then integrates feature information from different scales to enhance the network’s perception ability of crowd density. To the best of our knowledge, CMPNet is the first attempt to model the crowd structure and mine its semantics in the field of cross-modal crowd counting. The experimental results show that CMPNet achieves state-of-the-art performance on all RGB-T datasets, providing an effective solution for cross-modal crowd counting. We will release the code at https://github.com/KunChenKKK/CMPNet.

AIFormer: Adaptive Interaction Transformer for 3D Point Cloud Understanding

Recently, significant advancements have been made in 3D point cloud analysis by leveraging transformer architecture in 3D space. However, it remains challenging to effectively implement local and global learning within irregular and sparse structures of 3D point clouds. This paper presents the Adaptive Interaction Transformer (AIFormer), a novel hierarchical transformer architecture designed to enhance 3D point cloud analysis by fusing local and global features through the adaptive interaction of features. Specifically, AIFormer mainly consists of several stacked AIFormer Blocks. Each AIFormer module employs the Local Relation Aggregation Module and the Global Context Aggregation Module, respectively, to extract local details of relationships within the reference point and long-range dependencies between reference points. Then, the local and global features are fused using the Adaptive Interaction Module for adaptive interaction to optimize the point representation. Additionally, the AIFormer Block further designs geometric relation functions and contextual relative semantic encoding to enhance local and global feature extraction capabilities, respectively. Extensive experiments on three popular 3D point cloud datasets verify that AIFormer achieves state-of-the-art or comparable performances. Our comprehensive ablation study further validates the effectiveness and soundness of the AIFormer design.

Green apple detection method based on multi-dimensional feature extraction network model and Transformer module

To enhance fast and accurate detection of pollution-free green apples for food safety, this paper uses the DETR network as a framework to propose a new method for pollution-free green apple detection based on a multi-dimensional feature extraction network and Transformer module. Firstly, an improved DETR network main feature extraction module adopts the ResNet18 network and replaces some residual layers with deformable convolutions (DCNv2), enabling the model to better adapt to pollution-free fruit changes at different scales and angles, while eliminating the impact of microbial contamination on fruit testing; Subsequently, the extended spatial pyramid pooling model (DSPP) and multi-scale residual aggregation module (FRAM) are integrated, which help reduce feature noise and minimize the loss of underlying features during the feature extraction process. The fusion of the two modules enhances the model’s ability to detect objects of different scales, thereby improving the accuracy of near-color fruit detection; At the same time, in order to solve the problems of slow convergence speed and large calculation amount of the basic network model, the convergence speed of the overall network model is improved by replacing the attention mechanism of Transformer. Experimental results show that compared with the original DETR model, the proposed algorithm has improved in AP, AP50 and AP75 indicators, especially in the AP50 indicator, which has the most obvious improvement reaching a detection accuracy of 97.12%. In the meantime, the trained network model is deployed on the picking robot. Compared with the original DETR network model, its average detection accuracy is as high as 96.58%, and the detection speed is increased by about 51%. Mixed sample detection tests were carried out before and after the model deployment, and the detection rate of the proposed method for non-polluted fruits reached more than 0.95. enabling the picking robot to efficiently complete the task of picking green apples. The test results show that the algorithm proposed in this article exhibits great potential in the task of detecting pollution-free near-color fruits by the picking robot. It ensures pollution-free fruit picking and the application of AI in food safety.

DDAM-Net: A Difference-Directed Multi-Scale Attention Mechanism Network for Cultivated Land Change Detection.

Declining cultivated land poses a serious threat to food security. However, existing Change Detection (CD) methods are insufficient for overcoming intra-class differences in cropland, and the accumulation of irrelevant features and loss of key features leads to poor detection results. To effectively identify changes in agricultural land, we propose a Difference-Directed Multi-scale Attention Mechanism Network (DDAM-Net). Specifically, we use a feature extraction module to effectively extract the cropland's multi-scale features from dual-temporal images, and we introduce a Difference Enhancement Fusion Module (DEFM) and a Cross-scale Aggregation Module (CAM) to pass and fuse the multi-scale and difference features layer by layer. In addition, we introduce the Attention Refinement Module (ARM) to optimize the edge and detail features of changing objects. In the experiments, we evaluated the applicability of DDAM-Net on the HN-CLCD dataset for cropland CD and non-agricultural identification, with F1 and precision of 79.27% and 80.70%, respectively. In addition, generalization experiments using the publicly accessible PX-CLCD and SET-CLCD datasets revealed F1 and precision values of 95.12% and 95.47%, and 72.40% and 77.59%, respectively. The relevant comparative and ablation experiments suggested that DDAM-Net has greater performance and reliability in detecting cropland changes.

Underwater Acoustic Target Recognition Based on Sub-Regional Feature Enhancement and Multi-Activated Channel Aggregation

Feature selection and fusion in ship radiated noise-based underwater target recognition have remained challenging tasks. This paper proposes a novel feature extraction method based on multi-dimensional feature selection and fusion. Redundant features are filtered through feature visualization techniques. The Sub-regional Feature Enhancement modules (SFE) and Multi-activated Channel Aggregation modules (MCA) within the neural network are utilized to achieve underwater target recognition. Experimental results indicate that our network, named Sub-Regional Channel Aggregation Net (SRCA-Net), utilizing 3-s sound segments for ship radiated noise recognition, surpasses existing models, achieving an accuracy of 78.52% on the public DeepShip dataset.

TTMGNet: Tree Topology Mamba-Guided Network Collaborative Hierarchical Incremental Aggregation for Change Detection

Change detection (CD) identifies surface changes by analyzing bi-temporal remote sensing (RS) images of the same region and is essential for effective urban planning, ensuring the optimal allocation of resources, and supporting disaster management efforts. However, deep-learning-based CD methods struggle with background noise and pseudo-changes due to local receptive field limitations or computing resource constraints, which limits long-range dependency capture and feature integration, normally resulting in fragmented detections and high false positive rates. To address these challenges, we propose a tree topology Mamba-guided network (TTMGNet) based on Mamba architecture, which combines the Mamba architecture for effectively capturing global features, a unique tree topology structure for retaining fine local details, and a hierarchical feature fusion mechanism that enhances multi-scale feature integration and robustness against noise. Specifically, the a Tree Topology Mamba Feature Extractor (TTMFE) leverages the similarity of pixels to generate minimum spanning tree (MST) topology sequences, guiding information aggregation and transmission. This approach utilizes a Tree Topology State Space Model (TTSSM) to embed spatial and positional information while preserving the global feature extraction capability, thereby retaining local features. Subsequently, the Hierarchical Incremental Aggregation Module is utilized to gradually align and merge features from deep to shallow layers to facilitate hierarchical feature integration. Through residual connections and cross-channel attention (CCA), HIAM enhances the interaction between neighboring feature maps, ensuring that critical features are retained and effectively utilized during the fusion process, thereby enabling more accurate detection results in CD. The proposed TTMGNet achieved F1 scores of 92.31% on LEVIR-CD, 90.94% on WHU-CD, and 77.25% on CL-CD, outperforming current mainstream methods in suppressing the impact of background noise and pseudo-change and more accurately identifying change regions.

DAG-YOLO: A Context-Feature Adaptive fusion Rotating Detection Network in Remote Sensing Images

Object detection in remote sensing image (RSI) research has seen significant advancements, particularly with the advent of deep learning. However, challenges such as orientation, scale, aspect ratio variations, dense object distribution, and category imbalances remain. To address these challenges, we present DAG-YOLO, a one-stage context-feature adaptive weighted fusion network that incorporates through three innovative parts. First, we integrate 1D Gaussian Angle-coding with YOLOv5 to convert the angle regression task into a classification task, establishing a more robust rotating object detection baseline, GLR-YOLO. Second, we introduce the Dual Branch Context Adaptive Modeling module, which enhances feature extraction capabilities by capturing global context information. Third, we design an adaptive detect head with the Adaptive Global Feature Aggregation and Reweighting (AGFAR) module. AGFAR addresses feature inconsistency among different output layers of the Feature Pyramid Network, retaining useful semantic information and elevating detection accuracy. Extensive experiments on public datasets DOTA-v1.0, DOTA-v1.5, and UCAS-AOD showcase mAP scores of 77.75%, 73.79%, and 90.27%, respectively. Our proposed method has the best performance among the current mainstream SOTA methods, which proves its effectiveness in RSI object detection.

Fine-grained recognition algorithm of crop pests based on cross-layer bilinear aggregation and multi-task learning

Fine-grained recognition of crop pests is a crucial concern in the field of agriculture, as the accuracy of recognition and generalization ability directly affect the yield and quality of crops. Aiming at the characteristics of crop pests with a wide variety of species, small inter-class and large intra-class differences in external morphology, as well as the problems of uneven sample distribution and noisy labels in fine-grained image datasets under complex environments, we propose a fine-grained recognition model of crop pests (MT-MACLBPHSNet) based on cross-layer bilinear aggregation and multi-task learning, which consists of three key modules: the backbone network module, the cross-layer bilinear aggregation module, and the multi-task learning module. A new union loss function is designed in the primary task of the multi-task learning module, which is used to alleviate the two problems existing in the model training fine-grained image datasets. The experimental results show that the model effectively balances the model complexity and recognition accuracy in a comparative analysis with several existing excellent network models on the IP102-CP13 dataset, with the recognition accuracy reaching 75.37%, which is 7.06% higher than the Baseline model, and the F1-score reaching 67.06%. Additionally, the generalization of the model is also verified on the IP102-VP16 dataset, and the model outperforms most of the models in terms of recognition accuracy and generalization ability, which can provide an effective reference for fine-grained recognition of crop pests.

Monocular Depth and Ego-motion Estimation with Scale Based on Superpixel and Normal Constraints

Three-dimensional perception in intelligent virtual and augmented reality (VR/AR) and autonomous vehicles (AV) applications is critical and attracting significant attention. The self-supervised monocular depth and ego-motion estimation serves as a more intelligent learning approach that provides the required scene depth and location for 3D perception. However, the existing self-supervised learning methods suffer from scale ambiguity, boundary blur, and imbalanced depth distribution, limiting the practical applications of VR/AR and AV. In this article, we propose a new self-supervised learning framework based on superpixel and normal constraints to address these problems. Specifically, we formulate a novel 3D edge structure consistency loss to alleviate the boundary blur of depth estimation. To address the scale ambiguity of estimated depth and ego-motion, we propose a novel surface normal network for efficient camera height estimation. The surface normal network is composed of a deep fusion module and a full-scale hierarchical feature aggregation module. Meanwhile, to realize the global smoothing and boundary discriminability of the predicted normal map, we introduce a novel fusion loss which is based on the consistency constraints of the normal in edge domains and superpixel regions. Experiments are conducted on several benchmarks, and the results illustrate that the proposed approach outperforms the state-of-the-art methods in depth, ego-motion, and surface normal estimation.

PDeT: A Progressive Deformable Transformer for Photovoltaic Panel Defect Segmentation.

Defects in photovoltaic (PV) panels can significantly reduce the power generation efficiency of the system and may cause localized overheating due to uneven current distribution. Therefore, adopting precise pixel-level defect detection, i.e., defect segmentation, technology is essential to ensuring stable operation. However, for effective defect segmentation, the feature extractor must adaptively determine the appropriate scale or receptive field for accurate defect localization, while the decoder must seamlessly fuse coarse-level semantics with fine-grained features to enhance high-level representations. In this paper, we propose a Progressive Deformable Transformer (PDeT) for defect segmentation in PV cells. This approach effectively learns spatial sampling offsets and refines features progressively through coarse-level semantic attention. Specifically, the network adaptively captures spatial offset positions and computes self-attention, expanding the model's receptive field and enabling feature extraction across objects of various shapes. Furthermore, we introduce a semantic aggregation module to refine semantic information, converting the fused feature map into a scale space and balancing contextual information. Extensive experiments demonstrate the effectiveness of our method, achieving an mIoU of 88.41% on our solar cell dataset, outperforming other methods. Additionally, to validate the PDeT's applicability across different domains, we trained and tested it on the MVTec-AD dataset. The experimental results demonstrate that the PDeT exhibits excellent recognition performance in various other scenarios as well.

Implicit expression recognition enhanced table-filling for aspect sentiment triplet extraction

Aspect sentiment triplet extraction (ASTE) is a challenging task in aspect-based sentiment analysis (ABSA), involving the identification of aspect terms, opinion terms, and their corresponding sentiment polarities within comments to form triplets. The emergence of more realistic DMASTE datasets, featuring diverse domains, implicit aspect terms, and longer comments, poses challenges for existing methods. In particular, these methods struggle with recognizing implicit expressions effectively and capturing sufficient information. To overcome these hurdles, we propose an implicit expression recognition enhanced table-filling (IERET) method. This approach integrates modeling of overall implicit expression in sentences and employs a bidirectional information aggregation module to capture word pair information comprehensively. During the decoding process, a table-filling method accurately delineates aspect-opinion pair boundaries. Experimental results across in-domain, single-source cross-domain, and multi-source cross-domain on the DMASTE dataset demonstrate that our proposed IERET method achieves state-of-the-art performance.

RJAN: Region‐based joint attention network for 3D shape recognition

AbstractAs an essential field of multimedia and computer vision, 3D shape recognition has attracted much research attention in recent years. Multiview‐based approaches have demonstrated their superiority in generating effective 3D shape representations. Typical methods usually extract the multiview global features and aggregate them together to generate 3D shape descriptors. However, there exist two disadvantages: First, the mainstream methods ignore the comprehensive exploration of local information in each view. Second, many approaches roughly aggregate multiview features by adding or concatenating them together. The information loss for some discriminative characteristics limits the representation effectiveness. To address these problems, a novel architecture named region‐based joint attention network (RJAN) was proposed. Specifically, the authors first design a hierarchical local information exploration module for view descriptor extraction. The region‐to‐region and channel‐to‐channel relationships from different granularities can be comprehensively explored and utilised to provide more discriminative characteristics for view feature learning. Subsequently, a novel relation‐aware view aggregation module is designed to aggregate the multiview features for shape descriptor generation, considering the view‐to‐view relationships. Extensive experiments were conducted on three public databases: ModelNet40, ModelNet10, and ShapeNetCore55. RJAN achieves state‐of‐the‐art performance in the tasks of 3D shape classification and 3D shape retrieval, which demonstrates the effectiveness of RJAN. The code has been released on https://github.com/slurrpp/RJAN.

Adaptive multi-graph contrastive learning for bundle recommendation

Recently, recommending bundles - sets of items that complement each other - instead of individual items to users has drawn much attention in both academia and industry. Models based on Graph Neural Networks (GNNs) for bundle recommendation have achieved great success in capturing users’ preferences by modeling pairwise correlations among users, bundles, and items via information propagation on graphs. However, a notable limitation lies in their insufficient focus on explicitly modeling intricate ternary relationships. Additionally, the loose combination of node embeddings from different graphs tends to introduce noise, as it fails to consider disparities among the graphs. To this end, we propose a novel approach called Adaptive Multi-Graph Contrastive Learning for Bundle Recommendation (AMCBR). Specifically, AMCBR models ternary interactions by constructing multiple graphs, including a bundle preference graph based on direct user-bundle interactions, a collaborative neighborhoods graph featuring user-level and bundle-level subgraphs, and an item-level preference hypergraph capturing indirect user-bundle relationships through items. Then, (hyper)graph convolution is applied to each (hyper)graph to encode diverse potential preferences into node embeddings. To enhance the model’s robustness, an adaptive aggregation module is employed to assign varying weights to node embeddings from different graphs during the fusion process, which enriches the semantic and comprehensive information in the embeddings while mitigating potential noise. Finally, a contrastive learning strategy is proposed to jointly optimize the model, strengthening collaborative links between individual graphs. Extensive experiments on three real datasets demonstrate that AMCBR can outperform the state-of-the-art baselines on the Top-K recommendations.

SPFDNet: Water Extraction Method Based on Spatial Partition and Feature Decoupling

Extracting water information from remote-sensing images is of great research significance for applications such as water resource protection and flood monitoring. Current water extraction methods aggregated richer multi-level features to enhance the output results. In fact, there is a difference in the requirements for the water body and the water boundary. Indiscriminate multi-feature fusion can lead to perturbation and competition of information between these two types of features during the optimization. Consequently, models cannot accurately locate the internal vacancies within the water body with the external boundary. Therefore, this paper proposes a water feature extraction network with spatial partitioning and feature decoupling. To ensure that the water features are extracted with deep semantic features and stable spatial information before decoupling, we first design a chunked multi-scale feature aggregation module (CMFAM) to construct a context path for obtaining deep semantic information. Then, an information interaction module (IIM) is designed to exchange information between two spatial paths with two fixed resolution intervals and the two paths through. During decoding, a feature decoupling module (FDM) is developed to utilize internal flow prediction to acquire the main body features, and erasing techniques are employed to obtain boundary features. Therefore, the deep features of the water body and the detailed boundary information are supplemented, strengthening the decoupled body and boundary features. Furthermore, the integrated expansion recoupling module (IERM) module is designed for the recoupling stage. The IERM expands the water body and boundary features using expansion and adaptively compensates the transition region between the water body and boundary through information guidance. Finally, multi-level constraints are combined to realize the supervision of the decoupled features. Thus, the water body and boundaries can be extracted more accurately. A comparative validation analysis is conducted on the public datasets, including the gaofen image dataset (GID) and the gaofen2020 challenge dataset (GF2020). By comparing with seven SOTAs, the results show that the proposed method achieves the best results, with IOUs of 91.22 and 78.93, especially in the localization of water bodies and boundaries. By applying the proposed method in different scenarios, the results show the stable capability of the proposed method for extracting water with various shapes and areas.

PPDistiller: Weakly-supervised 3D point cloud semantic segmentation framework via point-to-pixel distillation

Despite the significant growth in the availability of 3D light detection and ranging (LiDAR) point cloud data in recent years, annotation remains expensive and time-consuming. This has led to an increasing demand for weakly-supervised semantic segmentation (WSSS) methods in applications such as autonomous driving, mapping, and robotics. Existing approaches typically rely solely on LiDAR point cloud data for WSSS, which often results in lower segmentation accuracy due to the sparsity of point clouds. To address these challenges, we propose a novel architecture, PPDistiller, which employs multiple teacher networks from different modalities. Compared to other WSSS and multimodal approaches, PPDistiller achieves superior segmentation accuracy with fewer annotations. This is facilitated through the novel Mean Multi-Teacher Framework (MMT), which incorporates multiple modalities and teachers. To address the issue of lacking 2D labels, we propose the Distance-CAM Self-Training (DCAM-ST) module, which utilizes sparse 3D weak annotations to produce accurate 2D pixel-level annotations. To enable adaptive fusion of 2D and 3D data, we introduce the Attention Point to Pixel Fusion (APPF) module, facilitating bidirectional transfer of cross-modal knowledge. Additionally, to fully leverage the spatial semantic information in point cloud, we propose the Pyramid Semantic-context Neighbor Aggregation (PSNA) module, aiming to exploit spatial and semantic correlations to improve performance. Extensive experimentation on SemanticKITTI, ScribbleKITTI and nuScenes datasets demonstrates the superiority of our proposed method. Compared to state-of-the-art fusion and weakly-supervised methods, PPDistiller achieves the highest mean Intersection over Union (mIoU) scores under both fully-supervised and weakly-supervised settings.

Enhanced Cross Layer Refinement Network for robust lane detection across diverse lighting and road conditions

With the rapid development of autonomous driving technology, lane detection, a key component of intelligent vehicle systems, is crucial for ensuring road safety and efficient vehicle navigation. In this paper, a new lane detection method is proposed to address the problem of degraded performance of existing lane detection methods when dealing with complex road environments. The proposed method evolves from the original Cross Layer Refinement Network (CLRNet) by incorporating two of our carefully designed core components: the Global Feature Optimizer (GFO) and the Adaptive Lane Geometry Aggregator (ALGA). The GFO is a multi-scale attention mechanism that mimics the human visual focusing ability, effectively filtering out unimportant information and focusing on the image regions most relevant to the task. The ALGA is a shape feature-aware aggregation module that utilizes the shape prior of lanes to enhance the correlation of anchor points in an image, better fusing global and local information. By integrating both components into CLRNet, an enhanced version called Enhanced CLRNet (E-CLRNet) is presented, which exhibits higher performance stability in complex roadway scenarios. Experiments on the CULane dataset reveal that E-CLRNet demonstrates superior performance stability over the original CLRNet in complex scenarios, including curves, shadows, missing lines, and dazzling light conditions. In particular, in the curves, the F1 score of E-CLRNet is improved by almost 3% over the original CLRNet. This study not only improves the accuracy and performance stability of lane detection but also provides a new solution for the application of autonomous driving technology in complex environments, which promotes the development of intelligent vehicle systems.