Multi-level Features Research Articles

MDFF-Net: Multi-level Dynamic Feature Fusion Network Combined with Deep Supervision Mechanism for Low-Light Image Enhancement

MDFF-Net: Multi-level Dynamic Feature Fusion Network Combined with Deep Supervision Mechanism for Low-Light Image Enhancement

Atmospheric turbulence-distorted OAM mode recognition via multi-scale dilated convolution with multi-level feature fusion

Vortex beams with orbital angular momentum (OAM) significantly enhance system capacity, and high-precision recognition of OAM mode through atmospheric turbulence (AT) channels can markedly improve the information transmission capability of free-space optical communication systems. In this paper, with a cylindrical lens-assisted distinguish between positive and negative OAM, a reliable neural network combining multi-scale dilated convolution (MSDC) unit and multi-level feature fusion (MLFF) module is proposed to detect high order AT-distorted OAM modes. The network fully exploits the features in light-intensity images to achieve a highest recognition accuracy of 99.4% for mode-orders from -20 to +20 in a hybrid ATs dataset (C n 2 = 5×10−16, 5×10−14, 5×10−12 m-2/3), and almost 96% even in strong turbulence. Experimental results on accuracy, efficiency, reliability, and robustness demonstrate that the proposed method excels and provides a trustworthy solution for complex AT-distorted OAM mode recognition.

PFB-Diff: Progressive Feature Blending diffusion for text-driven image editing

Diffusion models have demonstrated their ability to generate diverse and high-quality images, sparking considerable interest in their potential for real image editing applications. However, existing diffusion-based approaches for local image editing often suffer from undesired artifacts due to the latent-level blending of the noised target images and diffusion latent variables, which lack the necessary semantics for maintaining image consistency. To address these issues, we propose PFB-Diff, a Progressive Feature Blending method for Diffusion-based image editing. Unlike previous methods, PFB-Diff seamlessly integrates text-guided generated content into the target image through multi-level feature blending. The rich semantics encoded in deep features and the progressive blending scheme from high to low levels ensure semantic coherence and high quality in edited images. Additionally, we introduce an attention masking mechanism in the cross-attention layers to confine the impact of specific words to desired regions, further improving the performance of background editing and multi-object replacement. PFB-Diff can effectively address various editing tasks, including object/background replacement and object attribute editing. Our method demonstrates its superior performance in terms of editing accuracy and image quality without the need for fine-tuning or training. Our implementation is available at https://github.com/CMACH508/PFB-Diff.

High-precision and lightweight small-target detection algorithm for low-cost edge intelligence

The proliferation of edge devices driven by advancements in Internet of Things (IoT) technology has intensified the challenge of achieving high-precision small target detection, as it demands extensive computational resources. This amplifies the conflict between the need for precise detection and the requirement for cost-efficiency across numerous edge devices. To solve this problem, this paper introduces an enhanced target detection algorithm, MSGD-YOLO, built upon YOLOv8. The Faster Implementation of CSP Bottleneck with 2 convolutions (C2f) module is enhanced through the integration of the Ghost module and dynamic convolution, resulting in a more lightweight architecture while enhancing feature generation. Additionally, Spatial Pyramid Pooling with Enhanced Local Attention Network (SPPELAN) replaces Spatial Pyramid Pooling Fast (SPPF) to expand the receptive field, optimizing multi-level feature aggregation for improved performance. Furthermore, a novel Multi-Scale Ghost Convolution (MSGConv) and Multi-Scale Generalized Feature Pyramid Network (MSGPFN) are introduced to enhance feature fusion and integrate multi-scale information. Finally, four optimized dynamic convolutional detection heads are employed to capture target features more accurately and improve small target detection precision. Evaluation on the VisDrone2019 dataset shows that compared with YOLOv8-n, MSGD-YOLO improves mAP@50 and mAP@50–95 by 14.1% and 11.2%, respectively. In addition, the model not only achieves a 16.1% reduction in parameters but also attains a processing speed of 24.6 Frames Per Second (FPS) on embedded devices, thereby fulfilling real-time detection requirements.

MATNet: Multilevel attention-based transformers for change detection in remote sensing images

Remote sensing image change detection is crucial for natural disaster monitoring and land use change. As the resolution increases, the scenes covered by remote sensing images become more complex, and traditional methods have difficulties in extracting detailed information. With the development of deep learning, the field of change detection has new opportunities. However, existing algorithms mainly focus on the difference analysis between bi-temporal images, while ignoring the semantic information between images, resulting in the global and local information not being able to interact effectively. In this paper, we introduce a new transformer-based multilevel attention network (MATNet), which is capable of extracting multilevel features of global and local information, enabling information interaction and fusion, and thus modeling the global context more effectively. Specifically, we extract multilevel semantic features through the Transformer encoder, and utilize the Feature Enhancement Module (FEM) to perform feature summing and differencing on the multilevel features in order to better extract the local detail information, and thus better detect the changes in small regions. In addition, we employ a multilevel attention decoder (MAD) to obtain information in spatial and spectral dimensions, which can effectively fuse global and local information. In experiments, our method performs excellently on CDD, DSIFN-CD, LEVIR-CD, and SYSU-CD datasets, with F1 scores and OA reaching 95.67%∕87.75%∕90.94%∕86.82% and 98.95%∕95.93%∕99.11%∕90.53% respectively.

Regularity through randomness: Bioinspired quasi-amorphous photonic structures for multilevel information encryption

The biological world has produced a wealth of complex photonic materials offering abundant design inspirations. However, creating these intricate photonic structures intertwining order, disorder, and hierarchical arrangements remains challenging. Achieving this necessitates nano-fabrication technologies capable of integrating intended macroscopic properties while accommodating microscopic variability or randomness, akin to the operational principles observed in biological organisms. Here, we present quasi-amorphous photonic structure (QAPS) patterns with regulated sophisticated visual effects through hierarchically distributions of photonic elements with engineered randomness. Enabled by an infiltration-assisted (IFAST) inkjet printing method, the visual appearances of QAPS patterns are precisely controlled by collective harnessing of nanoscale resonant scattering of colloids, mesoscale interferences, and macroscale light transporting. The QAPS patterns exhibit isotropic diffuse colors, unprecedented iridescence and enable multichannel encrypted information that can be selectively identified using specific filters at particular specular illumination angles. Additionally, the intricate infiltration fields confer high adjustability in colloidal assembly, offering sophisticated packing geometries and color distributions at the microscale, which can serve as physical unclonable security features. The exceptional structural complexity and optical versatility of QAPSs enable the integration of multilevel security features within a single system, providing a promising roadmap to achieve larger information capacity, multichannel cryptography, and enhanced encryption levels.

Object-oriented U-GCN for open-pit mining extraction from high spatial resolution remote-sensing images of complex scenes

ABSTRACT Precise extraction of open-pit mines is crucial for resource management and ecological and environmental dynamic monitoring. Current methods for extracting open-pit mines encounter challenges such as low accuracy and difficulty detecting complex scenes of open-pit mining. To address these issues, this paper proposes an object-oriented intelligent extraction method for complex mining scenes using Gaofen-2(GF-2) high-resolution remote-sensing images, which expresses the pixel-level features at the object level and utilizes the unique feature propagation and aggregation capabilities of the graph structure for the extraction of the mines. First, an object-oriented feature expression strategy is introduced to express multi-level pixel-level features as object-level features by constructing objects with appropriate multi-resolution segmentation parameters. This approach effectively reduces the impact of isolated pixel noise and outliers on classification results. Second, this paper proposes a U-GCN-based open-pit mining extraction method that combines the powerful multi-level feature extraction capabilities of U-GCN to propagate and aggregate information in a graph structure, effectively modelling spatial relationships between different objects. This method achieves high-precision extraction of open-pit mining areas. In experiments conducted on two study areas of varying scales, the F1 scores for open-pit mine extraction reached 93.32% and 83.06%. Comparative experiments demonstrate that the proposed object-oriented U-GCN method performs superiorly in terms of accuracy, stability and robustness across mining scenes with different levels of complexity. The proposed open-pit mine extraction method offers new insights and methodologies for current extraction practices.

Piano performance evaluation dataset with multilevel perceptual features

This study aims to build a comprehensive dataset that enables the automatic evaluation of piano performances. In real-world piano performance, especially within the realm of classical piano music, we encounter a vast spectrum of performance variations. The challenge lies in how to effectively evaluate these performances. We must consider three critical aspects: (1) It is essential to gauge how performers express the music and how listeners perceive the performance, rather than focusing on the compositional characteristics of the musical piece. (2) Beyond fundamental elements like pitch and duration, we must also embrace higher-level features such as interpretation. (3) Such evaluation should be done by experts to discern the nuanced performances. Regrettably, there exists no dataset that addresses these challenging evaluation tasks. Therefore, we introduce a pioneering dataset PercePiano, annotated by music experts, with more extensive features capable of representing these nuanced aspects effectively. It encapsulates piano performance with a wide range of perceptual features that are recognized by musicians. Our evaluation benchmark includes a novel metric designed to accommodate the inherent subjectivity of perception. Furthermore, we propose an enhanced baseline framework that grounds performance on score data, aligning model predictions with human perception. Harnessing the aligned features enhances the baseline performance and proves to be adaptable to various model structures. In conclusion, our research opens new possibilities for comprehensive piano performance evaluation.

R-Net: Recursive decoder with edge refinement network for salient object detection

Fully convolutional neural (FCN) networks based on encoder-decoder structures are adopted for salient object detection (SOD) and achieve state-of-the-art performance. However, most of the previous works aggregate multilevel features layer by layer and ignore long-range dependencies of spatial context, leading to imbalanced feature processes and context separation. In addition, the blurry edges obtained from previous SOD methods inevitably influence the detection accuracy. To handle these problems, this study proposes an effective recursion-based SOD network R-Net adopting a partial cascade structure, which contains a recursive decoder module (RDM), a long-range dependency module (LRDM), and an edge refinement module (ERM). The RDM is composed of three subdecoders, which can scale different level features to the same size through recursive pooling and recursive upsampling for full-scale feature fusion. The LRDM bridges the encoder and decoder by weighting multilevel features to establish the long-range feature dependency of the spatial context. And the ERM attached to the decoder introduces the shallow reference feature to refine the blurry edges for obtaining delicate detection results. The SOD experimental results on DUTS-TE, HKU-IS, PASCAL-S, ECSSD, and DUT-OMRON datasets demonstrate that the proposed R-Net is more robust under different complex scenes compared with some state-of-the-art methods. The source codes are available at https://github.com/ZEROICEWANG/R-Net.

An end-to-end bi-objective approach to deep graph partitioning

Graphs are ubiquitous in real-world applications, such as computation graphs and social networks. Partitioning large graphs into smaller, balanced partitions is often essential, with the bi-objective graph partitioning problem aiming to minimize both the “cut” across partitions and the imbalance in partition sizes. However, existing heuristic methods face scalability challenges or overlook partition balance, leading to suboptimal results. Recent deep learning approaches, while promising, typically focus only on node-level features and lack a truly end-to-end framework, resulting in limited performance. In this paper, we introduce a novel method based on graph neural networks (GNNs) that leverages multilevel graph features and addresses the problem end-to-end through a bi-objective formulation. Our approach explores node-, local-, and global-level features, and introduces a well-bounded bi-objective function that minimizes the cut while ensuring partition-wise balance across all partitions. Additionally, we propose a GNN-based deep model incorporating a Hardmax operator, allowing the model to optimize partitions in a fully end-to-end manner. Experimental results on 12 datasets across various applications and scales demonstrate that our method significantly improves both partitioning quality and scalability compared to existing bi-objective and deep graph partitioning baselines.

PVT2DNet: Polyp segmentation with vision transformer and dual decoder refinement strategy

Colorectal carcinoma is a prevalent malignancy worldwide. Accurate polyp segmentation, along with endoscopic resection, can significantly reduce its incidence and mortality. Most polyp segmentation neural networks are CNN-based and single decoder strategy architectures, which learn limited robust representations. In this paper, we propose a novel network with the vision transformer and dual decoder refinement strategy called PVT2DNet to overcome some limitations of current networks and achieve more precise automated polyp segmentation. The PVT2DNet adopts a pyramid vision transformer encoder and enhances the multi-level features with the context-enhanced module (CEM). Moreover, instead of directly feeding features into a single decoder, we introduce a dual partial cascaded decoder refinement strategy to excavate more informative polyp cues. Extensive experimentations on five widely adopted datasets demonstrate the proposed network outperforms other state-of-the-art on most metrics.

Multi-scale feature reconstruction network for industrial anomaly detection

Unsupervised anomaly detection techniques, which operate without prior knowledge of anomalies, have garnered significant attention in industrial inspection due to their adaptability and generalization. Therefore, knowledge-based computer vision techniques have been broadly applied to identify unusual image patterns. However, real-time industrial applications present challenges such as limited anomalous samples, inadequate defect knowledge, and complex background textures. These factors lead to difficulties in accurately identifying defect regions, and conventional auto-encoder networks often struggle to overcome these issues.To address these limitations, we propose a multi-scale feature reconstruction (MSFR) network specifically designed for domain shift scenarios. Our approach employs a pyramidal vision transformer network (PVTN) to reconstruct multi-scale feature maps, capturing discriminative features at various scales. Additionally, a pre-trained module extracts multi-level features at the same scale, and a dedicated feature matching module enhances accuracy by improving the alignment probability between features. The MSFR strategy surpasses conventional auto-encoders by filtering pixel-level information at multiple depths.Empirical evaluations were conducted using benchmark datasets such as MVTec AD and AeBAD-S. Furthermore, an extensive ablation study demonstrates the effectiveness and viability of the proposed MSFR approach for industrial anomaly detection tasks. The experimental results show that the proposed model significantly outperforms recent approaches, making it highly suitable for real-world industrial applications, particularly in manufacturing.

Self-supervised learning-based re-parameterization instance segmentation for hazardous object in terahertz image

Recently, terahertz imaging technology has shown widespread potential applications in the public security screening field. Previous terahertz hazardous object detection techniques primarily relied on manual recognition and image processing methods. Some solutions incorporating deep learning technologies depend on large quantities of high-quality data, making it challenging to achieve low-cost and high-performance detection. To solve the issue of dependency on large amounts of high-quality data, we propose a diversified dynamic structure You Only Look Once (DDS-YOLO) model based on masked image modeling and structural reparameterization for the instance segmentation of hazardous objects in terahertz security inspection images. To address the scarcity and annotating difficulty problems of terahertz security inspection image samples, we combine automatic data strategies with masked image modeling for self-supervised learning. We propose a multilevel feature refinement fusion mechanism to enhance the quality of learned feature representations. The backbone parameter transfer and fine-tuning training strategies are employed to achieve hazardous object instance segmentation on the terahertz dataset. To address the low detection accuracy issue caused by the poor terahertz image quality, we develop a reparameterizable hierarchical structure for the backbone, improve a multi-scale feature-integrating neck, and design a dynamically decoupled head with lower computational requirements to enhance the performance of the instance segmentation model. Experimental results demonstrate that the proposed model accurately outputs detection boxes, categories, and segmentation masks for hazardous objects with minimal training samples. The comparative experimental results indicate that the proposed model outperforms existing state-of-the-art methods in terms of detection performance. The proposed DDS-YOLO model achieves 59.3 % in mask mean Average Precision (mAP) and 61.3 % in box mAP, and the model parameters and computational requirements also meet practical application scenarios.

Lightweight cross-modal transformer for RGB-D salient object detection

Recently, Transformer-based RGB-D salient object detection (SOD) models have pushed the performance to a new level. However, they come at the cost of consuming abundant resources, including memory and power, thus hindering their real-life applications. To remedy this situation, a novel lightweight cross-modal Transformer (LCT) for RGB-D SOD will be presented in this paper. Specifically, LCT will first reduce its parameters and computational costs by employing a middle-level feature fusion structure and taking a lightweight Transformer as the backbone. Then, with the aid of Transformers, it will compensate for performance degradation by effectively capturing the cross-modal and cross-level complementary information from the multi-modal input images. To this end, a cross-modal enhancement and fusion module (CEFM) with a lightweight channel-wise cross attention block (LCCAB) will be designed to capture the cross-modal complementary information effectively but with fewer costs. A bi-directional multi-level feature interaction module (Bi-MFIM) with a lightweight spatial-wise cross attention block (LSCAB) will be designed to capture the cross-level complementary context information. By virtue of CEFM and Bi-MFIM, the performance degradation caused by parameter reduction can be well compensated, thus boosting the performances. By doing so, our proposed model has only 2.8M parameters with 7.6G FLOPs and runs at 66 FPS. Furthermore, experimental results on several benchmark datasets show that our proposed model can achieve competitive or even better results than other models. Our code will be released on https://github.com/nexiakele/lightweight-cross-modal-Transformer-LCT-for-RGB-D-SOD.

MLFA-UNet: A multi-level feature assembly UNet for medical image segmentation

Medical image segmentation is crucial for accurate diagnosis and treatment in medical image analysis. Among the various methods employed, fully convolutional networks (FCNs) have emerged as a prominent approach for segmenting medical images. Notably, the U-Net architecture and its variants have gained widespread adoption in this domain. This paper introduces MLFA-UNet, an innovative architectural framework aimed at advancing medical image segmentation. MLFA-UNet adopts a U-shaped architecture and integrates two pivotal modules: multi-level feature assembly (MLFA) and multi-scale information attention (MSIA), complemented by a pixel-vanishing (PV) attention mechanism. These modules synergistically contribute to the segmentation process enhancement, fostering both robustness and segmentation precision. MLFA operates within both the network encoder and decoder, facilitating the extraction of local information crucial for accurately segmenting lesions. Furthermore, the bottleneck MSIA module serves to replace stacking modules, thereby expanding the receptive field and augmenting feature diversity, fortified by the PV attention mechanism. These integrated mechanisms work together to boost segmentation performance by effectively capturing both detailed local features and a broader range of contextual information, enhancing both accuracy and resilience in identifying lesions. To assess the versatility of the network, we conducted evaluations of MFLA-UNet across a range of medical image segmentation datasets, encompassing diverse imaging modalities such as wireless capsule endoscopy (WCE), colonoscopy, and dermoscopic images. Our results consistently demonstrate that MFLA-UNet outperforms state-of-the-art algorithms, achieving dice coefficients of 91.42%, 82.43%, 90.8%, and 88.68% for the MICCAI 2017 (Red Lesion), ISIC 2017, PH2, and CVC-ClinicalDB datasets, respectively.

The Study of Echocardiography of Left Ventricle Segmentation Combining Transformer and Convolutional Neural Networks.

Accurate prediction of echocardiographic parameters is essential for diagnosis and treatment of cardiac disease, especially for segmentation of the left ventricle to obtain measurements such as left ventricular ejection fraction and volume. However, manually outlining left ventricle on echocardiographic images is a time-consuming and physician experience-dependent task. Therefore, it is crucial to develop an accurate and efficient automatic segmentation tool. Therefore, we aimed to explore a model to perform echocardiography of left ventricle segmentation by combining transformer and convolutional neural networks (CNN).ResNet-50 was used in CNN branch. The encoder-decoder architecture was used for transformer branch, which was fused to the corresponding feature maps of the CNN branches. Fusion module was used to effectively combine feature information from the CNN and transformer. Bridge attention used to increase sensitivity and prediction accuracy of model. The entire network was trained end-to-end using the binary cross-entropy with logits loss L.In this work, we propose an automatic left ventricular (LV) segmentation model based on Transformer and CNN that efficiently captures global dependencies and spatial details and create a fusion module using CBAM that fuses Transformer and CNN features. In addition, attention is also computed using multi-level fusion features to obtain the final attention segmentation map. The model was trained and evaluated on a large cardiac image dataset, EchoNet-Dynamic, with test dice coefficient of 92.4%.The results show that our model can better segment left ventricle. We also tested our model on clinical patient ultrasound images, and visualization results proved effectiveness of the model.

Graph-based Saliency Detection: Integrating Multilevel Features and Analyzing the Distraction Using DNN

Graph-based Saliency Detection: Integrating Multilevel Features and Analyzing the Distraction Using DNN

Multi-Temporal Snow-Covered Remote Sensing Image Matching via Image Transformation and Multi-Level Feature Extraction

To address the challenge of image matching posed by significant modal differences in remote sensing images influenced by snow cover, this paper proposes an innovative image transformation-based matching method. Initially, the Pix2Pix-GAN conversion network is employed to transform remote sensing images with snow cover into images without snow cover, reducing the feature disparity between the images. This conversion facilitates the extraction of more discernible features for matching by transforming the problem from snow-covered to snow-free images. Subsequently, a multi-level feature extraction network is utilized to extract multi-level feature descriptors from the transformed images. Keypoints are derived from these descriptors, enabling effective feature matching. Finally, the matching results are mapped back onto the original snow-covered remote sensing images. The proposed method was compared to well-established techniques such as SIFT, RIFT2, R2D2, and ReDFeat and demonstrated outstanding performance. In terms of NCM, MP, Rep, Recall, and F1-measure, our method outperformed the state of the art by 177, 0.29, 0.22, 0.21, and 0.25, respectively. In addition, the algorithm shows robustness over a range of image rotation angles from −40° to 40°. This innovative approach offers a new perspective on the task of matching multi-temporal snow-covered remote sensing images.

Quaternion attention-based JND model for macrophotography image watermarking

With the advancement of imaging technology, macrophotography images (MPIs) have become a popular research topic. Unlike natural images, MPIs often feature sharp foregrounds and blurred backgrounds, leading to distinct perceptual characteristics in estimation. As the number of MPIs grows rapidly, concerns over image quality and security increase. Robust watermarking techniques have been introduced to address these challenges. Just Noticeable Difference (JND) has been widely used in quantization-based watermarking frameworks. However, existing JND models handle each image area with a single-level perceptual attention. Visual attention in Quaternion Discrete Wavelet Transform (QDWT), which can reflect the Multi-level perceptual attention feature. Therefore, we propose a new method called Quaternion Attention-based Just Noticeable Difference model for MPIs Watemarking (QAJnd-MW) for watermarking MPIs. This method uses visual attention mechanisms, recognizing that the HVS is more sensitive to attention regions. We generate a masking effect in the JND field. The input image undergoes QDWT to explore multi-scale features. The multi-scale feature maps, with multi-directional luminance and multi-channel color, help create local and global attention maps, which are fused to form the final attention map. Specifically, considering both attention-based masking effects, the quaternion attention-guided JND model is designed for a robust MPI watermarking framework, aiming to further improve MPI security. Extensive experiments on the MP2020 and Blur Detection datasets show that the proposed model significantly improves robustness against JPEG compression attacks, reducing the bit error rate (BER) by up to 12%. Additionally, the model performs well against other attacks, such as those in online social networks, with lower BER than current state-of-the-art techniques.

MLFGCN: short-term residential load forecasting via graph attention temporal convolution network.

Residential load forecasting is a challenging task due to the random fluctuations caused by complex correlations and individual differences. The existing short-term load forecasting models usually introduce external influencing factors such as climate and date. However, these additional information not only bring computational burden to the model, but also have uncertainty. To address these issues, we propose a novel multi-level feature fusion model based on graph attention temporal convolutional network (MLFGCN) for short-term residential load forecasting. The proposed MLFGCN model fully considers the potential long-term dependencies in a single load series and the correlations between multiple load series, and does not require any additional information to be added. Temporal convolutional network (TCN) with gating mechanism is introduced to learn potential long-term dependencies in the original load series. In addition, we design two graph attentive convolutional modules to capture potential multi-level dependencies in load data. Finally, the outputs of each module are fused through an information fusion layer to obtain the highly accurate forecasting results. We conduct validation experiments on two real-world datasets. The results show that the proposed MLFGCN model achieves 0.25, 7.58% and 0.50 for MAE, MAPE and RMSE, respectively. These values are significantly better than those of baseline models. The MLFGCN algorithm proposed in this paper can significantly improve the accuracy of short-term residential load forecasting. This is achieved through high-quality feature reconstruction, comprehensive information graph construction and spatiotemporal features capture.