Detail Enhancement Research Articles

Enhancing low‐light images with lightweight fused fixed‐directional filters network

AbstractDeep learning has made significant progress in the field of low‐light image enhancement. However, challenges remain, such as the substantial parameter consumption required for effective image enhancement. Inspired by multi‐scale geometric transformations in image detail enhancement, a novel model called the fixed‐directional filters network is proposed. Fixed‐directional filters network takes the original image as input and employs multiple branches for parallel processing. One branch uses conventional convolutional layers to extract features from the original image, while the other branches apply non‐linear mapping layers based on wavelet transforms. These wavelet transform branches capture the multi‐scale information of the image by combining different directions and convolutional kernels and utilize a trainable custom gamma mapping layer for non‐linear modulation to enhance specific regions of the image. The feature maps processed by each branch are merged through concatenation operations and then passed through convolutional layers to output the enhanced image. Using trainable mapping functions alone to enhance details significantly reduces the reliance on convolutional layers, effectively lowering the model's parameter count to only 13k parameters. Additionally, experiments demonstrate that fixed‐directional filters network significantly improves image quality, particularly in capturing image details and enhancing image contrast.

Retraction Note: Application of dynamic simulation technology based on image detail enhancement algorithm in urban 3D landscape

Retraction Note: Application of dynamic simulation technology based on image detail enhancement algorithm in urban 3D landscape

LoopNet for fine-grained fashion attributes editing

Generative Adversarial Networks (GANs) have revolutionized the field of image synthesis by transforming randomly sampled latent codes into high-fidelity synthesized images. However, current methods fall short in manipulating a wide range of fashion attributes due to semantic ambiguity and lack of disentanglement. This work focuses on fashion attribute editing by proposing an encoder-based GAN inversion method, namely LoopNet. To enable high-fidelity image inversion and fine-grained attribute editing, it refines edited images through two encoder–decoder stages, utilizing predefined directions from principal component analysis on latent codes and canny loss for detail enhancement. Experiments show LoopNet’s effectiveness in attribute disentanglement and manipulation, outperforming seven state-of-the-art image inversion methods.

AvatarWild: Fully controllable head avatars in the wild

Recent advancements in the field have resulted in significant progress in achieving realistic head reconstruction and manipulation using neural radiance fields (NeRF). Despite these advances, capturing intricate facial details remains a persistent challenge. Moreover, casually captured input, involving both head poses and camera movements, introduces additional difficulties to existing methods of head avatar reconstruction. To address the challenge posed by video data captured with camera motion, we propose a novel method, AvatarWild, for reconstructing head avatars from monocular videos taken by consumer devices. Notably, our approach decouples the camera pose and head pose, allowing reconstructed avatars to be visualized with different poses and expressions from novel viewpoints. To enhance the visual quality of the reconstructed facial avatar, we introduce a view-dependent detail enhancement module designed to augment local facial details without compromising viewpoint consistency. Our method demonstrates superior performance compared to existing approaches, as evidenced by reconstruction and animation results on both multi-view and single-view datasets. Remarkably, our approach stands out by exclusively relying on video data captured by portable devices, such as smartphones. This not only underscores the practicality of our method but also extends its applicability to real-world scenarios where accessibility and ease of data capture are crucial.

Room temperature NO2 gas sensor based mesoporous Zn/Al-layered double hydroxide: The effect of molar ratio

This study elucidates the fabrication and performance evaluation of mesoporous Zn/Al-layered double hydroxide (LDH) for nitrogen dioxide (NO2) efficient room temperature (RT) gas sensor through two-steps solution processed. The correlation between the microstructural analysis and gas sensing performance evaluation was clearly investigated. In detail, average porosity and surface area enhancement from 19.4 to 56.6 nm and 27.50 to 47.85 (m2/g) were perceived for molar ratios of 3:1 and 7:1 of Zn2+ to Al3+, respectively. This in turn resulted in gas sensor response enhancement from 10.21 to 16.41 % for the optimum device (molar ratio of 7:1) under gas concentration of 100 ppm. Continuously, the fabricated devices responded linearly to the applied gas concentration alteration wherein R2 average value of 0.995 was attained. In conjunction, the time-resolved characteristics revealed considerably fast response time along comparatively longer recovery window wherein the average rise/fall periods were estimated to be 2.98 and 19.3 sec., respectively.

Infrared and visible image fusion based on semi-global weighted least squares and guided edge-aware filters

Most of the existing image fusion methods are dedicated to extracting the respective private features in the source image to reconstruct to get a fused image that contains rich information. However, its neglect of the enhancement of texture details in the source image as well as the preservation of background structures, which makes the fusion result look less impressive. To this end, an infrared and visible image fusion method based on a semi-global weighted least squares (SGWLS) method and guided edge-aware filter (GEAF) is proposed. First, a SGWLS method is applied to decompose the source image into base and detail layers containing different scale information. Then, the detail layer are texture-enhanced by using the scale coefficients. Due to the lack of edge features in the original base layer, for this reason, a guided edge-aware filter is designed, in which the enhanced detail layer was used as the guidance image as a way to perform a quadratic feature enhancement fusion for the base layer in order to retain richer gradient information. Finally, reconstruction of the fused detail and base layers by using inverse transformation to obtain the fusion result. Compared with nine state-of-the-art fusion algorithms, a large number of experimental results demonstrate the superior performance of the proposed method in highlighting texture details and background features, as well as the obvious advantages in SF and AG metrics.

Mask-Guided Target Node Feature Learning and Dynamic Detailed Feature Enhancement for lncRNA-Disease Association Prediction.

Identifying new relevant long noncoding RNAs (lncRNAs) for various human diseases can facilitate the exploration of the causes and progression of these diseases. Recently, several graph inference methods have been proposed to predict disease-related lncRNAs by exploiting the topological structure and node attributes within graphs. However, these methods did not prioritize the target lncRNA and disease nodes over auxiliary nodes like miRNA nodes, potentially limiting their ability to fully utilize the features of the target nodes. We propose a new method, mask-guided target node feature learning and dynamic detailed feature enhancement for lncRNA-disease association prediction (MDLD), to enhance node feature learning for improved lncRNA-disease association prediction. First, we designed a heterogeneous graph masked transformer autoencoder to guide feature learning, focusing more on the features of target lncRNA (disease) nodes. The target nodes were increasingly masked as training progressed, which helps develop a more robust prediction model. Second, we developed a graph convolutional network with dynamic residuals (GCNDR) to learn and integrate the heterogeneous topology and features of all lncRNA, disease, and miRNA nodes. GCNDR employs an interlayer residual strategy and a residual evolution strategy to mitigate oversmoothing caused by multilayer graph convolution. The interlayer residual strategy estimates the importance of node features learned in the previous GCN encoding layer for nodes in the current encoding layer. Additionally, since there are dependencies in the importance of features of individual lncRNA (disease, miRNA) nodes across multiple encoding layers, a gated recurrent unit-based strategy is proposed to encode these dependencies. Finally, we designed a perspective-level attention mechanism to obtain more informative features of lncRNA and disease node pairs from the perspectives of mask-enhanced and dynamic-enhanced node features. Cross-validation experimental results demonstrated that MDLD outperformed 10 other state-of-the-art prediction methods. Ablation experiments and case studies on candidate lncRNAs for three diseases further proved the technical contributions of MDLD and its capability to discover disease-related lncRNAs.

Image inpainting by bidirectional information flow on texture and structure

Image inpainting aims to recover damaged regions of a corrupted image and maintain the integrity of the structure and texture within the filled regions. Previous popular approaches have restored images with both vivid textures and structures by introducing structure priors. However, the structure prior-based approaches meet the following main challenges: (1) the fine-grained textures suffer from adverse inpainting effects because they do not fully consider the interaction between structures and textures, (2) the features of the multi-scale objects in structural and textural information cannot be extracted correctly due to the limited receptive fields in convolution operation. In this paper, we propose a texture and structure bidirectional generation network (TSBGNet) to address the above issues. We first reconstruct the texture and structure of corrupted images; then, we design a texture-enhanced-FCMSPCNN (TE-FCMSPCNN) to optimize the generated textures. We also conjoin a bidirectional information flow (BIF) module and a detail enhancement (DE) module to integrate texture and structure features globally. Additionally, we derive a multi-scale attentional feature fusion (MAFF) module to fuse multi-scale features. Experimental results demonstrate that TSBGNet effectively reconstructs realistic contents and significantly outperforms other state-of-the-art approaches on three popular datasets. Moreover, the proposed approach yields promising results on the Dunhuang Mogao Grottoes Mural dataset.

Flotation froth image enhancement based on region decomposition and guided filtering

Extraction of information from froth images is important for automatic control of froth flotation. However, images captured by cameras often suffer from severe uneven lighting, which significantly reduces the quality of froth images. Low-quality images hinder the accurate extraction of froth information, thereby affecting the control of the froth flotation system. Hence, we propose an image enhancement method based on region decomposition and guided filtering to improve the quality of images. Initially, we separate the image into regions with sufficient and insufficient illumination based on reflectance. In regions with insufficient illumination, the guided filter is applied to direct pixels to acquire information from brighter points in their neighborhood. Conversely, in other regions, we regulate the magnitude of pixel variations to prevent overexposure. Finally, a detail enhancement method is proposed based on a multi-scale Gaussian pyramid and texture fusion to improve clarity and naturalness. The experiments show that the method we proposed surpasses several state-of-the-art algorithms on public datasets. In the field of flotation, our method effectively enhances the image quality. Compared to other enhancement methods under the same segmentation strategy, our method significantly improves segmentation accuracy, demonstrating its strong practical value. In addition, our method also shows advantages in terms of computational speed.

BMSeNet: Multiscale Context Pyramid Pooling and Spatial Detail Enhancement Network for Real-Time Semantic Segmentation.

Most real-time semantic segmentation networks use shallow architectures to achieve fast inference speeds. This approach, however, limits a network's receptive field. Concurrently, feature information extraction is restricted to a single scale, which reduces the network's ability to generalize and maintain robustness. Furthermore, loss of image spatial details negatively impacts segmentation accuracy. To address these limitations, this paper proposes a Multiscale Context Pyramid Pooling and Spatial Detail Enhancement Network (BMSeNet). First, to address the limitation of singular semantic feature scales, a Multiscale Context Pyramid Pooling Module (MSCPPM) is introduced. By leveraging various pooling operations, this module efficiently enlarges the receptive field and better aggregates multiscale contextual information. Moreover, a Spatial Detail Enhancement Module (SDEM) is designed, to effectively compensate for lost spatial detail information and significantly enhance the perception of spatial details. Finally, a Bilateral Attention Fusion Module (BAFM) is proposed. This module leverages pixel positional correlations to guide the network in assigning appropriate weights to the features extracted from the two branches, effectively merging the feature information of both branches. Extensive experiments were conducted on the Cityscapes and CamVid datasets. Experimental results show that the proposed BMSeNet achieves a good balance between inference speed and segmentation accuracy, outperforming some state-of-the-art real-time semantic segmentation methods.

Generalized denoising network LGCT-Net for various types of ESPI wrapped phase patterns

In this paper, we propose a generalized network based on our proposed Local-Global Channel Transformer (LGCT) module for denoising various types of ESPI wrapped phase patterns (including low-density, medium-density, high-density, variable-density, and discontinuous phase patterns). The Conv + BN + ReLU layer consists of convolution (Conv), batch normalization (BN), and the rectified linear unit (Relu) in series. The generalized network LGCT-Net interleaves four LGCT modules with five Conv + BN + ReLU layers in a dense connection manner. We propose the LGCT Module by stacking three Dilated-Group Convolution blocks (DGC block), a Contextual Transformer block (CoT block), and an Efficient Channel Attention block (ECA block). The LGCT module simultaneously leverages the local context extraction capability of convolutions and the powerful global information extraction capability of a transformer. Additionally, it performs feature extraction in both spatial and channel dimensions. We also create a diverse ESPI wrapped phase pattern denoising dataset with various densities, shapes, noise levels, and discontinuity. We successfully train the LGCT-Net without any preprocessing or postprocessing steps. We evaluate the performance of our method on simulated and experimental ESPI wrapped phase patterns with discontinuity and different densities. Then we compare it with previously published denoising methods PEARLS, HDCNN, ADCNN, and DBDNet quantitatively and qualitatively. The results show that our method facilitates the reduction of speckle noise and the enhancement of fine details while preserving structure and shape, outperforming the compared methods. In the end, we apply our method to dynamic measurements of nuclear graphite ESPI phase patterns at different times. And then performing phase unwrapping on the filtered phase patterns, we achieve successful results.

Infrared Image Enhancement Based on Adaptive Guided Filter and Global–Local Mapping

Infrared image enhancement technology plays a crucial role in improving image quality, addressing issues like low contrast, lack of sharpness, and poor visual effects within the original images. However, existing decomposition-based algorithms struggle with balancing detail enhancement, noise suppression, and utilizing global and local information effectively. This paper proposes an innovative method for enhancing details in infrared images using adaptive guided filtering and global–local mapping. Initially, the original image is decomposed into its base layer and detail layer through the adaptive guided filter and difference of the Gaussian filter. Subsequently, the detail layer undergoes enhancement using a detail gain factor. Finally, the base layer and enhanced detail layer are merged and remapped to a lower gray level. Experimental results demonstrate significant improvements in global and local contrast, as well as sharpness, with an average gradient enhancement of nearly 3% across various scenes.

Detailed-based dictionary learning for low-light image enhancement using camera response model for industrial applications

Images captured in low-light environments are severely degraded due to insufficient light, which causes the performance decline of both commercial and consumer devices. One of the major challenges lies in how to balance the image enhancement properties of light intensity, detail presentation, and colour integrity in low-light enhancement tasks. This study presents a novel image enhancement framework using a detailed-based dictionary learning and camera response model (CRM). It combines dictionary learning with edge-aware filter-based detail enhancement. It assumes each small detail patch could be sparsely characterised in the over-complete detail dictionary that was learned from many training detail patches using iterative ℓ1\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$${{\\ell}}_{1}$$\\end{document}-norm minimization. Dictionary learning will effectively address several enhancement concerns in the progression of detail enhancement if we remove the visibility limit of training detail patches in the enhanced detail patches. We apply illumination estimation schemes to the selected CRM and the subsequent exposure ratio maps, which recover a novel enhanced detail layer and generate a high-quality output with detailed visibility when there is a training set of higher-quality images. We estimate the exposure ratio of each pixel using illumination estimation techniques. The selected camera response model adjusts each pixel to the desired exposure based on the computed exposure ratio map. Extensive experimental analysis shows an advantage of the proposed method that it can obtain enhanced results with acceptable distortions. The proposed research article can be generalised to address numerous other similar problems, such as image enhancement for remote sensing or underwater applications, medical imaging, and foggy or dusty conditions.

Enhanced low-light image fusion through multi-stage processing with Bayesian analysis and quadratic contrast function

This manuscript introduces an innovative multi-stage image fusion framework that adeptly integrates infrared (IR) and visible (VIS) spectrum images to surmount the difficulties posed by low-light settings. The approach commences with an initial preprocessing stage, utilizing an Efficient Guided Image Filter for the infrared (IR) images to amplify edge boundaries and a function for the visible (VIS) images to boost local contrast and brightness. Utilizing a two-scale decomposition technique that incorporates Lipschitz constraints-based smoothing, the images are effectively divided into distinct base and detail layers, thereby guaranteeing the preservation of essential structural information. The process of fusion is carried out in two distinct stages: firstly, a method grounded in Bayesian theory is employed to effectively combine the base layers, so effectively addressing any inherent uncertainty. Secondly, a Surface from Shade (SfS) method is utilized to ensure the preservation of the scene's geometry by enforcing integrability on the detail layers. Ultimately a Choose Max principle is employed to determine the most prominent textural characteristics, resulting in the amalgamation of the base and detail layers to generate an image that exhibits a substantial enhancement in both clarity and detail. The efficacy of our strategy is substantiated by rigorous testing, showcasing notable progressions in edge preservation, detail enhancement, and noise reduction. Consequently, our method presents significant advantages for real-world applications in image analysis.

A two-stage image enhancement and dynamic feature aggregation framework for gastroscopy image segmentation

Accurate and reliable automatic segmentation of lesion areas in gastroscopy images can assist endoscopists in making diagnoses and reduce the possibility of missed or incorrect diagnoses. This paper presents a two-stage framework for segmenting gastroscopy images, which aims to improve the accuracy of medical image segmentation tasks using limited datasets. The proposed framework consists of two stages: the Image Enhancement Stage and the Lesion Segmentation Stage. First, in the Image Enhancement Stage, an image enhancement solution called TDC-Enhance is proposed to enrich the original small-scale gastroscopy image dataset. This solution performs Texture Enhancement, Detail Enhancement, and Color Enhancement on the original images. Then, in the Lesion Segmentation Stage, a multi-path automatic segmentation network for gastroscopy images, named DynaSiam, is introduced. DynaSiam comprises a Dependent Encoder, a Shared Encoder, and a Fusion Decoder. It learns feature information related to the lesion region by encoding the different enhanced images obtained in the Image Enhancement Stage as inputs to the multi-path network. Additionally, a Dynamic Feature Interaction (DFI) block is designed to capture and learn deeper image information, thereby improving the segmentation performance of the model. The experimental results show that the proposed method achieves a 90.80% mIoU, 92.71% Dice coefficient and 96.31% Accuracy. Other performance metrics also indicate the best performance, suggesting that the proposed model has significant potential for clinical analysis and diagnosis. Code and implementation details can be found on GitHub: https://github.com/kyasulee/DynaSiam.

Dual path features interaction network for efficient image super-resolution

Image super-resolution (SR) is a crucial task in computer vision that involves reconstructing a low-resolution (LR) image into its high-resolution (HR) counterpart. Transformer-based methods excel at establishing long-range dependency but face challenges with high-complexity computations and capturing fine-grained features. Conversely, CNN-based methods are advantageous in capturing fine-grained features but are limited by fixed receptive fields. Additionally, most SR methods suffer from channel redundancy, leading to higher computational overhead. In this paper, we propose a Dual Path Features Interaction Network (DPFINet) to achieve efficient image SR, which consists of two components: a) To alleviate the issue of feature channel redundancy, a Local–Global Features Modeling (LGFM) method is newly proposed, which concurrently models global and local features by splitting features along different channels. In LGFM, a Shift Window Linear Attention (SWLA) layer is adopted to effectively capture global information through a large shift window based on the split features. Meanwhile, a Multi-Scale Detail Enhancement (MSDE) layer is designed, where the split other features are encoded to facilitate detail reconstruction through an interactive fusion of semantic and local information, thereby addressing the limitations of SWLA in capturing fine-grained features. b) A Cross-Level Features Interaction (CLFI) method is proposed to fuse global and local features modeled by different network structures (SWLA and MSDE), where a novel residual fusion mechanism is designed to preserve both global and local information while complementing each other. Extensive experiments demonstrate that our method outperforms most state-of-the-art SR methods on five benchmark datasets. Notably, during inference, our approach improves performance by 0.41 dB and reduces memory consumption by approximately 79% compared to DiVANet (Behjati et al., 2023) on the Manga109 (×4) dataset.

Multi-stage coarse-to-fine progressive enhancement network for single-image HDR reconstruction

Compared with traditional imaging, high dynamic range (HDR) imaging technology can record scene information more accurately, thereby providing users higher quality of visual experience. Inverse tone mapping is a direct and effective way to realize single-image HDR reconstruction, but it usually suffers from some problems such as detail loss, color deviation and artifacts. To solve the problems, this paper proposes a multi-stage coarse-to-fine progressive enhancement network (named MSPENet) for single-image HDR reconstruction. The entire multi-stage network architecture is designed in a progressive manner to obtain higher-quality HDR images from coarse-to-fine, where a mask mechanism is used to eliminate the effects of over-exposure regions. Specifically, in the first two stages, two asymmetric U-Nets are constructed to learn the multi-scale information of input image and perform coarse reconstruction. In the third stage, a residual network with channel attention mechanism is constructed to learn the fusion of progressively transferred multi-level features and perform fine reconstruction. In addition, a multi-stage progressive detail enhancement mechanism is designed, including progressive gated recurrent unit fusion mechanism and multi-stage feature transfer mechanism. The former fuses the progressively transferred features with coarse HDR features to reduce the error stacking effect caused by multi-stage networks. Meanwhile, the latter fuses early features to supplement the lost information during each stage of feature delivery and combines features from different stages. Extensive experimental results show that the proposed method can reconstruct higher quality HDR images and effectively recover texture and color information in over-exposure regions compared to the state-of-the-art methods.

Multiple Resolutions Detail Enhancement Network for Real-Time Image Semantic Segmentation

Multiple Resolutions Detail Enhancement Network for Real-Time Image Semantic Segmentation

HEAL: High-Frequency Enhanced and Attention-Guided Learning Network for Sparse-View CT Reconstruction.

X-ray computed tomography (CT) imaging technology has become an indispensable diagnostic tool in clinical examination. However, it poses a risk of ionizing radiation, making the reduction of radiation dose one of the current research hotspots in CT imaging. Sparse-view imaging, as one of the main methods for reducing radiation dose, has made significant progress in recent years. In particular, sparse-view reconstruction methods based on deep learning have shown promising results. Nevertheless, efficiently recovering image details under ultra-sparse conditions remains a challenge. To address this challenge, this paper proposes a high-frequency enhanced and attention-guided learning Network (HEAL). HEAL includes three optimization strategies to achieve detail enhancement: Firstly, we introduce a dual-domain progressive enhancement module, which leverages fidelity constraints within each domain and consistency constraints across domains to effectively narrow the solution space. Secondly, we incorporate both channel and spatial attention mechanisms to improve the network's feature-scaling process. Finally, we propose a high-frequency component enhancement regularization term that integrates residual learning with direction-weighted total variation, utilizing directional cues to effectively distinguish between noise and textures. The HEAL network is trained, validated and tested under different ultra-sparse configurations of 60 views and 30 views, demonstrating its advantages in reconstruction accuracy and detail enhancement.

Underwater image enhancement via color correction and multi-feature image fusion

Abstract The light attenuation underwater causes the actual underwater images to suffer from color cast, low contrast, and weak illumination. To address these issues, an effective fusion-based method is proposed, which realizes color correction, brightness adjustment, contrast, and detail enhancement of underwater images. Concretely, we first design an adaptive color correction method via dominant color channel judgment and lower color channel compensation. Then, we detect the brightness of each input image and propose a gamma correction function based on the gradient of the cumulative histogram to adjust the brightness of the low-light images. Subsequently, global histogram stretching and adaptive fractional differentiation techniques are employed to process the brightness-adjusted image, and then the global contrast-enhanced version and detail-enhanced version are generated respectively. To integrate the advantages of both versions, a channel fusion method based on the Lab color space is used to fuse the luminance and color of the two versions separately. The experimental results demonstrate the effectiveness of the proposed method in improving the color and illumination of underwater images, as well as enhancing the clarity of images. Moreover, the testing results on multiple datasets validate the excellent stability of this method.