Laplacian Pyramid Research Articles

Single-image super-resolution based on an improved asymmetric Laplacian pyramid structure

Large-scale factor image super-resolution whose scale factor is greater than 4 is significant in real-world applications for single image super-resolution. The current image super-resolution techniques for large-scale factor, however, frequently upsample low-resolution images in a single pass, leading to edge artifacts in the reconstructed images. In this article, we provide an improved asymmetric Laplacian pyramid network to further realize large-scale factor image super-resolution and fully utilize various size features. A distinct architecture is applied at each layer of the pyramid for improving feature extraction. We extend the first layer of the pyramid with a lightweight transformer design, which enables the model to efficiently collect the contextual information in the sequence by utilizing the multi-headed attention mechanism. Additionally, we combine improved dense skip connections with recursive operations to form a deep dense recursive convolutional neural network that fuses low-level and high-level features while broadening the network's receptive field. The quantitative and qualitative analysis on benchmark dataset's demonstrate that our method provides superior performance in both PSNR and SSIM and is more in line with human visual perception.

A fine-grained recognition technique for identifying Chinese food images

As a crucial area of research in the field of computer vision, food recognition technology has become a core technology in many food-related fields, such as unmanned restaurants and food nutrition analysis, which are closely related to our healthy lives. Obtaining accurate classification results is the most important task in food recognition. Food classification is a fine-grained recognition process, which involves extracting features from a group of objects with similar appearances and accurately classifying them into different categories. In a such usage environment, the network is required to not only overview the overall image, but also capture the subtle details within it. In addition, since Chinese food images have unique texture features, the model needs to extract texture information from the image. However, existing CNN methods have not focused on and processed this information. To classify food as accurately as possible, this paper introduces the Laplace pyramid into the convolution layer and proposes a bilinear network that can perceive image texture features and multi-scale features (LMB-Net). The proposed model was evaluated on a public dataset, and the results demonstrate that LMB-Net achieves state-of-the-art classification performance.

Hierarchical cumulative network for unsupervised medical image registration

Unsupervised deep learning techniques have gained increasing popularity in deformable medical image registration However, existing methods usually overlook the optimal similarity position between moving and fixed images To tackle this issue, we propose a novel hierarchical cumulative network (HCN), which explicitly considers the optimal similarity position with an effective Bidirectional Asymmetric Registration Module (BARM). The BARM simultaneously learns two asymmetric displacement vector fields (DVFs) to optimally warp both moving images and fixed images to their optimal similar shape along the geodesic path. Furthermore, we incorporate the BARM into a Laplacian pyramid network with hierarchical recursion, in which the moving image at the lowest level of the pyramid is warped successively for aligning to the fixed image at the lowest level of the pyramid to capture multiple DVFs. We then accumulate these DVFs and up-sample them to warp the moving images at higher levels of the pyramid to align to the fixed image of the top level. The entire system is end-to-end and jointly trained in an unsupervised manner. Extensive experiments were conducted on two public 3D Brain MRI datasets to demonstrate that our HCN outperforms both the traditional and state-of-the-art registration methods. To further evaluate the performance of our HCN, we tested it on the validation set of the MICCAI Learn2Reg 2021 challenge. Additionally, a cross-dataset evaluation was conducted to assess the generalization of our HCN. Experimental results showed that our HCN is an effective deformable registration method and achieves excellent generalization performance.

Research on the scene design of virtual reality technology in film and television animation

Abstract In this paper, firstly, we study the modeling of film and television animation based on virtual reality technology and use quadratic polynomial inverse to control the intensity of light source to establish the lighting model. The image alignment is designed for the scene based on multi-resolution decomposition, and the filtered image layer is obtained by calculating the difference between two layers of Gaussian filtered images, and the Laplace pyramid is established to match the image resolution consistently. Then the scene is chunked and locally rendered, and a smooth transition between geometry and texture is achieved by morphing the geometry next to the texture. Finally, the scene effect, LOD detail level and interactive experience effect before and after simplification are compared and analyzed. The scene design of virtual reality technology basically scores 5-6 on each index, while the scene design of common technology only scores 3-4, indicating that the scene design of virtual reality technology works well.

Holistic Dynamic Frequency Transformer for image fusion and exposure correction

The correction of exposure-related issues is a pivotal component in enhancing the quality of images, offering substantial implications for various computer vision tasks. Historically, most methodologies have predominantly utilized spatial domain recovery, offering limited consideration to the potentialities of the frequency domain. Additionally, there has been a lack of a unified perspective towards low-light enhancement, exposure correction, and multi-exposure fusion, complicating and impeding the optimization of image processing. In response to these challenges, this paper proposes a novel methodology that leverages the frequency domain to improve and unify the handling of exposure correction tasks. Our method introduces Holistic Frequency Attention and Dynamic Frequency Feed-Forward Network, which replace conventional correlation computation in the spatial-domain. They form a foundational building block that facilitates a U-shaped Holistic Dynamic Frequency Transformer as a filter to extract global information and dynamically select important frequency bands for image restoration. Complementing this, we employ a Laplacian pyramid to decompose images into distinct frequency bands, followed by multiple restorers, each tuned to recover specific frequency-band information. The pyramid fusion allows a more detailed and nuanced image restoration process. Ultimately, our structure unifies the three tasks of low-light enhancement, exposure correction, and multi-exposure fusion, enabling comprehensive treatment of all classical exposure errors. Benchmarking on mainstream datasets for these tasks, our proposed method achieves state-of-the-art results, paving the way for more sophisticated and unified solutions in exposure correction.

Laplacian Salience-Gated Feature Pyramid Network for Accurate Liver Vessel Segmentation.

Liver vessels generated from computed tomography are usually pretty small, which poses major challenges for satisfactory vessel segmentation, including 1) the scarcity of high-quality and large-volume vessel masks, 2) the difficulty in capturing vessel-specific features, and 3) the heavily imbalanced distribution of vessels and liver tissues. To advance, a sophisticated model and an elaborated dataset have been built. The model has a newly conceived Laplacian salience filter that highlights vessel-like regions and suppresses other liver regions to shape the vessel-specific feature learning and to balance vessels against others. It is further coupled with a pyramid deep learning architecture to capture different levels of features, thus improving the feature formulation. Experiments show that this model markedly outperforms the state-of-the-art approaches, achieving a relative improvement of Dice score by at least 1.63% compared to the existing best model on available datasets. More promisingly, the averaged Dice score produced by the existing models on the newly constructed dataset is as high as 0.734±0.070 , which is at least 18.3% higher than that obtained from the existing best dataset under the same settings. These observations suggest that the proposed Laplacian salience, together with the elaborated dataset, can be helpful for liver vessel segmentation.

A lightweight object detection framework for underwater imagery with joint image restoration and color transformation

Traditional underwater detectors struggle to match the demands of visual observation and icon detection because of the underwater optical pictures' color distortion, uneven illumination, and excessive noise levels. In this paper, a lightweight deep-water object detection network is proposed, which consists of three key components, namely image restoration, color transformation, and object detection. First, we propose a lightweight attention module for the problems of color distortion and uneven illumination in underwater images. This module pays more attention to the areas with a greater influence of underwater color when dealing with different types of poor-quality images. Second, to reduce the computational complexity of object detection, we propose a color conversion module based on the laplacian pyramid decomposition. This module incorporates multi-scale features to effectively address the loss of details in the brightest or darkest regions. Finally, we propose a multi-scale dense residual object detection network. The network applies a multi-scale kernel to an improved residual module to realize the extraction and fusion of feature information of multiple receptive fields. Furthermore, to deal with the problem of underwater small-sample object detection, we propose a mutually exclusive loss function. The mutual exclusion loss promotes the update of network parameters so that the classes with different characteristics of samples generated by the network are mutually exclusive. We ran tests on the Pycharm platform to show that the suggested lightweight joint underwater object identification algorithm outperformed the most sophisticated ones. The proposed system was put to the test in an intricate underwater setting, achieving 92.56 percent accuracy and 19.2 frames per second.

Multi-Step Prediction of Typhoon Tracks Combining Reanalysis Image Fusion Using Laplacian Pyramid and Discrete Wavelet Transform with ConvLSTM

Typhoons often cause huge losses, so it is significant to accurately predict typhoon tracks. Nowadays, researchers predict typhoon tracks with the single step, while the correlation of adjacent moments data is small in long-term prediction, due to the large step of time. Moreover, recursive multi-step prediction results in the accumulated error. Therefore, this paper proposes to fuse reanalysis images at the similarly historical moment and predicted images through Laplacian Pyramid and Discrete Wavelet Transform to reduce the accumulated error. That moment is determined according to the difference in the moving angle at predicted and historical moments, the color histogram similarity between predicted images and reanalysis images at historical moments and so on. Moreover, reanalysis images are weighted cascaded and input to ConvLSTM on the basis of the correlation between reanalysis data and the moving angle and distance of the typhoon. And, the Spatial Attention and weighted calculation of memory cells are added to improve the performance of ConvLSTM. This paper predicted typhoon tracks in 12 h, 18 h, 24 h and 48 h with recursive multi-step prediction. Their MAEs were 102.14 km, 168.17 km, 243.73 km and 574.62 km, respectively, which were reduced by 1.65 km, 5.93 km, 4.6 km and 13.09 km, respectively, compared with the predicted results of the improved ConvLSTM in this paper, which proved the validity of the model.

Hyperspectral image classification Based on weakened Laplacian pyramid and guided filtering

ABSTRACT In recent years, methods of spatial-spectral feature extraction have been widely used in hyperspectral image (HSI) classification and have achieved good performance. However, the ways are often unable to distinguish effectively the boundary of the adjacent ground objects. Multi-scale feature fusion alleviates the problems to a certain extent, but in the case of limited samples, the previous methods often fail to achieve the desired effect. In this paper, we propose a multi-scale hyperspectral image classification method based on weakened Laplacian pyramid and guided filtering to nicely distinguish the boundary of the different ground objects. This structure first obtains multi-scale information by constructing a weakened Laplacian pyramid, and weakens the magnification of Gaussian blur in the upsampling process. While obtaining multi-scale images, it can not only retain enough image information to expand feature samples but also emphasize edge information in the edge extraction of the Laplacian pyramid. Then, the weakened Laplacian pyramid feature map is used as the guiding image of the guiding filter, which can better preserve the edge features and further improve the classification performance of HSI. Extensive experiments are carried out on four HSI data sets. The results show that our method is superior in classification accuracy and computational cost.

Laplacian Pyramid Fusion Network With Hierarchical Guidance for Infrared and Visible Image Fusion

The fusion of infrared and visible images combines the information from two complementary imaging modalities for various computer vision tasks. Many existing techniques, however, fail to maintain a uniform overall style and keep salient details of individual modalities simultaneously. This paper presents an end-to-end Laplacian Pyramid Fusion Network with hierarchical guidance (HG-LPFN) that takes advantage of pixel-level saliency reservation of Laplacian Pyramid and global optimization capability of deep learning. The proposed scheme generates hierarchical saliency maps through Laplacian Pyramid decomposition and modal difference calculation. In the pyramid fusion mode, all sub-networks are connected in a bottom-up manner. The sub-network for low-frequency fusion focuses on extracting universal features to produce an opposite style while sub-networks for high-frequency fusion determine how much the details of each modality will be retained. Taking the style, details, and background into consideration, we design a set of novel loss functions to supervise both low-frequency images and full-resolution images under the guidance of saliency maps. Experimental results on public datasets demonstrate that the proposed HG-LPFN outperforms the state-of-the-art image fusion techniques.

Strong non-uniformity correction algorithm based on spectral shaping statistics and LMS.

The existence of non-uniformity in infrared detector output images is a widespread problem that significantly degrades image quality. Existing scene-based non-uniformity correction algorithms typically struggle to balance strong non-uniformity correction with scene adaptability. To address this issue, we propose a novel scene-based algorithm that leverages the frequency characteristics of the non-uniformity, combine and improve single-frame stripe removal, multi-scale statistics, and least mean square (LMS) methods. Following the "coarse-to-fine" correction process, the coarse correction stage introduces an adaptive progressive correction strategy based on Laplacian pyramids. By improving 1-D guided filtering and high-pass filtering to shape high-frequency sub-bands, non-uniformity can be well separated from the scene, effectively suppressing ghosting. In the fine correction stage, we optimize the expected image estimation and spatio-temporal adaptive learning rates based on guided filtering LMS method. To validate the efficacy of our algorithm, we conduct extensive simulation and real experiments, demonstrating its adaptability to various scene conditions and its effectiveness in correcting strong non-uniformity.

A novel endoscopy image fusion system: combine white light imaging and compound band imaging.

White light imaging (WLI) is a commonly seen examination mode in endoscopy. The particular light in compound band imaging (CBI) can highlight delicate structures, such as capillaries and tiny structures on the mucosal surface. These two modes complement each other, and doctors switch between them manually to complete the examination. This paper proposes an endoscopy image fusion system to combine WLI and CBI. We add a real-time rotatable color wheel in the light source device of the AQ-200 endoscopy system to achieve rapid imaging of two modes at the same position of living tissue. The two images corresponding to the pixel level can avoid registration and lay the foundation for image fusion. We propose a multi-scale image fusion framework, which involves Laplacian pyramid (LP) and convolutional sparse representation (CSR) and strengthens the details in the fusion rule. Volunteer experiments and ex vivo pig stomach trials are conducted to verify the feasibility of our proposed system. We also conduct comparative experiments with other image fusion methods, evaluate the quality of the fused images, and verify the effectiveness of our fusion framework. The results show that our fused image has rich details, high color contrast, apparent structures, and clear lesion boundaries. An endoscopy image fusion system is proposed, which does not change the doctor's operation and makes the fusion of WLI and CBI optical staining technology a reality. We change the light source device of the endoscope, propose an image fusion framework, and verify the feasibility and effectiveness of our scheme. Our method fully integrates the advantages of WLI and CBI, which can help doctors make more accurate judgments than before. The endoscopy image fusion system is of great significance for improving the detection rate of early lesions and has broad application prospects.

Single UHD image dehazing via Interpretable Pyramid Network

Currently, most dehazing-based deep approaches are developed as an end-to-end manner to reconstruct a degraded image in an unintelligible fashion. For these dehazing models, the absence of ‘Credible Modeling’ to guide the network design is a barrier to be applied commercially in the open-world. To address this problem, we introduce the Taylor approximation principle as the soul and materialize this principle with the help of the Laplacian Pyramid. Specifically, we assume that the N paths of Laplacian pyramid model correspond to the N terms (sub-functions) in Taylor’s theorem. We attempt to use bottom paths and top paths to reconstruct the low-/high-frequency information of the clear image, respectively. Further, we develop a T-Unet module that focuses on regularizing the feature maps generated at bottom paths and design an attention sharing weight K to help approximate the Taylor high-order terms. Extensive experimental results demonstrate that our approach can run a 4K image on a single GPU with 24G RAM in real-time (80fps) and have unparalleled interpretability. The url of our code at https://github.com/zzr-idam/Interpretable-Pyramid-Network.

Implementation and Comparative Study of Pyramid-based Image Fusion Techniques for Lumbar Spine Images

Image fusion is a method of combining the necessary and relevant information from the set of source images into a single (fused) image which can be deemed to be more informative than the source. This paper discusses the implementation of various pyramid-based image fusion algorithms, such as the Laplacian pyramid, the ratio of the low-pass pyramid, the contrast pyramid, and the filter subtract decimate pyramid on multimodal CT and MR images of the lumbar spine. The fused images were evaluated using various objective evaluation quality metrics. The experimental results demonstrated that the ratio of the low pass pyramid achieved better performance compared to the other pyramids implemented, indicating that the fused image can also be used for further image fusion application or analysis purposes.

MSPIF: Multi-stage progressive visible and infrared image fusion with structures preservation

Infrared and visible image fusion is one of the hottest research in computer vision to improve image quality. Traditional fusion methods suffer from detail loss, low resolution, and application scenarios limitation. To address these problems, this paper proposes a multi-stage progressive visible and infrared image fusion strategy (MSPIF), including the following key stages: Firstly, the visible image is enhanced using a weighted fusion visible image enhancement algorithm. Secondly, the infrared and enhanced visible images are both decomposed by a pre-trained network, namely Retinex_Net, to obtain their respective illumination and reflectance components. Thirdly, the reflectance components are decomposed by discrete wavelet transform (DWT). The low-frequency components are fused by a weighted information entropy fusion strategy while the high-frequency components are fused by a local energy fusion strategy. Such two fused parts contribute to the final fused reflectance component with inverse DWT (IDWT). Meanwhile, the illumination components are fused using a multi-scale fusion strategy based on the Laplace pyramid. Finally, the improved result is generated by the fused reflectance and illumination components based on Retinex theory. Quantitative and qualitative results of experiments on the TNO, KAIST, and ROAD datasets show that the proposed MSPIF is effective and achieves good results with structures preservation, even is superior to existing methods.

Diagnosing and categorizing of pulmonary diseases using Deep learning conventional Neural network

Lung cancer is one of the major illnesses that contribute to millions of fatalities worldwide. Numerous deaths could be saved through the early identification and categorization of lung cancers. However, with traditional approaches, classification accuracy cannot be produced. To detect and classify lung diseases, a deep learning convolutional neural network model has been developed. LDDC, the customized local trilateral filter, is used for pre-processing the lung images from computing tomography for non-local trilateral filters. The region of interest for lung cancer was successfully restricted throughout the segmentation of the disease using hybrid fuzzy morphological procedures. To extract the deep seismic features, the Laplacian pyramid decomposition method was utilized for the segmented image. This paper covers an overall analysis of non-local trilateral filter Processing, hybrid fuzzy morphological techniques and analysis of patient and disease characteristics of LIDR- IDRI and FDA data of Group A (no co-AGA), P-value, Multi-mut Patient, Group B (with a co-AGA).

Pyramid Texture Filtering

We present a simple but effective technique to smooth out textures while preserving the prominent structures. Our method is built upon a key observation---the coarsest level in a Gaussian pyramid often naturally eliminates textures and summarizes the main image structures. This inspires our central idea for texture filtering, which is to progressively upsample the very low-resolution coarsest Gaussian pyramid level to a full-resolution texture smoothing result with well-preserved structures, under the guidance of each fine-scale Gaussian pyramid level and its associated Laplacian pyramid level. We show that our approach is effective to separate structure from texture of different scales, local contrasts, and forms, without degrading structures or introducing visual artifacts. We also demonstrate the applicability of our method on various applications including detail enhancement, image abstraction, HDR tone mapping, inverse halftoning, and LDR image enhancement. Code is available at https://rewindl.github.io/pyramid_texture_filtering/.

Adaptive hybrid pansharpening: a novel approach for combining two methods to achieve superior pansharpening performance

ABSTRACT This study proposes an adaptive strategy to improve the performances of two widely used pansharpening strategies, namely the Generalized Laplacian Pyramid (GLP) with Modulation Transfer Function (MTF)-matched filter and Context-Based Decision (CBD) injection scheme (i.e. MTF-GLP-CBD) and the Sparse Representation of Injected Details (i.e. SR-D). By considering the variance information of the PAN data, the suggested method determines which part of a pansharpened image should be produced by which of these procedures. The effectiveness of the suggested technique was assessed both qualitatively and quantitatively by comparing its performance to 31 other pansharpening strategies across three different test sites with diverse features. The outcomes of the experiments demonstrated the efficacy of the suggested approach. The experiments showed that the suggested approach not only surpassed the standard MTF-GLP-CBD and SR-D algorithms, but also all the other pansharpening algorithms employed. Of all pansharpening results, the results of the suggested approach led to the best quality index values. It can also be concluded that the suggested technique has the potential to enhance the effectiveness of other pansharpening algorithms.

An optimized design of the pointer meter image enhancement and automatic reading system in low illumination environment

As meter data becomes more and more important in the power industry, detection robots are led into substations for automatic collection of the pointer meters. However, the meter images captured in low illumination environments are unclear, resulting in poor recognition of the meter reading. A low-illumination image enhancement method based on virtual exposure is proposed in this paper, improving the dark and bright areas of low-illumination images, respectively. Then the image fusion was performed based on the Laplace pyramid to obtain clear meter images. In addition, the dial area was extracted using the Hough circle transform, and the pointer’s rotation center was fitted using the least squares approach. Finally, the straight line of the pointer was extracted, and the data reading was based on the line segment detector algorithm. Case studies show the above method has good robustness in low illumination environment, with high rate, and accuracy during the image enhancement and automatic reading of the pointer meter.

Low-light image enhancement based on virtual exposure

Under poor illumination, the image information captured by a camera is partially lost, which seriously affects the visual perception of the human. Inspired by the idea that the fusion of multiexposure images can yield one high-quality image, an adaptive enhancement framework for a single low-light image is proposed based on the strategy of virtual exposure. In this framework, the exposure control parameters are adaptively generated through a statistical analysis of the low-light image, and a virtual exposure enhancer constructed by a quadratic function is applied to generate several image frames from a single input image. Then, on the basis of generating weight maps by three factors, i.e., contrast, saturation and saliency, the image sequences and weight images are transformed by a Laplacian pyramid and Gaussian pyramid, respectively, and multiscale fusion is implemented layer by layer. Finally, the enhanced result is obtained by pyramid reconstruction rule. Compared with the experimental results of several state-of-the-art methods on five datasets, the proposed method shows its superiority on several image quality evaluation metrics. This method requires neither image calibration nor camera response function estimation and has a more flexible application range. It can weaken the possibility of overenhancement, effectively avoid the appearance of a halo in the enhancement results, and adaptively improve the visual information fidelity.