Rich Detail Research Articles

Remote Sensing Image Dehazing Using Heterogeneous Atmospheric Light Prior

Remote sensing images (RSIs) captured in haze weather will suffer from serious quality degradation with color distortion and contrast reduction, which creates numerous challenges for the utilization of RSIs. To address these issues, this paper proposes a novel haze removal algorithm, named HALP, for visible RSIs based on a heterogeneous atmospheric light prior and side window filter. HALP is comprised of two key components. Firstly, given the large imaging space of RSIs, the atmospheric light is treated as a globally non-uniform distribution instead of a global constant. Therefore, a simple and effective method for non-uniform atmospheric light estimation is presented, which utilizes the brightest pixel color in each local image patch as the atmospheric light of the local region. Secondly, a side window filter-based transmission estimation algorithm is proposed, which can effectively suppress the block effect in the transmission map caused by the large window of the minimum filter used in the dark channel algorithm. Experiments on both real-world and synthetic remote sensing haze images demonstrate the effectiveness of HALP. In terms of no-reference and full-reference image quality assessments, HALP yields excellent results, outperforming existing state-of-the-art algorithms, including physics-based and neural network-based methods. The visual comparison of dehazed results also shows that HALP can restore degraded RSIs with uneven haze, producing clear images with rich details and natural colors.

Guided Colorization Using Mono-Color Image Pairs.

Compared to color images captured by conventional RGB cameras, monochrome (mono) images usually have higher signal-to-noise ratios (SNR) and richer textures due to the lack of color filter arrays in mono cameras. Therefore, using a mono-color stereo dual-camera system, we can integrate the lightness information of target monochrome images with the color information of guidance RGB images to accomplish image enhancement in a colorization manner. In this work, based on two assumptions, we introduce a novel probabilistic-concept guided colorization framework. First, adjacent contents with similar luminance are likely to have similar colors. By lightness matching, we can utilize colors of the matched pixels to estimate the target color value. Second, by matching multiple pixels from the guidance image, if more of these matched pixels have similar luminance values to the target one, we can estimate colors with more confidence. Based on the statistical distribution of multiple matching results, we retain the reliable color estimates as initial dense scribbles and then propagate them to the rest of the mono image. However, for a target pixel, the color information provided by its matching results is quite redundant. Hence, we introduce a patch sampling strategy to accelerate the colorization process. Based on the analysis of the posteriori probability distribution of the sampling results, we can use much fewer matches for color estimation and reliability assessment. To alleviate incorrect color propagation in the sparsely scribbled regions, we generate extra color seeds according to the existed scribbles to guide the propagation process. Experimental results show that, our algorithm can efficiently and effectively restore color images with higher SNR and richer details from the mono-color image pairs, and achieves good performance in solving the color bleeding problem.

Promoting Monocular Depth Estimation by Multi-Scale Residual Laplacian Pyramid Fusion

Deep learning approach has achieved great success in monocular depth estimation. However, the learned deep network may produce a depth map with fewer details and incorrect global depth layout, especially when the learned network is applied to a high-resolution image. In order to generate a high-quality depth map with better global structure and richer details, we propose a multi-scale residual Laplacian pyramid fusion net (MS-RLap-FNet), to fuse the multi-scale depth maps estimated by the existing depth estimation models, for depth refinement. Our approach relies on a proposed multi-scale residual Laplacian pyramid decomposition of the multi-scale depth maps, and the fusion network modules to gradually refine the depth maps based on the decomposition from low to high resolution. Comprehensive experiments show that our method, by refining the depth maps based on three popular monocular depth estimation models (DPT, MiDas, SGR), outperforms the existing state-of-the-art methods both in quantity and quality on three public datasets with different image resolutions. The depth map refined by our method has better global depth layout with richer fine details.

Diffused Convolutional Neural Network for Hyperspectral Image Super-Resolution

With the rapid development of deep convolutional neural networks, super-resolution (SR) in hyperspectral image (HSI) has achieved good results. Current methods generally use two dimensional (2D) convolution for feature extraction, but they cannot effectively extract spectral information. Although three dimensional (3D) convolution can better characterize feature structure of HSI, it will lead to parameter redundancy, model complexity and severe memory shortage. To address above problems, we propose a new hyperspectral image super-resolution method, named diffused convolutional neural network (DCNN). Specifically, spectral convolutions have been added into the enhanced convolutional neural (ECN) block, and a series of spectral convolutions is introduced in the residual network to learn features in the channel direction of different depths. Further, Histogram of Oriented Gradient (HOG) and Local Binary Pattern (LBP) are used to retain the shape and texture information of the image respectively, which can well represent the spatial structure of the object. In order to effectively make use of the extracted shallow and deep features, a feature fusion strategy is employed to reinforce the reconstruction efficiency. Besides, an image enhancement module has been developed to diffuse the super-resolution image into the image space. Extensive evaluations and comparisons show that our DCNN approach can not only recover the HSI data with richer details, but also achieve superiority over several state-of-the-art methods.

Global Learnable Attention for Single Image Super-Resolution.

Self-similarity is valuable to the exploration of non-local textures in single image super-resolution (SISR). Researchers usually assume that the importance of non-local textures is positively related to their similarity scores. In this paper, we surprisingly found that when repairing severely damaged query textures, some non-local textures with low-similarity which are closer to the target can provide more accurate and richer details than the high-similarity ones. In these cases, low-similarity does not mean inferior but is usually caused by different scales or orientations. Utilizing this finding, we proposed a Global Learnable Attention (GLA) to adaptively modify similarity scores of non-local textures during training instead of only using a fixed similarity scoring function such as the dot product. The proposed GLA can explore non-local textures with low-similarity but more accurate details to repair severely damaged textures. Furthermore, we propose to adopt Super-Bit Locality-Sensitive Hashing (SB-LSH) as a preprocessing method for our GLA. With the SB-LSH, the computational complexity of our GLA is reduced from quadratic to asymptotic linear with respect to the image size. In addition, the proposed GLA can be integrated into existing deep SISR models as an efficient general building block. Based on the GLA, we constructed a Deep Learnable Similarity Network (DLSN), which achieves state-of-the-art performance for SISR tasks of different degradation types (e.g., blur and noise). Our code and a pre-trained DLSN have been uploaded to GitHub† for validation.

Bidirectional Translation Between UHD-HDR and HD-SDR Videos

With the popularization of ultra high definition (UHD) high dynamic range (HDR) displays, recent works focus on <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">upgrading</i> high definition (HD) standard dynamic range (SDR) videos to UHD-HDR versions, aiming to provides richer details and higher contrasts on advanced modern displays. However, joint considering the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">upgrading</i> & <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">downgrading</i> translations between two types of videos, which is practical in real applications, is generally neglected. On the one hand, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">downgrading</i> translation is the key to showing UHD-HDR videos on HD-SDR displays. On the other hand, considering both translations enables joint optimization and results in high quality translation. To this end, we propose the bidirectional translation network (BiT-Net), which jointly considers two translations in one network for the first time. In brief, BiT-Net is elaborately designed in an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">invertible</i> fashion that can be efficiently inferred along forward and backward directions for <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">downgrading</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">upgrading</i> tasks, respectively. Based on this framework, we divide each direction into three sub-tasks, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> , decomposition, structure-guided translation, and synthesis, to effectively translate the dynamic range and the high-frequency details. Benefiting from the dedicated architecture, our BiT-Net can work on 1) downgrading UHD-HDR videos, 2) upgrading existing HD-SDR videos, and 3) synthesizing UHD-HDR versions from the downgraded HD-SDR videos. Experiments show that the proposed method achieves state-of-the-art performances on all these three tasks.

Dual-Attention-Based Feature Aggregation Network for Infrared and Visible Image Fusion

Infrared and visible image fusion aims to produce fused images which retain rich texture details and high pixel intensity in the source images. In this paper, we propose a dual-attention based feature aggregation network for infrared and visible image fusion. Specifically, we first design a multi-branch channel attention based feature aggregation block by generating multiple branches to suppress useless features from different aspects. This block is also able to adaptively aggregate the meaningful features by exploiting the interdependencies between channel features. To gather more meaningful features during the fusion process, we further design a global-local spatial attention based feature aggregation block, for progressively integrating features of source images. After that, we introduce multi-scale structural similarity as loss function to evaluate the structural differences between the fused image and source images from multiple scales. In addition, the proposed network involves strong generalization ability since our fusion model is trained on the RoadScene dataset and tested directly on the TNO and MSRS datasets. Extensive experiments on these datasets demonstrate the superiority of our network compared to current state-of-the-art methods. The source code will be released at https://github.com/tangjunyang/Dual-attention.

SMNet: Synchronous Multi-Scale Low Light Enhancement Network With Local and Global Concern

Limited by objectively poor lighting conditions and hardware devices, low-light images with low visual quality and low visibility are inevitable in the real world. Accurate local details and reasonable global information play their essential and distinct roles in low-light image enhancement: local details contribute to fine textures, while global information is critical for a proper understanding of the global brightness level. In this paper, we focus on integrating local and global aspects to achieve high-quality low-light image enhancement by proposing the synchronous multi-scale low-light enhancement network (SMNet). A synchronous multi-scale representation learning structure and a global feature recalibration module are adopted in SMNet. Different from the traditional multi-scale feature learning architecture, SMNet carries out the multi-scale representation learning in a synchronous way: we first calculate the rough contextual representations in a top-down manner and then learn multi-scale representations in a bottom-up way to generate representations with rich local details. To acquire global brightness information, a global feature recalibration module (GFRM) is applied after the synchronous multi-scale representations to perceive and exploit proper global information by global pooling and projection to recalibrate channel weights globally. The synchronous multi-scale representation and GFRM compose the basic local-and-global block. Experimental results on mainstream real-world dataset LOL and synthetic dataset MIT-Adobe FiveK show that the proposed SMNet not only leads the way on objective metrics (0.41/2.31 improvement of PSNR on two datasets) but is also superior in subjective comparisons compared with typical SoTA methods. The code had already been uploaded to <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/linshideng/SMNet</uri> .

Edge Preserved Low-Rank SAR Image Despeckling via Hierarchical Prior Knowledge Regulation

Synthetic aperture radar (SAR) image despeckling is a challenging task as speckle noise is spatially correlated and signal-dependent, and appears as a grainy texture superimposed on images. Although traditional low-rank SAR image despeckling methods have shown promising performance, they have the problem of producing over-smoothed images with blurred edges due to their low-rank characteristics. In this paper, we propose a novel edge preserved SAR despeckling method named EP-LRSID, which can keep rich edge details while reducing speckle noise. Specifically, EP-LRSID takes a fresh look at the low-rank model, i.e., we can obtain structural edge information from residuals which is viewed as noise and simply disregarded by the traditional low-rank methods. To obtain discriminative edge information from residuals, the edge subspace is obtained in a manifold framework by using a dynamic affinity graph regularization. Moreover, a new hierarchical prior knowledge regulation is designed to make different kinds of pixels processed hierarchically, especially the strong scattering points in SAR images. By introducing this prior knowledge, our low-rank model can obtain more confidential low-rank parts and edge parts, thus structural information including edges can be better preserved in this way. Extensive experiments on several real and synthetic datasets demonstrate that EP-LRSID can achieve the highest despeckling performance with edge preservation than other state-of-the-art despeckling algorithms.

DBSAGAN: Dual Branch Split Attention Generative Adversarial Network for Super-Resolution Reconstruction in Remote Sensing Images

Super-resolution reconstruction methods emerge in an endless stream, but the models proposed by many researchers are not fit for certain types of images, such as remote sensing images. This is because remote sensing images have rich texture details and geometrical structures. Therefore, directly applying previous models to remote sensing images generates unsatisfactory artifacts. In this letter, we propose a dual branch split attention generative adversarial network (DBSAGAN) for super-resolution tasks on remote sensing images. Specifically, the proposed DBSAGAN adopts a dual branch split attention group as the cascading basic unit in the generator. In addition, we remove batch normalization layers in the basic unit to improve the generative ability of the network. To reduce the gap between the reconstructed and original images from the frequency domain, we innovatively use focal frequency loss to constrain the network. Experiments demonstrate that the proposed network outperforms existing state-of-the-art methods on the Gaofen-1 remote sensing image dataset.

DuFNet: Dual Flow Network of Real-Time Semantic Segmentation for Unmanned Driving Application of Internet of Things

The application of unmanned driving in the Internet of Things is one of the concrete manifestations of the application of artificial intelligence technology. Image semantic segmentation can help the unmanned driving system by achieving road accessibility analysis. Semantic segmentation is also a challenging technology for image understanding and scene parsing. We focused on the challenging task of real-time semantic segmentation in this paper. In this paper, we proposed a novel fast architecture for real-time semantic segmentation named DuFNet. Starting from the existing work of Bilateral Segmentation Network (BiSeNet), DuFNet proposes a novel Semantic Information Flow (SIF) structure for context information and a novel Fringe Information Flow (FIF) structure for spatial information. We also proposed two kinds of SIF with cascaded and paralleled structures, respectively. The SIF encodes the input stage by stage in the ResNet18 backbone and provides context information for the feature fusion module. Features from previous stages usually contain rich low-level details but high-level semantics for later stages. The multiple convolutions embed in Parallel SIF aggregate the corresponding features among different stages and generate a powerful global context representation with less computational cost. The FIF consists of a pooling layer and an upsampling operator followed by projection convolution layer. The concise component provides more spatial details for the network. Compared with BiSeNet, our work achieved faster speed and comparable performance with 72.34% mIoU accuracy and 78 FPS on Cityscapes Dataset based on the ResNet18 backbone.

Optimal Transport-Based Patch Matching for Image Style Transfer

State-of-the-art image style transfer methods have achieved impressive results by using neural networks. However, neural style transfer (NST) methods either ignore the local details of the style image by using the global statistics for style modeling or cannot fully use shallow features of neural networks, leading to the synthesized image having fewer details. In this study, we proposed a new patch-based style transfer method that directly operates in the image pixel domain without using any neural networks, achieving fascinating style transfer results with rich image details. The proposed method was derived from classic texture synthesis methods. Most previous methods rely on nearest neighbor search (NNS) for patch matching. However, this greedy strategy cannot guarantee the similarity of patch distributions between the synthesized image and the style image, which limits the expressiveness of textures. We solved this problem by proposing an optimal patch matching algorithm formed on the Optimal Transport (OT) theory, which theoretically guarantees the similarity of the patch distributions and gives a flexible style modeling method. Various qualitative and quantitative experiments demonstrated that the proposed method achieves better synthesized results than state-of-the-art style transfer methods, including NST and classic methods based on texture synthesis.

CADUI: Cross-Attention-Based Depth Unfolding Iteration Network for Pansharpening Remote Sensing Images

Pansharpening is an important technology for remote sensing imaging systems to obtain high-resolution multispectral (HRMS) images. It mainly obtains high-resolution multi-spectral (HRMS) images with uniform spectral distribution and rich spatial details by fusing low-resolution multi-spectral (LRMS) images and high-spatial-resolution panchromatic (PAN) images. Therefore, how to extract features completely and reconstruct images with high quality is critical to obtain ideal fusion images. In this paper, we propose a new pansharpening method, called the Cross Attention-based Depth Unfolding Iteration Network for Pan-sharpening remote sensing images (CADUI), which achieves the desired fusion effect by iteratively optimizing the deep prior regularization and combining it with a cross-attention mechanism. The network consists of two parts: optimized iterations of deep prior regularization (DEIN-Block) and cross-attention mechanism (CAFM-Block). Among them, DEIN-Block introduces the depth prior as an implicit regularization and improves the adaptability and representation ability of the relevant data of the reconstructed image through iteration. CAFM-Block realizes dual-branch fusion through cross-attention fusion and channel-attention fusion to achieve better fusion results. Simulation experiments and real experiments are carried out on the standard datasets QuikBird (QB) and WorldView-2 (WV2). Through quantitative comparison and qualitative analysis, it is proved that the method is superior to the existing methods.

An Efficient Medical Image Deep Fusion Model Based on Convolutional Neural Networks

Medical image fusion is considered the best method for obtaining one image with rich details for efficient medical diagnosis and therapy. Deep learning provides a high performance for several medical image analysis applications. This paper proposes a deep learning model for the medical image fusion process. This model depends on Convolutional Neural Network (CNN). The basic idea of the proposed model is to extract features from both CT and MR images. Then, an additional process is executed on the extracted features. After that, the fused feature map is reconstructed to obtain the resulting fused image. Finally, the quality of the resulting fused image is enhanced by various enhancement techniques such as Histogram Matching (HM), Histogram Equalization (HE), fuzzy technique, fuzzy type Π, and Contrast Limited Histogram Equalization (CLAHE). The performance of the proposed fusion-based CNN model is measured by various metrics of the fusion and enhancement quality. Different realistic datasets of different modalities and diseases are tested and implemented. Also, real datasets are tested in the simulation analysis.

Prototype-Guided Autoencoder for OCT-Based Fingerprint Presentation Attack Detection

Anti-spoofing ability is vital for fingerprint identification systems. Conventional fingerprint scanning devices can only obtain information from the fingertip surfaces, and their performance is susceptible to skin conditions and presentation attacks (PAs). However, optical coherence tomography (OCT) can scan subcutaneous tissue and obtain 3D fingerprint structures, naturally enhancing its PA detection (PAD) ability from the perspective of hardware. Existing unsupervised PAD methods are based on image reconstruction. However, the reconstruction error is easily affected by OCT noise and the rich details of OCT images. Therefore we propose feature-based reconstruction to alleviate this problem, called the prototype-guided autoencoder. The model consists of a memory module and a denoising autoencoder without the requirement of PA fingerprints. As only bona fide fingerprints are available during the training phase, the memory module contains the prototype features of the bona fide fingerprints. During the inference phase, as the prototype memory module is frozen, the reconstructed representation of the bona fide input is close to the bona fide fingerprint features. Calculating the distance between the original features and the prototype reconstructed representation of the sample can achieve PAD. To obtain a better decision making boundary, we propose a representation consistency constraint, which reduces the bona fide representation reconstruction distance closer, so that it is easier to differentiate between fingerprints and PAs.

Accurate Detection of Doppler Cardiograms With a Parameterized Respiratory Filter Technique Using a K-Band Radar Sensor

Cardiogram is one of the most important factors for health assessment. As a new expression of cardiograms, the Doppler cardiograms (DCGs) detected remotely via Doppler radar sensor (DRS) provide rich details of the heart motion. However, the existing techniques for measurement of DCG requires that the subject holds his/her breath to avoid the disturbance caused by respiration. To realize the accurate detection of DCGs in the presence of respiration, a high-linearity <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$K$ </tex-math></inline-formula> -band dc-coupled continuous-wave (CW) DRS with digital dc-tuning function was custom-designed and a novel parameterized respiratory filter (PRF) algorithm was proposed to precisely remove the respiratory signal. Both simulation and experimental results show that the proposed PRF algorithm has better DCG extraction performance than the conventional methods. Furthermore, the experimental results in normal and clinical environments show that the grained differential DCG (D-DCG) has promising potential in professional heart rate variability (HRV) analysis and heart diseases diagnosis, which means the proposed DCGs detection has the potential to be a convenient, comfortable, and reliable way for daily and clinical heart health assessment.

Energy justice, Just Transitions and Scottish energy policy: A re-grounding of theory in policy practice

Now, more than ever, there is a pressing need to focus overtly on justice issues in energy affairs as part of the socio-technical reimagining of energy futures. With reference to the energy justice and Just Transitions literatures - two popularised and increasingly intertwined concepts - this paper analyses how justice is conceptualised in policy practice through a case study analysis of energy policymaking in Scotland. It does so by drawing upon the analysis of 6 expert interviews and 148 legislative and government documents published between 2010 and 2020. Through the lens of this case study, the paper makes three contributions. First, it responds to appeals in the energy justice scholarship for practical applicability. Second, the paper re-grounds these justice literatures in political and economic realities within the Scottish case study, providing recommendations for academia, Scottish policy and wider contexts. Three, it reveals rich empirical detail. Throughout the analysis, four key thematic categories emerged, each articulating the manifestation of justice concerns in Scottish energy policy: affordability and fuel poverty, forms of recognition, energy in relation to political interests and decision-making, and the Just Transition as a cross-cutting theme.

Adaptive Hamming filter-based speckle triple-correlation imaging through the diffuser

A spatial filter for speckle triple-correlation imaging (STCI) is necessary to reduce high-frequency noise in retrieving the lost Fourier phase of the target hidden behind the diffuser. Existing methods mainly utilize the Gaussian filter with a rough window length, which suffers from the manual parameter and cannot preserve rich imaging detail for target reconstructions with different structures under noise conditions. This paper first presents the signal-to-noise ratio (SNR) model for STCI in real optical systems imaging through the diffuser, based on which we propose a new adaptive Hamming filter-based STCI. The presented SNR model provides physical priori to sense changes in the valid speckle area of different target structures and enables the adaptability of the proposed STCI. Both theoretical analyses and experimental results demonstrate that the proposed STCI is adaptive to different targets and performs better than the existing method under various low-SNR conditions.

VAIR—An epistemic model of socioscientific reasoning emerging from citizens engaged in a locally situated SSI

AbstractEngagement with socioscientific issues (SSI) has emerged as an important focus for science education. SSI have been successfully used to teach science concepts, but they also serve a role in helping learners develop a capacity to engage with science in everyday life. Here we present an epistemic framework for characterizing socioscientific reasoning (SSR), called the vAIR framework standing for virtuous Aims, Ideals, and Reasoning Practices. vAIR is a modification of the AIR framework for epistemic cognition. The “virtuous” framing of vAIR is to align with the language of Sosa's virtue epistemology, reflecting a capacity to engage in reasoning that promotes a collective praxis/enterprise. vAIR was developed from a study of citizens engaged in an authentic SSI to offer a deeper authenticity than might be achieved in the study of a classroom. The goal was to better understand the ontologically distinct components of SSR, as well as gain a better understanding of what “constructive” SSR might look like in the context of collective praxis/enterprise in the everyday world. The findings are presented through a series of vignettes that capture the rich detail of the context as well as the nuances between the dimensions of the vAIR model. vAIR, we argue, can serve as a tool for scaffolding the development and assessment of SSR in classrooms.

Weak illumination image enhancement algorithm based on cyclic generation countermeasure network

To solve the problem of missing or distorted detail texture when manually adjusting image parameters, a weak illumination image enhancement algorithm based on cyclic generation game network is proposed. The image features are normalized by Gaussian distribution. Combined with homomorphic filtering theory and defogging operation, the image is generated and denoised according to the network brightness to enhance the weak illumination image. The experimental results show that after using this method to process the image, the image entropy increases by 6.8%, the contrast increases by 27.5%, and the noise content decreases by 24.1%. It has better contrast. It can not only meet the enhancement effect of weak light, but also ensure the details of the image, so that the image has richer details and good visual appearance.