Intermediate Image Research Articles

DCPNet: Deformable Control Point Network for image enhancement

In this paper, we present a novel image enhancement network consisting of global and local color enhancement. The proposed network model is constructed using global transformation functions, which are formed by a set of piece-wise quadratic curves and a local color enhancement network based on the encoder–decoder network. To adaptively and dynamically control the ranges of each piece-wise curve, we introduce deformable control points (DCPs), which determine the overall structure of the global transformation functions. The parameters for piece-wise quadratic curve fitting and DCPs are estimated using the proposed DCP network (DCPNet). DCPNet processes a down-sampled image to derive the DCP parameters: The DCP offsets and the curve parameters. Then, we obtain a set of DCPs from the DCP offsets and connect each adjacent DCP pair by using the curve parameter to construct a global transformation function for each color channel. The original input images are then transformed based on the resulting transformation functions to obtain globally enhanced images. Finally, the intermediate image is fed into the local enhancement network, which has a U-Net architecture, to produce the spatially enhanced images. Extensive experimental results demonstrate the superiority of the proposed method over state-of-the-art methods in various image enhancement tasks, such as image retouching, tone-mapping, and underwater image enhancement.

Density Functional Investigations on 2D‐Be2C as an Anode for Alkali Metal‐Ion Batteries

ABSTRACTMetal‐ion batteries are in huge demand to cope with the increasing need for renewable energy, especially in automobiles. In this work, we apply first‐principle calculations to examine two‐dimensional beryllium carbide (2D‐Be2C) as a possible anode material for metal‐ion (Na and K) batteries. 2D‐Be2C is a semiconductor and becomes metallic by adsorbing metal ions. Negative adsorption energy indicates stable adsorption on the monolayer of Be2C. Alkali metal diffusion barrier and optimum path for minimum energy are studied within the framework of the climbing image nudged elastic band method. Here, six intermediate images are considered between the initial and final states. The lowest diffusion barriers for a single adsorbed Na and K atom are 0.016 and 0.026 eV, respectively. A maximum open circuit voltage of around 1 V is computed for K ions, whereas 0.5 V is for Na ions. Also, the maximum storage capacity of the Be2C monolayer is estimated at 1785 Ah/kg.

The InterVision Framework: An Enhanced Fine-Tuning Deep Learning Strategy for Auto-Segmentation in Head and Neck.

Adaptive radiotherapy (ART) workflows are increasingly adopted to achieve dose escalation and tissue sparing under dynamic anatomical conditions. However, recontouring and time constraints hinder the implementation of real-time ART workflows. Various auto-segmentation methods, including deformable image registration, atlas-based segmentation, and deep learning-based segmentation (DLS), have been developed to address these challenges. Despite the potential of DLS methods, clinical implementation remains difficult due to the need for large, high-quality datasets to ensure model generalizability. This study introduces an InterVision framework for segmentation. The InterVision framework can interpolate or create intermediate visuals between existing images to generate specific patient characteristics. The InterVision model is trained in two steps: (1) generating a general model using the dataset, and (2) tuning the general model using the dataset generated from the InterVision framework. The InterVision framework generates intermediate images between existing patient image slides using deformable vectors, effectively capturing unique patient characteristics. By creating a more comprehensive dataset that reflects these individual characteristics, the InterVision model demonstrates the ability to produce more accurate contours compared to general models. Models are evaluated using the volumetric dice similarity coefficient (VDSC) and the Hausdorff distance 95% (HD95%) for 18 structures in 20 test patients. As a result, the Dice score was 0.81 ± 0.05 for the general model, 0.82 ± 0.04 for the general fine-tuning model, and 0.85 ± 0.03 for the InterVision model. The Hausdorff distance was 3.06 ± 1.13 for the general model, 2.81 ± 0.77 for the general fine-tuning model, and 2.52 ± 0.50 for the InterVision model. The InterVision model showed the best performance compared to the general model. The InterVision framework presents a versatile approach adaptable to various tasks where prior information is accessible, such as in ART settings. This capability is particularly valuable for accurately predicting complex organs and targets that pose challenges for traditional deep learning algorithms.

Quaternion-Aware Low-Rank Prior for Blind Color Image Deblurring

Blind image deblurring is a critical and challenging task in the field of imaging science due to its severe ill-posedness. Appropriate prior information and regularizations are normally introduced to alleviate this problem. Inspired by the fact that the matrix representing a natural image is intrinsically low-rank or approximately low-rank, we employ the low-rank matrix approximation (LRMA) approach for tackling blind image deblurring problems with unknown kernels. When applied to color image restoration tasks, making use of the quaternion representation in the hypercomplex domain enables us to better illustrate the inner relationships among color channels and thus more accurately characterize color image structure. Following this idea, we develop a novel model for color image blind deblurring by implementing the quaternion representation to the LRMA method. This proposed model facilitates better results for blur kernel estimation through preserving the sharper color intermediate latent image, which is first implemented for addressing the blind color image deblurring problem. Extensive numerical experiments demonstrate that our proposed quaternion-aware low-rank prior model greatly improves the performance when compared with the conventional low-rank based scheme and outperforms some of the state-of-the-art methods in terms of some criteria and visual quality.

Into the depths: Unveiling ELAIS-N1 with LOFAR’s deepest sub-arcsecond wide-field images

We present the deepest wide-field 115–166 MHz image at sub-arcsecond resolution spanning an area of 2.5° × 2.5° centred at the ELAIS-N1 deep field. To achieve this, we improved the direction-independent (DI) and direction-dependent (DD) calibrations for the International LOw Frequency ARray (LOFAR) Telescope. This enhancement enabled us to efficiently process 32 h of data from four different 8-h observations using the high-band antennas (HBAs) of all 52 stations, covering baselines up to approximately 2000 km across Europe. The DI calibration was improved by using an accurate sky model and refining the series of calibration steps on the in-field calibrator, while the DD calibration was improved by adopting a more automated approach for selecting the DD calibrators and inspecting the self-calibration on these sources. For our brightest calibrators, we also added an additional round of self-calibration for the Dutch core and remote stations in order to refine the solutions for shorter baselines. To complement our highest resolution at 0.3″, we also made intermediate resolution wide-field images at 0.6″ and 1.2″. Our resulting wide-field images achieve a central noise level of 14 μJy beam−1 at 0.3″, doubling the depth and uncovering four times more objects than the Lockman Hole deep field image at comparable resolution but with only 8 h of data. Compared to LOFAR imaging without the international stations, we note that due to the increased collecting area and the absence of confusion noise, we reached a point-source sensitivity comparable to a 500-h ELAIS-N1 6″ image with 16 times less observing time. Importantly, we have found that the computing costs for the same amount of data are almost halved (to about 139 000 CPU h per 8 h of data) compared to previous efforts, though they remain high. Our work underscores the value and feasibility of exploiting all Dutch and international LOFAR stations to make deep wide-field images at sub-arcsecond resolution.

Secure image communication based on two-layer dynamic feedback encryption and DWT information hiding.

In response to the vulnerability of image encryption techniques to chosen plaintext attacks, this paper proposes a secure image communication scheme based on two-layer dynamic feedback encryption and discrete wavelet transform (DWT) information hiding. The proposed scheme employs a plaintext correlation and intermediate ciphertext feedback mechanism, and combines chaotic systems, bit-level permutation, bilateral diffusion, and dynamic confusion to ensure the security and confidentiality of transmitted images. Firstly, a dynamically chaotic encryption sequence associated with a secure plaintext hash value is generated and utilized for the first round of bit-level permutation, bilateral diffusion, and dynamic confusion, resulting in an intermediate ciphertext image. Similarly, the characteristic values of the intermediate ciphertext image are used to generate dynamically chaotic encryption sequences associated with them. These sequences are then employed for the second round of bit-level permutation, bilateral diffusion, and dynamic confusion to gain the final ciphertext image. The ciphertext image hidden by DWT also provides efficient encryption, higher level of security and robustness to attacks. This technology offers indiscernible secret data insertion, rendering it challenging for assailants to spot or extract concealed information. By combining the proposed dynamic closed-loop feedback secure image encryption scheme based on the 2D-SLMM chaotic system with DWT-based hiding, a comprehensive and robust image encryption approach can be achieved. According to the results of theoretical research and experimental simulation, our encryption scheme has dynamic encryption effect and reliable security performance. The scheme is highly sensitive to key and plaintext, and can effectively resist various common encryption attacks and maintain good robustness. Therefore, our proposed encryption algorithm is an ideal digital image privacy protection technology, which has a wide range of practical application prospects.

Visible–infrared person re-identification via patch-mixed cross-modality learning

Visible–infrared person re-identification (VI-ReID) aims to retrieve images of the same pedestrian from different modalities, where the challenges lie in the significant modality discrepancy. To alleviate the modality gap, recent methods generate intermediate images by GANs, grayscaling, or mixup strategies. However, these methods could introduce extra data distribution, and the semantic correspondence between the two modalities is not well learned. In this paper, we propose a Patch-Mixed Cross-Modality framework (PMCM), where two images of the same person from two modalities are split into patches and stitched into a new one for model learning. A part-alignment loss is introduced to regularize representation learning, and a patch-mixed modality learning loss is proposed to align between the modalities. In this way, the model learns to recognize a person through patches of different styles, thereby the modality semantic correspondence can be inferred. In addition, with the flexible image generation strategy, the patch-mixed images freely adjust the ratio of different modality patches, which could further alleviate the modality imbalance problem. On two VI-ReID datasets, we report new state-of-the-art performance with the proposed method.

Off-axis reflective microscope objective with a centimeter scale field of view and micron resolution

Microscope objectives with wide field-of-view (FOV) and high resolution are urgently needed for the frontier research in life sciences. However, traditional transmission microscope objectives typically have a narrow FOV and severe chromatic aberration. A new off-axis reflective microscope objective with a centimeter scale FOV and micron resolution is proposed in this paper. This objective, with its simple structure, can operate over a wide wavelength range. A design method for a wide FOV optical system is presented, which can eliminate the obstruction of the rays and control the intermediate image plane. Using this method, we design a novel off-axis four-mirror microscope objective with a FOV of 10 mm × 1.5 mm and a numerical aperture of 0.33.

Joint dual-teacher distillation and unsupervised fusion for unpaired real-world image dehazing

Existing learning-based dehazing algorithms struggle to deal with real world hazy images for lack of paired clean data. Moreover, most dehazing methods require significant computation and memory. To address the above problems, we propose a joint dual-teacher knowledge distillation and unsupervised fusion framework for single image dehazing in this paper. First, considering the complex degradation factors in real-world hazy images, two synthetic-to-real dehazing networks are explored to generate two preliminary dehazing results with the heterogeneous distillation strategy. Second, to get more qualified ground truth, an unsupervised adversarial fusion network is proposed to refine the preliminary outputs of teachers with unpaired clean images. In particular, the unpaired clean images are enhanced to deal with the dim artifacts. Furthermore, to alleviate the structure distortion in the unsupervised adversarial training, we constructed an intermediate image to constrain the output of the fusion network. Finally, considering the memory storage and computation overhead, an end-to-end lightweight student network is trained to learn the mapping from the original hazy image to the output of the fusion network. Experimental results demonstrate that the proposed method achieves state-of-the-art performance on real-world hazy images in terms of no-reference image quality assessment and the parameters.

IGIE-net: Cross-modality person re-identification via intermediate modality image generation and discriminative information enhancement

Given an RGB image (or an IR image) of a pedestrian, the task of cross-modality person re-identification aims to retrieve images of a specific pedestrian from an IR (or RGB) gallery. However, the huge modality discrepancy between RGB and IR images significantly affects the performance of cross-modality person re-identification. For the purpose of reducing the impact of modality discrepancy and extracting more discriminative pedestrian features, a new cross-modality person re-identification network is proposed by us, which is based on intermediate modality image generation and discriminative information enhancement (IGIE-Net). Specifically, we design an intermediate modality image generation module (IMIGM) to effectively weaken the impact of modality discrepancy, which first extracts the modality-specific and modality-shared information contained in RGB and IR images separately, and then generates intermediate RGB and intermediate IR images by adaptively fusing original RGB and original IR images with their corresponding modality-specific and modality-shared information separately. In addition, we also design a discriminative information enhancement module (DIEM) to obtain more discriminative pedestrian representations by enhancing the discriminative information contained in the deep features of pedestrians. Extensive experiments on the publicly available SYSU-MM01 and RegDB datasets indicate that the performance of IGIE-Net reaches the current advanced level.

Blind image deblurring using both L0 and L1 regularization of Max-min prior

Image deblurring involves estimating the blur kernel from intermediate latent images to recover their corresponding sharp, deblurred versions. In recent years, sophisticated priors such as dark channel, extreme channel, local maximum gradient, and patch-wise minimal pixels have demonstrated promising effectiveness. In this study, we propose a Max-min (Mm) prior for effective blind image deblurring. This work is motivated by the observation that the 1−Mm map of blurred images is less sparse than that of the corresponding clean images. The difference between the highest and lowest intensities around dominant edges is greater than in smooth areas. We theoretically and empirically demonstrate that blurring greatly diminishes this inherent characteristic. This property enables us to establish a new energy function and propose a blur kernel estimation model using L0 and L1 regularization of the Mm prior. The proposed method employs a linear operator to compute the Mm map, combined with an efficient optimization scheme to handle various specific scenarios. Through visual and quantitative experiments, we show that the proposed algorithm outperforms state-of-the-art algorithms. In terms of deblurring quality, robustness, and computational efficiency, the developed model surpasses comparable methods. The implemented code is available at https://github.com/eqtedaei/mm-prior-deblur

Exploiting high-quality reconstruction image encryption strategy by optimized orthogonal compressive sensing

Compressive sensing is favored because it breaks through the constraints of Nyquist sampling law in signal reconstruction. However, the security defects of joint compression encryption and the problem of low quality of reconstructed image restoration need to be solved urgently. In view of this, this paper proposes a compressive sensing image encryption scheme based on optimized orthogonal measurement matrix. Utilizing a combination of DWT and OMP, along with chaos, the proposed scheme achieves high-security image encryption and superior quality in decryption reconstruction. Firstly, the orthogonal optimization method is used to improve the chaotic measurement matrix. Combined with Part Hadamard matrix, the measurement matrix with strong orthogonal characteristics is constructed by Kronecker product. Secondly, the original image is sparsely represented by DWT. Meanwhile, Arnold scrambling is used to disturb the correlation between its adjacent pixels. Following this, the image is compressed and measured in accordance with the principles of compressive sensing and obtain the intermediate image to be encrypted. Finally, the chaotic sequence generated based on 2D-LSCM is used to perform on odd-even interleaved diffusion and row-column permutation at bit-level to obtain the final ciphertext. The experimental results show that this scheme meets the cryptographic requirements of obfuscation, diffusion and avalanche effects, and also has a large key space, which is sufficient to resist brute-force cracking attacks. Based on the sparse and reconstruction algorithm of compressive sensing proposed in this paper, it has better image restoration quality than similar algorithms. Consequently, the compressive sensing image encryption scheme enhances both security and reconstruction quality, presenting promising applications in the evolving landscape of privacy protection for network big data.

A Lightweight Image Cryptosystem for Cloud-Assisted Internet of Things

Cloud computing and the increasing popularity of 5G have greatly increased the application of images on Internet of Things (IoT) devices. The storage of images on an untrusted cloud has high security and privacy risks. Several lightweight cryptosystems have been proposed in the literature as appropriate for resource-constrained IoT devices. These existing lightweight cryptosystems are, however, not only at the risk of compromising the integrity and security of the data but also, due to the use of substitution boxes (S-boxes), require more memory space for their implementation. In this paper, a secure lightweight cryptography algorithm, that eliminates the use of an S-box, has been proposed. An algorithm termed Enc, that accepts a block of size n divides the block into L n R bits of equal length and outputs the encrypted block as follows: E=L⨂R⨁R, where ⨂ and ⨁ are exclusive-or and concatenation operators, respectively, was created. A hash result, hasR=SHA256P⨁K, was obtained, where SHA256, P, and K are the Secure Hash Algorithm (SHA−256), the encryption key, and plain image, respectively. A seed, S, generated from enchash=Enchashenc,K, where hashenc is the first n bits of hasR, was used to generate a random image, Rim. An intermediate image, intimage=Rim⨂P, and cipher image, C=Encintimage,K, were obtained. The proposed scheme was evaluated for encryption quality, decryption quality, system sensitivity, and statistical analyses using various security metrics. The results of the evaluation showed that the proposed scheme has excellent encryption and decryption qualities that are very sensitive to changes in both key and plain images, and resistance to various statistical attacks alongside other security attacks. Based on the result of the security evaluation of the proposed cryptosystem termed Hash XOR Permutation (HXP), the study concluded that the security of the cryptography algorithm can still be maintained without the use of a substitution box.

Wide field-of-view light-field head-mounted display for virtual reality applications

Light-field head-mounted displays (HMDs) can resolve vergence-accommodation conflicts but suffer from limited display pixels, causing a narrow field-of-view (FOV). This study proposes a wide-FOV light-field HMD with a 5.5-inch-diagonal 4 K display for virtual reality applications. By adjusting the pitch of elemental images to control the eye relief and creating a virtual intermediate image, horizontal and vertical FOVs of 68.8° and 43.1°, respectively, can be achieved using a monocular optical bench prototype.

Deformable registration of preoperative MR and intraoperative long-length tomosynthesis images for guidance of spine surgery via image synthesis

PurposeImproved integration and use of preoperative imaging during surgery hold significant potential for enhancing treatment planning and instrument guidance through surgical navigation. Despite its prevalent use in diagnostic settings, MR imaging is rarely used for navigation in spine surgery. This study aims to leverage MR imaging for intraoperative visualization of spine anatomy, particularly in cases where CT imaging is unavailable or when minimizing radiation exposure is essential, such as in pediatric surgery. MethodsThis work presents a method for deformable 3D-2D registration of preoperative MR images with a novel intraoperative long-length tomosynthesis imaging modality (viz., Long-Film [LF]). A conditional generative adversarial network is used to translate MR images to an intermediate bone image suitable for registration, followed by a model-based 3D-2D registration algorithm to deformably map the synthesized images to LF images. The algorithm’s performance was evaluated on cadaveric specimens with implanted markers and controlled deformation, and in clinical images of patients undergoing spine surgery as part of a large-scale clinical study on LF imaging. ResultsThe proposed method yielded a median 2D projection distance error of 2.0 mm (interquartile range [IQR]: 1.1–3.3 mm) and a 3D target registration error of 1.5 mm (IQR: 0.8–2.1 mm) in cadaver studies. Notably, the multi-scale approach exhibited significantly higher accuracy compared to rigid solutions and effectively managed the challenges posed by piecewise rigid spine deformation. The robustness and consistency of the method were evaluated on clinical images, yielding no outliers on vertebrae without surgical instrumentation and 3% outliers on vertebrae with instrumentation. ConclusionsThis work constitutes the first reported approach for deformable MR to LF registration based on deep image synthesis. The proposed framework provides access to the preoperative annotations and planning information during surgery and enables surgical navigation within the context of MR images and/or dual-plane LF images.

Design of a Spaceborne, Compact, Off-Axis, Multi-Mirror Optical System Based on Freeform Surfaces

Based on the application requirements of high spectral resolutions, high spatial resolutions and wide swatches, a new-generation, high-performance, spaceborne, hyperspectral imaging spectrometer (NGHSI) with a spatial resolution of 15 m and a swatch of 90 km is proposed. The optical system of the NGHSI has a focal length of 1128 mm, an F-number of three, a field of view (FOV) of 7.32° and a slit length of 144 mm. A new off-axis, multi-mirror telescope structure with intermediate images is put forward, which solves the design problem that realizes secondary imaging and good telecentricity at the same time. And a new off-axis lens-compensation Offner configuration is adopted to address the challenge of the high-fidelity design of spectral imaging systems with long slit lengths. The relationship between X-Y polynomials and aberration coefficients is analyzed, and the X-Y polynomial freeform surfaces are used to correct the off-axis aberrations. The design results show that the image quality of the telescope system is close to the diffraction limit. The smile, known as the spectral distortion along the line, and keystone, which is the magnification difference for different wavelengths, of the spectral imaging system are less than 1/10 pixel size. The complete optical system of the NGHSI, including the telescope system and the spectral imaging system, has excellent imaging quality and the layout is compact and reasonable, which realizes the miniaturization design.

Trajectory correction enables free-running chemical shift encoded imaging for accurate cardiac proton-density fat fraction quantification at 3T

BackgroundMetabolic diseases can negatively alter epicardial fat accumulation and composition, which can be probed using quantitative cardiac chemical shift encoded(CSE) MRI by mapping proton-density fat fraction (PDFF). To obtain motion-resolved high-resolution PDFF maps, we proposed a free-running cardiac CSE-MRI framework at 3T. To employ faster bipolar readout gradients, a correction for gradients imperfections was added using the gradient impulse response function (GIRF) and evaluated on intermediate images and PDFF quantification. MethodsTen minutes free-running cardiac 3D radial CSE-MRI acquisitions were compared in vitro and in vivo at 3T. Monopolar and bipolar readout gradients schemes provided 8 echoes (TE1/ΔTE = 1.16/1.96ms) and 13 echoes (TE1/ΔTE = 1.12/1.07ms), respectively. Bipolar-gradients free-running cardiac fat and water images and PDFF maps were reconstructed with or without GIRF-correction. PDFF values were evaluated in silico, in vitro on a fat/water phantom, and in vivo in 10 healthy volunteers and three diabetic patients. ResultsIn monopolar mode, fat-water swaps were demonstrated in silico and confirmed in vitro. Using bipolar readout gradients, PDFF quantification was reliable and accurate with GIRF correction with a mean bias of 0.03% in silico and 0.36% in vitro while it suffered from artifacts without correction, leading to a PDFF bias of 4.9% in vitro and swaps in vivo. Using bipolar readout gradients, in vivo PDFF of epicardial adipose tissue was significantly lower than in subcutaneous fat (80.4±7.1% vs 92.5±4.3%, P<0.0001). ConclusionAiming for an accurate PDFF quantification, high-resolution free-running cardiac CSE-MRI imaging proved to benefit from bipolar echoes with k-space trajectory correction at 3T. This free-breathing acquisition framework enables to investigate epicardial adipose tissue PDFF in metabolic diseases.

Single-Image SVBRDF Estimation Using Auxiliary Renderings as Intermediate Targets.

Recently, single-image SVBRDF capture is formulated as a regression problem, which uses a network to infer four SVBRDF maps from a flash-lit image. However, the accuracy is still not satisfactory since previous approaches usually adopt endto-end inference strategies. To mitigate the challenge, we propose "auxiliary renderings" as the intermediate regression targets, through which we divide the original end-to-end regression task into several easier sub-tasks, thus achieving better inference accuracy. Our contributions are threefold. First, we design three (or two pairs of) auxiliary renderings and summarize the motivations behind the designs. By our design, the auxiliary images are bumpiness-flattened or highlight-removed, containing disentangled visual cues about the final SVBRDF maps and can be easily transformed to the final maps. Second, to help estimate the auxiliary targets from the input image, we propose two mask images including a bumpiness mask and a highlight mask. Our method thus first infers mask images, then with the help of the mask images infers auxiliary renderings, and finally transforms the auxiliary images to SVBRDF maps. Third, we propose backbone UNets to infer mask images, and gated deformable UNets for estimating auxiliary targets. Thanks to the well designed networks and intermediate images, our method outputs better SVBRDF maps than previous approaches, validated by the extensive comparisonal and ablation experiments.

Locality Preservation for Unsupervised Multimodal Change Detection in Remote Sensing Imagery.

Multimodal change detection (MCD) is a topic of increasing interest in remote sensing. Due to different imaging mechanisms, the multimodal images cannot be directly compared to detect the changes. In this article, we explore the topological structure of multimodal images and construct the links between class relationships (same/different) and change labels (changed/unchanged) of pairwise superpixels, which are imaging modality-invariant. With these links, we formulate the MCD problem within a mathematical framework termed the locality-preserving energy model (LPEM), which is used to maintain the local consistency constraints embedded in the links: the structure consistency based on feature similarity and the label consistency based on spatial continuity. Because the foundation of LPEM, i.e., the links, is intuitively explainable and universal, the proposed method is very robust across different MCD situations. Noteworthy, LPEM is built directly on the label of each superpixel, so it is a paradigm that outputs the change map (CM) directly without the need to generate intermediate difference image (DI) as most previous algorithms have done. Experiments on different real datasets demonstrate the effectiveness of the proposed method. Source code of the proposed method is made available at https://github.com/yulisun/LPEM.

Alignment method of intermediate image plane imperfect imaging near-infrared catadioptric system

Alignment method of intermediate image plane imperfect imaging near-infrared catadioptric system