Visible Imaging Research Articles

Face Super-resolution Reconstruction based on Adaptive Global Residual Network

At present, some face super-resolution works do not consider the degradation factors of face images in reality, and the resolution of reconstructed face images is not high enough, and there are problems such as blur, smoothness, artifacts, and unclear details. This paper uses adaptive activation function and global attention mechanism to propose an adaptive global residual network AGRNet for face super-resolution reconstruction; then uses a formula with random parameters to simulate the process of face degradation in reality, and uses multi-scale discriminators and composite loss functions to expand it to AGRNet-HR, which can reconstruct degraded faces in reality with higher resolution. The SSIM value and PSNR value of AGRNet on the Helen test set are 0.8352 and 27.54 respectively, the LPIPS value of AGRNet-HR on the CelebAHQ test set is 0.2633, and the FID value on the real test set composed of low-resolution faces and old photos in CelebA is 26.27. Comparing the index values and image visual effects of the experimental results with mainstream methods, it shows that AGRNet and AGRNet-HR are more competitive, and the effectiveness of the key modules of the model is verified through ablation experiments.

Using deep feature distances for evaluating the perceptual quality of MR image reconstructions.

Commonly used MR image quality (IQ) metrics have poor concordance with radiologist-perceived diagnostic IQ. Here, we develop and explore deep feature distances (DFDs)-distances computed in a lower-dimensional feature space encoded by a convolutional neural network (CNN)-as improved perceptual IQ metrics for MR image reconstruction. We further explore the impact of distribution shifts between images in the DFD CNN encoder training data and the IQ metric evaluation. We compare commonly used IQ metrics (PSNR and SSIM) to two "out-of-domain" DFDs with encoders trained on natural images, an "in-domain" DFD trained on MR images alone, and two domain-adjacent DFDs trained on large medical imaging datasets. We additionally compare these with several state-of-the-art but less commonly reported IQ metrics, visual information fidelity (VIF), noise quality metric (NQM), and the high-frequency error norm (HFEN). IQ metric performance is assessed via correlations with five expert radiologist reader scores of perceived diagnostic IQ of various accelerated MR image reconstructions. We characterize the behavior of these IQ metrics under common distortions expected during image acquisition, including their sensitivity to acquisition noise. All DFDs and HFEN correlate more strongly with radiologist-perceived diagnostic IQ than SSIM, PSNR, and other state-of-the-art metrics, with correlations being comparable to radiologist inter-reader variability. Surprisingly, out-of-domain DFDs perform comparably to in-domain and domain-adjacent DFDs. A suite of IQ metrics, including DFDs and HFEN, should be used alongside commonly-reported IQ metrics for a more holistic evaluation of MR image reconstruction perceptual quality. We also observe that general vision encoders are capable of assessing visual IQ even for MR images.

The Lyman Continuum Escape Fraction of Star-forming Galaxies at 2.4 ≲ z ≲ 3.0 from UVCANDELS

Abstract The UltraViolet Imaging of the Cosmic Assembly Near-infrared Deep Extragalactic Legacy Survey Fields (UVCANDELS) survey is a Hubble Space Telescope (HST) Cycle-26 Treasury Program, allocated in total 164 orbits of primary Wide-Field Camera 3 Ultraviolet and VISible light F275W imaging with coordinated parallel Advanced Camera for Surveys F435W imaging, on four of the five premier extragalactic survey fields: GOODS-N, GOODS-S, EGS, and COSMOS. We introduce this survey by presenting a comprehensive analysis of the absolute escape fraction ( f esc abs ) of Lyman continuum radiation through stacking the UV images of a population of star-forming galaxies with secure redshifts at 2.4 ≤ z ≤ 3.0. Our stacking benefits from the catalogs of high-quality spectroscopic redshifts compiled from archival ground-based data and HST slitless spectroscopy, carefully vetted by dedicated visual inspection efforts. We develop a robust stacking method to apply to 10 samples of in total 56 galaxies, and perform detailed Monte Carlo simulations of the intergalactic medium (IGM) attenuation, to take into account the sample variance of the mean IGM transmission when measuring f esc abs . The full stack at z ≈ 2.44 from 28 galaxies places a stringent 1σ upper limit of f esc abs ≲ 5 % , whereas the full stack at z ≈ 2.72 of an equal number of galaxies gives an upper limit of f esc abs ≲ 26 % at 1σ confidence level. These new F275W and F435W imaging mosaics from UVCANDELS have been made publicly available on the Barbara A. Mikulski Archive for Space Telescopes.

Multimodal near‐infrared molecular imaging of ex vivo endometrial carcinoma via CD47‐based targeted tracer

AbstractThe detection and complete eradication of early‐stage small tumors during hysteroscopy remains a significant clinical challenge in preserving fertility for young women with endometrial cancer (EC). The purpose of this study is to verify the feasibility of CD47 as an optical molecular imaging (OMI) target for human EC and to achieve precise localization and identification in hysteroscopic surgery. The results demonstrated that CD47 was overexpressed in EC through bioinformatics, immunohistochemistry, and qRT‐PCR. In EC cell lines, CD47‐targeted near‐infrared photoimmunotherapy (NIR‐PIT) induced cytotoxicity in a light dose‐dependent manner. Laser confocal microscopy revealed that CD47 intervention significantly increased the phagocytic effect of macrophages on EC cells. In the mice model of partial tumor resection mediated by CD47‐targeted OMI, compared to group A (immune therapy alone), group C (NIR‐PIT treatment) mice showed a reduced tumor recurrence rate after NIR‐PIT intervention. However, the difference did not reach statistical significance. We then evaluated the effect of CD47‐targeted NIR‐PIT maintenance therapy on tumor recurrence in mice. The results indicated that, compared to untreated animals, the tumor growth rate was slower in the NIR‐PIT group using CD47‐Alexa Fluor 790 (CD47‐AF790), allowing for more sustained tumor control. The freshly isolated whole uterus specimens from EC patients were co‐incubated with CD47‐AF790, and a significantly enhanced contrast of NIR visible images of tumor tissue was observed, demonstrating high sensitivity and specificity (tumor‐to‐background ratio >5.05). Finally, under fluorescence microscopy, specific fluorescent signals are observed on tumor cells. In conclusion, accurate localization and excision of EC can be accomplished by employing CD47 optical molecular contrast agents with OMI technology. This method shows potential as a viable and promising approach for the precise diagnosis of EC.

Training State-of-the-Art Deep Learning Algorithms with Visible and Extended Near-Infrared Multispectral Images of Skin Lesions for the Improvement of Skin Cancer Diagnosis

An estimated 60,000 people die annually from skin cancer, predominantly melanoma. The diagnosis of skin lesions primarily relies on visual inspection, but around half of lesions pose diagnostic challenges, often necessitating a biopsy. Non-invasive detection methods like Computer-Aided Diagnosis (CAD) using Deep Learning (DL) are becoming more prominent. This study focuses on the use of multispectral (MS) imaging to improve skin lesion classification of DL models. We trained two convolutional neural networks (CNNs)—a simple CNN with six two-dimensional (2D) convolutional layers and a custom VGG-16 model with three-dimensional (3D) convolutional layers—using a dataset of MS images. The dataset included spectral cubes from 327 nevi, 112 melanomas, and 70 basal cell carcinomas (BCCs). We compared the performance of the CNNs trained with full spectral cubes versus using only three spectral bands closest to RGB wavelengths. The custom VGG-16 model achieved a classification accuracy of 71% with full spectral cubes and 45% with RGB-simulated images. The simple CNN achieved an accuracy of 83% with full spectral cubes and 36% with RGB-simulated images, demonstrating the added value of spectral information. These results confirm that MS imaging provides complementary information beyond traditional RGB images, contributing to improved classification performance. Although the dataset size remains a limitation, the findings indicate that MS imaging has significant potential for enhancing skin lesion diagnosis, paving the way for further advancements as larger datasets become available.

Feature extraction and fusion algorithm for infrared visible light images based on residual and generative adversarial network

Feature extraction and fusion algorithm for infrared visible light images based on residual and generative adversarial network

LEFuse: Joint low-light enhancement and image fusion for nighttime infrared and visible images

LEFuse: Joint low-light enhancement and image fusion for nighttime infrared and visible images

Intelligent segmentation of wildfire region and interpretation of fire front in visible light images from the viewpoint of an unmanned aerial vehicle (UAV)

Intelligent segmentation of wildfire region and interpretation of fire front in visible light images from the viewpoint of an unmanned aerial vehicle (UAV)

Piezo-Actuated Distributed Bragg Reflector–Based Tunable Fabry–Pérot Filter for Visible Light Hyperspectral Imaging

Piezo-Actuated Distributed Bragg Reflector–Based Tunable Fabry–Pérot Filter for Visible Light Hyperspectral Imaging

GANSD: A generative adversarial network based on saliency detection for infrared and visible image fusion

GANSD: A generative adversarial network based on saliency detection for infrared and visible image fusion

CycleFusion: Automatic Annotation and Graph-to-Graph Transaction Based Cycle-Consistent Adversarial Network for Infrared and Visible Image Fusion

CycleFusion: Automatic Annotation and Graph-to-Graph Transaction Based Cycle-Consistent Adversarial Network for Infrared and Visible Image Fusion

Visible Digital Image Watermarking Using Single Candidate Optimizer

With the advent of internet technologies, accessing information has become remarkably facile, while concurrently precipitating copyright conundrums. This predicament can be ameliorated by embedding copyright information within digital images, a methodology termed digital image watermarking. Artificial intelligence optimization algorithms are extensively employed in myriad problem-solving scenarios, yielding efficacious outcomes. This study proposes a visible digital image watermarking method utilizing the Single Candidate Optimizer (SCO). Contrary to many prevalent metaheuristic optimization algorithms, SCO, introduced in 2024, is not population-based. The fitness function of SCO is designed to maximize the resemblance between the watermarked image and both the host and watermark images. Experiments were conducted on images commonly utilized in image processing, and the results were evaluated using eight quality metrics. Additionally, the obtained numerical results were juxtaposed with those from well-known and widely-used genetic algorithms, differential evolution algorithms, and artificial bee colony optimization algorithms. The findings demonstrate that SCO outperforms the others in visible digital image watermarking. Furthermore, due to its non-population-based nature, SCO is significantly faster compared to its counterparts.

Online monitoring of a high-power CW laser optical mirror based on infrared and visible image features visualization

In high-power CW (continuous wave) laser systems, laser damage to optical mirrors often occurs, and it is urgent to develop timely and effective monitoring technologies to avoid laser damage or emergency stop at the early damage stage. In this paper, an online monitoring method based on the combination of infrared and visible images is proposed. The proposed method is based on the feature recognition of infrared and visible images. After setting an appropriate threshold, a decision model can be employed to enable real-time monitoring and health status evaluation of optical mirrors. Here, the thresholds for abnormal and dangerous temperatures were established at 100°C and 200°C, respectively. The results show that the temperature of the optical mirror would significantly increase after damage, with the surface temperature of the element reaching up to 250°C. At the same time, the damaged area could be captured by a visible camera and the changes in gray values were displayed in the visible image. Online monitoring of the health status of optical elements can be achieved by assessing changes in infrared image temperature, visible image spot position, and area. This monitoring method serves as an early warning method for potential optical elements damage.

MFFOD: Multidomain Feature Fusion Object Detector for Infrared Images

Abstract Infrared imagery surpasses the limitations of visible light images and finds widespread applications in fields such as military reconnaissance and security surveillance. Recent studies on infrared target detection aim to preserve local features and global representations to the greatest extent. However, compared to visible light images, infrared images exhibit inherent challenges such as insufficient texture information and coarse boundaries, which introduce new difficulties to this research. To address these issues, this paper introduces additional information cues from the perspective of enriching feature map information. Specifically, we propose a multidomain feature fusion object detector (MFFOD), whose backbone feature extraction network consists of a convolutional branch and a fast Fourier transform (FFT) branch. This hybrid domain representation enables the extraction of both domain-specific information and global high-frequency and low-frequency information with minimal computational overhead. Furthermore, in the intermediate layers of the network, we have carefully designed a feature injection module that enables comprehensive interaction between channel features and spatial features within a single feature map. Experimental results demonstrate that MFFOD achieves average detection accuracies of 88.97%, 90.32%, and 99.25% on three significant infrared scene datasets, outperforming existing target detection methods. We hope that this general detection algorithm will provide a robust reference for future infrared target detection research.

Toward visible ultrafast imaging with a synchronously pumped switching wave Kerr frequency comb

The pursuit of real-time, high-resolution imaging at visible wavelength has long been hampered by the limitations of traditional laser sources. Existing visible ultrafast lasers often suffer from limited repetition rates, or complex pulse shaping requirements, hindering their applicability for advanced imaging techniques. This work introduces a novel ultrafast imaging technology using a 775 nm near visible Kerr frequency comb. Kerr frequency combs arise from the formation of stable localized dissipative structures in coherently driven Kerr resonators. Compared to conventional visible lasers, Kerr frequency combs offer unique advantages, such as adjustable spectral bandwidth, precise frequency control, and robust coherence. Particularly, Kerr frequency combs generated using a synchronously pumped scheme can offer flexible repetition rates from the order of hundreds of MHz to tens of GHz, overcoming the limited repetition rates of the existing visible imaging lasers. In this work, we realize a Kerr frequency comb source centred at 775 nm through switching wave generation and use this source in a proof-of-concept demonstration of ultrafast biological imaging. Experimentally, we are able to achieve a 500 MHz line-scan rate and demonstrate a new approach to high speed imaging of biological samples with unprecedented speed and versatility. We envisage our work will pave the way for exciting new discoveries in various fields, propelling us closer to a future where ultrafast dynamics can be visualized in real-time and with exquisite detail.

The 5-year results of the Stratified Cancer Active Surveillance programme for men with prostate cancer.

To report 5-year outcomes from the STRATified CANcer Surveillance (STRATCANS) programme based on progression risks using National Institute for Health and Clinical Excellence (NICE) Cambridge Prognostic Group (CPG) at diagnosis, prostate specific antigen density and magnetic resonance imaging (MRI) visibility. Men with CPG1 and CPG2 disease selecting active surveillance (AS) were included into STRATCANS and allocated to one of three increasing follow-up intensities. Outcome measures were: (i) treatment for CPG≥3 progression (main outcome), (ii) any treatment, (iii) conversion to watchful waiting (WW), (iv) patient self-attrition, and (v) mortality. A total of 297 men (median age 66.0 years) were reviewed. The median (interquartile range, mean) follow-up for men still on AS was 4.9 (2.7-7.6, 5.3) years. In the cohort, 38.0% were CPG2 and 25.0% Grade Group (GG) 2 at AS entry. Overall, 214/297 (72.1%) remained treatment free: 158 (53.1%) were still on AS, 17 (5.7%) died of other causes, and 39 (13.1%) progressed to WW/discharge. Only 10 (3.4%) left AS from anxiety. There were no cancer deaths or metastatic events. In all, 80 men (26.9%) converted to treatment due to biopsy/MRI progression but only 35 (11.7%) of these reached CPG≥3 disease. Treatment for CPG≥3 occurred in 7.6% of CPG1 and 18.5% of CPG2 disease and 9.9% of GG1 and 17.5% of GG2 disease. By STRATCANS tier, treatment for CPG≥3 disease was 4.7% in STRATCANS 1, 12.9% in STRATCANS 2, and 27.4% in STRATCANS 3 (P < 0.001). STRATCANS had an area under the curve (AUC) of 0.74 for predicting CPG≥3 progression out-performing stratification by GG (AUC 0.64), CPG (0.69) and Likert score (0.51) alone or a combination of MRI visibility and GG (0.64). Longitudinal data have allowed further refinement of the STRATCANS schedule. The STRATCANS 5-year outcomes demonstrate that a simple risk stratified surveillance using a prognostically meaningful endpoint is safe, durable, has low treatment rates, high patient compliance and appropriately tailors monitoring based on risks of progression. A website and implementation toolkit are now available.

High-resolution Observations of an X-1.0 White-light Flare with Moving Flare Ribbons

Abstract We analyze high-resolution observations of an X-1.0 white-light flare, triggered by a filament eruption, on 2022 October 2. The full process of filament formation and subsequent eruption was captured in the Hα passband by the Visible Imaging Spectrograph (VIS) on board the Goode Solar Telescope (GST) within its center field of view. White-light emissions appear in flare ribbons following the filament eruption and Hα ribbon brightening. GST Broadband Filter Imager data show that the continuum intensity, as compared to the nearby quiet-Sun area, has increased by up to 20% in the photospheric TiO band around 7057 Å. The Helioseismic and Magnetic Imager on board the Solar Dynamics Observatory reported 10% contrast enhancement in the continuum near Fe i 6173 Å line. The separation motion of two white-light kernels is recorded by the high-cadence GST/TiO images and is well accompanied by the motion of the VIS Hα flare ribbon leading edge. One kernel, located in a 150 Gauss field within a granulation area, exhibited an average apparent motion speed of 55 km s−1, which is the highest average speed ever reported. The other kernel drifted at 9 km s−1 in an 800 Gauss magnetic field area. Hard X-ray (HXR) emissions reaching up to 300 keV have been observed for this flare. The simultaneous occurrence of high-cadence HXR, microwave, and white-light emissions strongly suggests that the energetic particles from the flare directly contribute to the heating. The inverted HXR energy flux density corresponding to 10% TiO brightening is 2.07 ± 0.23 × 1011 erg cm−2 s−1 during the flare peak.

Alternative illumination system for extreme ultraviolet mask inspection based on Fourier synthesis technology

This study proposes an alternative illuminator based on the Fourier synthesis technology that provides a powerful and flexible way of controlling the coherent properties of illumination for extreme ultraviolet mask inspection. The illuminator achieves coherence control by programming the incident beam scanning a Fresnel zone plate and thus can provide free pupil-fill patterns. In this work, a visible laser-based laboratory microscopic imaging platform has been developed using the illuminator. The spatial resolution and the dense-line image contrast were experimentally evaluated for various coherence factors when a disk and a dipole pupil-fill pattern were applied, respectively. The results are in good agreement with the theoretical calculation of the Rayleigh criterion and the contrast transfer function, which validates the proposed new illuminator. The reliable laser-based imaging platform sheds light on designing and improving EUV mask inspection systems based on synchrotron radiation light sources. The proposed new alternative illuminator will be used in an EUV microscopy at the Shanghai Synchrotron Radiation Facility in future work.

An Infrared and Visible Image Fusion Network Based on Res2Net and Multiscale Transformer

The aim of infrared and visible image fusion is to produce a composite image that can highlight the infrared targets and maintain plentiful detailed textures simultaneously. Despite the promising fusion performance of current deep-learning-based algorithms, most fusion algorithms highly depend on convolution operations, which limits their capability to represent long-range contextual information. To overcome this challenge, we design a novel infrared and visible image fusion network based on Res2Net and multiscale Transformer, called RMTFuse. Specifically, we devise a local feature extraction module based on Res2Net (LFE-RN) in which dense connections are adopted to reuse the information that might be lost in convolution operation and a global feature extraction module based on multiscale Transformer (GFE-MT) which is composed of a Transformer module and a global feature integration module (GFIM). The Transformer module extracts the coarse-to-fine semantic features of the source images, while GFIM is used to further aggregate the hierarchical features to strengthen contextual feature representations. Furthermore, we employ the pre-trained VGG-16 network to compute the loss of features with different depths. Massive experiments on mainstream datasets indicate that RMTFuse is superior to the state-of-the-art methods in both subjective and objective assessments.

Design of a Multifunctional Resin-Based Outdoor Spherical Robot Shell for Ultrahigh Visible to Near-Infrared Transmittance and Mid-Infrared Radiative Cooling.

As robots undertake increasingly complex tasks, such as real-time visible image sensing, environmental analysis, and weather monitoring under harsh conditions, design of an appropriate robot shell has become crucial to ensure the reliability of internal electronic components. Several key factors, such as the cooling efficiency, visible transparency, mechanical performance, and weathering resistance of the shell material, are proposed in this research to ensure future robot functionality. In this study, a polymeric double-layered shell for fabrication by stereolithography 3D printing was designed, featuring a porous outer layer and a spherical inner shell. The inner spherical shell provides approximately 90% transmission in the visible to near-infrared wavelength range (450-1050 nm) and ensures the proper functioning of the optical devices, such as cameras, lidar, and solar cells, inside the robot. In addition, the inner shell material displays high emittance in the mid-infrared range (5-20 μm) to facilitate effective radiative cooling and protect the robot control system from thermal damage. The 3D-printed inner shell is exposed to a real environment for three months, and its stable optical and mechanical performance confirms its weather resistance ability. Moreover, the 3D-printed outer robot shell promotes mechanical strength while the robot is moving. The optimal 50% porous outer shell is designed to protect the inner shell from continuous moving impact. Finite element simulations are also used to show that the 50% porosity of the outer shell significantly reduces the strain energy upon impact. Compared with a conventional single-layer design with a strain energy of 130 mJ, the double-layered shell with 50% porosity exhibits a reduced strain energy of 22.09 mJ. This double-layered design, which offers excellent weather resistance, high visible transparency, and effective radiative cooling, is promising for future applications in both land and water robot shells.