Visible Imaging Research Articles

Correction: Application of visible light polarization imaging in normal temperature measurement

Correction: Application of visible light polarization imaging in normal temperature measurement

IRLF-SRNet: A super-resolution network based on local–global feature enhance-refine for camera-array based infrared light field images

Super resolution (SR) is essential for the camera-array based infrared light field (IRLF) images. However, the existing SR methods are mainly designed for visible LF images and are unsuitable for IRLF. This is because the sub-aperture images (SAIs) in IRLF hold blurred edges and pixel-level shifts. Artifacts and unbalanced SR performance may exist. In this paper, we propose a novel SR network, named IRLF-SRnet, to mitigate the above problems. It includes two designs: (1) local–global feature enhance-refine strategy, and (2) multi-branch cross-view correlation modeling block. The former uses global features from all SAIs to enhance the local feature from one SAI, which enhances the edges in each SAI implicitly. Then, it refines the global feature with the enhanced local feature, which further refines the edges. The latter first divides and sends the features of SAIs into multiple branches, and then models the cross-view correlation among features in each branch by bidirectional convolutional long-short term memory (BiConvLSTM), so that be aware of the pixel-level shifts. Qualitative and quantitative results show that our method outperforms state-of-the-art methods, with more balanced SR performance and fewer artifacts.

Nanopix-v1: A compact miniaturized coded-aperture gamma imager

This article explores the complex challenges associated with detecting and identifying radioactive hotspots in the nuclear industry using gamma cameras. Gamma cameras facilitate the localization of radioactive hotspots by overlaying a visualization of radioactivity onto a visible image. In the nuclear industry, gamma rays are observed in an energy range spanning from 59.5 keV (Am-241) to 1.3 MeV (Co-60).The motivation for this exploration stems from a specific challenge: characterizing hot-cells inaccessible to human operators. At the ORANO La Hague radioactive waste reprocessing plant, access to these restricted regions is possible through apertures with diameters ranging from 8 to 10 cm. In collaboration, the Nanopix-v1, characterized by its compact form as a coded mask gamma imaging system, with dimensions of 8 × 5.1 × 4.3 cm³ and a mass of 268 g, was developed through the synergistic efforts of CEA List and X-ray Imaging Europe (XIE) in partnership with ORANO.This article conducts a detailed examination of the laboratory and in situ performance of the Nanopix-v1, carried out within the challenging operational environment of the La Hague radioactive waste reprocessing plant. The focal point of the article is a comprehensive exploration of the technological components supporting the Nanopix-v1, along with an explanation of the coded mask imaging localization methodologies that underpinned the rigorous evaluation process. The performance of the gamma camera is evaluated through a series of carefully conducted trials, aimed at evaluating the angular resolution and minimum measurement time, including laboratory conditions, experimentation within a metrological Cs-137 irradiator, and practical implementation at the ORANO La Hague site.

Advancing Volcanic Activity Monitoring: A Near-Real-Time Approach with Remote Sensing Data Fusion for Radiative Power Estimation

The global, near-real-time monitoring of volcano thermal activity has become feasible through thermal infrared sensors on various satellite platforms, which enable accurate estimations of volcanic emissions. Specifically, these sensors facilitate reliable estimation of Volcanic Radiative Power (VRP), representing the heat radiated during volcanic activity. A critical factor influencing VRP estimates is the identification of hotspots in satellite imagery, typically based on intensity. Different satellite sensors employ unique algorithms due to their distinct characteristics. Integrating data from multiple satellite sources, each with different spatial and spectral resolutions, offers a more comprehensive analysis than using individual data sources alone. We introduce an innovative Remote Sensing Data Fusion (RSDF) algorithm, developed within a Cloud Computing environment that provides scalable, on-demand computing resources and services via the internet, to monitor VRP locally using data from various multispectral satellite sensors: the polar-orbiting Moderate Resolution Imaging Spectroradiometer (MODIS), the Sea and Land Surface Temperature Radiometer (SLSTR), and the Visible Infrared Imaging Radiometer Suite (VIIRS), along with the geostationary Spinning Enhanced Visible and InfraRed Imager (SEVIRI). We describe and demonstrate the operation of this algorithm through the analysis of recent eruptive activities at the Etna and Stromboli volcanoes. The RSDF algorithm, leveraging both spatial and intensity features, demonstrates heightened sensitivity in detecting high-temperature volcanic features, thereby improving VRP monitoring compared to conventional pre-processed products available online. The overall accuracy increased significantly, with the omission rate dropping from 75.5% to 3.7% and the false detection rate decreasing from 11.0% to 4.3%. The proposed multi-sensor approach markedly enhances the ability to monitor and analyze volcanic activity.

Freeform-surface-based optical design of a broadband compressive spectral imager with co-aperture coding

Compressive spectral imaging (CSI) is an advanced computational imaging approach to reconstruct the three-dimensional (3D) spatio-spectral data cube of a target scene through a single or a few snapshots. However, limited by the response range of the image detector, the existing CSI systems mostly work within narrow spectral bands, such as the visible or shortwave-infrared (SWIR) spectral band. The work band of the CSI system constrains the detection capacity for the targets under complex environments (such as rain, snow, haze, etc.). In addition, most of the current CSI prototypes lack engineering design for practical applications. This paper develops a novel, to the best of our knowledge, optical design scheme of a broadband CSI system with co-aperture coding to simultaneously realize visible multi-spectral imaging (10 channels) and SWIR super-resolution imaging (16×). The freeform surfaces are used to design the front-end reflective objective lens, thus significantly improving the image quality and spatial modulation precision of the system. By means of performance evaluation and tolerance analysis, excellent image quality and manufacturability of the proposed system are demonstrated.

An infrared and visible image fusion using knowledge measures for intuitionistic fuzzy sets and Swin Transformer

The objectives of infrared and visible image fusion are to generate a single image that includes significant objects and rich texture information. However, the current deep-learning methods ignore uncertainty in the decision-making process during the fusion phase. To address this issue, we propose a novel infrared and visible image fusion method using a Swin Transformer and knowledge measures of intuitionistic fuzzy sets (IFSs) named SWKIF-Fusion. This model employs a Swin Transformer-based pre-trained module for feature extraction, which is the most effective module for modeling long-range dependencies. The fusion process of SWKIF-Fusion integrates the proposed knowledge measure of IFSs. IFSs inherently possess a high capability to handle uncertainty, and the knowledge measure of IFSs provides uncertainty quantification. This integration in the fusion phase mitigates the uncertainty in the decision-making process. This fusion of IFSs, knowledge measures, and the Swin Transformer-based deep learning model enhances the overall performance, as demonstrated through experiments on the TNO, Roadscene, OTCBVS, M3FD, and MSRS datasets. This study also fills the gap in developing knowledge measures for IFSs by proposing novel general construction methods. It presents a theoretically sound framework for knowledge measures of IFSs using well-established mathematical concepts such as t-norms, t-conorms, automorphisms, and aggregation operators.

Reinforcing Visual Content Integrity through Image Restoration and AI Recognition

A number of factors can cause photographs to deteriorate over time, losing their clarity and aesthetic appeal. The goal of this research is to help restore priceless memories because images act as memory triggers and better image quality can increase clarity and visual comfort. This research is important because it may aid in the recovery and preservation of special memories that were captured in damaged photos. Restoring an image's visual quality has the power to arouse feelings and create closer ties to the past.AI has made great progress in several areas in the recent past. However, questions concerning the veracity and authenticity of visual content have been raised due to the possibility of false information and the spread of deepfakes. We set out to create an automated system that could accurately distinguish between photographs taken by hand and images created by artificial intelligence as a solution to this problem. Furthermore, there are some useful advantages to improving image quality, including better vitality, higher clarity, and less eye strain when viewing the restored photos. All things considered, this study shows how deep learning using GANs works well for image restoration, emphasising its usefulness in retrieving priceless memories and improving visual content for wider uses. The findings of this study demonstrate the potential benefits that AI-driven image restoration methods could have on our daily interactions with visual information in the digital age as well as our personal life.

Explainable multi-layer COSFIRE filters robust to corruptions and boundary attack with application to retina and palmprint biometrics

We propose a novel and versatile computational approach, based on hierarchical COSFIRE filters, that addresses the challenge of explainable retina and palmprint recognition for automatic person identification. Unlike traditional systems that treat these biometrics separately, our method offers a unified solution, leveraging COSFIRE filters’ trainable nature for enhanced selectivity and robustness, while exhibiting explainability and resilience to decision-based black-box adversarial attack and partial matching. COSFIRE filters are trainable, in that their selectivity can be determined with a one-shot learning step. In practice, we configure a COSFIRE filter that is selective for the mutual spatial arrangement of a set of automatically selected keypoints of each retina or palmprint reference image. A query image is then processed by all COSFIRE filters and it is classified with the reference image that was used to configure the COSFIRE filter that gives the strongest similarity score. Our approach, tested on the VARIA and RIDB retina datasets and the IITD palmprint dataset, achieved state-of-the-art results, including perfect classification for retina datasets and a 97.54% accuracy for the palmprint dataset. It proved robust in partial matching tests, achieving over 94% accuracy with 80% image visibility and over 97% with 90% visibility, demonstrating effectiveness with incomplete biometric data. Furthermore, while effectively resisting a decision-based black-box adversarial attack and impervious to imperceptible adversarial images, it is only susceptible to highly perceptible adversarial images with severe noise, which pose minimal concern as they can be easily detected through histogram analysis in preprocessing. In principle, the proposed learning-free hierarchical COSFIRE filters are applicable to any application that requires the identification of certain spatial arrangements of moderately complex features, such as bifurcations and crossovers. Moreover, the selectivity of COSFIRE filters is highly intuitive; and therefore, they provide an explainable solution.

Infrared and visible image fusion method based on target enhancement and rat swarm optimization

In order to solve the target ambiguity and information loss in the fusion results of traditional infrared and visible images, a fusion method of infrared and visible images based on the target enhancement and mouse swarm optimization, which is abbreviated as TERSFuse. Firstly, in order to reduce the loss of the original image details in the fusion results, the infrared contrast enhancement module and the visible image enhancement module based on the brightness perception are constructed respectively. Secondly, the infrared and visible enhanced images were decomposed by using the Laplace pyramid transform to obtain the corresponding high and low frequency images. In order to make the fusion result fully retain the original information, the "maximum absolute value" rule is used to fuse the infrared and visible high frequency images, and the low frequency images are fused by calculating the weight coefficient. Finally, the image reconstruction module based on the rat swarm optimization is designed to achieve the adaptive allocation of weight parameters of high frequency and low frequency image reconstruction, and then improve the visual effect of the fused image. In order to verify the advantages of the present algorithm, the experimental results show that the present algorithm not only obtains the good visual effects, but also can retains the rich edge texture and contrast information of the original image.

Efficient multi-level cross-modal fusion and detection network for infrared and visible image

With the rapid development of uncrewed aerial vehicle (UAV) technology, detecting aerial images has found significant applications across various domains. However, existing algorithms overlook the impact of illumination on target detection, resulting in less satisfactory detection performance under low-light conditions. We propose EfficientFuseDet, a visible and infrared image fusion detection network to overcome this issue. First, an effective multilevel cross-modal fusion network called EfficientFuse is presented to combine complementary information from both modalities better. EfficientFuse captures local dependencies and global contextual information in shallow and deep layers, seamlessly combining complimentary local and global features throughout the network. The generated fused images can exhibit clear target contours and abundant texture information. Second, we propose a detection network called AFI-YOLO, which employs an inverted residual vision transformer backbone (IRViT) to effectively address the challenges associated with background interference in fused images. We design an efficient feature pyramid network (EFPN) that efficiently integrates multiscale information, enhancing multiscale detection capability using aerial images. A reparameterization decoupling head (RepHead) is proposed to further improve target classification and localization precision. Finally, experiments on the DroneVehicle dataset indicate that the detection accuracy using fused images can reach 47.2 %, which is higher than that observed with visible light images of 45 %. Compared to state-of-the-art detection algorithms, EfficientFuseDet exhibits a slight decrease in speed. However, it demonstrates superior detection capabilities and effectively enhances the detection accuracy using aerial images under low-light conditions.

Multi-spectral image fusion for moving object detection

This study delves into the problem of moving object detection in infrared and visible images. While existing approaches primarily focus on single-task detection using single-spectral image data, such as thermal infrared or visible images, they often ignore the differences between different spectral images and the interconnectedness of related tasks such as image fusion and segmentation. To tackle these problems, we present a novel multi-spectral image fusion network with quality and semantic awareness for moving object detection (MOD), particularly in scenarios where ground truth labels for both infrared and visible images are unavailable. Our approach fuses multi-spectral images, incorporating additional subtasks in the image fusion to obtain content and quality perception of infrared and visible images. In addition, we design a novel residual global perception module (RGPM) and multi-spectral fusion loss, which can capture more hidden features and contextual information across various scales. This enhanced capability leads to more precise detection and tracking of moving objects, particularly in challenging situations involving occlusions and dynamic backgrounds. Compared with single-spectral moving object detection optimization, the hurdles of utilizing deep learning for multi-spectral image fusion, e.g., without ground truth labels and harmful noise, are significantly mitigated. Extensive quantitative and qualitative comparative experiments demonstrate its effectiveness, robustness, and superior performance compared to contemporary methods. Concisely, the proposed fusion representation learning has gained 44.2%,31.2%,36.3%,41.6%,5.3%,31.2%,76.2% on EI, SF, DF, AG, MI, SD and Nabf metrics compared with the best competitors.

Dual-Mode Imaging System for Early Detection and Monitoring of Ocular Surface Diseases.

The global prevalence of ocular surface diseases (OSDs), such as dry eyes, conjunctivitis, and subconjunctival hemorrhage (SCH), is steadily increasing due to factors such as aging populations, environmental influences, and lifestyle changes. These diseases affect millions of individuals worldwide, emphasizing the importance of early diagnosis and continuous monitoring for effective treatment. Therefore, we present a deep learning-enhanced imaging system for the automated, objective, and reliable assessment of these three representative OSDs. Our comprehensive pipeline incorporates processing techniques derived from dual-mode infrared (IR) and visible (RGB) images. It employs a multi-stage deep learning model to enable accurate and consistent measurement of OSDs. This proposed method has achieved a 98.7% accuracy with an F1 score of 0.980 in class classification and a 96.2% accuracy with an F1 score of 0.956 in SCH region identification. Furthermore, our system aims to facilitate early diagnosis of meibomian gland dysfunction (MGD), a primary factor causing dry eyes, by quantitatively analyzing the meibomian gland (MG) area ratio and detecting gland morphological irregularities with an accuracy of 88.1% and an F1 score of 0.781. To enhance convenience and timely OSD management, we are integrating a portable IR camera for obtaining meibography during home inspections. Our system demonstrates notable improvements in expanding dual-mode image-based diagnosis for broader applicability, effectively enhancing patient care efficiency. With its automation, accuracy, and compact design, this system is well-suited for early detection and ongoing assessment of OSDs, contributing to improved eye healthcare in an accessible and comprehensible manner.

Pain Experience and Image Quality with Curved versus Standard Compression for Breast Cancer Screening Mammography: A Randomized Controlled Trial.

Background A curve-shaped compression paddle could reduce the pain experienced by some women at breast cancer screening. Purpose To compare curved and standard compression systems in terms of pain experience and image quality in mammography screening. Materials and Methods In this randomized controlled trial conducted between October 2021 and February 2022, participants screened at three screening sites in the Netherlands were randomized to either a curved-paddle or sham-paddle group. The sham paddle was a standard paddle that was presented as a new paddle. At a standard screening examination, one additional image was acquired with a curved or sham paddle. Pain was measured on a numerical rating scale (range, 0-10). Participants provided a pain score after compression with the standard and test paddles, resulting in two scores per participant. Differences in pain scores were compared between groups using analysis of covariance, adjusting for pain score after standard-paddle compression. Two radiographers and two radiologists performed unblinded paired comparisons of curved-paddle vs standard-paddle images, using standard image quality criteria (radiographers evaluated 1246 image pairs using 12 criteria; radiologists evaluated 320 image pairs using six criteria). The one-sample Wilcoxon signed-rank test was used to determine if there was a significant preference for either paddle. Results In total, 2499 female participants (mean age, 61.6 years ± 7.1 [SD]) were studied; 1250 in the curved-paddle group and 1249 in the sham-paddle group. The mean pain score decreased by an additional 0.19 points in the curved-paddle group compared with the sham-paddle group (95% CI: 0.09, 0.28; P < .001). In terms of image quality, the observers showed no preference or a preference for the standard paddle. Decreased image contrast (range Bonferroni-corrected P values: P < .001 to P > .99) and visibility of structures were the main concerns for curved-paddle images. Conclusion The use of the curved paddle resulted in a minimal pain reduction during mammography breast compression but image quality was reduced. © RSNA, 2024 Supplemental material is available for this article.

CMFuse: Cross-Modal Features Mixing via Convolution and MLP for Infrared and Visible Image Fusion

CMFuse: Cross-Modal Features Mixing via Convolution and MLP for Infrared and Visible Image Fusion

DGFusion: An effective dynamic generalizable network for infrared and visible image fusion

The objective of infrared and visible image fusion is to generate a unified image that highlights prominent targets and retains intricate texture details, even in scenarios with imbalanced source image information. However, current image fusion algorithms primarily consider factors like illumination, restricting their applicability to certain scenes and compromising their adaptability. To tackle the issue, this paper proposes the DGFusion, which utilizes TWSSLoss to balance the contribution of source images in the fused output, effectively mitigating the limitations linked to relying solely on illumination guidance. Additionally, modality-complement feature attention harmonizer (MCFAH) facilitates cross-modal channel attention learning. This process assigns weights to features and accomplishes fusion by exchanging cross-modal differential information, thereby enriching each feature with details from the other modality. Furthermore, the multi convolution attentive net (MCAN) dynamically adjusts the contributions of features from different modalities. It prioritizes the most expressive characteristics to accentuate complementary information, enabling efficient fusion. In conclusion, our method outperforms seven state-of-the-art alternatives in terms of preserving target details and retaining texture information. Rigorous generalization experiments across five diverse datasets demonstrate the robustness of our DGFusion model in various scenarios.

Investigating laser beam shadowing and powder particle dynamics in directed energy deposition through high-fidelity modelling and high-speed imaging

Laser beam shadowing in directed energy deposition process occurs when in-flight powder particles partially block the laser beam, preventing it from fully reaching the melt pool. This blockage alters the beam's profile, introducing spatio-temporal randomness in heat transfer to the melt pool and affecting its stability. This study utilizes a novel high-fidelity multiphysics model to quantitatively analyze gas-powder stream dynamics and laser-particle interactions. It specifically focuses on dynamic variations in laser beam shadowing, attenuation, and reflection of the laser beam radiant energy, along with powder particles temperature prediction from in-flight heating, accounting for multiple reflection phenomena. Gas-particle dynamics is modeled using a fully coupled CFD-DEM approach, while laser-particle in-flight interactions are predicted using a ray-tracing-based photon discretization method. Modeling predicted that interactions between the laser beam and the in-flight powder stream could result in energy losses up to approximately 70 % due to shadowing and attenuation. The study comprehensively explored the spatio-temporal variations in the attenuated laser beam profile, revealing significant deviations from its initial Gaussian profile and substantial stochasticity and randomness over time. To confirm the modeling results and correlate computational predictions with experimental data, high-speed visible imaging of the powder stream, both with and without the melt pool, along with laser beam attenuation measurements using a photodiode system, were conducted. The model's predictions of particle velocity and laser beam attenuation characteristics aligned with the results from these experiments. Modeling and high-speed imaging demonstrated that in-flight heating caused some particles to melt and few others to reach the vaporization threshold. High-speed imaging further validated these numerical findings by examining the interaction of heated particles with the melt pool across different phases (solid, liquid, and partially vaporized). A significant observation was the increased velocities of partially vaporized particles due to recoil pressure from selective vaporization, which led to the formation of deep craters in the melt pool resulting from high-speed impact.

GRDATFusion: A gradient residual dense and attention transformer infrared and visible image fusion network for smart city security systems in cloud and fog computing

AbstractThe infrared and visible fusion technology holds a pivotal position in smart city for cloud and fog computing, particularly in security system. By fusing infrared and visible image information, this technology enhances target identification, tracking and monitoring precision, bolstering overall system security. However, existing deep learning‐based methods rely heavily on convolutional operations, which excel at extracting local features but have limited receptive fields, hampering global information capture. To overcome this difficulty, we introduce GRDATFusion, a novel end‐to‐end network comprising three key modules: transformer, gradient residual dense and attention residual. The gradient residual dense module extracts local complementary features, leveraging a dense‐shaped network to retain potentially lost information. The attention residual module focuses on crucial input image details, while the transformer module captures global information and models long‐range dependencies. Experiments on public datasets show that GRDATFusion outperforms state‐of‐the‐art algorithms in qualitative and quantitative assessments. Ablation studies validate our approach's advantages, and efficiency comparisons demonstrate its computational efficiency. Therefore, our method makes the security systems in smart city with shorter delay and satisfies the real‐time requirement.

Detection of Protective Coatings Applied on Baroque Amber Artworks: Case Studies

Amber has been used to create decorative items for centuries, but its degradation presents challenges for conservators. This study identifies substances historically used to protect amber objects, especially those from 17th and 18th century Gdansk workshops. Despite their historical value, information on amber conservation is scarce. Traditional substances are noted, but their exact compositions and effects on amber remain unclear. Synthetic resins, introduced in the late 19th century, also degrade, complicating conservation due to their removal difficulty and interference with amber identification. This research aimed to develop methods for detecting and analyzing protective coatings on amber objects using macroscopic and microscopic techniques. Initial methods included analytical photography under visible and UV light and reflectance imaging spectroscopy (RIS) to assess the surface. Raman spectroscopy (RS) and X-ray fluorescence spectroscopy (XRF) were used for detailed analysis. RS provided precise layer-specific information but was sensitive to surface conditions, while XRF quickly identified inorganic compounds but not organic materials. Examining amber objects from Polish collections using this methodology revealed various protective substances, including synthetic resins and nitrocellulose varnishes. This research contributes to amber conservation by proposing a comprehensive material analysis approach, essential for developing effective conservation strategies for these historic objects.

ITFuse: An interactive transformer for infrared and visible image fusion

Infrared and visible image fusion (IVIF) has attracted increasing attention from the community because of its pleasing results in downstream applications. However, most existing deep fusion models are either feature–level fusion or image–level fusion, leading to information loss. In this paper, we propose an interactive transformer for IVIF, termed ITFuse. In contrast to previous algorithms, ITFuse consists of feature interactive modules (FIMs) and a feature reconstruction module (FRM) to alternatively extract and integrate important features. Specifically, to adequately exploit the common properties of different source images, we design a residual attention block (RAB) for mutual feature representation. To aggregate the distinct characteristics that existed in the corresponding input images, we leverage interactive attention (ITA) to incorporate the complementary information for comprehensive feature preservation and interaction. In addition, cross-modal attention (CMA) and transformer block (TRB) are presented to fully merge the capitalized features and construct long-range relationships. Furthermore, we devise a pixel loss and a structural loss to train the proposed deep fusion model in an unsupervised manner for excess performance amelioration. Massive experiments on popular databases illustrate that our ITFuse performs better than other representative state-of-the-art methods in terms of both qualitative and quantitative assessments. The source code of the proposed method is available at https://github.com/tthinking/ITFuse.

Emulsion-Based Multi-Phase Integrated Microbeads with Inner Multi-Interface Structure Enable Dual-Modal Imaging for Precision Endovascular Embolization.

Microsphere-based embolic agents have gained prominence in transarterial embolization (TAE) treatment, a critical minimally invasive therapy widely applied for a variety of diseases such as hypervascular tumors and acute bleeding. However, the development of microspheres with long-term, real-time, and repeated X-ray imaging as well as ultrasound imaging remains challenging. In this study, emulsion-based dual-modal imaging microbeads with a unique internal multi-interface structure is developed for TAE treatment. The embolic microbeads are fabricated from a solidified oil-in-water (O/W) emulsion composed of crosslinked CaAlg-based aqueous matrix and dispersed radiopaque iodinated oil (IO) droplets through a droplet-based microfluidic fabrication method. The CaAlg-IO microbeads exhibit superior X-ray imaging visibility due to the incorporation of exceptionally high iodine level up to 221 mgImL-1, excellent ultrasound imaging capability attributed to the multi-interface structure of the O/W emulsion, great microcatheter deliverability thanks to their appropriate biomechanical properties and optimal microbead density, and extended drug release behavior owing to the biodegradation nature of the embolics. Such an embolic agent presents a promising emulsion-based platform to utilize multi-phased structures for improving endovascular embolization performance and assessment capabilities.