Visible Imaging Research Articles

Lightweight Infrared Image Denoising Method Based on Adversarial Transfer Learning.

A lightweight infrared image denoising method based on adversarial transfer learning is proposed. The method adopts a generative adversarial network (GAN) framework and optimizes the model through a phased transfer learning strategy. In the initial stage, the generator is pre-trained using a large-scale grayscale visible light image dataset. Subsequently, the generator is fine-tuned on an infrared image dataset using feature transfer techniques. This phased transfer strategy helps address the problem of insufficient sample quantity and variety in infrared images. Through the adversarial process of the GAN, the generator is continuously optimized to enhance its feature extraction capabilities in environments with limited data. Moreover, the generator structure incorporates structural reparameterization technology, edge convolution modules, and progressive multi-scale attention block (PMAB), significantly improving the model's ability to recognize edge and texture features. During the inference stage, structural reparameterization further optimizes the network architecture, significantly reducing model parameters and complexity and thereby improving denoising efficiency. The experimental results of public and real-world datasets demonstrate that this method effectively removes additive white Gaussian noise from infrared images, showing outstanding denoising performance.

CNN-Based Integrated Framework for Enhanced Diabetic Retinopathy Detection

Background: Diabetic Retinopathy (DR), a significant cause of vision loss globally, is characterized by retinal damage caused by diabetes. Early detection is vital to prevent irreversible blindness, yet challenges remain in accurately identifying DR stages and enhancing blood vessel visibility in fundus images. This paper aims to develop an early detection methodology for DR, addressing the need for early diagnosis and the difficulties in distinguishing DR severity through fundus imaging. The challenges in the early detection of Diabetic Retinopathy (DR) and enhancing blood vessel visibility in fundus images are multifaceted, including issues such as data imbalance, image noise, and complex patterns. By addressing these challenges through advanced ML techniques and image processing methodologies, the proposed methodology in the paper aims to overcome the limitations in early detection and severity assessment of DR, contributing to improved patient outcomes and vision preservation. Methods: This study utilizes Machine Learning (ML) to analyze complex patterns in fundus color images of DR, employing spatial domain filtering to reduce image noise and address data imbalances across DR severity levels through data augmentation. A Convolutional Neural Network (CNN), enhanced with a Gabor filter, is applied for stage-specific DR detection and to pinpoint infected areas. The dataset includes 1000 color fundus images, with a 70:30 split for training and testing, respectively. The adoption of a Gabor filtering technique aims to refine the model’s performance further. Results: The incorporation of a CNN with a Gabor filter has shown outstanding efficacy in detecting DR from fundus color images, achieving a training accuracy of 97.5%, validation accuracy of 96.5%, Cohen kappa score of 89.76%, and testing accuracy of 95.87%. This method effectively illustrates the disease-affected areas in the fundus images provided. Conclusion: Through a comparative analysis of image processing techniques, this research highlights the advantages of using advanced DR analysis for image preprocessing. The proposed CNN-Gabor filter approach demonstrates significant success in identifying diabetic retinopathy in fundus color photographs and accurately delineating the affected regions, contributing valuable insights to the field of medical image processing.

Euclid preparation. LIII. LensMC, weak lensing cosmic shear measurement with forward modelling and Markov Chain Monte Carlo sampling

is a weak lensing shear measurement method developed for and Stage-IV surveys. It is based on forward modelling in order to deal with convolution by a point spread function (PSF) with comparable size to many galaxies, sampling the posterior distribution of galaxy parameters via Markov chain Monte Carlo, and marginalisation over nuisance parameters for each of the 1.5 billion galaxies observed by We quantified the scientific performance through high-fidelity images based on the Flagship simulations and emulation of the VIS images, realistic clustering with a mean surface number density of $ arcmin^ for galaxies, and $ arcmin^ for stars, and a diffraction-limited chromatic PSF with a full width at half maximum of $ $ and spatial variation across the field of view. measured objects with a density of $ arcmin^ in $4500 deg^2 $. The total shear bias was broken down into measurement (our main focus here) and selection effects (which will be addressed in future work). We found measurement multiplicative and additive biases of $m_1=(-3.6 $, $m_2=(-4.3 $, $c_1=(-1.78 $, and $c_2=(0.09 $; a large detection bias with a multiplicative component of $1.2 $ and an additive component of $-3 $; and a measurement PSF leakage of $ $ and $ $. When model bias is suppressed, the obtained measurement biases are close to requirement and largely dominated by undetected faint galaxies ($-5 $). Although significant, model bias will be straightforward to calibrate given its weak sensitivity on galaxy morphology parameters. is publicly available at

Multi‐domain pseudo‐reference quality evaluation for infrared and visible image fusion

AbstractInfrared and visible image fusion involves merging the advantages of infrared and visible images to generate a composite image that encompasses thermal radiation as well as intricate texture details. Infrared and visible image fusion has garnered increasing attention, with numerous fusion methods proposed. However, how to fairly perceive the performance of fused image remains a contentious topic. This paper is dedicated to solving this problem from two perspectives (e.g., subjective and objective aspects). Firstly, an infrared and visible fusion image quality assessment dataset was constructed, including 60 pairs of infrared and visible images captured in various scenes, along with 540 fusion images with different types and degrees of distortions. Additionally, a subjective evaluation dataset of 16,200 subjective scores by 30 participants was further provided for the fused image. Secondly, to overcome the challenging assessment for infrared and visible fusion images without a real reference image, an interesting multi‐domain pseudo‐reference image quality assessment model (MPIQAM) is proposed, by comprehensively considering the thermal radiation information distortion, texture information distortion, and overall naturalness of the fused image. The proposed MPIQAM was compared with 18 mainstream objective metrics, and the experimental findings showcased a commendable level of competitiveness.

Infrared image super-resolution reconstruction based on visible light image guidance and recursive fusion

Due to the limitations of hardware equipment, infrared images generally have problems such as low resolution, blurred details and poor visual quality during acquisition. Using visible light images to guide the super-resolution reconstruction of infrared images is an effective way to improve the resolution of infrared images. However, the imaging principles of visible light images and infrared images are different, resulting in differences in detail information between the two images, so problems such as blur and ghosting may occur during reconstruction. This paper proposes an infrared image super-resolution network based on visible light image guidance and recursive fusion. In this network, a flow Fourier residual module is designed, and modules of different depths are used to extract information of different frequencies in visible light images and infrared images, so that each module focuses on the appropriate frequency information. At the same time, a hybrid attention module is used to obtain detail information in multimodal images from channel and spatial perspectives, and fuse them in a complementary way, which helps to eliminate the generation of artifacts. On this basis, a global recursive fusion branch is designed to consider the correlation between multi-layer features, adaptively fuse multi-layer features, and generate clearer high-resolution infrared images. Experimental results show that compared with the comparative methods, this method performs better in objective evaluation indicators; in terms of subjective visual comparison, the images reconstructed by this method have clearer textures and fewer artifacts, and better object distinction in complex environments.

LFIR-YOLO: Lightweight Model for Infrared Vehicle and Pedestrian Detection

The complexity of urban road scenes at night and the inadequacy of visible light imaging in such conditions pose significant challenges. To address the issues of insufficient color information, texture detail, and low spatial resolution in infrared imagery, we propose an enhanced infrared detection model called LFIR-YOLO, which is built upon the YOLOv8 architecture. The primary goal is to improve the accuracy of infrared target detection in nighttime traffic scenarios while meeting practical deployment requirements. First, to address challenges such as limited contrast and occlusion noise in infrared images, the C2f module in the high-level backbone network is augmented with a Dilation-wise Residual (DWR) module, incorporating multi-scale infrared contextual information to enhance feature extraction capabilities. Secondly, at the neck of the network, a Content-guided Attention (CGA) mechanism is applied to fuse features and re-modulate both initial and advanced features, catering to the low signal-to-noise ratio and sparse detail features characteristic of infrared images. Third, a shared convolution strategy is employed in the detection head, replacing the decoupled head strategy and utilizing shared Detail Enhancement Convolution (DEConv) and Group Norm (GN) operations to achieve lightweight yet precise improvements. Finally, loss functions, PIoU v2 and Adaptive Threshold Focal Loss (ATFL), are integrated into the model to better decouple infrared targets from the background and to enhance convergence speed. The experimental results on the FLIR and multispectral datasets show that the proposed LFIR-YOLO model achieves an improvement in detection accuracy of 4.3% and 2.6%, respectively, compared to the YOLOv8 model. Furthermore, the model demonstrates a reduction in parameters and computational complexity by 15.5% and 34%, respectively, enhancing its suitability for real-time deployment on resource-constrained edge devices.

Infrared and Visible Image Fusion via Sparse Representation and Guided Filtering in Laplacian Pyramid Domain

The fusion of infrared and visible images together can fully leverage the respective advantages of each, providing a more comprehensive and richer set of information. This is applicable in various fields such as military surveillance, night navigation, environmental monitoring, etc. In this paper, a novel infrared and visible image fusion method based on sparse representation and guided filtering in Laplacian pyramid (LP) domain is introduced. The source images are decomposed into low- and high-frequency bands by the LP, respectively. Sparse representation has achieved significant effectiveness in image fusion, and it is used to process the low-frequency band; the guided filtering has excellent edge-preserving effects and can effectively maintain the spatial continuity of the high-frequency band. Therefore, guided filtering combined with the weighted sum of eight-neighborhood-based modified Laplacian (WSEML) is used to process high-frequency bands. Finally, the inverse LP transform is used to reconstruct the fused image. We conducted simulation experiments on the publicly available TNO dataset to validate the superiority of our proposed algorithm in fusing infrared and visible images. Our algorithm preserves both the thermal radiation characteristics of the infrared image and the detailed features of the visible image.

Fusion of visible and fluorescence imaging through deep neural network for color value prediction of pelletized red peppers.

A non-destructive method for determining the color value of pelletized red peppers is crucial for pepper processing factories. This study aimed to investigate the potentiality of visible and fluorescence images for the determination of color value of pelletized red pepper. The imaging problem, caused by the cylindrical shape and irregular cross-sectional features of the pelletized red peppers, was reduced through the extraction of an approximate plane region. To integrate the information in the visible and fluorescence images, a baseline convolutional neural network (CNN) architecture was designed and low level, middle level, and high level fusion models (denoted as LL-CNN, ML-CNN, and HL-CNN, respectively) were developed upon the baseline CNN. The effects of input image size and color space were examined. According to the training result, CNN fusion models were developed using visible image in L*a*b* color space and fluorescence image in RGB color space using 56×56 input image size. Among the three types of CNN fusion models, the HL-CNN obtained the best performance, resulting in Rv 2 of 0.828 and RMSEV of 0.351. This study suggests that the fusion of visible and fluorescence image through CNN is a practical approach to save testing time and replace traditional destructive method. The low cost and compact structure of the imaging systems can maintain the commercial appeal of pepper industry.

Optimization method of road target detection based on infrared and visible light image fusion

Optimization method of road target detection based on infrared and visible light image fusion

DBSF-Net: Infrared Image Colorization Based on the Generative Adversarial Model with Dual-Branch Feature Extraction and Spatial-Frequency-Domain Discrimination

Thermal infrared cameras can image stably in complex scenes such as night, rain, snow, and dense fog. Still, humans are more sensitive to visual colors, so there is an urgent need to convert infrared images into color images in areas such as assisted driving. This paper studies a colorization method for infrared images based on a generative adversarial model. The proposed dual-branch feature extraction network ensures the stability of the content and structure of the generated visible light image; the proposed discrimination strategy combining spatial and frequency domain hybrid constraints effectively improves the problem of undersaturated coloring and the loss of texture details in the edge area of the generated visible light image. The comparative experiment of the public infrared visible light paired data set shows that the algorithm proposed in this paper has achieved the best performance in maintaining the consistency of the content structure of the generated image, restoring the image color distribution, and restoring the image texture details.

Denoiser Learning for Infrared and Visible Image Fusion.

Infrared image (IR) and visible image (VI) fusion creates fusion images that contain richer information and gain improved visual effects. Existing methods generally use the operators of manual design, such as intensity and gradient operators, to mine the image information. However, it is hard for them to achieve a complete and accurate description of information, which limits the image fusion performance. To this end, a novel information measurement method is proposed to achieve IR and VI fusion. Its core idea is to guide a generator in achieving image fusion by learning the denoisers. Specifically, by using denoisers to restore fusion images with different noise interference to source images, a mutual competition relationship is formed between denoisers, which helps the generator thoroughly explore the data specificity of the source images and guide it to achieve more accurate feature representation. In addition, a semantic adaptive measurement loss function is proposed to constrain the generator, which fuses semantic information adaptively by considering the semantic information density of different source images. The results of quantitative and qualitative experiments have shown that the proposed method can achieve a higher quality information fusion and has a faster fusion speed on three public datasets when compared with advanced methods.

Frequency division multiplexing-based light-emitting diode identification expansion design in visible light imaging positioning system

Frequency division multiplexing-based light-emitting diode identification expansion design in visible light imaging positioning system

Operando Spectroelectrochemical Imaging of Concentration Fields and Tafel Kinetics in Microfluidic Electrochemical Devices.

With the development of microtechnologies for energy conversion and storage, mass transfer and micromolar concentration variations need to be measured at the microscale. These advances need to be accompanied by novel imaging techniques with the capability of achieving high spatial resolution while detecting very small signal variations (less than 0.1%). Thus, in this study, a new microscopy technique is proposed based on a combination of electrochemical impedance spectroscopy (EIS) and visible imaging spectroscopy to measure the concentration fields at the micromolar scale in operando microfluidic fuel cells (MFCs). This technique exploits EIS modulation and Fourier analysis to reduce the noise during concentration field imaging. A mass transfer model in the periodic regime is derived to validate the measurements and to estimate the Tafel kinetics and mass diffusivities during potassium permanganate reduction from only one potential measurement. The proposed imaging method and mathematical framework presented in this study can be used to study binary electrochemical reactions without gas production.

Development of an online prediction system for soil organic matter and soil moisture content based on multi-modal fusion

Realizing accurate determination of in situ soil organic matter (SOM) content and soil moisture (SM) content in the field is of great importance for improving agricultural production efficiency. However, the feature information of a single sensor is limited, and there is still a certain gap in modeling accuracy compared to traditional laboratory methods. The variations of SM in the field also interfere with data collection, limiting the application of single sensor. In order to realize the efficient detection of in-situ SOM and SM content, this study developed an online detection system for SOM and SM content based on the fusion of characteristic wavelengths, visible images and thermal imaging image features. Firstly, based on a vehicle-mounted platform, a visible-thermal imaging camera and a characteristic wavelength integration device were integrated in a deep pine plow to realize the simultaneous acquisition of in-situ soil multi-sensor data. Then, a lightweight multimodal network was constructed to obtain thermal visible light image features and characteristic wavelength features through branch networks, achieving deep fusion of different modal data and predicting SOM and SM content. Finally, output the forecasting results of SOM and SM content, and transmit the predictive information to the cloud platform for storage. It was verified that the Multi-modal proposed in this study worked best in the laboratory environment, with a predicted R2 of 0.91 and RMSE of 2.9 g/kg for SOM, and a predicted R2 of 0.92 and RMSE of 0.77 % for SM. Compared with single image or characteristic spectral data, the fusion of visible image and characteristic wavelength effectively improved the prediction accuracy of SOM. The real-time prediction of SM was realized by fusing thermal imaging data, and the elimination of the moisture effect of visible images and characteristic wavelengths was also realized with the help of deep fusion of Multi-modal network. After field validation, the R2 of the Multi-modal system was 0.84 and the RMSE was 5.0 g/kg for SOM, and the R2 of the SM was 0.88 and the RMSE was 1.03 %. Despite the differences in soil types between the validation field and the sampling field, the system still demonstrated strong generalization and achieved high accuracy prediction of in-situ SOM and SM in the field, which effectively improves the efficiency and applicability of field. The efficiency and applicability of soil testing effectively improve the efficiency and provide technical guidance for precision management in the field.

Adaptive superpixel segmentation and pigment identification of colored relics based on visible spectral images

This work explores the extraction of the spatial distribution and chemical composition information of pigments in colored relics through visible spectral images. An adaptive superpixel segmentation method is proposed to extract the spatial distribution information of pigments. Quadtree decomposition is applied to generate nonuniform initial seed points based on the image homogeneity. These seed points are used as the initial cluster centers in an extended simple linear iterative clustering (SLIC) algorithm for visible spectral images to create superpixels of varying sizes that reflect the homogeneity. Each superpixel is subsequently treated as an individual area in the colored relics, and a pigment identification method based on the visible spectral reflectance is proposed to identify the pigments in these areas. A standard reference database is constructed using samples that simulate the painting process of ancient wall paintings in the Mogao Grottoes. Geometric features, which are characterized by the linear combination of the normalized visible spectral reflectance, its slope and its curvature, are designed to represent the chemical composition of pigments. The geometric features of the superpixels are compared with those of the pigments in the database using the Euclidean distance to determine the pigments in each area of the colored relics. This work is expected to provide scientific guidance for pigment selection in the color restoration of colored relics.

SFINet: A semantic feature interactive learning network for full-time infrared and visible image fusion

Infrared and visible image fusion aims to combine data from various source images to generate a high-quality image. Nevertheless, numerous fusion methods often prioritize visual quality above semantic information. To address this problem, we present a Semantic Feature Interactive Learning Network (SFINet) for full-time infrared and visible images. The SFINet encompasses an image fusion network and an image segmentation network through a Semantic Feature Interaction (SFI) module. The image fusion network employs Multi-scale Feature Extraction (MFE) modules to capture global and local information at multiple scales. Meanwhile, it performs an adaptive fusion of complementary information using a Dual Attention Feature Fusion (DAFF) module. The image segmentation network guides the image fusion network using the SFI module for semantic feature interaction. Comparative results prove that the proposed method is superior to state-of-the-art (SOTA) models in image fusion and semantic segmentation tasks. The code is available at https://github.com/songwenhao123/SFINet.

Rectifying inhomogeneous illumination in digital images using a latent light manifestation algorithm

ABSTRACT Rectification of illumination is vital in enhancing the visibility of images acquired in suboptimal lighting conditions. Still, it remains a challenging yet intriguing process as many existing algorithms fail to meet expectations. Hence, a latent light manifestation (LLM) algorithm is introduced, beginning with a conversion to the HSV space, processing the V channel using the Naka-Rushton and exponential equations. Next, the two output images are blended using a two-step range-expansion blending approach. After that, a statistics-based tonality adjustment method and a dynamic range extension process are applied. Finally, a transfer to the RGB space is performed to produce the output image. The LLM is tested using three datasets, compared with eight contemporary algorithms, and the quality evaluations are carried out via five performance measures. The results denote that the developed LLM consistently outperforms existing algorithms, scoring the best according to the used measures and yielding visually pleasing results.

Low-light image enhancement guided by multi-domain features for detail and texture enhancement

Low-light image enhancement holds significant value in the fields of computer vision and image processing, such as in applications like surveillance photography and medical imaging. Images captured in low-light environments typically suffer from significant noise levels, low contrast, and color distortion. Although existing low-light image enhancement techniques can improve image brightness and contrast to some extent, they often introduce noise or result in over-enhancement, leading to the loss of detail and texture. This paper introduces an innovative approach to low-light image enhancement by fusing spatial and frequency domain features while optimizing them with multiple loss functions. The core of the algorithm lies in its multi-branch feature extraction, multi-loss function constraints, and a carefully designed model structure. In particular, the model employs an encoder-decoder architecture, where the encoder extracts spatial features from the image, the Fourier feature extraction module captures frequency domain information, and the histogram feature encoder-decoder module processes global brightness distribution. These extracted features are then fused and reconstructed in the decoder to produce the enhanced image. In terms of loss functions, the algorithm combines perceptual loss, structural similarity loss, Fourier loss, and histogram loss to ensure comprehensive and natural enhancement effects. The novelty of this algorithm lies not only in its multi-branch feature extraction design but also in its unique model structure, which synergistically improves image quality across different domains, effectively preventing over-enhancement, and ultimately achieving a balanced enhancement of brightness, details, and texture. Experimental results on multiple datasets, including SIDD, LOL, MIT-Adobe-FiveK, and LOL-v2-synthetic, demonstrate that the proposed method outperforms existing techniques in terms of image detail, texture, and brightness. Specifically, it achieves a PSNR of 27.52 dB on the LOL dataset, surpassing Wavelet Diffusion by 1.19 dB. Additionally, on the LOL-v2-synthetic dataset, it achieves a PSNR of 29.56 dB, exceeding Wavelet Diffusion by 3.06 dB. These results demonstrate a significant enhancement in the visual quality of low-light images.

A Dataset of Visible Light and Thermal Infrared Images for Health Monitoring of Caged Laying Hens in Large-Scale Farming.

Considering animal welfare, the free-range laying hen farming model is increasingly gaining attention. However, in some countries, large-scale farming still relies on the cage-rearing model, making the focus on the welfare of caged laying hens equally important. To evaluate the health status of caged laying hens, a dataset comprising visible light and thermal infrared images was established for analyses, including morphological, thermographic, comb, and behavioral assessments, enabling a comprehensive evaluation of the hens' health, behavior, and population counts. To address the issue of insufficient data samples in the health detection process for individual and group hens, a dataset named BClayinghens was constructed containing 61,133 images of visible light and thermal infrared images. The BClayinghens dataset was completed using three types of devices: smartphones, visible light cameras, and infrared thermal cameras. All thermal infrared images correspond to visible light images and have achieved positional alignment through coordinate correction. Additionally, the visible light images were annotated with chicken head labels, obtaining 63,693 chicken head labels, which can be directly used for training deep learning models for chicken head object detection and combined with corresponding thermal infrared data to analyze the temperature of the chicken heads. To enable the constructed deep-learning object detection and recognition models to adapt to different breeding environments, various data enhancement methods such as rotation, shearing, color enhancement, and noise addition were used for image processing. The BClayinghens dataset is important for applying visible light images and corresponding thermal infrared images in the health detection, behavioral analysis, and counting of caged laying hens under large-scale farming.

Autoencoder-based image fusion network with enhanced channels and feature saliency

The existing deep learning based infrared and visible image fusion technologies have made significant progress, but there are still many problems need to be solved, such as information loss (targets and texture, etc.) of both infrared and visible images, noise and artifacts existing in fused image. To address these issues in fusion, an infrared and visible image fusion method based on autoencoder network is proposed in this paper. Firstly, novel enhanced channels are designed and input parallelly with source images into the network to enhance the specific features and reduce information loss in feature fusion. Then, the feature maps are obtained by the encoder. Next, a feature fusion method based on feature saliency is proposed, using a pre-trained classifier to measure the saliency of features, and the fused image is obtained by the decoder finally. Experimental results demonstrate that the targets are obvious and the textures are plentiful in the fused images generated by the proposed method. Also, the objective metrics of the proposed method are higher than the state of the art methods, which demonstrate that the proposed method is effective to fuse the infrared and visible images.