Visible Imaging Research Articles

Independent Tri-Spectral Integration for Intelligent Ship Monitoring in Ports: Bridging Optical, Infrared, and Satellite Insights

Image-based ship monitoring technology has extensive applications, and is widely used in various aspects of port management, including illegal activity surveillance, vessel identification at entry and exit points, channel and berth management, unmanned vessel control, and incident warning and emergency response. However, most current ship identification technologies rely on a single information source, reducing detection accuracy in the complex and variable marine environment. To address this issue, this paper proposes a knowledge transfer-based ship identification system integrating three modules. The system enables synchronized monitoring of visible light coastal images, satellite cloud images, and infrared spectrum images, thereby mitigating problems such as low detection accuracy and poor adaptability of image recognition. Additionally, extensive supplementary experiments were conducted to evaluate the effectiveness of the preprocessing and data augmentation modules as well as the transfer learning module. The study also discusses the limitations of current deep learning-based ship monitoring models, particularly their poor adaptability to image recognition and inability to achieve all-weather, round-the-clock monitoring. Experimental results based on three ship monitoring datasets demonstrate that the proposed system, by integrating three distinct detection conditions, outperforms other models with an F1-score of 98.74%, approximately 10% higher than most existing ship detection systems.

Infrared and visible image fusion based on saliency detection and deep multi-scale orientational features

Infrared and visible image fusion based on saliency detection and deep multi-scale orientational features

Real-Time Visualization of the Gastrointestinal Tract during Nasogastric Tube Placement: Pilot study of a New Video-Assisted System

Background and aims: Nasogastric tubes are commonly inserted blindly, leading to complications. Existing NG tube placement assisting systems faced safety and cost-effectiveness issues. This study aims to evaluate a new assisting system using a camera probe inserted into the nasogastric tube to provide real-time visualization. Patients and methods: Thirty patients requiring nasogastric tube placement were prospectively included. The primary objective was to determine the success rate of nasogastric tube placement, while the secondary objectives included the assessment of the usability and safety of this system. Results: Our findings revealed a high success rate of 96.7%. The median time to complete nasogastric tube placement was 3.8 min. No serious complications were observed during the 7-day follow-up period. Operator’s feedback indicated this system helps facilitate nasogastric tube placement, identify the gastric mucosa and safety landmark, and ease of camera wire removal. However, image visibility received a slightly lower score due to gastrointestinal secretions entering the nasogastric tube through the side hole. To address this, air insufflation was used to enhance visibility in 13 patients. Conclusions: This video-assisted system provides real-time visualization of the gastrointestinal tract during tube insertion and has been shown to enhance the safety and effectiveness of nasogastric tube placement.

Conti-Fuse: A novel continuous decomposition-based fusion framework for infrared and visible images

For better explore the relations of inter-modal and inner-modal, even in deep learning fusion framework, the concept of decomposition plays a crucial role. However, the previous decomposition strategies (base & detail or low-frequency & high-frequency) are too rough to present the common features and the unique features of source modalities, which leads to a decline in the quality of the fused images. The existing strategies treat these relations as a binary system, which may not be suitable for the complex generation task (e.g. image fusion). To address this issue, a continuous decomposition-based fusion framework (Conti-Fuse) is proposed. Conti-Fuse treats the decomposition results as few samples along the feature variation trajectory of the source images, extending this concept to a more general state to achieve continuous decomposition. This novel continuous decomposition strategy enhances the representation of complementary information of inter-modal by increasing the number of decomposition samples, thus reducing the loss of critical information. To facilitate this process, the continuous decomposition module (CDM) is introduced to decompose the input into a series continuous components. The core module of CDM, State Transformer (ST), is utilized to efficiently capture the complementary information from source modalities. Furthermore, a novel decomposition loss function is also designed which ensures the smooth progression of the decomposition process while maintaining linear growth in time complexity with respect to the number of decomposition samples. Extensive experiments demonstrate that our proposed Conti-Fuse achieves superior performance compared to the state-of-the-art fusion methods.

Water vapor and soot spatial characteristics retrieve of axisymmetric optically-thin laminar diffusion flame based on visible and near-infrared multi-spectral light field imaging

The comprehension of the distribution of gaseous species and soot particles plays a pivotal role in investigating the combustion process of a flame. A highly effective method to accomplish this is by extracting visible and near-infrared emission information from flames. In this study, we present a novel near-infrared multi-spectral light field imaging model that enables the concurrent extraction of gas and soot property distributions within a flame. A synthetic test of ethylene diffusion flame is assessed using the proposed reconstruction method. The mole fraction of gaseous water, together with the flame temperature and soot volume fraction, are decoupled spectrally using near-infrared and visible wavelengths. The results demonstrate a reliably retrieved temperature range of 1400 K to 2050 K, accurately reconstructing the actual distributions of soot volume fraction and gaseous water mole fraction. Minor influences on the imaging results and property reconstruction are observed due to uncertainties arising from the reconstruction method, absorption function, reconstruction wavelength for H2O mole fraction, and signal-to-noise ratio. This study serves as a theoretical guide for the future development of practical near-infrared multi-spectral light field imaging techniques for rapid and robust flame diagnostic purposes related to soot and gas properties.

Bilateral enhancement network with signal-to-noise ratio fusion for lightweight generalizable low-light image enhancement

Low-light image enhancement aims to enhance the visibility and contrast of low-light images while eliminating complex degradation issues such as noise, artifacts, and color distortions. Most existing low-light image enhancement methods either focus on quality while neglecting computational efficiency or have limited learning and generalization capabilities. To address these issues, we propose a Bilateral Enhancement Network with signal-to-noise ratio fusion, called BiEnNet, for lightweight and generalizable low-light image enhancement. Specifically, we design a lightweight Bilateral enhancement module with SNR (Signal-to-Noise Ratio) Fusion (BSF), which serves the SNR map of the input low-light image as the interpolation weights to dynamically fuse global brightness features and local detail features extracted from a bilateral network and achieve differentiated enhancement across different regions. To improve the network’s generalization ability, we propose a Luminance Normalization (LNM) module for preprocessing and a Dual-Exposure Processing (DEP) module for post-processing. LNM divides the channels of input features into luminance-related channels and luminance independent channels, and reduces the inconsistency of the degradation distribution of input low-light images by only normalizing the luminance-related channels. DEP learns overexposure and underexposure corrections simultaneously by employing the ReLU activation function, inverting operation, and residual network, which can improve the robustness of enhancement effects under different exposure conditions while reducing network parameters. Experiments on the LOL-V1 dataset shows BiEnNet significantly increased PSNR by 8.6\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\%$$\\end{document} and SSIM by 3.6\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\%$$\\end{document} compared to FLW-Net, reduced parameters by 98.78\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\%$$\\end{document}, and improved computational speed by 52.64\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\%$$\\end{document} compared to the classical KIND.

PSNet: A non-uniform illumination correction method for underwater images based pseudo-siamese network

Autonomous underwater vehicles (AUVs) based on visual perception play an important role in maritime operations. However, underwater environment often suffers from poor lighting conditions, making the utilization of artificial light sources necessary. This reliance on artificial lighting frequently results in non-uniform illumination. Furthermore, the absorption and scattering effects of water cause further degradation, such as color distortion and blurring of details. To address these challenges, we propose a pseudo-siamese network, named PSNet, designed for underwater optical image enhancement. PSNet separates the non-uniformly illuminated layer from the optimally uniformly illuminated image and utilizes a cascading iteration strategy to enhance the image details. To achieve a better balance prediction quality, we introduce structure loss and residual reconstruction loss as additional guides for model learning. Additionally, we incorporate a color consistency loss to mitigate color distortion. To address the lack of training data, we develop a non-uniform illumination model and generate a dataset that includes both non-uniformly illuminated layers and uniformly illuminated images. Through comprehensive experimental evaluations, PSNet significantly enhances the visual quality of underwater optical images and consistently outperforms state-of-the-art approaches in multiple performance metrics.

Visually relevant on-bench through-focus analysis of intraocular lenses

Cataract surgery involves the implantation of an intraocular lens (IOL) to replace the opacified crystalline lens. Monofocal IOLs, the most common type, are intended to have the eye in focus at a given distance, usually at infinity. Simultaneous vision IOLs (SVIOLs) and extended depth of focus (EDOF) aim to minimize postoperative dependence on spectacles by providing either multiple foci or an extended depth of focus. These lenses utilize a variety of diffractive and refractive designs to achieve varied focal depths. While common optical testing methods based on the IOL’s modulation transfer function (MTF) or resolving power at best focus are essential for quality control, they do not fully address the lenses’ performance requirements in daily visual tasks such as reading in a variety of distances. The purpose of this work was to introduce a visually relevant on-bench test method, which includes an image analysis technique and a visual acuity-related image quality metric, to evaluate the through-focus performance of different commercially available IOLs. This method consists of recording a series of optotype images in a realistic eye model with the IOL, adjusting the stimulus vergence through a focus-tunable lens. We compare the results obtained with mono-focal, enhanced mono-focal, EDOF, and (diffractive) trifocal IOLs.

Short-Wave Infrared (SWIR) Imaging for Robust Material Classification: Overcoming Limitations of Visible Spectrum Data

This paper presents a novel approach to material classification using short-wave infrared (SWIR) imaging, aimed at applications where differentiating visually similar objects based on material properties is essential, such as in autonomous driving. Traditional vision systems, relying on visible spectrum imaging, struggle to distinguish between objects with similar appearances but different material compositions. Our method leverages SWIR’s distinct reflectance characteristics, particularly for materials containing moisture, and demonstrates a significant improvement in accuracy. Specifically, SWIR data achieved near-perfect classification results with an accuracy of 99% for distinguishing real from artificial objects, compared to 77% with visible spectrum data. In object detection tasks, our SWIR-based model achieved a mean average precision (mAP) of 0.98 for human detection and up to 1.00 for other objects, demonstrating its robustness in reducing false detections. This study underscores SWIR’s potential to enhance object recognition and reduce ambiguity in complex environments, offering a valuable contribution to material-based object recognition in autonomous driving, manufacturing, and beyond.

SICFuse: Swin Transformer integrated with invertible neural network and correlation coefficient assistance for infrared and visible image fusion

SICFuse: Swin Transformer integrated with invertible neural network and correlation coefficient assistance for infrared and visible image fusion

Illumination enhancement discriminator and compensation attention based low-light visible and infrared image fusion

Infrared and visible image fusion is an important image enhancement technology, aiming to generate high-quality fused images with prominent targets and rich textures in extreme environments. However, most existing image fusion methods are designed for infrared and visible images under normal lighting. At night, due to severe degradation of visible images, existing fusion methods have deficiencies in texture details and visual perception, which affects subsequent visual applications. To this end, this paper proposes a three-discriminator infrared and visible image fusion method based on GAN network. Specifically, this method adds an illumination enhancement discriminator based on the GAN-based dual discriminator fusion network. The input of this discriminator is the fused image generated by the generator and the low-light enhanced visible light image. By fighting in the third discriminator, it is ensured that the fused image output by the generator achieves the expected effect on the brightness information. In addition, this method also proposes a compensation attention module to convey the multi-scale features extracted by the feature extraction network and ensure that the fused image contains important detailed texture information. Compared with other fusion methods on public data sets such as MSRS, M3FD, Roadscence and TNO, the fusion results of this paper perform better in both quantitative measurement and qualitative effects. It also performs better in enhancing the brightness information.

BG-YOLO: A Bidirectional-Guided Method for Underwater Object Detection.

Degraded underwater images decrease the accuracy of underwater object detection. Existing research uses image enhancement methods to improve the visual quality of images, which may not be beneficial in underwater image detection and lead to serious degradation in detector performance. To alleviate this problem, we proposed a bidirectional guided method for underwater object detection, referred to as BG-YOLO. In the proposed method, a network is organized by constructing an image enhancement branch and an object detection branch in a parallel manner. The image enhancement branch consists of a cascade of an image enhancement subnet and object detection subnet. The object detection branch only consists of a detection subnet. A feature-guided module connects the shallow convolution layers of the two branches. When training the image enhancement branch, the object detection subnet in the enhancement branch guides the image enhancement subnet to be optimized towards the direction that is most conducive to the detection task. The shallow feature map of the trained image enhancement branch is output to the feature-guided module, constraining the optimization of the object detection branch through consistency loss and prompting the object detection branch to learn more detailed information about the objects. This enhances the detection performance. During the detection tasks, only the object detection branch is reserved so that no additional computational cost is introduced. Extensive experiments demonstrate that the proposed method significantly improves the detection performance of the YOLOv5s object detection network (the mAP is increased by up to 2.9%) and maintains the same inference speed as YOLOv5s (132 fps).

Calibration of visible and near-infrared spectral imaging technology to predict the quality evolution of retail fresh pork bellies with different fat content

This study investigates quality changes occurred in sliced pork belly with different fat content during refrigerated storage, and the potential of spectral imaging technology in predicting quality properties. Pork bellies with different fat levels (low ’LF’, medium ’MF’ and high ’HF’) were selected from slaughtering houses and directly transferred to the laboratory. The sliced bellies were packed in modified atmosphere packages with high oxygen levels (80 %) and the essential visual and olfactory characteristics, microbiological load, pH, lipid oxidation and colour values were assessed throughout 20 days of refrigerated storage. The spectral images of all belly samples were acquired in the wavelength range from 386 to 1015 nm. Results revealed significant quality losses throughout storage attributed primarily to lipid oxidation and colour changes. The HF bellies showed lower L* and higher a* values than LF and MF. Additionally, the LF and MF bellies, with higher proportion of polyunsaturated fatty acids, showed higher lipid oxidation compared to the HF bellies throughout storage. The appropriate combination of spectral preprocessing, together with the appropriate selection of the region of interest, facilitated the development of robust models to predict the visual appearance, odour, lipid oxidation, and a* values of belly slices during refrigerated storage. The obtained results demonstrated the potential of spectral imaging for predicting quality characteristics of sliced fresh pork bellies during refrigerated storage.

Diagnostic study of atmospheric pressure inductively coupled plasma temperature field based on image and relative spectral line method

Atmospheric pressure inductively coupled plasma (ICP) is widely applied in the production of high-purity coatings and powders. This paper innovatively presents a dual-band imaging ICP temperature diagnostic system that integrates visible light image processing technology and relative spectral line method. The system utilizes a dual-band imaging unit to simultaneously acquire monochromatic grayscale images of ICP at two different wavelengths using a single CCD camera. By establishing the correspondence between the emission intensity received by the emission spectrometer and the grayscale images recorded by the CCD camera, the two-dimensional (2D) electron excitation temperature (EET) field distribution of ICP is obtained by the relative spectral line method. The experimental results demonstrate that the maximum EET is located near to the center line of the RF-driven coil. Additionally, the EET in ICP decreases gradually from the center of the induction coil to the periphery. As the radio frequency (RF) power increases, the maximum EET also increases, and the high-temperature region expands. The accuracy of this method is validated by comparing it with the results obtained from the Boltzmann plot method. Therefore, this method can quickly obtain the transient EET field distribution of ICP, which is significant for optimizing the application of ICP in material processing.

Research and Design of an Active Light Source System for UAVs Based on Light Intensity Matching Model

The saliency feature is a key factor in achieving vision-based tracking for multi-UAV control. However, due to the complex and variable environments encountered during multi-UAV operations—such as changes in lighting conditions and scale variations—the UAV’s visual features may degrade, especially under high-speed movement, ultimately resulting in failure of the vision tracking task and reducing the stability and robustness of swarm flight. Therefore, this paper proposes an adaptive active light source system based on light intensity matching to address the issue of visual feature loss caused by environmental light intensity and scale variations in multi-UAV collaborative navigation. The system consists of three components: an environment sensing and control module, a variable active light source module, and a light source power module. This paper first designs the overall framework of the active light source system, detailing the functions of each module and their collaborative working principles. Furthermore, optimization experiments are conducted on the variable active light source module. By comparing the recognition effects of the variable active light source module under different parameters, the best configuration is selected. In addition, to improve the robustness of the active light source system under different lighting conditions, this paper also constructs a light source color matching model based on light intensity matching. By collecting and comparing visible light images of different color light sources under various intensities and constructing the light intensity matching model using the comprehensive peak signal-to-noise ratio parameter, the model is optimized to ensure the best vision tracking performance under different lighting conditions. Finally, to validate the effectiveness of the proposed active light source system, quantitative and qualitative recognition comparison experiments were conducted in eight different scenarios with UAVs equipped with active light sources. The experimental results show that the UAV equipped with an active light source has improved the recall of yoloV7 and RT-DETR recognition algorithms by 30% and 29.6%, the mAP50 by 21% and 19.5%, and the recognition accuracy by 13.1% and 13.6, respectively. Qualitative experiments also demonstrated that the active light source effectively improved the recognition success rate under low lighting conditions. Extensive qualitative and quantitative experiments confirm that the UAV active light source system based on light intensity matching proposed in this paper effectively enhances the effectiveness and robustness of vision-based tracking for multi-UAVs, particularly in complex and variable environments. This research provides an efficient and computationally effective solution for vision-based multi-UAV systems, further enhancing the visual tracking capabilities of multi-UAVs under complex conditions.

ENHANCING UNDERWATER IMAGE QUALITY: EVALUATING COMBINATIVE APPROACHES FOR EFFECTIVE IN SEAGRASS BED ECOSYSTEM

The Complex underwater characteristics, challenges for image processing tasks. These images often have poor visibility due to low contrast, light scattering and various types of interference. There is a lack of exploration into the effectiveness of existing underwater image enhancement methods, particularly in the context of seagrass ecosystems, allows for further investigation. This study aims to explore and evaluate the effectiveness of various methods in underwater image enhancement, including Colour Balanced, CLAHE, and Unsharp Masking and their combinations, starting with converting video data from UTS devices into two-dimensional images. Furthermore, the quality of images taken from underwater cameras placed in a complex and wild seagrass meadow environment was improved using the proposed method, and the quality was evaluated by the SSIM value. The results show that the CLAHE method has the highest average SSIM value of 0.898. Meanwhile, the combined Color Balanced-CLAHE method achieved an SSIM value of 0.683 in a separate evaluation. This combination is an innovative approach to address complex underwater image quality problems, providing a more specific and adaptive solution. Overall, the proposed method is able to improve the visual quality of images on aspects such as clarity, color, and visibility of objects in the image

Nighttime visible and infrared image fusion based on adversarial learning

The task of infrared–visible image fusion (IVIF) aims to integrate multi-modal complementary information and facilitate other downstream tasks, especially under some harsh circumstances. To tackle the challenges of preserving significant information and enhancing visual effects under nighttime conditions, we propose a novel IVIF method based on adversarial learning, namely AdvFusion. It consists of an autoencoder-based generator and a dual discriminator. In particular, the multi-scale features of source images are firstly extracted by ResNet, and then aggregated based on the attention mechanisms and nest connection strategy to generate the fused images. Meanwhile, a global and local dual discriminator structure is designed to minimize the distance between the illumination distributions of the reference images and fused images, which achieves contrast enhancement within fused images and helps to uncover hidden cues in darkness. Moreover, a color loss is utilized to maintain color balance of each fused image, while the widely used perceptual loss and gradient loss are employed to maintain content consistency between the source and fused images. Extensive experiments conducted on five datasets demonstrate that our AdvFusion can achieve promising results compared with the state-of-the-art IVIF methods in terms of both visual effects and quantitative metrics. Furthermore, AdvFusion can also boost the performance of semantic segmentation on MSRS dataset and object detection on M3FD dataset.

MACT: Underwater image color correction via Minimally Attenuated Channel Transfer

Underwater images usually show reduced quality due to the underwater environment where light propagation is affected by scattering and absorption, severely limiting the effectiveness of underwater images in practical applications. To effectively deal with the problem of poor underwater image quality, this paper proposes an innovative Minimally Attenuated Channel Transfer (MACT) method that effectively recovers color distortion and enhances the visibility of underwater images. In underwater images captured from natural scenes, specific color channels are often observed to be severely attenuated. To compensate for the information loss caused by channel attenuation, our color correction method selects the channel with the most minor degradation in the degraded image as the reference channel. Subsequently, we employ the reference channel and the color compensation factor obtained by dual-mean difference to perform adaptive color compensation on different color-degraded channels. Finally, we balance the histogram distribution of the compensated color channels by a linear stretching operation. Extensive experimental results on three benchmark datasets demonstrate that our preprocessing method achieves better performance. The project page is available at https://www.researchgate.net/publication/384252681_2024-MACT.

ALFusion: Adaptive fusion for infrared and visible images under complex lighting conditions

In the task of infrared and visible image fusion, source images often exhibit complex and variable characteristics due to scene illumination. To address the challenges posed by complex lighting conditions and enhance the quality of fused images, we develop the ALFusion method, which combines dynamic convolution and Transformer for infrared and visible image fusion. The core idea is to utilize the adaptability of dynamic convolution coupled with the superior long-range modeling capabilities of the Transformer to design a hybrid feature extractor. This extractor dynamically and comprehensively captures features from source images under various lighting conditions. To enhance the effectiveness of mixed features, we integrate an elaborate multi-scale attention enhancement module into the skip connections of the U-Net architecture. In this module, we use convolutional kernels of various sizes to expand the receptive field and incorporate an attention mechanism to enhance and highlight the combined use of dynamic convolution and the Transformer. Considering the performance in advanced visual tasks, an illumination detection network is integrated into the loss function, strategically balancing the pixel fusion ratio and optimizing visual impacts under varied lighting conditions, leveraging information from different source images. The fusion results indicate that our method consistently delivers superior background contrast and enhanced texture and structural features across diverse lighting conditions. Ablation and complementary experiments have been conducted, further affirming the effectiveness of our proposed method and highlighting its potential in advanced visual tasks.

Research on multi-scale fusion image enhancement and improved YOLOv5s lightweight ROV underwater target detection method

Underwater target detection technology is very important for Marine resources detection and underwater environment perception. The problems of low-quality underwater images, a large amount of calculation, and low accuracy of target detection models still need to be solved. Firstly, a new underwater image enhancement algorithm based on multi-scale fusion is proposed. The algorithm combines the improved white balance algorithm and the improved CLAHE (Contrast Limited Adaptive Histogram Equalization) and realizes the multi-scale fusion strategy by introducing different weight maps, aiming to improve the visual effect and quality of underwater images. Secondly, to further improve the accuracy and efficiency of underwater target detection, an improved YOLOv5s lightweight underwater target detection algorithm is proposed. The algorithm combines the Ghost convolution module, EMA(Efficient Multi-Scale Attention) mechanism, and CARAFE (Content-Aware ReAssembly of FEatures) up-sampling mode. The comparison on the URPC(Underwater Robot Perception Challenge) public dataset shows that the proposed algorithm has achieved significant improvement in terms of model size, parameter amount, calculation amount, and mAP@0.5. Thirdly, to verify the effectiveness of the image enhancement algorithm and underwater target detection algorithm proposed in this paper, the ROV(Remote Operated Vehicle) is used to conduct experiments in the experimental pool. By designing underwater target detection experiments before and after image enhancement, the results show that the enhanced image significantly improves the detection ability of underwater targets. Finally, underwater target detection software is developed to provide strong support for the application of underwater-related fields.