Visible Imaging Research Articles

Research on multi-scale fusion image enhancement and improved YOLOv5s lightweight ROV underwater target detection method.

Underwater target detection technology is very important for Marine resources detection and underwater environment perception. The problems of low-quality underwater images, a large amount of calculation, and low accuracy of target detection models still need to be solved. Firstly, a new underwater image enhancement algorithm based on multi-scale fusion is proposed. The algorithm combines the improved white balance algorithm and the improved CLAHE (Contrast Limited Adaptive Histogram Equalization) and realizes the multi-scale fusion strategy by introducing different weight maps, aiming to improve the visual effect and quality of underwater images. Secondly, to further improve the accuracy and efficiency of underwater target detection, an improved YOLOv5s lightweight underwater target detection algorithm is proposed. The algorithm combines the Ghost convolution module, EMA(Efficient Multi-Scale Attention) mechanism, and CARAFE (Content-Aware ReAssembly of FEatures) up-sampling mode. The comparison on the URPC(Underwater Robot Perception Challenge) public dataset shows that the proposed algorithm has achieved significant improvement in terms of model size, parameter amount, calculation amount, and mAP@0.5. Thirdly, to verify the effectiveness of the image enhancement algorithm and underwater target detection algorithm proposed in this paper, the ROV(Remote Operated Vehicle) is used to conduct experiments in the experimental pool. By designing underwater target detection experiments before and after image enhancement, the results show that the enhanced image significantly improves the detection ability of underwater targets. Finally, underwater target detection software is developed to provide strong support for the application of underwater-related fields.

Multi-Level Cross-Modal Interactive-Network-Based Semi-Supervised Multi-Modal Ship Classification

Ship image classification identifies the type of ships in an input image, which plays a significant role in the marine field. To enhance the ship classification performance, various research focuses on studying multi-modal ship classification, which aims at combining the advantages of visible images and infrared images to capture complementary information. However, the current methods simply concatenate features of different modalities to learn complementary information, which neglects the multi-level correlation between different modalities. Moreover, the existing methods require a large amount of labeled ship images to train the model. How to capture the multi-level cross-modal correlation between unlabeled and labeled data is still a challenge. In this paper, a novel semi-supervised multi-modal ship classification approach is proposed to solve these issues, which consists of two components, i.e., multi-level cross-modal interactive network and semi-supervised contrastive learning strategy. To learn comprehensive complementary information for classification, the multi-level cross-modal interactive network is designed to build local-level and global-level cross-modal feature correlation. Then, the semi-supervised contrastive learning strategy is employed to drive the optimization of the network with the intra-class consistency constraint based on supervision signals of unlabeled samples and prior label information. Extensive experiments on the public datasets demonstrate that our approach achieves state-of-the-art semi-supervised classification effectiveness.

Contrast-enhanced thin-slice abdominal CT with super-resolution deep learning reconstruction technique: evaluation of image quality and visibility of anatomical structures.

To compare image quality and visibility of anatomical structures on contrast-enhanced thin-slice abdominal CT images reconstructed using super-resolution deep learning reconstruction (SR-DLR), deep learning-based reconstruction (DLR), and hybrid iterative reconstruction (HIR) algorithms. This retrospective study included 54 consecutive patients who underwent contrast-enhanced abdominal CT. Thin-slice images (0.5mm thickness) were reconstructed using SR-DLR, DLR, and HIR. Objective image noise and contrast-to-noise ratio (CNR) for liver parenchyma relative to muscle were assessed. Two radiologists independently graded image quality using a 5-point rating scale for image noise, sharpness, artifact/blur, and overall image quality. They also graded the visibility of small vessels, main pancreatic duct, ureters, adrenal glands, and right adrenal vein on a 5-point scale. SR-DLR yielded significantly lower objective image noise and higher CNR than DLR and HIR (P < .001). The visual scores of SR-DLR for image noise, sharpness, and overall image quality were significantly higher than those of DLR and HIR for both readers (P < .001). Both readers scored significantly higher on SR-DLR than on HIR for visibility for all structures (P < .01), and at least one reader scored significantly higher on SR-DLR than on DLR for visibility for all structures (P < .05). SR-DLR reduced image noise and improved image quality of thin-slice abdominal CT images compared to HIR and DLR. This technique is expected to enable further detailed evaluation of small structures.

Fusion of UAV-Acquired Visible Images and Multispectral Data by Applying Machine-Learning Methods in Crop Classification

The sustainable development of agriculture is closely related to the adoption of precision agriculture techniques, and accurate crop classification is a fundamental aspect of this approach. This study explores the application of machine learning techniques to crop classification by integrating RGB images and multispectral data acquired by UAVs. The study focused on five crops: rice, soybean, red bean, wheat, and corn. To improve classification accuracy, the researchers extracted three key feature sets: band values and vegetation indices, texture features extracted from a grey-scale co-occurrence matrix, and shape features. These features were combined with five machine learning models: random forest (RF), support vector machine (SVM), k-nearest neighbour (KNN) based, classification and regression tree (CART) and artificial neural network (ANN). The results show that the Random Forest model consistently outperforms the other models, with an overall accuracy (OA) of over 97% and a significantly higher Kappa coefficient. Fusion of RGB images and multispectral data improved the accuracy by 1–4% compared to using a single data source. Our feature importance analysis showed that band values and vegetation indices had the greatest impact on classification results. This study provides a comprehensive analysis from feature extraction to model evaluation, identifying the optimal combination of features to improve crop classification and providing valuable insights for advancing precision agriculture through data fusion and machine learning techniques.

Super-oscillation sub-diffraction focusing with emulated atmospheric turbulence

The super-oscillation (SO) phenomena successfully applied to super-resolution optical telescopes have yet to study the existence of atmospheric turbulence (AT). In this paper, we first experimentally investigate the sub-diffraction focusing of the SO light field in the atmospheric-like turbulence. The 137-element adaptive optics (AO) system is utilized to correct the dynamic wavefront aberration produced by the AT and obtain the AO closed-loop RMS of the residual wavefront error ∼λ/15. For a proofed telescope (clear aperture 12 mm and focal length 1000 mm) @λ=632.8 nm, the FWHM of the experimental SO spot under the modest AT (wavefront error RMS ∼0.57λ) is about 0.79 times of the Airy spot compared with the non-AT result of 0.76 times, while the strong AT (wavefront error RMS ∼1.35λ) erases the sub-diffraction focusing effect of the SO field and the average side-lobe intensity increases significantly. This study is promising for super-resolving telescope, synthetic aperture for visible imaging, etc.

Enhancing Image Dehazing Efficiency with Dual Colour Space Attentional Deep Networks

Aims and Background: Image dehazing is an essential task in computer-vision, aimed at enhancing the clarity and visibility of images degraded by fog and haze. Traditional methods often struggle with handling the complex scattering effects of haze and maintaining the natural colours of the scene. Objectives and Methodology: To address these challenges, we propose a novel approach termed Dual Colour Space Attentional Deep Network (DCSADN) for efficient image dehazing. Our method leverages the advantages of two-colour spaces: RGB and YCbCr, to boost the network's skill to capture both the luminance and chrominance information, thereby improving the dehazing performance. We employ a multi-stage training strategy to optimize the performance of the DCSADN. This multi-stage approach ensures that the network generalizes well across different types of haze conditions. Extensive experiments demonstrate the superiority of our method over state-of-the-art dehazing techniques. Results: The results indicate that our approach not only achieves higher quantitative scores in terms of Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM) but also produces dehazed images with more natural colours and details. Conclusion: The findings reveal that the dual colour space processing and attentional modules are crucial for the enhanced performance of our method and providing a valuable tool for improving image quality in hazy conditions.

SIFusion: Lightweight infrared and visible image fusion based on semantic injection.

The objective of image fusion is to integrate complementary features from source images to better cater to the needs of human and machine vision. However, existing image fusion algorithms predominantly focus on enhancing the visual appeal of the fused image for human perception, often neglecting their impact on subsequent high-level visual tasks, particularly the processing of semantic information. Moreover, these fusion methods that incorporate downstream tasks tend to be overly complex and computationally intensive, which is not conducive to practical applications. To address these issues, a lightweight infrared and visible light image fusion method known as SIFusion, which is based on semantic injection, is proposed in this paper. This method employs a semantic-aware branch to extract semantic feature information, and then integrates these features into the fused features through a Semantic Injection Module (SIM) to meet the semantic requirements of high-level visual tasks. Furthermore, to simplify the complexity of the fusion network, this method introduces an Edge Convolution Module (ECB) based on structural reparameterization technology to enhance the representational capacity of the encoder and decoder. Extensive experimental comparisons demonstrate that the proposed method performs excellently in terms of visual appeal and advanced semantics, providing satisfactory fusion results for subsequent high-level visual tasks even in challenging scenarios.

Effect of detachment on Magnum-PSI ELM-like pulses: direct observations and qualitative results

Abstract Conditions similar to those at the end of the divertor leg in a tokamak were replicated in the linear plasma machine Magnum-PSI. The neutral pressure in the target chamber is then increased to cause the target to transition from an attached to a detached state. Superimposed to this steady state regime, edge localised mode (ELM)-like pulses are reproduced, resulting in a sudden increase in plasma temperature and density, such that the heat flux increases transiently by half an order of magnitude. Visible light emission, target thermography, and Thomson scattering are used to demonstrate that the higher the neutral pressure the more energy is removed from the ELM-like pulse in the volume. If the neutral pressure is sufficiently high, the ELM-like pulse can be prevented from affecting the target and the plasma energy is fully dissipated in the volume instead (ID 4 in table 1). The visible light images allow the division of the ELM-plasma interaction process of ELM energy dissipation into 3 ‘stages’ ranging from no dissipation to full dissipation (the target plasma is detached). In the second publication related to this study, spectroscopic data is analysed with a Bayesian approach, to acquire insights into the significance of molecular processes in dissipating the plasma energy and particles.

MSL-CCRN: Multi-stage self-supervised learning based cross-modality contrastive representation network for infrared and visible image fusion

Infrared and visible image fusion (IVIF) facing different information in two modal scenarios, the focus of research is to better extract different information. In this work, we propose a multi-stage self-supervised learning based cross-modality contrastive representation network for infrared and visible image fusion (MSL-CCRN). Firstly, considering that the scene differences between different modalities affect the fusion of cross-modal images, we propose a contrastive representation network (CRN). CRN enhances the interaction between the fused image and the source image, and significantly improves the similarity between the meaningful features in each modality and the fused image. Secondly, due to the lack of ground truth in IVIF, the quality of directly obtained fused image is seriously affected. We design a multi-stage fusion strategy to address the loss of important information in this process. Notably, our method is a self-supervised network. In fusion stage I, we reconstruct the initial fused image as the new view of fusion stage II. In fusion stage II, we use the fused image obtained in the previous stage to carry out three-view contrastive representation, thereby constraining the feature extraction of the source image. This makes the final fused image introduce more important information in the source image. Through a large number of qualitative, quantitative experiments and downstream object detection experiments, our propose method shows excellent performance compared with most advanced methods.

FLFusionSR: a fast and lightweight fusion and super-resolution network for infrared and visible images on edge devices

FLFusionSR: a fast and lightweight fusion and super-resolution network for infrared and visible images on edge devices

Dual-Mode Stretchable Emitter with Programmable Emissivity and Air Permeability.

Materials with anisotropic emission characteristics have attracted considerable attention for thermal management. Although many dual-mode emitters have been developed for this purpose in the form of textiles, multilayer films, and photonic structures, multiple functionalities are essential for their versatile applications. Herein, a highly stretchable dual-mode emitter with programmable emissivity and air permeability is presented. The emitter comprises a planar Ge2Sb2Te5 (GST) cavity on one side of a perforated elastomer substrate and an infrared-reflecting metal layer on the other side. With a laser-induced phase transition from amorphous to crystalline GST, the emitter exhibits a large emissivity difference of 0.52 between both sides. The dual-mode emitter remains highly stable without mechanical failure after repeated stretching cycles to a strain of 50%. This air-permeable and stretchable emitter can be attached to any curved surface, including the human body. The GST-side emissivity can be programmed into an arbitrary emissivity pattern using a spatially modulated laser beam, ultimately enabling the printing of mutually independent visible and thermal images in a single emitter. This study provides a promising structure for multispectral optical security as well as thermal management.

LEAFusion: An Infrared and Visible Light Image Fusion Network Resilient to Harsh Light Environment Interference Based on Harsh Light Environment Aware

LEAFusion: An Infrared and Visible Light Image Fusion Network Resilient to Harsh Light Environment Interference Based on Harsh Light Environment Aware

Improving Image Quality and Visualization of Hepatocellular Carcinoma in Arterial Phase Imaging Using Contrast Enhancement-Boost Technique.

This study aimed to evaluate the image quality and visualization of hepatocellular carcinoma (HCC) on arterial phase computed tomography (CT) using the contrast enhancement (CE)-boost technique. This retrospective study included 527 consecutive patients who underwent dynamic liver CT between June 2021 and February 2022. Quantitative and qualitative image analyses were performed on 486 patients after excluding 41 patients. HCC conspicuity was evaluated in 40 of the 486 patients with at least one HCC in the liver. Iodinated images obtained by subtracting nonenhanced images from arterial phase images were combined to generate CE-boost images. For quantitative image analysis, image noise, signal-to-noise ratio (SNR), and contrast-to-noise ratio (CNR) were measured for the liver, pancreas, muscles, and aorta. For qualitative analysis, the overall image quality and noise were graded using a 3-point scale. Artifact, sharpness, and HCC lesion conspicuity were assessed using a 5-point scale. The paired-sample t test was used to compare quantitative measures, whereas the Wilcoxon signed-rank test was used to compare qualitative measures. The mean SNR and CNR of the aorta, liver, pancreas, and muscle were significantly higher, and the image noise was significantly lower in the CE-boost images than in the conventional images (P < 0.001). The mean CNR of HCC was also significantly higher in the CE-boost images than in the conventional images (P < 0.001). In the qualitative analysis, CE-boost images showed higher scores for HCC lesion conspicuity than conventional images (P < 0.001). The overall image quality and visibility of HCC were improved using the CE-boost technique.

Semantic Guided Matting Net

Abstract Human matting refers to extracting human parts from natural images with high quality, including human detail information such as hair, glasses, hats, etc. This technology plays an essential role in image synthesis and visual effects in the film industry. When the green screen is not available, the existing human matting methods need the help of additional inputs (such as trimap, background image, etc.), or the model with high computational cost and complex network structure, which brings great difficulties to the application of human matting in practice. To alleviate such problems, we use a segmentation network as the foundation and use multiple branches to achieve human segmentation, contour detail extraction, and information fusion. We also propose a foreground probability map module, which uses the feature maps in the segmentation network to pre-estimate the foreground probabilities of each pixel and obtain Semantic Guided Matting Net. Under the condition that only a single image is needed as the input, the human matting task can be realized by making full use of the semantic information in the image. We validate our method on the P3M-10k dataset. Compared with the benchmark, our method has made significant improvements in various evaluation indicators.

ALFusion: Adaptive fusion for infrared and visible images under complex lighting conditions

ALFusion: Adaptive fusion for infrared and visible images under complex lighting conditions

Synthetic measurements of runaway electron synchrotron emission in the SPARC tokamak.

With plasma currents up to 8.7 MA, the SPARC tokamak runs the risk of forming multi-MA beams of relativistic "runaway" electrons (REs), which could damage plasma facing components if unmitigated. The infrared (IR) and visible imaging and visible spectroscopy systems in SPARC are designed with measurements of synchrotron emission from REs in mind. Synchrotron radiation is emitted by REs along their direction of motion, opposite the plasma current. Matched clockwise and counterclockwise wide views are proposed to detect synchrotron and background radiation, allowing observation of RE synchrotron emission in both plasma current configurations. Due to SPARC's high toroidal magnetic field strength, 12.2T on axis, the synchrotron light spectrum is expected to peak in the visible-IR wavelength range. The synthetic diagnostic tool, Synchrotron Orbit-Following Toolkit, is used to model synchrotron images and spectra for three scenarios, with appropriate magnetic equilibria for each: REs generated during plasma current ramp-up, steady-state flat-top (although unlikely, but serving as a reference), and disruptions. Required time resolutions, achievable spatial coverage, and appropriate spectral ranges for various RE energies are assessed.

Research on the veneer defect image enhancement algorithm based on AMEF-AGC.

Aiming at the veneer defect image acquisition process is prone to the problems of blurred edges, inconspicuous contrast and distortion, which cannot show the defects clearly.To improve image analyzability and clarity, a veneer defect image enhancement method based on AMEF-AGC is proposed herein. First, a veneer defect image is subjected to Gamma correction to obtain multiple underexposed image sequences for which Gaussian and Laplacian pyramids are constructed to determine the weights of the multiple exposure sequence group images.Multiscale fusion is then performed based on these weights. Second, the fused image is converted into the HSV color space, where contrast and brightness enhancements are performed for the luminance component, and then converted back to the RGB color space to obtain an enhanced veneer defect image. Solid wood panels selected herein were pine, poplar and birch with defects including live knots, dead knots, and cracks.Compared with those obtained using several algorithms including AMEF, AGC, improved AGC and GC, this algorithm achieved 6.93% and 5.4% improvements in PSNR and SSIM metrics, respectively.Results demonstrated that the proposed method effectively enhanced veneer defect images with blurred and distorted edges, improved image clarity and visual quality, and made defect parts and details of the image clearer.

Contemporaneous high-angular-resolution imaging of the AGB star W Hya in vibrationally excited H2O lines and visible polarized light with ALMA and VLT/SPHERE-ZIMPOL

Aims. We present contemporaneous high-angular-resolution millimeter imaging and visible polarimetric imaging of the nearby asymptotic giant branch (AGB) star W Hya to better understand the dynamics and dust formation within a few stellar radii. Methods. The star W Hya was observed in two vibrationally excited H2O lines at 268 and 251 GHz with Atacama Large Millimeter/submillimeter Array (ALMA) at a spatial resolution of 16 × 20 mas and at 748 and 820 nm at a resolution of 26 × 27 mas with the Very Large Telescope (VLT)/Spectro-Polarimetric High-contrast Exoplanet REsearch (SPHERE)-Zurich Imaging Polarimeter (ZIMPOL). Results. ALMA’s high spatial resolution allowed us to image strong emission of the vibrationally excited H2O line at 268 GHz (v2 = 2, JKa, Kc = 65, 2–74, 3) over the stellar surface instead of absorption against the continuum, which is expected for thermal excitation. Strong, spotty emission was also detected along and just outside the stellar disk limb at an angular distance of ∼40 mas (∼1.9 R⋆), extending to ∼60 mas (∼2.9 R⋆). Another H2O line (v2 = 2, JKa, Kc = 92, 8–83, 5) at 251 GHz with a similar upper-level energy was tentatively identified, which shows absorption over the stellar surface. This suggests that the emission over the surface seen in the 268 GHz H2O line is suprathermal or even maser emission. The estimated gas temperature and H2O density are consistent with the radiatively pumped masers. The 268 GHz H2O line reveals global infall at up to ∼15 km s−1 within 2–3 R⋆, but outflows at up to ∼8 km s−1 are also present. The polarized intensity maps obtained in the visible reveal clumpy dust clouds forming within ∼40 mas (∼1.9 R⋆) with a particularly prominent cloud in the SW quadrant and a weaker cloud in the east. The 268 GHz H2O emission overlaps very well with the visible polarized intensity maps, which suggests that both the nonthermal and likely maser H2O emission and the dust originate from dense, cool pockets in the inhomogeneous atmosphere within ∼2–3 R⋆.

Non-destructive estimation for Kyoho grape shelf-life using Vis/NIR hyperspectral imaging and deep learning algorithm

Grape shelf-life estimation is a substantial challenge for the grape industry. The objective of this study is to investigate the potential of grape shelf-life estimation using HSI technique and a deep learning algorithm. The visible and near-infrared (400.68–1001.61 nm) hyperspectral reflectance images data of grape samples was acquired and preprocessed with different spectral preprocessing methods. Additionally, a stacked denoising autoencoder (SDAE)-based deep learning algorithm was developed to extract deep features from pixel-level hyperspectral data of grapes, and then these features were used as inputs to establish support vector machine (SVM) models for estimating grape shelf-life. Furthermore, SVM, one-dimensional convolutional neural network (1D CNN) and long short-term memory (LSTM) models were used as traditional machine learning and end to end models for comparison. The results demonstrated that the SDAE-SVM model achieved reasonable recognition accuracy of 100 % and 98.125 % for the shelf-life of grapes in the training and test sets, respectively. The overall results suggested that SDAE-based deep learning method can be used as a powerful tool to deal with large-scale hyperspectral data as well as this research confirms the feasibility of non-destructive estimation for grapes shelf-life by the combination of HSI technique and deep learning method, which would provide a valuable guidance for shelf-life estimation of other postharvest fruit.

Diagnosing Fungal Infection in Wheat Kernels by Integrating Spectroscopic Technology and Digital Color Imaging System: Artificial Neural Network, Principal Component Analysis and Correlation Feature Selection Techniques

ABSTRACTContamination of cereal grain, especially wheat, with fungal infections can cause significant economic impacts and it endangers the health of humans and livestock. This study aims to appraise the UV/VIS–NIR and digital color (RGB) imaging systems and spectroscopic methodology to detect wheat kernels infected by fungi such as Penicillium expansum and Fusarium graminearum. NIR spectra of 190–1100 nm at 10 nm intervals, visible color reflectance images and non‐visible reflectance images of wheat kernels in the ultraviolet and near‐infrared ranges were applied to develop the multi‐layer perceptron (MLP) artificial neural network model. The optimum wavelengths were selected by application of the principal component analysis (PCA) after preprocessing the raw spectra. A confusion matrix was used in the correlation feature selection method (CFS) for the decision tree classifier of selected features. The results showed that the four UV wavelengths of 310, 330, 400, and 410 nm were the best wavelengths using PCA to distinguish healthy and unhealthy wheat kernels. Considering the intensity of the wavelengths as the neural network inputs, samples were classified into healthy and unhealthy categories with an accuracy of 90.9%. Also, 18 features of color images in RGB, LAB, HSV, HSI, YCbCr, and YIQ spaces provided the highest average accuracy of 44.4% in classifying healthy and infected wheat kernels by using a CCD Proline camera in the ultraviolet range. In contrast, other cameras in the visible and invisible range showed low accuracy. Furthermore, the best classification accuracy of the healthy and infected samples by the use of the CFS method was obtained at 88.1%. Based on the findings, spectroscopic methodology proved to be highly effective for detecting, classifying and automatic cleaning of various agricultural seeds, with a particular emphasis on wheat kernals.