Visible Imaging Research Articles

Multi-objective optimization for vector quantization via genetic algorithm

The issue of image compression continues to be a subject of ongoing research within the domain of image processing, particularly in the context of medical applications. The quality of the decompressed image can vary depending on the accuracy of the compression technique, resulting in either fine or distorted details. Therefore, the diagnostic procedure performed by medical professionals is contingent upon the precision of the compression and decompression process. In addition, the compression of medical images serves to decrease the amount of storage required, thereby enabling faster transmission over computer networks through the reduction of their bit size. This paper proposes a hybrid mix of the discrete wavelet transform (DWT) technique and vector quantization (VQ) to improve the compression technique of medical images. The aim of the proposed compression technique is to preserve diagnostic image information while achieving a high compression ratio. First, noise in medical images is caused by salt pepper noise. At the same time, the edges of the images are maintained in sharpness and context. Then, a lossless compression method is applied to the wavelet coefficients of the subband with the lowest frequency, while the thresholding method was used to efficiently construct coefficients for high-frequency sub-bands. This process will produce a traditional VQ, which is estimated via the Genetic Algorithm (GA) with fuzzy clustering. While Arithmetic encoded theory was being utilized to quantize coefficients, the proposed compression technique was evaluated by dividing the image into two levels and three levels of sub-bands, respectively, in two different scenarios. Comparing the decompressed image to the filtered image by means of different evaluation metrics, the proposed method can enhance compression performance and strike a balance between compression ratio and image visual quality.

MACCNet: Multiscale Attention and Cross- Convolutional Network for Infrared and Visible Image Fusion

MACCNet: Multiscale Attention and Cross- Convolutional Network for Infrared and Visible Image Fusion

Exploring Ear Recognition: Image Fusion and Deep Learning with Thermal and Visible Images

Exploring Ear Recognition: Image Fusion and Deep Learning with Thermal and Visible Images

Image Fusion Method Based on Snake Visual Imaging Mechanism and PCNN

The process of image fusion is the process of enriching an image and improving the image’s quality, so as to facilitate the subsequent image processing and analysis. With the increasing importance of image fusion technology, the fusion of infrared and visible images has received extensive attention. In today’s deep learning environment, deep learning is widely used in the field of image fusion. However, in some applications, it is not possible to obtain a large amount of training data. Because some special organs of snakes can receive and process infrared information and visible information, the fusion method of infrared and visible light to simulate the visual mechanism of snakes came into being. Therefore, this paper takes into account the perspective of visual bionics to achieve image fusion; such methods do not need to obtain a significant amount of training data. However, most of the fusion methods for simulating snakes face the problem of unclear details, so this paper combines this method with a pulse coupled neural network (PCNN). By studying two receptive field models of retinal nerve cells, six dual-mode cell imaging mechanisms of rattlesnakes and their mathematical models and the PCNN model, an improved fusion method of infrared and visible images was proposed. For the proposed fusion method, eleven groups of source images were used, and three non-reference image quality evaluation indexes were compared with seven other fusion methods. The experimental results show that the improved algorithm proposed in this paper is better overall than the comparison method for the three evaluation indexes.

Light-weight Fusion Network for UAV Visible-light and Infrared Images based on Real-time Brain-like Intelligent Computing

Abstract. Multiple sensors equipped on the Unmanned Aerial Vehicle (UAV) enables the acquisition of multi-modal and multi-source remote sensing data. UAV remote sensing usually faces with real-time or near-real-time tasks in complex and highly dynamic environments, such as disaster monitoring, traffic management, border patrol and so on. Under these conditions, the image fusion algorithm needs to be high efficiency, precision and reliability. In this paper, we proposed an intelligent real-time fusion network for UAV multi-source remote sensing data based on AI brain-like chips, and deployed the algorithm on the UAV platform to achieve online high-efficiency computing. Firstly, we have developed a novel image fusion algorithm named SFNet for infrared and visible image fusion based on ShuffleNetv2. Then, we use ZCA and l1-norm to process the remodeled deep feature. The weight maps are generated by bi-cubic interpolation and soft-max operation. Finally, the fused image is reconstructed by weighted-average operation. The proposed SFNet is deployed on the Lynxi KA200 brain computing chip, and a comprehensive inference test is carried out with UAV remote sensing data. Several State-Of-The-Art (SOTA) data fusion algorithms are deployed on the same chip for experimental comparison. The proposed SFNet is proved to have faster inference speed and better feature extraction results on brain-like chips. It is more suitable for real-time UAV remote sensing image fusion tasks.

Recognition of Plastic Film in Terrain-Fragmented Areas Based on Drone Visible Light Images

In order to meet the growing demand for food and achieve food security development goals, contemporary agriculture increasingly depends on plastic coverings such as agricultural plastic films. The remote sensing-based identification of these plastic films has gradually become a necessary tool for agricultural production management and soil pollution prevention. Addressing the challenges posed by the complex terrain and fragmented land parcels in karst mountainous regions, as well as the frequent presence of cloudy and foggy weather conditions, the extraction efficacy of mulching films is compromised. This study utilized a DJI Mavic 2 Pro UAV to capture visible light images in an area with complex terrain features such as peaks and valleys. A plastic film sample dataset was constructed, and the U-Net deep learning model parameters integrated into ArcGIS Pro were continuously modified and optimized to achieve precise plastic film identification. The results are as follows: (1) Sample quantity significantly affects recognition performance. When the sample size is 800, the accuracy of plastic film extraction notably improves, with area accuracy reaching 91%, a patch quantity accuracy of 96.38%, and an IOU and F1-score of 85.89% and 94.20%, respectively, compared to the precision achieved with a sample size of 300; (2) Different learning rates, batch sizes, and iteration numbers have a certain impact on the training effectiveness of the U-Net model. The most suitable model parameters improved the training effectiveness, with the highest training accuracy achieved at a learning rate of 0.001, a batch size of 10, and 25 iterations; (3) Comparative experiments with the Support Vector Machine (SVM) model validate the suitability of U-Net model parameters and sample datasets for precise identification in rugged terrains with fragmented spatial distribution, particularly in karst mountainous regions. This underscores the applicability of the U-Net model in recognizing plastic film coverings in karst mountainous regions, offering valuable insights for agricultural environmental health assessment and green planting management in farmlands.

Observational signatures of charged rotating traversable wormhole: shadows and light rings with different accretions

In this work, we investigate the optical appearance of qualitatively new observational features of accretion disk images around the charged rotating traversable wormhole (TWH) space-time for different spin, throat, and charge values. To accomplish this, we first consider the Hamilton–Jacobi method to derive the geodesic equations for the motion of photons and study the effects of parameters on the photon orbit in the observer’s sky. We found that each parameter affects the size and shape of the wormhole (WH) shadow and flatness is observed in the shadow because of spin and other parameters. To produce shadow images of sufficient visual quality but within manageable computational times, we adopt the ray-trace procedure and characterize the significant features of light trajectories on the observer’s screen, depending on the interaction between the space-time structure and the accretion disk. In addition, we consider the static spherically symmetric accretion flow model to observe the specific intensity around the traversable WH space-time geometry. It is found that the intensity and positions of the photon ring vary with respect to the involved parameters. In future observation, this type of study may provide a fertile playground to test the nature of compact objects, specifically the WH in the strong-field regime.

3D Poissonian image deblurring via patch-based tensor logarithmic Schatten-p minimization

In medical and biological image processing, multi-dimensional images are often corrupted by blur and Poisson noise. In this paper, we first propose a new tensor logarithmic Schatten-p (t-log-Sp ) low-rank measure and a tensor iteratively reweighted Schatten-p minimization algorithm for minimizing such measure. Furthermore, we adopt this low-rank measure to regularize the non-local tensors formed by similar 3D image patches and develop a patch-based non-local low-rank model. The data fidelity term of the model characterizes the Poisson noise distribution and blur operator. The optimization model is further solved by an alternating minimization technique combined with variable splitting. Experimental results tested on 3D fluorescence microscope images show that the proposed patch-based tensor logarithmic Schatten-p minimization method outperforms state-of-the-art methods in terms of image evaluation metrics and visual quality.

MRASFusion: A multi-scale residual attention infrared and visible image fusion network based on semantic segmentation guidance

To address the challenges of inadequate preservation of prominent targets, poor retention of texture details, and unsatisfactory reconstruction of image backgrounds in image fusion. In this paper, a multi-scale residual attention fusion network based on semantic segmentation guidance is proposed, termed as MRASFusion. First of all, Swin Transformer segmentation mask with high precision, and strong scalability is adopted to avoid the inefficiency and error of manual segmentation mask. The mask generated by semantic segmentation is used to construct a loss function to guide the image fusion process. Secondly, in order to maintain the integrity of contextual information and texture details, a new feature extraction module is proposed to fully extract the meaningful features. Finally, the fused image is obtained by reconstructing the extracted features. To verify the effectiveness of the method, MRASFusion is qualitatively and quantitatively compared with nine state-of-the-art fusion methods on TNO and RoadScene datasets. Experimental results indicate that our method has demonstrated satisfactory performance in image fusion tasks, exhibiting superior capabilities in preserving target information and retaining texture details. Furthermore, our fusion results have brought some performance improvements for advanced vision tasks, i.e., improved accuracy for the object detection, which provides a better foundation for solving real-world problems.

Geographical and seasonal variations of gravity wave activities in the upper mesosphere measured by space-borne imaging of molecular oxygen nightglow

Geographical and seasonal variations of gravity wave events in the upper mesosphere were investigated using the nightglow imaging data obtained by the Visible and near-Infrared Spectral Imager (VISI) on the Ionosphere, Mesosphere, upper Atmosphere and Plasmasphere (IMAP) onboard the International Space Station (ISS). The nadir-imaging data of the O2(0–0) atmospheric band (762 nm) with the typical emission peak around 95 km altitude was used to investigate small-scale waves (horizontal wavelengths less than ~ 200 km) on a global scale. To detect gravity wave events, the variance of high-pass filtered nightglow images within a local 100 km radius was evaluated, with a threshold set at three times the standard deviation from the average variance of the background level. A data screening algorithm that evaluates the variance of upwelling contamination light emission was also introduced to remove contaminated data. Applying the variance filter and data screening algorithm to a nearly 3-year data set, from November 2012 to August 2015, occurrence maps of wave events for four seasons were derived. The occurrence maps show a higher frequency of wave events in winter high latitudes (> 40° N/S), considerably attributed to gravity wave activity associated with the polar night jet. Hot spots were observed near orographic sources in winter high latitudes, including the eastern part of North America, Europe, and the southern Andes. In the summer hemisphere, hot spots were detected at mid-to-high latitudes such as North America, Europe, and the eastern side of the Eurasian continent, and at equatorial latitudes just above the intertropical convection zone (ITCZ). They are likely gravity waves from deep convection that arise from mid-latitude summertime thunderstorms and the ITCZ, respectively. During the equinox seasons, hot spots were detected near convective sources such as the Amazon Rainforest, Congo Rainforest, and the Indochina peninsula.Graphical

Image Segmentation Technology Based on Ant Colony Algorithm

Image segmentation is a key task in computer vision, with applications ranging from medical diagnosis to autonomous driving. The Ant Colony Algorithm (ACO), modeled after ant foraging behavior, has emerged as a viable segmentation methodology. However, ACO-based segmentation algorithms frequently generate segmented outputs with jagged or uneven boundaries, which reduces their interpretability and usability. To alleviate this problem, they study the use of boundary-smoothing approaches in ACO-based segmentation. In this paper, they investigate image segmentation technology based on the Ant Colony Algorithm, with a focus on border smoothing. They examine the fundamentals of ACO and its application to image segmentation, emphasizing its strengths and limits. They also look at several boundary smoothing strategies, such as morphological operations, edge-preserving filters, and active contours (snakes), and how they affect segmentation performance. Through experimental validation and comparative analysis, they show that boundary smoothing improves the accuracy and visual quality of segmented images produced by ACO-based segmentation algorithms. These results help to design more robust and visually appealing segmentation algorithms, which have potential applications in medical imaging, remote sensing, and industrial automation.

Infrared and visible image fusion method based on visual saliency objects and fuzzy region attributes

Infrared and visible image fusion method based on visual saliency objects and fuzzy region attributes

Enhancing Power Lines Detection Using Deep Learning and Feature-Level Fusion of Infrared and Visible Light Images

AbstractThe detection of power lines is critical for flight safety, especially for drones and low-flying aircraft. Power line detection models help prevent collisions, reducing potential damage and preserving lives, while also safeguarding critical infrastructure. This has led to significant research efforts to develop accurate detection models. In this study, we employ paired infrared–visible power line datasets to train three distinct deep learning models. The first two models are sequential deep learning models based on VGG16 and AlexNet networks. They are tailored for detection in visible images, while they were optimized again for infrared images. For the third model, we introduce an innovative deep learning architecture utilizing Functional Application Programming Interface, affording us the flexibility to construct a multi-input model with shared layers. Our proposed model accepts paired images (visible and infrared) as inputs. Then, a feature-level fusion process is applied to merge the extracted features from both inputs and generate an enriched feature map. This approach amalgamates the advantages of visible images, which boast high resolution and rich texture features, with infrared images, which excel in high contrast and clear vision under adverse environmental conditions. Comparing the outcomes of the three models, our proposed model emerges as the front runner, boasting an impressive accuracy rate of 99.37%. Moreover, real-time processing was adopted by conducting ablation experiments to optimize the model and reduce the number of trainable parameters, resulting in an inference speed of 2.7 milliseconds per frame.

High-visibility Ghost Imaging with Phase-controlled Discrete Classical Light Sources

Abstract In this paper, we take phase modulation to create discrete phase-controlled sources and realize the superbunching effect by a phase-correlated method. From theoretical and numerical simulations, we find the space translation invariance of the bunching effect is a key point for the ghost imaging realization. Experimentally, we create the orderly phase-correlated discrete sources which can realize high-visibility second-order ghost imaging than the result with chaotic sources. Moreover, some factors affecting the visibility of ghost image are discussed in detail.

SCAKD: a knowledge distillation framework based on spatial-corner attention for infrared and visible image fusion

SCAKD: a knowledge distillation framework based on spatial-corner attention for infrared and visible image fusion

CABnet: A channel attention dual adversarial balancing network for multimodal image fusion

Infrared and visible image fusion aims to generate informative images by leveraging the distinctive strengths of infrared and visible modalities. These fused images play a crucial role in subsequent downstream tasks, including object detection, recognition, and segmentation. However, complementary information is often difficult to extract. Existing generative adversarial network-based methods generate fused images by modifying the distribution of source images' features to preserve instances and texture details in both infrared and visible images. Nevertheless, these approaches may result in a degradation of the fused image quality when the original image quality is low. Considering the balance of information from different modalities can improve the quality of the fused image. Hence, we introduce CABnet, a Channel Attention dual adversarial Balancing network. CABnet incorporates a channel attention mechanism to capture crucial channel features, thereby, enhancing complementary information. It also employs an adaptive factor to control the mixing distribution of infrared and visible images, which ensures the preservation of instances and texture details during the adversarial process. To enhance efficiency and reduce reliance on manual labeling, our training process adopts a semi-supervised learning strategy. Through qualitative and quantitative experiments across multiple datasets, CABnet surpasses existing state-of-the-art methods in fusion performance, notably achieving a 51.3% enhancement in signal to noise ratio and a 13.4% improvement in standard deviation.

Modality specific infrared and visible image fusion based on multi-scale rich feature representation under low-light environment

Infrared (IR) and Visible (VIS) image fusion is a critical task in computer vision, which aims to produce high-quality fused images by combining two modal images with complementary information. However, current image fusion techniques neglect the severe degradation of texture and details in low-light environments and do not account for human visual perception. In this paper, a multi-scale image fusion algorithm adapted to the low-light environment is proposed. Firstly, the light distribution of the low-light image is estimated and adjusted based on the grey value probability distribution, resulting in an enhanced VIS image with rich details. Simultaneously, considering the imaging characteristics of the two modal images, a superpixel method is utilized to obtain a saliency map for the IR image, and a multi-scale feature extraction scheme is employed to establish a VIS detail map. Finally, we employ entropy to measure the amount of information in the saliency and detail maps to guide the fusion process. Experimental results from qualitative and quantitative evaluations demonstrate that the fused image conforms to human visual perception, maintains a uniform distribution of pixel intensity, and preserves rich texture features at the edge details.

Image fusion using Y-net-based extractor and global-local discriminator

Although some deep learning-based image fusion approaches have realized promising results, how to extract information-rich features from different source images while preserving them in the fused image with less distortions remains challenging issue that needs to be addressed. Here, we propose a well worked-out GAN-based scheme with multi-scale feature extractor and global-local discriminator for infrared and visible image fusion. We use Y-Net as the backbone architecture to design the generator network, and introduce the residual dense block (RDblock) to yield more realistic fused images for infrared and visible images by learning discriminative multi-scale representations that are closer to the essence of different modal images. During feature reconstruction, the cross-modality shortcuts with contextual attention (CMSCA) are employed to selectively aggregate features at different scales and different levels to construct information-rich fused images with better visual effect. To ameliorate the information content of the fused image, we not only constrain the structure and contrast information using structural similarity index, but also evaluate the intensity and gradient similarities at both feature and image levels. Two global-local discriminators that combine global GAN with PatchGAN as a unified architecture help to dig for finer differences between the generated image and reference images, which force the generator to learn both the local radiation information and pervasive global details in two source images. It is worth mentioning that image fusion is achieved during confrontation without fusion rules. Lots of assessment tests demonstrate that the reported fusion scheme achieves superior performance against state-of-the-art works in meaningful information preservation.

MINM: Marine Intelligent Netting Monitoring Using Multi-Scattering Model and Multi-Space Transformation

Marine intelligent net tank aquaculture monitoring plays an important role in improving aquaculture efficiency, environmental monitoring efficiency, and environmental safety. The underwater environment has complex light, often with problems such as scattering and absorption, resulting in poor image quality, making it difficult to accurately analyze and judge the aquaculture environment. Improving marine intelligent net tank aquaculture monitoring has the following three advantages: 1) better observation and monitoring of the aquaculture process, timely detection of problems and abnormalities, to protect the benefits of aquaculture and product quality. 2) more convenient and rapid monitoring of the aquaculture environment, improving monitoring efficiency and reducing monitoring costs. 3) effective monitoring of the underwater environment around the farm, and timely detection of foreign pollution, harmful substances, and other problems, to protect the safety of the aquaculture environment. Therefore, in order to solve the two degradation problems of scattering and absorption in the process of marine smart net farm monitoring, we propose a marine smart net farm monitoring method using multiple scattering models and multiple spatial transformations, called MINM. Specifically, inspired by the image chromatic aberration correction method, we design a color correction method in the multicolor space, which is implemented by using the Lab and RGB color space by performing contrast-constrained adaptive histogram equalization and gray world assumptions, respectively, to correct color shifts in different color models. Based on this, we propose a de-scattering method using a multi-scattering model, which eliminates the effect of scattering on underwater imaging by embedding a complete multi-scattering underwater imaging model to guide the extraction of different features in the multi-scattering model. To obtain more qualified results, we also propose an efficient perceptual fusion to mix the output of the de-scattering and color correction. Thus, our method can take advantage of multiple scattering models and multiple spatial transformations to effectively improve the visual quality of underwater images, producing enhanced results that fit the complete underwater imaging model and have bio-visual characteristics. In extensive experimental demonstrations, our MINM method has shown higher performance than the state-of-the-art methods in terms of both visual quality and quantitative metrics. All experimental results and datasets in this paper are available from the following website: https://github.com/An-Shunmin/MINM.

Conditional generative model with skip-connection structure for low-light image enhancement

Vision-guided coal mine robots often encounter challenges in low-light environments, capturing images marked by poor visibility and significant loss of detail, which complicates the advancement of smart mine safety production. Despite various advancements, achieving high visibility and detailed images through robotic vision sensors remains a formidable task. This paper introduces a conditional generative model incorporating a skip-connection structure to address these issues. We present a novel approach utilizing conditional Generative Adversarial Networks (GNAs) aimed at improving the visibility of image content. This involves an encoder-decoder framework with skip-connections functioning as the generator, and the discriminator, both tailored to enhance image detail. In addition, we implement loss functions integrated with boundary equilibrium constraints to counter the issue of model collapse in conditional GANs, thereby enhancing training stability and image fidelity. Our experimental findings demonstrate that our model delivers competitive results against other state-of-the-art low-light image enhancement models, alongside a standardized training regimen that ensures rapid and consistent convergence.