Visible Imaging Research Articles

해부학적 카메라와 의학 이미지에 관한 이론적 고찰

This is an attempt to consider how, when physical optical apparatus penetrate the body, the visibility of internal body images, considered invisible, leads to the formation of a subject with a certain kind of ‘Optical Unconscious’ in humans. The internal body images refer to the “anatomical camera” based on medical devices such as X-rays, endoscopes, ultrasounds, CT/MRI/PET, and the “medical images” they represent. Unlike medical images that inevitably require interpretation, the emergence of internal body images, not easily accessible in everyday life, crossing the boundaries between film cameras and medical cameras, can be considered a symptomatic phenomenon. Even though the appearance of these images has become frequent on screens around the turn of the 2020s, they cannot simply be seen as new images. The reason is that the history of optical apparatus resonates with humans' desire to capture the body and movement. In terms of media archaeology, reviewing individual theories about scattered internal body images in the field of film history, and reconciling the discontinuity of anatomical cameras and medical images, reveal the affinity and complementarity of optical apparatus that were divided into science and art, while also gathering the origins of various apparatus that were concealed or unified in film history.

Generating bimodal privacy-preserving data for face recognition

The performance of state-of-the-art face recognition systems depends crucially on the availability of large-scale training datasets. However, increasing privacy concerns nowadays accompany the collection and distribution of biometric data, which has already resulted in the retraction of valuable face recognition datasets. The use of synthetic data represents a potential solution, however, the generation of privacy-preserving facial images useful for training recognition models is still an open problem. Generative methods also remain bound to the visible spectrum, despite the benefits that multispectral data can provide. To address these issues, we present a novel identity-conditioned generative framework capable of producing large-scale recognition datasets of visible and near-infrared privacy-preserving face images. The framework relies on a novel identity-conditioned dual-branch style-based generative adversarial network to enable the synthesis of aligned high-quality samples of identities determined by features of a pretrained recognition model. In addition, the framework incorporates a novel filter to prevent samples of privacy-breaching identities from reaching the generated datasets and improve both identity separability and intra-identity diversity. Extensive experiments on six publicly available datasets reveal that our framework achieves competitive synthesis capabilities while preserving the privacy of real-world subjects. The synthesized datasets also facilitate training more powerful recognition models than datasets generated by competing methods or even small-scale real-world datasets. Employing both visible and near-infrared data for training also results in higher recognition accuracy on real-world visible spectrum benchmarks. Therefore, training with multispectral data could potentially improve existing recognition systems that utilize only the visible spectrum, without the need for additional sensors.

Initial Fire Detection Method for Intelligent CCTV Using Deep Learning Technology

Pre-prediction and early detection are important for the prevention of fires. Hence, the integration of intelligent CCTV image analysis technology, deep learning, and artificial intelligence is actively being studied. Intelligent CCTV can analyze visible light images of fires, but the accuracies of the detection targets are low. To solve this problem, a dual sensing experimental device was constructed by combining a visible light sensor and a thermal image sensor, and an algorithm was implemented to predict fire events. An algorithm was constructed to detect three classes (Fire, Smoke, and Person) through RGB sensors and thermal imaging sensors. Additionally, a CNN deep learning method and the YOLOv5 method were used to build a classification neural network. The dual sensing device was able to act as a trigger to predict and detect a fire based on the size of the flame and changes in temperature.

LRFE-CL: A self-supervised fusion network for infrared and visible image via low redundancy feature extraction and contrastive learning

Aiming to amalgamate the distinct yet complementary attributes inherent to infrared and visible spectra, Infrared and Visible Image Fusion (IVIF) endeavors to preserve an optimal intensity distribution while curtailing the superfluous data within the resultant image. Nonetheless, it is an incontrovertible reality that the majority of extant methods are prone to engendering an undesirable redundancy of features, in addition to a marked degradation throughout the fusion process. To address the issue, we proposed a self-supervised fusion network for infrared and visible image via low redundancy feature extraction and contrastive learning, termed as LRFE-CL. Specifically, the Low Redundancy Feature Extraction Network (SSFN-LRAE) has been developed to effectively extract low-redundancy features from infrared and visible images using the self-supervised strategy and the Channel Shift Attention Module (CSAM). Subsequently, the Contrastive Fusion Decoder (CFDN) is designed to fuse multi-layer features extracted from the SSFN-LRAE. To prevent feature degradation, a novel regularized contrastive loss mechanism targeting multi-layer features has been formulated to optimize our CFDN, ensuring the fusion of comprehensive texture details and brightness information. Our comprehensive experimental results unequivocally demonstrate the exceptional performance of our method, both in terms of visual effects and fusion performance.

MLP-based multimodal tomato detection in complex scenarios: Insights from task-specific analysis of feature fusion architectures

Accurate and efficient tomato detection is essential for the practical deployment of robotic picking in practical agricultural applications, but it still remains significantly challenging to detect tomatoes in complex scenarios with fluctuating light, overlapping fruits, and occlusion from branches and leaves when solely using RGB images. The recent development of RGB-D sensors has brought one promising opportunity to adopt multimodal fusion for implementing high-quality fruit detection. However, the feasibility of the existing multimodal fusion and feature extraction architectures for lightweight tomato detection tasks, especially in complex agricultural scenarios, raises questions that need to be explored. As a remedy, we proposed a multimodal fusion encoder that leveraged depth and near-infrared modalities to assist RGB images in making full use of multimodal data. Moreover, the encoder contained a plug-and-play structure capable of being implemented as MLP-based (Multi-Layer Perceptron), ViT-based (Vision Transformer), or CNN-based (Convolutional Neural Networks) architectures. Furthermore, we developed a lightweight experimental detection framework based on YOLOv7-tiny by means of integrating the multimodal fusion encoder, and YOLO-DNA (Depth and Near-infrared Assisted) was put forward based on the MLP-based architecture after conducting comprehensive analysis of the aforementioned three architectures. In addition, a tomato multimodal dataset containing visible, depth, and near-infrared images was established. Experimental results demonstrated that YOLO-DNA achieved mAP0.5 of 98.13% and mAP0.5:0.95 of 74.0%, an average increase of 5.01% in mAP0.5 and 14.55% in mAP0.5:0.95 over mainstream lightweight detection models, with a detection speed of 37.12 FPS, meeting the demand of real-time tomato detection. This finding has the potential to advance research on fruit detection in the field of intelligent agricultural harvesting.

Measurement method for live chicken shank length based on improved ResNet and fused multi-source information

Phenotypic parameters are crucial reference indicators in poultry breeding. However, the chickens shank length is still manually measured, which is time-consuming and labor-intensive. Additionally, the measurement results are difficult to unify due to the subjective factors of different individuals. To address this issue, this paper proposed a method for live chicken shank length measurement (SLM). It enriches chicken shank feature by fusing visible images and infrared images. The fusion images are then input into a deep regression model based on the improved ResNet. The measurement model used ResNet as its backbone and introduces Squeeze-and-Excitation (SE) blocks and a Spatial Pyramid Pooling (SPP) block, resulting in more precise and stable shank length measurements. The average coefficient of variation, average floating error, average standard deviation and Pearson correlation coefficient for shank length measurements using the fusion images are 0.21 %, 0.49 %, 0.181 mm and 0.996, respectively, compared with using single visible or infrared image, the accuracy and stability are obviously improved. That indicated combining deep learning model and fusion information, the SLM proposed in this paper can achieve a more precise, reliable and standardized measurement of live chicken shank length.

Misalignment fusion network for parsing infrared and visible urban scenes

In the realm of infrared and visible scene parsing, satisfactory performance has been achieved by leveraging the complementary nature of infrared and visible imaging modalities. Existing methods have employed various strategies to fuse cross-modality features. However, these strategies typically integrate features at the same levels (the depths of network) neglecting the potential interactions across different levels. To address this limitation, we introduce a novel concept called misalignment fusion, which involves merging multimodality feature maps at distinct levels. In line with this concept, we propose a misalignment fusion network (MFNet) specifically designed for the task of infrared and visible urban scene parsing. Our network incorporates a misalignment-guided fusion module to integrate cross-modality features, as well as an adaptive refined selective fusion module to combine the predicted segmentation maps obtained from two parallel-branch decoders. Numerous experiments have been conducted to evaluate the performance of the proposed MFNet. The results consistently demonstrate that our approach surpasses existing state-of-the-art methods in the field of infrared and visible urban scene parsing.

Self-supervised learning for RGB-D object tracking

Recently, there has been a growing interest in RGB-D object tracking thanks to its promising performance achieved by combining visual information with auxiliary depth cues. However, the limited volume of annotated RGB-D tracking data for offline training has hindered the development of a dedicated end-to-end RGB-D tracker design. Consequently, the current state-of-the-art RGB-D trackers mainly rely on the visual branch to support the appearance modelling, with the depth map utilised for elementary information fusion or failure reasoning of online tracking. Despite the achieved progress, the current paradigms for RGB-D tracking have not fully harnessed the inherent potential of depth information, nor fully exploited the synergy of vision-depth information. Considering the availability of ample unlabelled RGB-D data and the advancement in self-supervised learning, we address the problem of self-supervised learning for RGB-D object tracking. Specifically, an RGB-D backbone network is trained on unlabelled RGB-D datasets using masked image modelling. To train the network, the masking mechanism creates a selective occlusion of the input visible image to force the corresponding aligned depth map to help with discerning and learning vision-depth cues for the reconstruction of the masked visible image. As a result, the pre-trained backbone network is capable of cooperating with crucial visual and depth features of the diverse objects and background in the RGB-D image. The intermediate RGB-D features output by the pre-trained network can effectively be used for object tracking. We thus embed the pre-trained RGB-D network into a transformer-based tracking framework for stable tracking. Comprehensive experiments and the analysis of the results obtained on several RGB-D tracking datasets demonstrate the effectiveness and superiority of the proposed RGB-D self-supervised learning framework and the following tracking approach.

AMPLIFYING AQUATIC VISUALS

Our proposed fusion-based approach aims to combat the degradation of underwater images, addressing issues such as diminished colors and indistinguishable objects. Leveraging contrast stretching and Auto White Balance, the technique significantly improves contrast and color, offering a straightforward yet effective solution to enhance visibility in aquatic images, crucial for applications like video surveillance in outdoor computer vision systems. This straightforward approach plays a pivotal role in enhancing image visibility, contributing significantly to applications like video surveillance and other outdoor computer vision systems. Our dehazing process builds upon two essential statistical observations related to haze-free images and haze. By applying dark channel prior and guided filter to the decomposed image, we effectively estimate atmosphere light, resulting in a dehazed output. This method tackles the intricacies of haze, contributing to the generation of high-quality images with improved details and clarity. Beyond image enhancement, our approach showcases versatility by extending into object classification, demonstrating the broader impact and potential of our method in the realm of outdoor computer vision systems.

Research on Methods of Micro-target Detection in Various Application Scenarios

This paper provides a comprehensive introduction and comparison of the research on small target detection algorithms in different application scenarios and systematically explores the application of deep learning algorithms for small target detection in remote sensing imagery, visible imagery, and infrared imagery. In remote sensing images, the Deconvolution R-CNN algorithm is used to improve the accuracy of small target detection by adding a deconvolution layer. In the field of visible images, a Gaussian Mixture Model (GMM)-based approach is introduced to achieve fast detection of small targets with SIFT features and a GMM target detector. In infrared images, a comprehensive algorithm adapted to low signal-to-noise ratio and complex background is proposed to automatically detect tiny targets through background suppression and threshold selection optimization. In this paper, we compare and analyze the performance of these methods in different scenarios, and propose directions for future research, including expanding the training dataset, applying super-resolution methods to improve the image resolution, and enhancing the universality of the algorithms. This paper provides a comprehensive overview of research in the field of small target detection, as well as an outlook on future research directions, which provides a useful reference for the further development of the field.

Concrete Highway Crack Detection Based on Visible Light and Infrared Silicate Spectrum Image Fusion.

Cracks provide the earliest and most immediate visual response to structural deterioration of asphalt pavements. Most of the current methods for crack detection are based on visible light sensors and convolutional neural networks. However, such an approach obviously limits the detection to daytime and good lighting conditions. Therefore, this paper proposes a crack detection technique cross-modal feature alignment of YOLOV5 based on visible and infrared images. The infrared spectrum characteristics of silicate concrete can be an important supplement. The adaptive illumination-aware weight generation module is introduced to compute illumination probability to guide the training of the fusion network. In order to alleviate the problem of weak alignment of the multi-scale feature map, the FA-BIFPN feature pyramid module is proposed. The parallel structure of a dual backbone network takes 40% less time to train than a single backbone network. As determined through validation on FLIR, LLVIP, and VEDAI bimodal datasets, the fused images have more stable performance compared to the visible images. In addition, the detector proposed in this paper surpasses the current advanced YOLOV5 unimodal detector and CFT cross-modal fusion module. In the publicly available bimodal road crack dataset, our method is able to detect cracks of 5 pixels with 98.3% accuracy under weak illumination.

Unpaired high-quality image-guided infrared and visible image fusion via generative adversarial network

Current infrared and visible image fusion (IVIF) methods lack ground truth and require prior knowledge to guide the feature fusion process. However, in the fusion process, these features have not been placed in an equal and well-defined position, which causes the degradation of image quality. To address this challenge, this study develops a new end-to-end model, termed unpaired high-quality image-guided generative adversarial network (UHG-GAN). Specifically, we introduce the high-quality image as the reference standard of the fused image and employ a global discriminator and a local discriminator to identify the distribution difference between the high-quality image and the fused image. Through adversarial learning, the generator can generate images that are more consistent with high-quality expression. In addition, we also designed the laplacian pyramid augmentation (LPA) module in the generator, which integrates multi-scale features of source images across domains so that the generator can more fully extract the structure and texture information. Extensive experiments demonstrate that our method can effectively preserve the target information in the infrared image and the scene information in the visible image and significantly improve the image quality.

A weight induced contrast map for infrared and visible image fusion

Fusion involves merging details from both infrared (IR) and visible images to generate a unified composite image that offers richer and more valuable information than either individual image. Surveillance, navigation, remote sensing, and military applications require various imaging modalities, including visible and IR, to oversee specific scenes. These sensors provide supplementary data and improve situational understanding, so it is essential to fuse this information into a single image. Fusing IR and visible images presents several challenges due to the differences in imaging modalities, data characteristics, and the need for accurate and meaningful integration of information. In this context, a novel image fusion architecture focuses on enhancing prominent targets, with the objective of integrating thermal information from infrared images into visible images while preserving textural details within the visible images. Initially, in the proposed algorithm, the images from different sensors are divided into components of high and low frequencies using a Guided filter and an Average filter, respectively. A unique contrast detection mechanism is proposed that is capable of preserving the contrast information from the original images. Further, the contrast details of the IR and visible images are enhanced using local standard deviation filtering and local range filtering, respectively. We have developed a new weight map construction strategy that can effectively preserve the supplemental data from both the original images. These weights and gradient details of the source images are utilized to preserve the salient feature details of the images acquired from the various modalities. A decision-making approach is utilized among the high-frequency components of the original images to retain the prominent feature details of the source images. Finally, the salient feature details and the prominent feature details are integrated to generate the fused image. The developed technique is validated using both subjective and quantitative perspectives. The developed approaches provide EN, MI, Nabf, and SD of 6.86815, 13.73269, 0.15390, and 78.16158 respectively against deep learning-based approaches. Also, the proposed algorithm provides EN, MI, Nabf, FMIw, and Qabf against 6.86815, 13.73269, 0.15390, 0.41634 and 0.47196 respectively against existing traditional fusion methods. It is observed that the developed technique provides adequate accuracy against twenty-seven state-of-the-art techniques.

An integrated enhancement method to improve image visibility and remove color cast for sand-dust image

AbstractSand-dust color images suffer from poor image visibility and serious color cast that significantly affect the performance of outdoor computer vision systems. Therefore, this paper proposes an integrated enhancement method for the sand-dust image. The proposed method improves the image visibility and removes the sand-dust color cast. It integrates two main processes in two different color models. The adaptive gray world-blue channel (AGW-B) is utilized in the Red-Green-Blue (RGB) color model to remove the sand-dust color cast. Then, the contrast limited adaptive histogram equalization with normalized intensity and saturation correction (CLAHE-NISC) is conducted in a Hue-Saturation-Intensity (HSI) color model to enhance the image visibility. Sand-dust images with weak, medium, and extreme sand-dust color casts were utilized in the subjective and objective evaluations. Results show that the proposed method produced better and clearer enhanced images than the other four current sand-dust image enhancement methods.

Inter-Channel Correlation Modeling and Improved Skewed Histogram Shifting for Reversible Data Hiding in Color Images

Reversible data hiding (RDH) is an advanced data protection technology that allows the embedding of additional information into an original digital medium while maintaining its integrity. Color images are typical carriers for information because of their rich data content, making them suitable for data embedding. Compared to grayscale images, color images with their three color channels (RGB) enhance data embedding capabilities while increasing algorithmic complexity. When implementing RDH in color images, researchers often exploit the inter-channel correlation to enhance embedding efficiency and minimize the impact on image visual quality. This paper proposes a novel RDH method for color images based on inter-channel correlation modeling and improved skewed histogram shifting. Initially, we construct an inter-channel correlation model based on the relationship among the RGB channels. Subsequently, an extended method for calculating the local complexity of pixels is proposed. Then, we adaptively select the pixel prediction context and design three types of extreme predictors. The improved skewed histogram shifting method is utilized for data embedding and extraction. Finally, experiments conducted on the USC-SIPI and Kodak datasets validate the superiority of our proposed method in terms of image fidelity.

The Deep Learning Methods for Fusion Infrared and Visible Images: A Survey

The Deep Learning Methods for Fusion Infrared and Visible Images: A Survey

Scene recovery: Combining visual enhancement and resolution improvement

Visibility enhancement of outdoor images under complex imaging conditions has been a crucial task for computer vision and received growing attention. However, existing image enhancement methods could result in typical block-like artifacts or color distortion. The undesirable impurities might also be significantly magnified after the enhancement task, further reducing the image quality. For enhancing and super-resolving complex real-world degradation, we propose a simultaneous visual enhancement and resolution improvement (VERI) variational scene recovery model for jointly enhancing image visibility and improving the resolution of the degraded image. Particularly, we estimate the scattering light map for degradation images to achieve clean scene radiance and simultaneously seek a high-quality image through a deep super-resolution network. The semi-proximal alternating direction method of multipliers (sPADMM) algorithm is employed for efficiently solving the minimization problems in the proposed model. Extensive experiments illustrate the effectiveness and robustness of the proposed method in dealing with various scenes, such as haze, sandstorm, underwater or low illumination.

Deep Learning-Based Forest Fire Risk Research on Monitoring and Early Warning Algorithms

With the development of image processing technology and video analysis technology, forest fire monitoring technology based on video recognition is more and more important in the field of forest fire prevention and control. The objects currently applied to forest fire video image monitoring system monitoring are mainly flames and smoke. This paper proposes a forest fire risk monitoring and early warning algorithm, which integrates a deep learning model, infrared monitoring and early warning, and forest fire weather index. The algorithm first obtains the current visible image and infrared image of the same forest area, utilizing a smoke detection model based on deep learning to detect smoke in the visible image, and obtains the confidence level of the occurrence of fire in said visible image. Then, it determines whether the local temperature value of said infrared image exceeds a preset warning value, and obtains a judgment result based on the infrared image. It calculates again a current FWI based on environmental data, and determines a current fire danger level based on the current FWI. Finally, it determines whether or not to carry out a fire warning based on said fire danger level, said confidence level of the occurrence of fire in said visible image, and said judgment result based on the infrared image. The experimental results show that the accuracy of the algorithm in this paper reaches 94.12%, precision is 96.1%, recall is 93.67, and F1-score is 94.87. The algorithm in this paper can improve the accuracy of smoke identification at the early stage of forest fire danger occurrence, especially by excluding the interference caused by clouds, fog, dust, and so on, thus improving the fire danger warning accuracy.

RSTFusion: an end-to-end fusion network for infrared and visible images based on residual swin transfomer

RSTFusion: an end-to-end fusion network for infrared and visible images based on residual swin transfomer

S2CANet: A self-supervised infrared and visible image fusion based on co-attention network

Existing methods for infrared and visible image fusion (IVIF) often overlook the analysis of common and distinct features among source images. Consequently, this study develops A self-supervised infrared and visible image fusion based on co-attention network, incorporating auxiliary networks and backbone networks in its design. The primary concept is to transform both common and distinct features into common features and reconstructed features, subsequently deriving the distinct features through their subtraction. To enhance the similarity of common features, we designed the fusion block based on co-attention (FBC) module specifically for this purpose, capturing common features through co-attention. Moreover, fine-tuning the auxiliary network enhances the image reconstruction effectiveness of the backbone network. It is noteworthy that the auxiliary network is exclusively employed during training to guide the self-supervised completion of IVIF by the backbone network. Additionally, we introduce a novel estimate for weighted fidelity loss to guide the fused image in preserving more brightness from the source image. Experiments conducted on diverse benchmark datasets demonstrate the superior performance of our S2CANet over state-of-the-art IVIF methods.