Visible Imaging Research Articles

FusionOC: Research on optimal control method for infrared and visible light image fusion

Infrared and visible light image fusion can solve the limitations of single-type visual sensors and can boost the target detection performance. However, since the traditional fusion strategy lacks the controllability and feedback mechanism, the fusion model cannot precisely perceive the relationship between the requirements of the fusion task, the fused image quality, and the source image features. To this end, this paper establishes a fusion model based on the optimal controlled object and control mode called FusionOC. This method establishes two types of mathematical models of the controlled objects by verifying the factors and conflicts affecting the quality of the fused image. It combines the image fusion model with the quality evaluation function to determine the two control factors separately. At the same time, two proportional-integral-derivative (PID) control and regulation modes based on the backpropagation (BP) neural network are designed according to the control factor characteristics. The fusion system can adaptively select the regulation mode to regulate the control factor according to the user requirements or the task to make the fusion system perceive the connection between the fusion task and the result. Besides, the fusion model employs the feedback mechanism of the control system to perceive the feature difference between the fusion result and the source image, realize the guidance of the source image feature to the entire fusion process, and improve the fusion algorithm's generalization ability and intelligence level when handling different fusion tasks. Experimental results on multiple public datasets demonstrate the advantages of FusionOC over advanced methods. Meanwhile, the benefits of our fusion results in object detection tasks have been demonstrated.

Identification of conservation and restoration materials for iron relics through ultraviolet-induced visible luminescence imaging and pyrolysis-gas chromatography/mass spectrometry

Understanding the previous protection and restoration efforts and the current state of cultural relics is essential before compiling a conservation and restoration plan. The lack of detailed archival records for some early conservation operations, the identification of restoration materials necessitates the use of scientific analytical methods. In this study, the composition and spatial distribution of historical restoration materials on five iron relics were investigated through pyrolysis-gas chromatography/mass spectrometry (Py-GC/MS) and ultraviolet-induced visible luminescence imaging (UVL). The relics studied were iron weight 20791, iron adze head 2335, and iron axe 2334 from the Gansu Provincial Museum, iron sword D0008 from the Zhaotong Municipal Museum, and iron sword 450 from the National Museum of China. All five relics had undergone restoration without accompanying archival records. UVL revealed the distribution of various conservation materials. Notably, two distinct layers of the conservation material were observed on iron axe 2334. Differences in the fluorescence color and intensity of iron sword 450 provided information regarding the sampling strategy. The samples were collected under ultraviolet light emitting diode illumination to ensure representativeness and minimize damage to the relics. Through Py-GC/MS, the coating materials for iron weight 20791 and iron adze head 2335 were identified as boiled tung oil mixed with rosin resin. Iron axis 2334 had a two-layer coating: a base layer of boiled tung oil and a top layer of shellac. The coating material for iron sword D0008 was determined to be paraffin wax. The protective layer of iron sword 450 included multiple materials, including shellac, polystyrene, and bisphenol-A-type epoxy resin. This study confirms that UVL combined with Py-GC/MS serves as an effective technique for analyzing historical restoration materials. UVL guided the selection of representative samples for Py-GC/MS, reducing the time and amount of sampling required and minimizing further damage to the relics. This research provides valuable data for the restoration archives of five iron artifacts, offering a scientific basis for conservators to evaluate conservation methods, devise future conservation strategies, and exclude ineffective conservation materials.

Infrared and visible image fusion based on hybrid multi-scale decomposition and adaptive contrast enhancement

Effectively fusing infrared and visible images enhances the visibility of infrared target information while capturing visual details. Balancing the brightness and contrast of the fusion image adequately has posed a significant challenge. Moreover, preserving detailed information in fusion images has been problematic. To address these issues, this paper proposes a fusion algorithm based on multi-scale decomposition and adaptive contrast enhancement. Initially, we present a hybrid multi-scale decomposition method aimed at extracting valuable information comprehensively from the source image. Subsequently, we advance an adaptive base layer optimization approach to regulate the brightness and contrast of the resultant fusion image. Lastly, we design a weight mapping rule grounded in saliency detection to integrate small-scale layers, thereby conserving the edge structure within the fusion outcome. Both qualitative and quantitative experimental results affirm the superiority of the proposed method over 11 state-of-the-art image fusion methods. Our method excels in preserving more texture and achieving higher contrast, which proves advantageous for monitoring tasks.

FusionNGFPE: An Image Fusion Approach Driven by Non-global Fuzzy Pre-enhancement Framework

The majority of prevailing image fusion methods employ a global strategy, often resulting in a reduction of contrast. This study addresses this issue by proposing a novel image fusion approach called FusionNGFPE, specifically designed for the structural characteristics of infrared (IR) imagery. The approach introduces a contrast equalization algorithm based on the Fourth-order Partial Differential Equation (FPDE) to enhance background regions effectively. Considering the inherent differences between IR and visible (VIS) images, we developed a hybrid fusion strategy that combines the Expectation Maximization (EM) algorithm and Principal Component Analysis (PCA). Comparative analysis with state-of-the-art fusion methods shows that our proposed algorithm achieves superior performance in both qualitative and quantitative evaluations. To further demonstrate the practical significance of FusionNGFPE, we integrated this fusion framework into the RGBT target tracking task using the VOT-RGBT and OTCBVS datasets. Extensive comparative experiments confirm that the FusionNGFPE framework integrates seamlessly with the tracking task, significantly improving tracking accuracy across diverse scenarios.

Enhanced antibacterial effect of natural tannin stabilized silver nano particles against human pathogens: A target toward FtsZ proteins

BackgroundTannins are the polyphenolic group of plant compounds having strong antimicrobial potential. Research on human pathogens using silver nanoparticles for antimicrobial purposes has opened up new possibilities in nanomedicine. ObjectiveThe present study was concerned with combining the effectiveness of the two by the green synthesis of silver nanoparticles with plant tannin. MethodsSynthesis of silver nanoparticles was done based on the tannin content of the selected plants. Characterization of the synthesized nanoparticles has been performed through UV–VIS, FTIR spectroscopy, Zeta potential, XRD analysis and FEGSEM imaging. The antibacterial potentiality of the nanoparticles was checked against two most susceptible bacteria Staphylococcus epidermidis and Salmonella typhi. ResultsAmong the studied plants, Phyllanthus emblica showed highest tannin content and best bactericidal properties. The nanoparticles synthesized with P. emblica showed the highest zone of inhibition against the studied bacteria. An in-silico comparative molecular docking study of the bioactive compounds from the selected plants was performed against the FtsZ protein of S. epidermidis and YfdX protein of S. typhi. Three compounds namely Isocorilagin,1(β), 6-di-o-galloylglucose and Hamamelitannin appeared as the best inhibitors of the said proteins and among them, Isocorilagin, a natural tannin showed the best docking score of -10.2 Kcal/mol with FtsZ protein. Further molecular dynamics simulation studies of the FtsZ protein-Isocorilagin complex support its stability indicating Isocorilagin as the natural inhibitor of the pathogenic bacterial protein FtsZ. ConclusionThese results concluded that synthesis of nanoparticles with plant tannin is a cost effective green approach and the synthesised nanoparticles appear as efficient antimicrobial agent.

Prospective Comparison of FOCUS MUSE and Single-Shot Echo-Planar Imaging for Diffusion-Weighted Imaging in Evaluating Thyroid-Associated Ophthalmopathy.

To prospectively compare single-shot (SS) echo-planar imaging (EPI) and field-of-view optimized and constrained undistorted single-shot multiplexed sensitivity-encoding (FOCUS MUSE) for diffusion-weighted imaging (DWI) in evaluating thyroid-associated ophthalmopathy (TAO). SS EPI and FOCUS MUSE DWIs were obtained from 39 patients with TAO (18 male; mean ± standard deviation: 48.3 ± 13.3 years) and 26 healthy controls (9 male; mean ± standard deviation: 43.0 ± 18.5 years). Two radiologists scored the visual image quality using a 4-point Likert scale. The image quality score, signal-to-noise ratio (SNR), contrast-to-noise ratio (CNR), and apparent diffusion coefficient (ADC) of extraocular muscles (EOMs) were compared between the two DWIs. Differences in the ADC of EOMs were also evaluated. The performance of discriminating active from inactive TAO was assessed using receiver operating characteristic curves. The correlation between ADC and clinical activity score (CAS) was analyzed using Spearman correlation. Compared with SS EPI DWI, FOCUS MUSE DWI demonstrated significantly higher image quality scores (P < 0.001), a higher SNR and CNR on the lateral rectus muscle (LRM) and medial rectus muscle (MRM) (P < 0.05), and a non-significant difference in the ADC of the LRM and MRM. Active TAO showed higher ADC than inactive TAO and healthy controls with both SS EPI and FOCUS MUSE DWIs (P < 0.001). Inactive TAO and healthy controls did not show a significant ADC difference with both DWIs. Compared with SS EPI DWI, FOCUS MUSE DWI demonstrated better discrimination of active from inactive TAO (AUC: 0.925 vs. 0.779; P = 0.007). The ADC was significantly correlated with CAS in SS EPI DWI (r = 0.391, P < 0.001) and FOCUS MUSE DWI (r = 0.645, P < 0.001). FOCUS MUSE DWI provides better images for evaluating EOMs and better performance in diagnosing active TAO than SS EPI DWI. The application of FOCUS MUSE will facilitate the DWI evaluation of TAO.

Volcanic tremor associated with successive gas emission activity at a boiling pool: Analyses of seismic array and visible image data recorded at Iwo-Yama in Kirishima Volcanic complex, Japan

Volcanic tremors are often observed during volcanic activity and volcanic eruptions, and their generation processes provide clues for understanding volcanic fluid activity underground and eruption dynamics. However, tremors are characterized by continuous oscillations that mask P- and S-waves; hence few studies have precisely located the source, which is the most fundamental information for understanding the generation mechanism. In this study, we focus on volcanic tremors excited by continuous gas emissions occurring at a vent called Y2a in Iwo-Yama, the Kirishima Volcanic Complex, Japan, to clarify the source process of the tremor as well as gas emission activity. We simultaneously observed the volcanic tremor by deploying a small aperture array consisting of six seismometers and the gas emission activity by using a newly developed visual IoT system that can be operated without commercial electricity. MUSIC analysis locates the tremor at depths ranging from the ground surface to approximately 200 m beneath the Y2a and Y2b vents, which are approximately 30 m apart, for approximately four months from November 2021 to February 2022. The source locations of the tremors in the 2 Hz (1.2–2.6 Hz), 4 Hz (3–4 Hz), and 5 Hz (4–5.5 Hz) ranges show some differences and changes with time. The source location tends to become deeper when the 2 Hz amplitude is large. The infrasound generated by gas emission activity is dominant in the tremor signals, which are recognized in the wave propagation velocity with an acoustic velocity of 330 m/s when the 2 Hz amplitude is small. The visual IoT system succeeded in detecting long-term changes in the gas emission activity, and we found that the 2 Hz amplitude of tremor was well correlated with the amount of hot water in the boiling pool of Y2a, which was controlled by precipitation and evaporation during non-rainy days. From these observations, we infer that the volcanic tremor is generated by resonance of volcanic gas and hot water in a crack-like structure beneath Y2a. The resonance was triggered by the counterforces of the gas emissions in the boiling pool, and the infrasound was dominant during periods of hot water depletion in the boiling pool. Temporal changes in the source depths may be caused by changes in the fluid properties, configuration of the resonator and/or the strengths of the underground sources and infrasound. Our simultaneous observations of seismic array and visual IoT system clarify that even the continuous gas emission activity that looks stable is controlled by external sources such as precipitation.

Fast Sand-dust Image Enhancement Algorithm by Blue Channel Compensation and Guided Image Filtering

Computer vision systems are significantly impacted by the color variation and low visibility of images taken under sand-dust conditions To address the aforementioned issues, we provide a quick and efficient algorithm to improve photos taken under sand-dust situations. The loss value in the blue channel is first adjusted for. Subsequently, the image deteriorated by sand and dust is color corrected using white balance technology. Lastly, the picture contrast and edge correctness are improved using guided image filtering, and the image detail information is improved by using an adaptive method to determine the detail layer's magnification factor. The approach can successfully restore the fading features of sand-dust-deteriorated photos, according to the experimental results on a sizable number of degraded images by the dust in a brief amount of time, and enhance image sharpness. The suggested method may greatly improve the photos taken during sand-dust weather conditions, as shown by experimental findings via qualitative and quantitative assessments, and the outcomes are superior to those of other ways.

Good Practices in using Instructional Images to Enhance Young Children's Linguistic Skills from the Viewpoint of Language Education Experts

The increasing emphasis on improving early childhood language education necessitates a deeper exploration of effective pedagogical tools, particularly visual aids. This study aims to examine the perspectives of language education specialists regarding best practices for employing images to enhance language development in early childhood settings. A targeted sample of 33 experts in language teaching, all working in early childhood education institutions in northern Jordan, was selected for the study. Qualitative research methodology was adopted, utilizing semi-structured interviews and grounded theory for data analysis. The findings identified a series of best practices for the use of instructional images to foster language development in young learners. Key practices included avoiding overly explicit interpretations of images by teachers, cultivating teachers’ confidence in children’s ability to interpret and engage with images independently, providing immediate feedback after children view images, allowing ample opportunities for children to articulate their interpretations of image content, ensuring clear and visible image presentation, selecting images that resonate with children’s real-life experiences, and accommodating individual differences among children. Based on these results, the study put forth several recommendations aimed at enhancing language acquisition through the strategic use of visual aids. The research highlights the critical role of creating a learning environment that effectively integrates images as a tool for promoting language skills in early childhood, encouraging a more interactive and engaging approach to language education.

A Multi-Scale Feature Fusion Based Lightweight Vehicle Target Detection Network on Aerial Optical Images

Vehicle detection with optical remote sensing images has become widely applied in recent years. However, the following challenges have remained unsolved during remote sensing vehicle target detection. These challenges include the dense and arbitrary angles at which vehicles are distributed and which make it difficult to detect them; the extensive model parameter (Param) that blocks real-time detection; the large differences between larger vehicles in terms of their features, which lead to a reduced detection precision; and the way in which the distribution in vehicle datasets is unbalanced and thus not conducive to training. First, this paper constructs a small dataset of vehicles, MiVehicle. This dataset includes 3000 corresponding infrared and visible image pairs, offering a more balanced distribution. In the infrared part of the dataset, the proportions of different vehicle types are as follows: cars, 48%; buses, 19%; trucks, 15%; freight, cars 10%; and vans, 8%. Second, we choose the rotated box mechanism for detection with the model and we build a new vehicle detector, ML-Det, with a novel multi-scale feature fusion triple cross-criss FPN (TCFPN), which can effectively capture the vehicle features in three different positions with an mAP improvement of 1.97%. Moreover, we propose LKC–INVO, which allows involution to couple the structure of multiple large kernel convolutions, resulting in an mAP increase of 2.86%. We also introduce a novel C2F_ContextGuided module with global context perception, which enhances the perception ability of the model in the global scope and minimizes model Params. Eventually, we propose an assemble–disperse attention module to aggregate local features so as to improve the performance. Overall, ML-Det achieved a 3.22% improvement in accuracy while keeping Params almost unchanged. In the self-built small MiVehicle dataset, we achieved 70.44% on visible images and 79.12% on infrared images with 20.1 GFLOPS, 78.8 FPS, and 7.91 M. Additionally, we trained and tested our model on the following public datasets: UAS-AOD and DOTA. ML-Det was found to be ahead of many other advanced target detection algorithms.

AerialIRGAN: unpaired aerial visible-to-infrared image translation with dual-encoder structure

Due to the high cost of equipment and the constraints of shooting conditions, obtaining aerial infrared images of specific targets is very challenging. Most methods using Generative Adversarial Networks for translating visible images to infrared greatly depend on registered data and struggle to handle the diversity and complexity of scenes in aerial infrared targets. This paper proposes a one side end-to-end unpaired aerial visible-to-infrared image translation algorithm, termed AerialIRGAN. AerialIRGAN introduces a dual-encoder structure, where one encoder is designed based on the Segment Anything Model to extract deep semantic features from visible images, and the other encoder is designed based on UniRepLKNet to capture small-scale patterns and sparse patterns from visible images. Subsequently, AerialIRGAN constructs a bridging module to deeply integrate the features of both encoders and their corresponding decoders. Finally, AerialIRGAN proposes a structural appearance consistency loss to guide the synthetic infrared images to maintain the structure of the source image while possessing distinct infrared characteristics. The experimental results show that compared to the existing typical infrared image generation algorithms, the proposed method can generate higher-quality infrared images and achieve better performance in both subjective visual description and objective metric evaluation.

Observations of Water Frost on Mars With THEMIS: Application to the Presence of Brines and the Stability of (Sub)Surface Water Ice

AbstractCharacterizing the exchange of water between the Martian atmosphere and the (sub)surface is a major challenge for understanding the mechanisms that regulate the water cycle. Here we present a new data set of water ice detected on the Martian surface with the Thermal Emission Imaging System (THEMIS). The detection is based on the correlation between bright blue‐white patterns in visible images and a temperature measured in the infrared that is too warm to be associated with ice and interpreted instead as water ice. Using this method, we detect ice down to 21.4°S, 48.4°N, on pole‐facing slopes at mid‐latitudes, and on any surface orientation poleward of 45° latitude. Water ice observed with THEMIS is most likely seasonal rather than diurnal. Our data set is consistent with near‐infrared frost detections and predictions by the Mars Planetary Climate Model. Water frost average temperature is 170 K, and the maximum temperature measured is 243 K, lower than the water ice melting point. Melting of pure water ice on the surface is unlikely due to cooling by latent heat during its sublimation. However, 243 THEMIS images show frosts that are hot enough to form brines if salts are present on the surface. The water vapor pressure at the surface, calculated from the ice temperature, indicates a dry atmosphere in early spring, during the recession of the ice cap. The large amount of water vapor released by the sublimation of warm frost cannot stabilize subsurface ice at mid‐latitudes.

Exploring fusion domain: Advancing infrared and visible image fusion via IDFFN-GAN

Infrared (IR) and visible (VI) images are crucial in applications such as surveillance and night vision, where each modality provides complementary information—IR captures thermal details, while VI captures textures. Fusing these images is essential to combine their strengths, resulting in a more comprehensive and informative image. In this work, we introduce the “fusion domain” concept, a unique distribution that optimally blends IR and VI features. Our primary contribution is developing the Intermediate Domain Feature Fusion Network (IDFFN), which employs an MLP to learn the optimal domain factor for this fusion. Integrated with the IDFFN-GAN’s dual discriminators, our approach refines the fusion process to produce images that better preserve thermal and texture details from the proposed fusion and gradient domains. Experimental results demonstrate that our method achieves a 5% boost in Entropy (EN), a 50% rise in Spatial Frequency (SF), a 58% improvement in Average Gradient (AG), a 10% enhancement in Standard Deviation (SD), and a remarkable 198% increase in NAB/F compared to state-of-the-art techniques, ensuring superior retention of specific features from both modalities.

MMAE: A universal image fusion method via mask attention mechanism

As an important carrier of data, images contain a huge amount of information. The purpose of image fusion is to integrate the information from source images into a single image. Since the source images are from the same scene, there is much redundant information between them. Common fusion methods do not filter this information and the fusion process is often disturbed. This leads to degradation in the quality of the reconstructed fused image. To solve this problem, this paper explores the strategies of information filtering and fusion control, and proposes a universal image fusion method based on the mask attention mechanism. It can be divided into pre-training stage and formal fusion stage. In the pre-training stage, coarse-grained mask maps are generated which are employed to improve the mask autoencoder and the mask attention mechanism. In the image formal fusion stage, with the help of coarse-grained mask maps, the mask autoencoder changes the process of random masking and discards redundant features between source images. Meanwhile the mask attention mechanism focuses on the distinctions between various source images and retains effective complementary information. Qualitative and quantitative extension experiments on different modal datasets validate the applicability of the model in multi-focus image fusion, infrared and visible image fusion, and medical image fusion. Our method achieves excellent performance in all these tasks and performs better than existing fusion methods. Our code is publicly available at https://github.com/xiangxiang-wang/MMAE.

Combining UAV Multi-Source Remote Sensing Data with CPO-SVR to Estimate Seedling Emergence in Breeding Sunflowers

In order to accurately obtain the seedling emergence rate of breeding sunflower and to assess the quality of sowing as well as the merit of sunflower varieties, a method of extracting the sunflower seedling emergence rate using multi-source remote sensing information from unmanned aerial vehicles is proposed. Visible and multispectral images of sunflower seedlings were acquired using a UAV. The thresholding method was used to segment the excess green image of the visible image into vegetation and non-vegetation, to obtain the center point of the vegetation to generate a buffer, and to mask the visible image to achieve weed removal. The components of color models such as the hue–saturation value (HSV), green-relative color space (YCbCr), cyan-magenta-yellow-black (CMYK), and CIELAB color space (L*A*B) models were compared and analyzed. The A component of the L*A*B model was preferred for the optimization of K-means clustering to segment sunflower seedlings and mulch using the genetic algorithm, and the segmentation accuracy was improved by 4.6% compared with the K-means clustering algorithm. All told, 10 geometric features of sunflower seedlings were extracted using segmented images, and 10 vegetation indices and 48 texture features of sunflower seedlings were calculated based on multispectral images. The Pearson’s correlation coefficient method was used to filter the three types of features, and the geometric feature set, the vegetation index set, the texture feature set, and the preferred feature set were constructed. The construction of a sunflower plant number estimation model using the crested porcupine optimizer–support vector machine is proposed and compared with the sunflower plant number estimation models constructed based on decision tree regression, BP neural network, and support vector machine regression. The results show that the accuracy of the model based on the preferred feature set is higher than that of the other three feature sets, indicating that feature screening can improve the accuracy and stability of models; assessed using the CPO-SVR model, the accuracy of the preferred feature set was the highest, with an R² of 0.94, an RMSE of 5.16, and an MAE of 3.03. Compared to the SVR model, the value of the R2 is improved by 3.3%, the RMSE decreased by 18.3%, and the MAE decreased by 18.1%. The results of the study can be cost-effective, accurate, and reliable in terms of obtaining the seedling emergence rate of sunflower field breeding.

PTPFusion: A progressive infrared and visible image fusion network based on texture preserving

Infrared and visible image fusion aims to provide a more comprehensive image for downstream tasks by highlighting the main target and maintaining rich texture information. Image fusion methods based on deep learning suffer from insufficient multimodal information extraction and texture loss. In this paper, we propose a texture-preserving progressive fusion network (PTPFusion) to extract complementary information from multimodal images to solve these issues. To reduce image texture loss, we design multiple consecutive texture-preserving blocks (TPB) to enhance fused texture. The TPB can enhance the features by using a parallel architecture consisting of a residual block and derivative operators. In addition, a novel cross-channel attention (CCA) fusion module is developed to obtain complementary information by modeling global feature interactions via cross-queries mechanism, followed by information fusion to highlight the feature of the salient target. To avoid information loss, the extracted features at different stages are merged as the output of TPB. Finally, the fused image will be generated by the decoder. Extensive experiments on three datasets show that our proposed fusion algorithm is better than existing state-of-the-art methods.

SDAM: A dual attention mechanism for high-quality fusion of infrared and visible images.

Image fusion of infrared and visible images to obtain high-quality fusion images with prominent infrared targets has important applications in various engineering fields. However, current fusion processes encounter problems such as unclear texture details and imbalanced infrared targets and texture detailed information, which lead to information loss. To address these issues, this paper proposes a method for infrared and visible image fusion based on a specific dual-attention mechanism (SDAM). This method employs an end-to-end network structure, which includes the design of channel attention and spatial attention mechanisms. Through these mechanisms, the method can fully exploit the texture details in the visible images while preserving the salient information in the infrared images. Additionally, an optimized loss function is designed to combine content loss, edge loss, and structure loss to achieve better fusion effects. This approach can fully utilize the texture detailed information of visible images and prominent information in infrared images, while maintaining better brightness and contrast, which improves the visual effect of fusion images. Through conducted ablation experiments and comparative evaluations on public datasets, our research findings demonstrate that the SDAM method exhibits superior performance in both subjective and objective assessments compared to the current state-of-the-art fusion methods.

MSADRCN: meta-learning based joint super-resolution fusion of infrared and visible images

MSADRCN: meta-learning based joint super-resolution fusion of infrared and visible images

An in-vehicle real-time infrared object detection system based on deep learning with resource-constrained hardware

Advanced driver assistance systems primarily rely on visible images for information. However, in low-visibility weather conditions, such as heavy rain or fog, visible images struggle to capture road conditions accurately. In contrast, infrared (IR) images can overcome this limitation, providing reliable information regardless of external lighting. Addressing this problem, we propose an in-vehicle IR object detection system. We optimize the you only look once (YOLO) v4 object detection algorithm by replacing its original backbone with MobileNetV3, a lightweight feature extraction network, resulting in the MobileNetV3-YOLOv4 model. Furthermore, we replace traditional pre-processing methods with an Image Enhancement Conditional Generative Adversarial Network inversion algorithm to enhance the pre-processing of the input IR images. Finally, we deploy the model on the Jetson Nano, an edge device with constrained hardware resources. Our proposed method achieves an 82.7% mean Average Precision and a frame rate of 55.9 frames per second on the FLIR dataset, surpassing state-of-the-art methods. The experimental results confirm that our approach provides outstanding real-time detection performance while maintaining high precision.

IV-YOLO: A Lightweight Dual-Branch Object Detection Network.

With the rapid growth in demand for security surveillance, assisted driving, and remote sensing, object detection networks with robust environmental perception and high detection accuracy have become a research focus. However, single-modality image detection technologies face limitations in environmental adaptability, often affected by factors such as lighting conditions, fog, rain, and obstacles like vegetation, leading to information loss and reduced detection accuracy. We propose an object detection network that integrates features from visible light and infrared images-IV-YOLO-to address these challenges. This network is based on YOLOv8 (You Only Look Once v8) and employs a dual-branch fusion structure that leverages the complementary features of infrared and visible light images for target detection. We designed a Bidirectional Pyramid Feature Fusion structure (Bi-Fusion) to effectively integrate multimodal features, reducing errors from feature redundancy and extracting fine-grained features for small object detection. Additionally, we developed a Shuffle-SPP structure that combines channel and spatial attention to enhance the focus on deep features and extract richer information through upsampling. Regarding model optimization, we designed a loss function tailored for multi-scale object detection, accelerating the convergence speed of the network during training. Compared with the current state-of-the-art Dual-YOLO model, IV-YOLO achieves mAP improvements of 2.8%, 1.1%, and 2.2% on the Drone Vehicle, FLIR, and KAIST datasets, respectively. On the Drone Vehicle and FLIR datasets, IV-YOLO has a parameter count of 4.31 M and achieves a frame rate of 203.2 fps, significantly outperforming YOLOv8n (5.92 M parameters, 188.6 fps on the Drone Vehicle dataset) and YOLO-FIR (7.1 M parameters, 83.3 fps on the FLIR dataset), which had previously achieved the best performance on these datasets. This demonstrates that IV-YOLO achieves higher real-time detection performance while maintaining lower parameter complexity, making it highly promising for applications in autonomous driving, public safety, and beyond.