Proposed Fusion Method Research Articles

Audio-Visual Kinship Verification: A New Dataset and a Unified Adaptive Adversarial Multimodal Learning Approach.

Facial kinship verification refers to automatically determining whether two people have a kin relation from their faces. It has become a popular research topic due to potential practical applications. Over the past decade, many efforts have been devoted to improving the verification performance from human faces only while lacking other biometric information, for example, speaking voice. In this article, to interpret and benefit from multiple modalities, we propose for the first time to combine human faces and voices to verify kinship, which we refer it as the audio-visual kinship verification study. We first establish a comprehensive audio-visual kinship dataset that consists of familial talking facial videos under various scenarios, called TALKIN-Family. Based on the dataset, we present the extensive evaluation of kinship verification from faces and voices. In particular, we propose a deep-learning-based fusion method, called unified adaptive adversarial multimodal learning (UAAML). It consists of the adversarial network and the attention module on the basis of unified multimodal features. Experiments show that audio (voice) information is complementary to facial features and useful for the kinship verification problem. Furthermore, the proposed fusion method outperforms baseline methods. In addition, we also evaluate the human verification ability on a subset of TALKIN-Family. It indicates that humans have higher accuracy when they have access to both faces and voices. The machine-learning methods could effectively and efficiently outperform the human ability. Finally, we include the future work and research opportunities with the TALKIN-Family dataset.

Baby cry recognition based on WOA-VMD and an improved Dempster–Shafer evidence theory

Background and objectiveConflict may happen when more than one classifier is used to perform prediction or classification. The recognition model error leads to conflicting evidence. These conflicts can cause decision errors in a baby cry recognition and further decrease its recognition accuracy. Thus, the objective of this study is to propose a method that can effectively minimize the conflict among deep learning models and improve the accuracy of baby cry recognition. MethodsAn improved Dempster–Shafer evidence theory (DST) based on Wasserstein distance and Deng entropy was proposed to reduce the conflicts among the results by combining the credibility degree between evidence and the uncertainty degree of evidence. To validate the effectiveness of the proposed method, examples were analyzed, and applied in a baby cry recognition. The Whale optimization algorithm-Variational mode decomposition (WOA-VMD) was used to optimally decompose the baby cry signals. The deep features of decomposed components were extracted using the VGG16 model. Long Short-Term Memory (LSTM) models were used to classify baby cry signals. An improved DST decision method was used to obtain the decision fusion. ResultsThe proposed fusion method achieves an accuracy of 90.15% in classifying three types of baby cry. Improvement between 2.90% and 4.98% was obtained over the existing DST fusion methods. Recognition accuracy was improved by between 5.79% and 11.53% when compared to the latest methods used in baby cry recognition. ConclusionThe proposed method optimally decomposes baby cry signal, effectively reduces the conflict among the results of deep learning models and improves the accuracy of baby cry recognition.

Feature Extraction and Classification Analysis of High-Dimensional Biological Data Based on Dimensionality Reduction Fusion Method

Identification and extraction of characterized information from complex high-dimensional biological data is a very meaningful issue. The dimensionality reduction fusion method based on random forest, feature extraction and neural network is proposed to recognize and classify two datasets of mRNA and lncRNA. It is shown that the proposed fusion method achieved accurate identification/classification of cancer and non-cancer groups, and simultaneously selected identity variables that have biological relevance to lung cancer (tumor) as potential biomarkers from a large number of variables. It is considered as an effective tool and theoretical support for lung cancer identification in clinical application, and it can be extended to other kinds of cancer or biological data. Ultimately, an advanced method for feature extraction and classification analysis of high-dimensional data is provided.

A data and physical model joint driven method for lithium-ion battery remaining useful life prediction under complex dynamic conditions

Accurate remaining useful life (RUL) prediction of batteries plays an important role in battery management. The existing methods mainly rely on the battery test data under ideal operating conditions to show good performance. However, the actual operating conditions of batteries are usually complex, which bring great challenges to the RUL prediction. To improve the accuracy and the generalization ability of RUL prediction method, a fusion method of electrochemical–thermal model (ECT) and unscented kalman filter (UKF) is proposed for lithium-ion battery. Firstly, an ECT–based capacity degradation model is established by coupling pseudo two dimensions (P2D) electrochemistry model, 3D thermal model and solid electrolyte interface (SEI) formation model. Secondly, the UKF method is used to update iteratively the parameters of model to improve its accuracy. Finally, several sets of operating data of lithium-ion battery under different conditions are used for RUL prediction, followed by the comparative analysis of different models and algorithms. As a result, the proposed fusion method not only exhibits better accuracy in RUL prediction under ideal conditions, but also shows excellent accuracy and generalization for other dynamic stochastic operating conditions.

Harmonizing Image Forgery Detection & Localization: Fusion of Complementary Approaches.

Image manipulation is easier than ever, often facilitated using accessible AI-based tools. This poses significant risks when used to disseminate disinformation, false evidence, or fraud, which highlights the need for image forgery detection and localization methods to combat this issue. While some recent detection methods demonstrate good performance, there is still a significant gap to be closed to consistently and accurately detect image manipulations in the wild. This paper aims to enhance forgery detection and localization by combining existing detection methods that complement each other. First, we analyze these methods' complementarity, with an objective measurement of complementariness, and calculation of a target performance value using a theoretical oracle fusion. Then, we propose a novel fusion method that combines the existing methods' outputs. The proposed fusion method is trained using a Generative Adversarial Network architecture. Our experiments demonstrate improved detection and localization performance on a variety of datasets. Although our fusion method is hindered by a lack of generalization, this is a common problem in supervised learning, and hence a motivation for future work. In conclusion, this work deepens our understanding of forgery detection methods' complementariness and how to harmonize them. As such, we contribute to better protection against image manipulations and the battle against disinformation.

A Dual-Tree–Complex Wavelet Transform-Based Infrared and Visible Image Fusion Technique and Its Application in Tunnel Crack Detection

Computer vision methods have been widely used in recent years for the detection of structural cracks. To address the issues of poor image quality and the inadequate performance of semantic segmentation networks under low-light conditions in tunnels, in this paper, infrared images are used, and a preprocessing method based on image fusion technology is developed. First, the DAISY descriptor and the perspective transform are applied for image alignment. Then, the source image is decomposed into high- and low-frequency components of different scales and directions using DT-CWT, and high- and low-frequency subband fusion rules are designed according to the characteristics of infrared and visible images. Finally, a fused image is reconstructed from the processed coefficients, and the fusion results are evaluated using the improved semantic segmentation network. The results show that using the proposed fusion method to preprocess images leads to a low false alarm rate and low missed detection rate in comparison to those using the source image directly or using the classical fusion algorithm.

Design of multi-objective guided filter infrared thermal image fusion for hypervelocity impact damages detection

Spacecraft hypervelocity impact damage assessment is becoming increasingly important as the space debris environment deteriorates. Common single-target guided filtering methods are hardly capable of coping with the complex damage infrared thermal image fusion requirements of hypervelocity impacts. In this paper, a two-layer infrared reconstructed image fusion algorithm based on Multi-objective guided filtering (MOGF) is proposed. The aim is to keep the filtered input and output images similar while reducing thermal image noise and retaining as much of the tiny defects as possible in the fusion process. MOGF combines a multi-objective evolutionary optimization algorithm to simultaneously optimize multiple guided filter cost functions in the form of vectors in order to obtain the optimal set of guided filter weight parameters. Given the problem of irregular frontier surfaces in practical multi-objective optimization problems, a weight vector adaptive adjustment strategy is introduced to obtain a more uniform population. Finally, the optimal weight set is passed to the full pixel fusion layer to complete the complex damage infrared reconstructed image fusion. The experiments demonstrate that the proposed fusion method considering multiple targets has higher performance in hypervelocity impact complex damage fusion compared to the general single target fusion method.

Audio-Visual Fusion Based on Interactive Attention for Person Verification.

With the rapid development of multimedia technology, personnel verification systems have become increasingly important in the security field and identity verification. However, unimodal verification systems have performance bottlenecks in complex scenarios, thus triggering the need for multimodal feature fusion methods. The main problem with audio-visual multimodal feature fusion is how to effectively integrate information from different modalities to improve the accuracy and robustness of the system for individual identity. In this paper, we focus on how to improve multimodal person verification systems and how to combine audio and visual features. In this study, we use pretrained models to extract the embeddings from each modality and then perform fusion model experiments based on these embeddings. The baseline approach in this paper involves taking the fusion feature and passing it through a fully connected (FC) layer. Building upon this baseline, we propose three fusion models based on attentional mechanisms: attention, gated, and inter-attention. These fusion models are trained on the VoxCeleb1 development set and tested on the evaluation sets of the VoxCeleb1, NIST SRE19, and CNC-AV datasets. On the VoxCeleb1 dataset, the best system performance achieved in this study was an equal error rate (EER) of 0.23% and a detection cost function (minDCF) of 0.011. On the evaluation set of NIST SRE19, the EER was 2.60% and the minDCF was 0.283. On the evaluation set of the CNC-AV set, the EER was 11.30% and the minDCF was 0.443. These experimental results strongly demonstrate that the proposed fusion method can significantly improve the performance of multimodal character verification systems.

Filter bank second-order underdamped stochastic resonance analysis for implementing a short-term high-speed SSVEP detection

ObjectiveThe progression of brain-computer interfaces (BCIs) has been propelled by breakthroughs in neuroscience, signal processing, and machine learning, marking it as a dynamic field of study over the past few decades. Nevertheless, the nonlinear and non-stationary characteristics of steady-state visual evoked potentials (SSVEPs), coupled with the incongruity between frequently employed linear techniques and nonlinear signal attributes, resulted in the subpar performance of mainstream non-training algorithms like canonical correlation analysis (CCA), multivariate synchronization index (MSI), and filter bank CCA (FBCCA) in short-term SSVEP detection. MethodsTo tackle this problem, the novel fusions of common filter bank analysis, CCA dimensionality reduction methods, USSR models, and MSI recognition models are used in SSVEP signal recognition. ResultsUnlike conventional linear techniques such as CCA, MSI, and FBCCA, the filter bank second-order underdamped stochastic resonance (FBUSSR) analysis demonstrates superior efficacy in the detection of short-term high-speed SSVEPs. ConclusionThis research enlists 32 subjects and uses a public dataset to assess the proposed approach, and the experimental outcomes indicate that the non-training method can attain greater recognition precision and stability. Furthermore, under the conditions of the newly proposed fusion method and light stimulation, the USSR model exhibits the most optimal enhancement effect. SignificanceThe findings of this study underscore the expansive potential for the application of BCI systems in the realm of neuroscience and signal processing.

LAND SUBSIDENCE IN WUHAN REVEALED USING A MULTI-SENSOR INSAR TIME SERIES FUSION APPROACH

Abstract. Satellite Interferometric Synthetic Aperture Radar (InSAR) is widely utilized for topographic, geological, and natural resource investigations. However, many existing InSAR studies on ground deformation are limited to relatively short observation periods and single sensors. This paper introduces a novel method for fusing multi-sensor InSAR time series data, specifically designed to address scenarios involving partial overlaps and temporal gaps. The method employs a new Power Exponential Knothe Model (PEKM) to fit and fuse overlaps in the deformation curves, while a Long Short-Term Memory (LSTM) neural network predicts and fuses data during temporal gaps in the series. In this study, the city of Wuhan in China was selected as the experimental area. SAR datasets from COSMO-SkyMed (2011–2015), TerraSAR-X (2015–2019), and Sentinel-1 (2019–2021) were fused to map long-term surface deformation over the past decade. An independent InSAR time series analysis from 2011 to 2020, based on 230 COSMO-SkyMed scenes, was used as a reference for comparison. The correlation coefficient between the fusion algorithm’s results and the reference data is 0.87 in the time overlapping region and 0.97 in the time-interval dataset. The overall correlation coefficient of 0.78 demonstrates that the proposed algorithm achieves a similar trend as the reference deformation curve. Based on the long time series settlement results obtained through fusion, a detailed analysis of the causes of settlement was conducted for several subsidence zones. The subsidence in the Houhu area is primarily attributed to the consolidation and compression of soft soil. Soil mechanics were employed to estimate the expected completion time of subsidence and calculate the degree of consolidation for each year. The COSMO-SkyMed PSInSAR results indicate that the area has entered the late stage of consolidation and compression, gradually stabilizing over time. Furthermore, the subsidence curve observed in the area around Xinrong reveals that the construction of an underground section of subway Line 21 has caused significant settlement in that particular region. The high temporal granularity of the PSInSAR time series also enables precise detection of a rebound phase following a major flooding event in 2016. The experimental results demonstrate the accuracy of the proposed fusion method in providing robust time series for analyzing long-term land subsidence mechanisms. Additionally, these findings unveil previously unknown characteristics of land subsidence in Wuhan, thus clarifying the relationship with urban causative factors.

CrossFuse: A novel cross attention mechanism based infrared and visible image fusion approach

Multimodal visual information fusion aims to integrate the multi-sensor data into a single image which contains more complementary information and less redundant features. However the complementary information is hard to extract, especially for infrared and visible images which contain big similarity gap between these two modalities. The common cross attention modules only consider the correlation, on the contrary, image fusion tasks need focus on complementarity (uncorrelation). Hence, in this paper, a novel cross attention mechanism (CAM) is proposed to enhance the complementary information. Furthermore, a two-stage training strategy based fusion scheme is presented to generate the fused images. For the first stage, two auto-encoder networks with same architecture are trained for each modality. Then, with the fixed encoders, the CAM and a decoder are trained in the second stage. With the trained CAM, features extracted from two modalities are integrated into one fused feature in which the complementary information is enhanced and the redundant features are reduced. Finally, the fused image can be generated by the trained decoder. The experimental results illustrate that our proposed fusion method obtains the SOTA fusion performance compared with the existing fusion networks. The codes of our fusion method will be available soon.

Within-Season Crop Identification by the Fusion of Spectral Time-Series Data and Historical Crop Planting Data

Crop mapping at an earlier time within the growing season benefits agricultural management. However, crop spectral information is very limited at the early crop phenological stages, leading to difficulties for within-season crop identification. In this study, we proposed a deep learning-based fusion method for crop mapping within the growing season, which first learned a priori information (i.e., pre-season crop types) from historical crop planting data and then integrated the a priori information with the satellite-derived crop types estimated from spectral times-series data. We expect that preseason crop types provided by crop rotation patterns is an effective supplement to spectral information to generate reliable crop maps in the early growing season. We tested the proposed fusion method at three representative sites in the U.S. with different crop rotation intensities and one site with cloudy weather conditions in the Sichuan Province of China. The experimental results showed that the fusion method incorporated the strengths of pre-season crop type estimates and the spectral-based crop type estimates and thus achieved higher crop classification accuracy than the two estimates throughout the growing season. We found that pre-season crop estimates had a higher accuracy in the scenarios with either nearly continuous planting or half-time planting of the same crop. In addition, the historical crop type data strongly affected the performance of pre-season crop estimates, suggesting that high-quality historical crop planting data are particularly important for crop identification at earlier times in the growing season. Our study highlighted the great potential for near real-time crop mapping through the fusion of spectral information and crop rotation patterns.

Infrared and Visible Image Fusion Based on Mask and Cross-Dynamic Fusion

Both single infrared and visible images have respective limitations. Fusion technology has been developed to conquer these restrictions. It is designed to generate a fused image with infrared information and texture details. Most traditional fusion methods use hand-designed fusion strategies, but some are too rough and have limited fusion performance. Recently, some researchers have proposed fusion methods based on deep learning, but some early fusion networks cannot adaptively fuse images due to unreasonable design. Therefore, we propose a mask and cross-dynamic fusion-based network called MCDFN. This network adaptively preserves the salient features of infrared images and the texture details of visible images through an end-to-end fusion process. Specifically, we designed a two-stage fusion network. In the first stage, we train the autoencoder network so that the encoder and decoder learn feature extraction and reconstruction capabilities. In the second stage, the autoencoder is fixed, and we employ a fusion strategy combining mask and cross-dynamic fusion to train the entire fusion network. This strategy is conducive to the adaptive fusion of image information between infrared images and visible images in multiple dimensions. On the public TNO dataset and the RoadScene dataset, we selected nine different fusion methods to compare with our proposed method. Experimental results show that our proposed fusion method achieves good results on both datasets.

Optimized Convolutional Fusion for Multimodal Neuroimaging in Alzheimer's Disease Diagnosis: Enhancing Data Integration and Feature Extraction.

Multimodal neuroimaging has gained traction in Alzheimer's Disease (AD) diagnosis by integrating information from multiple imaging modalities to enhance classification accuracy. However, effectively handling heterogeneous data sources and overcoming the challenges posed by multiscale transform methods remains a significant hurdle. This article proposes a novel approach to address these challenges. To harness the power of diverse neuroimaging data, we employ a strategy that leverages optimized convolution techniques. These optimizations include varying kernel sizes and the incorporation of instance normalization, both of which play crucial roles in feature extraction from magnetic resonance imaging (MRI) and positron emission tomography (PET) images. Specifically, varying kernel sizes allow us to adapt the receptive field to different image characteristics, enhancing the model's ability to capture relevant information. Furthermore, we employ transposed convolution, which increases spatial resolution of feature maps, and it is optimized with varying kernel sizes and instance normalization. This heightened resolution facilitates the alignment and integration of data from disparate MRI and PET data. The use of larger kernels and strides in transposed convolution expands the receptive field, enabling the model to capture essential cross-modal relationships. Instance normalization, applied to each modality during the fusion process, mitigates potential biases stemming from differences in intensity, contrast, or scale between modalities. This enhancement contributes to improved model performance by reducing complexity and ensuring robust fusion. The performance of the proposed fusion method is assessed on three distinct neuroimaging datasets, which include: Alzheimer's Disease Neuroimaging Initiative (ADNI), consisting of 50 participants each at various stages of AD for both MRI and PET (Cognitive Normal, AD, and Early Mild Cognitive); Open Access Series of Imaging Studies (OASIS), consisting of 50 participants each at various stages of AD for both MRI and PET (Cognitive Normal, Mild Dementia, Very Mild Dementia); and whole-brain atlas neuroimaging (AANLIB) (consisting of 50 participants each at various stages of AD for both MRI and PET (Cognitive Normal, AD). To evaluate the quality of the fused images generated via our method, we employ a comprehensive set of evaluation metrics, including Structural Similarity Index Measurement (SSIM), which assesses the structural similarity between two images; Peak Signal-to-Noise Ratio (PSNR), which measures how closely the generated image resembles the ground truth; Entropy (E), which assesses the amount of information preserved or lost during fusion; the Feature Similarity Indexing Method (FSIM), which assesses the structural and feature similarities between two images; and Edge-Based Similarity (EBS), which measures the similarity of edges between the fused and ground truth images. The obtained fused image is further evaluated using a Mobile Vision Transformer. In the classification of AD vs. Cognitive Normal, the model achieved an accuracy of 99.00%, specificity of 99.00%, and sensitivity of 98.44% on the AANLIB dataset.

A fusion method for capacity estimation of fast-charging lithium-ion battery under varied degradation mode

Recently, lithium-ion batteries with fast-charging capability have been gradually equipped in electric products. Capacity estimation specially developed for fast-charging batteries is still an open question, and most of the existing works are proposed for batteries charged under the 1C rate. However, incorrect fast-charging strategies may result in Li plating leading to battery capacity aging rapidly and degradation mode variation. Moreover, unprecise sampling by BMS increases the difficulty of accurately estimating battery capacity in such scenarios. Thus, a fusion prognostic method based on ensemble learning is proposed for the above issues in fast-charging batteries. Firstly, measurement-based health features are extracted from the portion charging phase. Then, the accuracy of the validation dataset-based strategy is proposed to achieve time-varying weight allocation. Finally, a fusion prognostic model is constructed based on the above weight allocation strategy and health features. The effectiveness of the proposed fusion method is verified by a self-design fast-charging battery dataset that maintains stable and reliable performance under sparse sampling and degradation mode variation conditions. In addition, the robustness and adaptability are validated by the comparison experiments with traditional ensemble learning-based methods, in which accuracy is improved by 27.5% minimally.

Two large-exposure-ratio image fusion by improved morphological segmentation.

The fusion of two large-exposure-ratio images, especially in the rocket launch field, is a challenging task because of fast-moving objects and differential features from daily scenes. Based on the large-exposure-ratio images, we propose a principle of halo formation at the boundaries of over-exposed areas. To avoid the halos in the fusion images, an improved morphological segmentation (IMS) method is developed to segment the over-exposed regions and boundaries. The IMS method is inspired by the mountain topography and builds a bridge between the 3D patches and quadratic polynomial coefficients. An improved multiscale method with segmentation in high-exposed images is proposed. In the rocket launch observation experiment, we constructed a two-camera simultaneous imaging system to avoid the dynamic scenes. The result of our proposed fusion method could best preserve the details and colors of the flames in low-exposed images and has the best subjective observation. The objective matrices also demonstrate superior edge and contrast performances over mainstream methods.

Enhanced defect detection in NDE using registration aided heterogeneous data fusion

The degradation of systems in service, including pipelines, over time highlights the critical need for reliable and accurate defect detection to ensure safe operations. However, single modality-based Nondestructive Evaluation (NDE) data used in practical applications often suffers from noise contamination and errors caused by various factors like lift-off/standoff distances, probe drift, scanning speed, variation in data acquisition rates, and poor sensor sensitivity. The presence of agnostic noise types poses a challenge in extracting defect signals. To address these challenges, this paper presents an automated NDE theory-based data fusion framework aimed at enhancing the detection of surface and near-surface defects in magnetizable and conductive specimens. The Magnetic Flux Leakage (MFL) and Eddy Current (EC) based NDE sensing methods demonstrate the highly heterogeneous nature of noise distributions. Given the heterogeneity of the inspection methods, a screening rule is proposed to determine the conditions under which fusion would be beneficial. An important aspect of the proposed fusion method is registration, which ensures accurate alignment of multi-sensor image data. Two registration methods are proposed in this study as performing blind fusion without registration leads to erroneous results. The first registration method is translational, whereas the second method is registration based on linear optimal transport (OT) which proves to be effective in the boundary conditions. Finally, the registered source images from the EC and MFL modalities are fused using pixel-based fusion algorithms, including transform domain and spatial domain-based methods. Qualitative and quantitative assessments demonstrate that the registered fusion results exhibit higher accuracy and reliability compared to unregistered fused results and source images. Although the fusion method is applied to MFL and EC data in this paper, it is also suitable for other NDE modalities.

Non-contact human respiratory rate measurement based on two-level fusions of video and FMCW radar information

In this paper, we propose a robust respiratory rate (RR) measurement method using a two-level fusion of video and FMCW (frequency modulated continuous wave) radar information. Specifically, the video pixel displacement signals and the radar phase variation signals are extracted independently in shoulders and chest regions, which are then denoised by the serial-CEEMDAN (SCEEMDAN) method to obtain the corresponding single modal RR values. Next, at the feature level, the multivariate singular spectrum analysis (MSSA) is employed to extract the shared respiratory components in the video and radar modalities. Finally, at the decision level, we calculate the target RR value based on a signal-to-noise ratio (SNR) weighting of all single-modality and MSSA results. The proposed two-level fusion method is evaluated on a self-collected dataset, which includes 15 healthy subjects acting five challenging activities. The experimental results show that the proposed fusion method significantly outperform that of the single modality.

MDF-Net for abnormality detection by fusing X-rays with clinical data

This study investigates the effects of including patients’ clinical information on the performance of deep learning (DL) classifiers for disease location in chest X-ray images. Although current classifiers achieve high performance using chest X-ray images alone, consultations with practicing radiologists indicate that clinical data is highly informative and essential for interpreting medical images and making proper diagnoses. In this work, we propose a novel architecture consisting of two fusion methods that enable the model to simultaneously process patients’ clinical data (structured data) and chest X-rays (image data). Since these data modalities are in different dimensional spaces, we propose a spatial arrangement strategy, spatialization, to facilitate the multimodal learning process in a Mask R-CNN model. We performed an extensive experimental evaluation using MIMIC-Eye, a dataset comprising different modalities: MIMIC-CXR (chest X-ray images), MIMIC IV-ED (patients’ clinical data), and REFLACX (annotations of disease locations in chest X-rays). Results show that incorporating patients’ clinical data in a DL model together with the proposed fusion methods improves the disease localization in chest X-rays by 12% in terms of Average Precision compared to a standard Mask R-CNN using chest X-rays alone. Further ablation studies also emphasize the importance of multimodal DL architectures and the incorporation of patients’ clinical data in disease localization. In the interest of fostering scientific reproducibility, the architecture proposed within this investigation has been made publicly accessible(https://github.com/ChihchengHsieh/multimodal-abnormalities-detection).

Strawberry Defect Identification Using Deep Learning Infrared–Visible Image Fusion

Feature detection of strawberry multi-type defects and the ripeness stage faces huge challenges because of color diversity and visual similarity. Images from hyperspectral near-infrared (NIR) information sources are also limited by their low spatial resolution. In this study, an accurate RGB image (with a spatial resolution of 2048×1536 pixels) and NIR image (ranging from 700–1100 nm in wavelength, covering 146 bands, and with a spatial resolution of 696×700 pixels) fusion method was proposed to improve the detection of defects and features in strawberries. This fusion method was based on a pretrained VGG-19 model. The high-frequency parts of original RGB and NIR image pairs were filtered and fed into the pretrained VGG-19 simultaneously. The high-frequency features were extracted and output into ReLU layers; the l1-norm was used to fuse multiple feature maps into one feature map, and area pixel averaging was introduced to avoid the effect of extreme pixels. The high- and low-frequency parts of RGB and NIR were summed into one image according to the information weights at the end. In the validation section, the detection dataset included expanded 4000 RGB images and 4000 NIR images (training and testing set ratio was 4:1) from 240 strawberry samples labeled as mud contaminated, bruised, both defects, defect-free, ripe, half-ripe, and unripe. The detection neural network YOLOv3-tiny operated on RGB-only, NIR-only, and fused image input modes, achieving the highest mean average precision of 87.18% for the proposed method. Finally, the effects of different RGB and NIR weights on the detection results were also studied. This research demonstrated that the proposed fusion method can greatly improve the defect and feature detection of strawberry samples.