Fusion Stage Research Articles

MissModal: Increasing Robustness to Missing Modality in Multimodal Sentiment Analysis

Abstract When applying multimodal machine learning in downstream inference, both joint and coordinated multimodal representations rely on the complete presence of modalities as in training. However, modal-incomplete data, where certain modalities are missing, greatly reduces performance in Multimodal Sentiment Analysis (MSA) due to varying input forms and semantic information deficiencies. This limits the applicability of the predominant MSA methods in the real world, where the completeness of multimodal data is uncertain and variable. The generation-based methods attempt to generate the missing modality, yet they require complex hierarchical architecture with huge computational costs and struggle with the representation gaps across different modalities. Diversely, we propose a novel representation learning approach named MissModal, devoting to increasing robustness to missing modality in a classification approach. Specifically, we adopt constraints with geometric contrastive loss, distribution distance loss, and sentiment semantic loss to align the representations of modal-missing and modal-complete data, without impacting the sentiment inference for the complete modalities. Furthermore, we do not demand any changes in the multimodal fusion stage, highlighting the generality of our method in other multimodal learning systems. Extensive experiments demonstrate that the proposed method achieves superior performance with minimal computational costs in various missing modalities scenarios (flexibility), including severely missing modality (efficiency) on two public MSA datasets.

Uncertainty-aware incomplete multimodal fusion for few-shot Central Retinal Artery Occlusion classification

Central Retinal Artery Occlusion (CRAO) is a rare and severe ophthalmic disease that remains challenging to accurately diagnose and classify in clinical practice. The low incidence rate of CRAO makes it difficult to gather a large-scale dataset for training deep-learning models in CRAO classification. Even though the integration of multimodal information has shown the potential to enhance classification performance, the acquisition of complete multimodal data shows significant challenges. This is mainly due to limitations in medical resources and examination costs. Consequently, existing deep learning approaches are unable to learn discriminative features for CRAO classification. In this work, we propose a novel deep-learning methodology that takes advantage of multi-task learning and trustworthy fusion to improve the classification of CRAO with incomplete multimodal data. In the feature extraction stage, we design a multi-task framework that simultaneously performs classification and lesion segmentation tasks. The segmentation task helps the image encoder learn more disease-related features. In particular, text annotations of lesion regions are used to compute the similarity between text features and image features as an auxiliary loss to learn discriminative representations for fine-grained classification. In the multimodal fusion stage, we propose a trustworthy fusion strategy to learn effective joint representations from incomplete multimodal data, in which the model’s predictive uncertainty is used to adaptively weigh modality-specific features. We evaluate our method on a self-collected dataset and compare its performance with other state-of-the-art approaches. The results show the superiority of the proposed method with a mean accuracy of 90.31 %.

Multi-modal fusion architecture search for camera-based semantic scene completion

Camera-based Semantic scene completion (SSC) aims to infer the 3D volumetric occupancy and semantic categories of a scene simultaneously from a single RGB image. The main challenge of camera-based SSC is the lack of geometry information compared with RGB-D SSC. Although the estimated depth from RGB image will help SSC to some extent, the depth prediction quality is far from the demand of SSC. To solve this problem, we propose a NAS-based multi-modal fusion method to incorporate the semantic and geometry information from other intermediate representations (predicted depth and predicted 2D segmentation) to form a more robust 2D feature representation. A key idea of this design is that explicit 2D semantic information could alleviate the misleading information of 3D distortions introduced by estimated depth. Specifically, we propose the Confidence-Block to automatically learn an optimal architecture for routing and obtaining the depth prediction confidence. We propose the two-level fusion search space by decomposing the fusion search space into fusion stage search space and fusion operation search space. Moreover, we propose a confidence-aware 2D–3D projection module to alleviate the 3D projection error. Extensive experiments show that our method outperforms the state-of-the-art method by a large margin using a single RGB image on NYU and NYUCAD datasets.

Precise Control of Two-Dimensional Hexagonal Platelets via Scalable, One-Pot Assembly Pathways Using Block Copolymers with Crystalline Side Chains.

Crystallization-driven self-assembly (CDSA) of block copolymers (BCPs) in selective solvents provides a promising route for direct access to two-dimensional (2D) platelet micelles with excellent uniformity, although significant limitations also exist for this robust approach, such as tedious, multistep procedures, and low yield of assembled materials. Herein, we report a facile strategy for massively preparing 2D, highly symmetric hexagonal platelets with precise control over their dimensions based on BCPs with crystalline side chains. Mechanistic studies unveiled that the formation of hexagonal platelets was subjected to a hierarchical self-assembly process, involving an initial stage of formation of kinetically trapped spheres upon cooling driven by solvophobic interactions, and a second stage of fusion of such spheres to the 2D nuclei to initiate the lateral growth of hexagonal platelets via sequential particle attachments driven by thermodynamically ordered reorganization of the BCP upon aging. Moreover, the size of the developed 2D hexagonal platelets could be finely regulated by altering the copolymer concentration over a broad concentration range, enabling scale-up to a total solids concentration of at least 6% w/w. Our work reveals a new mechanism to create uniform 2D core-shell nanoparticles dictated by crystallization and particle fusion, while it also provides an alternative facile strategy for the design of soft materials with precise control of their dimensions, as well as for the scalability of the derived nanostructures.

A Small Intestinal Stromal Tumor Detection Method Based on an Attention Balance Feature Pyramid.

Small intestinal stromal tumor (SIST) is a common gastrointestinal tumor. Currently, SIST diagnosis relies on clinical radiologists reviewing CT images from medical imaging sensors. However, this method is inefficient and greatly affected by subjective factors. The automatic detection method for stromal tumors based on computer vision technology can better solve these problems. However, in CT images, SIST have different shapes and sizes, blurred edge texture, and little difference from surrounding normal tissues, which to a large extent challenges the use of computer vision technology for the automatic detection of stromal tumors. Furthermore, there are the following issues in the research on the detection and recognition of SIST. After analyzing mainstream target detection models on SIST data, it was discovered that there is an imbalance in the features at different levels during the feature fusion stage of the network model. Therefore, this paper proposes an algorithm, based on the attention balance feature pyramid (ABFP), for detecting SIST with unbalanced feature fusion in the target detection model. By combining weighted multi-level feature maps from the backbone network, the algorithm creates a balanced semantic feature map. Spatial attention and channel attention modules are then introduced to enhance this map. In the feature fusion stage, the algorithm scales the enhanced balanced semantic feature map to the size of each level feature map and enhances the original feature information with the original feature map, effectively addressing the imbalance between deep and shallow features. Consequently, the SIST detection model's detection performance is significantly improved, and the method is highly versatile. Experimental results show that the ABFP method can enhance traditional target detection methods, and is compatible with various models and feature fusion strategies.

In-situ Raman spectroscopy study on coal pyrolysis and subsequent char low temperature oxidation and gasification

Thermal conversion is pivotal for clean and efficient coal utilization. Evolution of solid structure profoundly affects the gasification mechanism. However, in-situ study on coal structure evolution during gasification is limited. In this study, in-situ Raman spectroscopy was used to investigate the carbon structure evolution during thermal conversion. Results revealed two stages in coal pyrolysis: polycondensation and aromatic ring fusion. During polycondensation, numerous side chain groups break, while aromatic compounds condensed and rings fused in the aromatic ring fusion stage, increasing molecular size. In char gasification, larger particles required more time to achieve stable shrinkage, thus delaying coal char graphitization process. Low temperature oxidation led to decreased order due to oxygen-containing structure formation, evolving into a graphite-like structure as disordered carbon diminished. In addition, oxygen-containing groups acted as active oxidation sites. The insights into real-time structure evolution enhance understanding of the particle behavior during thermal conversion of coal.

Colliding heavy nuclei take multiple identities on the path to fusion

The properties of superheavy elements probe extremes of physics and chemistry. They are synthesised at accelerator laboratories using nuclear fusion, where two atomic nuclei collide, stick together (capture), then with low probability evolve to a compact superheavy nucleus. The fundamental microscopic mechanisms controlling fusion are not fully understood, limiting predictive capability. Even capture, considered to be the simplest stage of fusion, is not matched by models. Here we show that collisions of 40Ca with 208Pb, experience an ‘explosion’ of mass and charge transfers between the nuclei before capture, with unexpectedly high probability and complexity. Ninety different partitions of the protons and neutrons between the projectile-like and target-like nuclei are observed. Since each is expected to have a different probability of fusion, the early stages of collisions may be crucial in superheavy element synthesis. Our interpretation challenges the current view of fusion, explains both the successes and failures of current capture models, and provides a framework for improved models.

Real-time gesture-based control of UAVs using multimodal fusion of FMCW radar and vision

Gesture-based control has gained prominence as an intuitive and natural means of interaction with unmanned aerial vehicles (UAVs). This paper presents a real-time gesture-based control system for UAVs that leverages the multimodal fusion of Frequency Modulated Continuous Wave (FMCW) radar and vision sensors, aiming to enhance user experience through precise and responsive UAV control via hand gestures. The research focuses on developing an effective fusion framework that combines the complementary advantages of FMCW radar and vision sensors. FMCW radar provides robust range and velocity measurements, while vision sensors capture fine-grained visual information. By integrating data from these modalities, the system achieves a comprehensive understanding of hand gestures, resulting in improved gesture recognition accuracy and robustness. The proposed system comprises three main stages: data acquisition, gesture recognition, and multimodal fusion. In the data acquisition stage, synchronized data streams from FMCW radar and vision sensors are captured. Then, machine learning algorithms are employed in the gesture recognition stage to classify and interpret hand gestures. Finally, the multimodal fusion stage aligns and fuses the data, creating a unified representation that captures the spatial and temporal aspects of hand gestures, enabling real-time control commands for the UAV. Experimental results demonstrate the system‘s effectiveness in accurately recognizing and responding to hand gestures. The multimodal fusion of FMCW radar and vision sensors enables a robust and versatile gesture-based control interface.

Data fabric and digital twins: An integrated approach for data fusion design and evaluation of pervasive systems

Recent technological improvements have made it possible for pervasive computing intelligent environments, augmented by sensors and actuators, to offer services that support society's aims for a wide variety of applications. This requires the fusion of data gathered from multiple sensors to convert them into information to obtain valuable knowledge. Poor implementation of data fusion hinders the appropriate actions from being taken and offering the appropriate support to users and environment needs, particularly relevant in the healthcare domain. Data fusion poses challenges that are mainly related to the quality of the data or data sources, the definition of a data fusion process and evaluating the data fusion carried out. There is also a lack of holistic engineering frameworks to address these challenges. These frameworks should be able to support automated methods of extracting knowledge from information, selecting algorithms and techniques, assessing information and evaluating information fusion systems in an automatic and standardized manner. This work proposes a holistic framework to improve data fusion in pervasive systems, addressing the issues identified by means of two processes: the first of which guides the design of the system architecture and focuses on data management. It is based on a previous proposal that integrated aspects of Data Fabric and Digital Twins to solve data management and data contextualization and representation issues, respectively. The extension of the previous proposal presented here was mainly defined by integrating aspects and techniques from different well-known multi-sensor data fusion models. The previous proposal identified high-level data processing activities and was intended to facilitate their traceability to components in the system architecture. However, the previously defined stages are not completely adequate in a data fusion process and the data processing tasks to be performed in each stage are not described in detail, especially in the data fusion stages. The second process of the framework deals with evaluating data fusion systems and is based on international standards to ensure the quality of the data fusion tasks performed by such systems. This process also offers guidelines for designing the architecture of an evaluation subsystem to automatically perform data fusion evaluation in runtime as part of the system. To illustrate the proposal, a system for preventing the spread of COVID-19 in nursing homes is described that was developed using the proposed guidelines It is also illustrated by a description of how the data fusion tasks it supports are evaluated by the proposed evaluation process. The overall evaluation of the data fusion performed by this system was considered satisfactory, which indicates that the proposal facilitates the design and development of data fusion systems and helps to achieve the necessary quality requirements.

Strawberry ripeness detection based on YOLOv8 algorithm fused with LW-Swin Transformer

Identifying the ripeness of strawberries can be challenging due to their complex growth environment, interference from light intensity, and shading caused by strawberry aggregation. To address these issues, this study aims to develop an algorithm for accurately detecting and classifying ripe strawberries. This study proposed a novel LS-YOLOv8s model for detecting and grading the ripeness of strawberries, which is based on the YOLOv8s deep learning algorithm and incorporates the LW-Swin Transformer module. To improve the performance of the model, two new random variables were introduced in the contrast enhancement process to control the enhancement effect. The dataset was expanded from 1089 to 7515 images, which increased the diversity of the data and reduced the risk of over fitting the model. Additionally, the Swin Transformer module was added to the TopDown Layer2 during the feature fusion stage to capture long distance dependencies in the input data and improve the generalization capability of the model with the use of a multi-headed self-attention mechanism. Finally, a more efficient feature fusion network was achieved by introducing a residual network with learnable parameters and scaled normalization into the original residual structure of the Swin Transformer. To evaluate the effectiveness of LS-YOLOv8s for strawberry ripeness detection, we collected a dataset of strawberry images from a strawberry planting base. The dataset was split using the 5-fold cross-validation approach, which improved the model evaluation process. Experimental results showed that LS-YOLOv8s better than other models, with a 1.6 %, 33.5 %, and 3.4 % improvement in mAP0.5 on the validation set compared to YOLOv5s, CenterNet, and SSD, respectively. Moreover, LS-YOLOv8s achieved better detection precision and speed than YOLOv8m with only approximately 51.93 % of the number of parameters used, achieving 94.4 % detection precision and 19.23fps detection speed, improving by 0.5 % and 6.56fps, respectively. The LS-YOLOv8s model can provide reliable theoretical support for detecting strawberry targets, evaluating their ripeness, and automating the strawberry picking process for orchard management.

Multi-modal person re-identification based on transformer relational regularization

For robust multi-modal person re-identification (re-ID) models, it is crucial to effectively utilize the complementary information and constraint relationships among different modalities. However, current multi-modal methods often overlook the correlation between modalities at the feature fusion stage. To address this issue, we propose a novel multimodal person re-ID method called Transformer Relation Regularization (TRR). Firstly, we introduce an adaptive collaborative matching module that facilitates the exchange of useful information by mining feature correspondences between modalities. This module allows for the integration of complementary information, enhancing the re-ID performance. Secondly, we propose an enhanced embedded module that corrects general information using discriminative information within each modality. By leveraging this approach, we improve the model’s stability in challenging multi-modal environments. Lastly, we propose an adaptive triple loss to enhance sample utilization efficiency and mitigate the problem of inconsistent representation among multimodal samples. This loss function optimizes the model’s ability to distinguish between different individuals, leading to improved re-ID accuracy. Experimental results on several challenging visible-infrared person re-ID benchmark datasets demonstrate that our proposed TRR method achieves optimal performance. Additionally, extensive ablation studies validate the effective contribution of each component to the overall model. In summary, our proposed TRR method effectively leverages complementary information, addresses the correlation between modalities, and improves the re-ID performance in multi-modal scenarios. The results obtained from various benchmark datasets and the comprehensive analysis support the efficacy of our approach.

Annotate and retrieve in vivo images using hybrid self-organizing map

Multimodal retrieval has gained much attention lately due to its effectiveness over uni-modal retrieval. For instance, visual features often under-constrain the description of an image in content-based retrieval; however, another modality, such as collateral text, can be introduced to abridge the semantic gap and make the retrieval process more efficient. This article proposes the application of cross-modal fusion and retrieval on real in vivo gastrointestinal images and linguistic cues, as the visual features alone are insufficient for image description and to assist gastroenterologists. So, a cross-modal information retrieval approach has been proposed to retrieve related images given text and vice versa while handling the heterogeneity gap issue among the modalities. The technique comprises two stages: (1) individual modality feature learning; and (2) fusion of two trained networks. In the first stage, two self-organizing maps (SOMs) are trained separately using images and texts, which are clustered in the respective SOMs based on their similarity. In the second (fusion) stage, the trained SOMs are integrated using an associative network to enable cross-modal retrieval. The underlying learning techniques of the associative network include Hebbian learning and Oja learning (Improved Hebbian learning). The introduced framework can annotate images with keywords and illustrate keywords with images, and it can also be extended to incorporate more diverse modalities. Extensive experimentation has been performed on real gastrointestinal images obtained from a known gastroenterologist that have collateral keywords with each image. The obtained results proved the efficacy of the algorithm and its significance in aiding gastroenterologists in quick and pertinent decision making.

Feature specific progressive improvement for salient object detection

Benefiting from deep learning, Salient Object Detection (SOD) has made much progress. However, most existing methods adopt the same strategy to extract salient cues from different feature levels without fully considering their differences in the feature extraction stage and/or suffer from the accumulation of noise and dilution of spatial details in the feature fusion stage. These two problems hinder the further improvement in performance. In this paper, we propose an effective SOD model, PiNet, which can address the above problems via two novel mechanisms in the network: level-specific feature extraction and progressive refinement of saliency. We have designed the customized feature extraction components for each level of features—enabling us to extract better saliency cues from multi-level features. The saliency feature refinement in the branches follows a coarse-to-fine process, where the refined features progressively contain more location cues, internal and boundary details. Through short connections, the extracted saliency cues in different branches are selectively transmitted and integrated, which well mitigates the accumulation of noisy information and the dilution of detailed information. By using four different backbones, we verify our model has good adaptability and can make accurate saliency predictions under different pretrained models. Extensive experiments on five public datasets demonstrate that PiNet outperforms 19 state-of-the-art (SOTA) methods in SOD, with its small model size (56.1 MB) and fast inference speed (47 FPS).

ASENN: attention-based selective embedding neural networks for road distress prediction

This study proposes an innovative neural network framework, ASENN (Attention-based Selective Embedding Neural Network), for the prediction of pavement deterioration. Considering the complexity and uncertainty associated with the pavement deterioration process, two fundamental frameworks, SEL (Selective Embedding Layer) and MDAL (Multi-Dropout Attention Layer), are combined to enhance feature abstraction and prediction accuracy. This approach is significant while analyzing the pavement deterioration process due to the high variability of the contributing deterioration factors. These factors, represented as tabular data, undergo filtering, embedding, and fusion stages in the SEL, to extract crucial features for an effective representation of pavement deterioration. Further, multiple attention-weighted combinations of raw data are obtained through the MDAL. Several SELs and MDALs were combined as basic cells and layered to form an ASENN. The experimental results demonstrate that the proposed model outperforms existing tabular models on four road distress parameter datasets corresponding to cracking, deflection, international roughness index, and rutting. The optimal number of cells was determined using different ablation settings. The results also show that the feature learning capabilities of the ASENN model improved as the number of cells increased; however, owing to the limited combination space of feature fields, extreme depths were not preferred. Furthermore, the ablation investigation demonstrated that MDAL can improve performance, particularly on the cracking dataset. Notably, compared with mainstream transformer models, ASENN requires significantly less storage and achieves faster execution speed.

GLTF-Net: Deep-Learning Network for Thick Cloud Removal of Remote Sensing Images via Global–Local Temporality and Features

Remote sensing images are very vulnerable to cloud interference during the imaging process. Cloud occlusion, especially thick cloud occlusion, significantly reduces the imaging quality of remote sensing images, which in turn affects a variety of subsequent tasks using the remote sensing images. The remote sensing images miss ground information due to thick cloud occlusion. The thick cloud removal method based on a temporality global–local structure is initially suggested as a solution to this problem. This method includes two stages: the global multi-temporal feature fusion (GMFF) stage and the local single-temporal information restoration (LSIR) stage. It adopts the fusion feature of global multi-temporal to restore the thick cloud occlusion information of local single temporal images. Then, the featured global–local structure is created in both two stages, fusing the global feature capture ability of Transformer with the local feature extraction ability of CNN, with the goal of effectively retaining the detailed information of the remote sensing images. Finally, the local feature extraction (LFE) module and global–local feature extraction (GLFE) module is designed according to the global–local characteristics, and the different module details are designed in this two stages. Experimental results indicate that the proposed method performs significantly better than the compared methods in the established data set for the task of multi-temporal thick cloud removal. In the four scenes, when compared to the best method CMSN, the peak signal-to-noise ratio (PSNR) index improved by 2.675, 5.2255, and 4.9823 dB in the first, second, and third temporal images, respectively. The average improvement of these three temporal images is 9.65%. In the first, second, and third temporal images, the correlation coefficient (CC) index improved by 0.016, 0.0658, and 0.0145, respectively, and the average improvement for the three temporal images is 3.35%. Structural similarity (SSIM) and root mean square (RMSE) are improved 0.33% and 34.29%, respectively. Consequently, in the field of multi-temporal cloud removal, the proposed method enhances the utilization of multi-temporal information and achieves better effectiveness of thick cloud restoration.

A Synthesizing Semantic Characteristics Lung Nodules Classification Method Based on 3D Convolutional Neural Network.

Early detection is crucial for the survival and recovery of lung cancer patients. Computer-aided diagnosis system can assist in the early diagnosis of lung cancer by providing decision support. While deep learning methods are increasingly being applied to tasks such as CAD (Computer-aided diagnosis system), these models lack interpretability. In this paper, we propose a convolutional neural network model that combines semantic characteristics (SCCNN) to predict whether a given pulmonary nodule is malignant. The model synthesizes the advantages of multi-view, multi-task and attention modules in order to fully simulate the actual diagnostic process of radiologists. The 3D (three dimensional) multi-view samples of lung nodules are extracted by spatial sampling method. Meanwhile, semantic characteristics commonly used in radiology reports are used as an auxiliary task and serve to explain how the model interprets. The introduction of the attention module in the feature fusion stage improves the classification of lung nodules as benign or malignant. Our experimental results using the LIDC-IDRI (Lung Image Database Consortium and Image Database Resource Initiative) show that this study achieves 95.45% accuracy and 97.26% ROC (Receiver Operating Characteristic) curve area. The results show that the method we proposed not only realize the classification of benign and malignant compared to standard 3D CNN approaches but can also be used to intuitively explain how the model makes predictions, which can assist clinical diagnosis.

EXABSUM: a new text summarization approach for generating extractive and abstractive summaries

Due to the exponential growth of online information, the ability to efficiently extract the most informative content and target specific information without extensive reading is becoming increasingly valuable to readers. In this paper, we present 'EXABSUM,' a novel approach to Automatic Text Summarization (ATS), capable of generating the two primary types of summaries: extractive and abstractive. We propose two distinct approaches: (1) an extractive technique (EXABSUMExtractive), which integrates statistical and semantic scoring methods to select and extract relevant, non-repetitive sentences from a text unit, and (2) an abstractive technique (EXABSUMive), which employs a word graph approach (including compression and fusion stages) and re-ranking based on keyphrases to generate abstractive summaries using the source document as an input. In the evaluation conducted on multi-domain benchmarks, EXABSUM outperformed extractive summarization methods and demonstrated competitiveness against abstractive baselines.

Improved Pest-YOLO: Real-time pest detection based on efficient channel attention mechanism and transformer encoder

Insect infestations and pests inflict significant losses in agriculture, substantially augmenting the demand for automated pest detection and early pest management in the cultivation process. However, the task of multi-class pest identification, involving both localization and classification, is exceptionally challenging due to the small size, great similarity, and environmental variability of pests. This paper presents an enhanced version of our previous work, Pest-YOLO, aimed at improving accuracy while maintaining real-time pest detection. The improved Pest-YOLO incorporates two key advancements: an efficient channel attention (ECA) mechanism for improved feature extraction and a transformer encoder for capturing global features. We replace the original squeeze-excitation attention mechanism with the ECA mechanism, effectively improving the model's ability to extract essential features from pest images. Additionally, we introduce the transformer encoder to the convolutional neural network (CNN) architecture to enhance its capability to capture global contextual information. To further enhance the expressiveness of features for small targets like agricultural pests, we propose a feature fusion method called cross-stage feature fusion (CSFF). This method significantly improves the representation of small targets during the feature fusion stage. Through experiments on the Pest24 dataset, our method achieves an impressive mean average precision of 73.4%, surpassing the performance of state-of-the-art methods. These results demonstrate the effectiveness of our improved Pest-YOLO model in accurately detecting pests in real-time scenarios.

Even-Order Taylor Approximation-Based Feature Refinement and Dynamic Aggregation Model for Video Object Detection

Video object detection (VOD) is a sophisticated visual task. It is a consensus that is used to find effective supportive information from correlation frames to boost the performance of the model in VOD tasks. In this paper, we not only improve the method of finding supportive information from correlation frames but also strengthen the quality of the features extracted from the correlation frames to further strengthen the fusion of correlation frames so that the model can achieve better performance. The feature refinement module FRM in our model refines the features through the key–value encoding dictionary based on the even-order Taylor series, and the refined features are used to guide the fusion of features at different stages. In the stage of correlation frame fusion, the generative MLP is applied in the feature aggregation module DFAM to fuse the refined features extracted from the correlation frames. Experiments adequately demonstrate the effectiveness of our proposed approach. Our YOLOX-based model can achieve 83.3% AP50 on the ImageNet VID dataset.

Improving the Detection and Positioning of Camouflaged Objects in YOLOv8

Camouflaged objects can be perfectly hidden in the surrounding environment by designing their texture and color. Existing object detection models have high false-negative rates and inaccurate localization for camouflaged objects. To resolve this, we improved the YOLOv8 algorithm based on feature enhancement. In the feature extraction stage, an edge enhancement module was built to enhance the edge feature. In the feature fusion stage, multiple asymmetric convolution branches were introduced to obtain larger receptive fields and achieve multi-scale feature fusion. In the post-processing stage, the existing non-maximum suppression algorithm was improved to address the issue of missed detection caused by overlapping boxes. Additionally, a shape-enhanced data augmentation method was designed to enhance the model’s shape perception of camouflaged objects. Experimental evaluations were carried out on camouflaged object datasets, including COD and CAMO, which are publicly accessible. The improved method exhibits enhancements in detection performance by 8.3% and 9.1%, respectively, compared to the YOLOv8 model.