Adaptive Information Fusion Research Articles

Multiobject Tracking Algorithm With Adaptive Noise Calculation and Information Fusion in Crowded Environments

Multiobject Tracking Algorithm With Adaptive Noise Calculation and Information Fusion in Crowded Environments

MFPIDet: improved YOLOV7 architecture based on multi-scale feature fusion for prohibited item detection in complex environment

Prohibited item detection is crucial for the safety of public places. Deep learning, one of the mainstream methods in prohibited item detection tasks, has shown superior performance far beyond traditional prohibited item detection methods. However, most neural network architectures in deep learning still lack sufficient local feature representation ability for overlapping and small targets, and ignore the problem of semantic conflicts caused by direct feature fusion. In this paper, we propose MFPIDet, a novel prohibited item detection neural network architecture based on improved YOLOV7 to achieve reliable prohibited item detection in complex environments. Specifically, a multi-scale attention module (MAM) backbone is proposed to filter the redundant information of target regions and further applied to enhance the local feature representation ability of overlapping objects. Here, to reduce the redundant information of target regions, a squeeze-excitation (SE) block is used to filter the background. Then, aiming at enhancing the feature expression ability of overlapping objects, a multi-scale feature extraction module (MFEM) is designed for local feature representation. In addition, to obtain richer context information, We design an adaptive fusion feature pyramid network (AF-FPN) to combine the adaptive context information fusion module (ACIFM) with the feature fusion module (FFM) to improve the neck structure of YOLOV7. The proposed method is validated on the PIDray dataset, and the tested results showed that our method obtained the highest mAP (68.7%), which is improved by 3.5% than YOLOV7 methods. Our approach provides a new design pattern for prohibited item detection in complex environments and shows the development potential of deep learning in related fields.

SWFormer: A scale-wise hybrid CNN-Transformer network for multi-classes weed segmentation

Weeds in rapeseed field are an important factor in crop yield reduction and economic loss. Thus, Precision Agriculture is an important task for sustainable agriculture and weed management. At present, deep learning techniques have shown great potential for image-based detection and classification in various crops and weeds. However, the inherent limitations of traditional convolutional neural networks pose significant challenges due to the locally similarity of weeds and crops in color, shape and texture. To address this issue, we introduce SWFormer, a scale-wise hybrid CNN-Transformer network. SWFormer leverages the distinct strengths of both convolutional and transformer architectures. Convolutional structures excel at extracting short-range dependency information among pixels, whereas transformer structures are adept at capturing global dependency relationships. Additionally, we propose two innovative modules. Firstly, the Scale-wise Cascade Convolution (SWCC) module is designed to capture multiscale features and expand the receptive field. Secondly, the Adaptive Semantic Aggregation (ASA) module facilitates adaptive and effective information fusion across two distinct feature maps. Our experiments were conducted on the publicly available cropandweed dataset and SB20 dataset. it yields improved performance over other mainstream segmentation models. Specifically, SWFormer with 52.33M/527.51GFLOPs achieves an mAP of 76.54% and an accuracy of 83.95% on the cropandweed dataset. For the SB20 dataset, it attains an mAP of 61.24% and an accuracy of 79.47%. Overall, the evaluation clearly demonstrates our proposed SWFormer is conducive to promoting further research in the area of Precision Agriculture.

Information Monitoring and Adaptive Information Fusion of Multisource Fusion Navigation Systems in Complex Environments

Information Monitoring and Adaptive Information Fusion of Multisource Fusion Navigation Systems in Complex Environments

Adaptive information fusion network for multi‐modal personality recognition

AbstractPersonality recognition is of great significance in deepening the understanding of social relations. While personality recognition methods have made significant strides in recent years, the challenge of heterogeneity between modalities during feature fusion still needs to be solved. This paper introduces an adaptive multi‐modal information fusion network (AMIF‐Net) capable of concurrently processing video, audio, and text data. First, utilizing the AMIF‐Net encoder, we process the extracted audio and video features separately, effectively capturing long‐term data relationships. Then, adding adaptive elements in the fusion network can alleviate the problem of heterogeneity between modes. Lastly, we concatenate audio‐video and text features into a regression network to obtain Big Five personality trait scores. Furthermore, we introduce a novel loss function to address the problem of training inaccuracies, taking advantage of its unique property of exhibiting a peak at the critical mean. Our tests on the ChaLearn First Impressions V2 multi‐modal dataset show partial performance surpassing state‐of‐the‐art networks.

Across Sessions and Subjects Domain Adaptation for Building Robust Myoelectric Interface.

Gesture interaction via surface electromyography (sEMG) signal is a promising approach for advanced human-computer interaction systems. However, improving the performance of the myoelectric interface is challenging due to the domain shift caused by the signal's inherent variability. To enhance the interface's robustness, we propose a novel adaptive information fusion neural network (AIFNN) framework, which could effectively reduce the effects of multiple scenarios. Specifically, domain adversarial training is established to inhibit the shared network's weights from exploiting domain-specific representation, thus allowing for the extraction of domain-invariant features. Effectively, classification loss, domain diversence loss and domain discrimination loss are employed, which improve classification performance while reduce distribution mismatches between the two domains. To simulate the application of myoelectric interface, experiments were carried out involving three scenarios (intra-session, inter-session and inter-subject scenarios). Ten non-disabled subjects were recruited to perform sixteen gestures for ten consecutive days. The experimental results indicated that the performance of AIFNN was better than two other state-of-the-art transfer learning approaches, namely fine-tuning (FT) and domain adversarial network (DANN). This study demonstrates the capability of AIFNN to maintain robustness over time and generalize across users in practical myoelectric interface implementations. These findings could serve as a foundation for future deployments.

Towards Adaptive Information Fusion in Graph Convolutional Networks

Graph Convolutional Networks (GCNs) have gained great popularity in tackling various analytic tasks on graph and network data. However, some recent studies raise concerns about whether GCNs can optimally integrate node features and topological structures in a complex graph. In this paper, we first present an experimental investigation. Surprisingly, our experimental results clearly show that the capability of the state-of-the-art GCNs in fusing node features and topological structures is distant from optimal or even satisfactory. The weakness may severely hinder the capability of GCNs in some classification tasks, since GCNs may not be able to adaptively learn some deep correlation information between topological structures and node features. Can we remedy the weakness and design a new type of GCNs that can retain the advantages of the state-of-the-art GCNs and, at the same time, enhance the capability of fusing topological structures and node features substantially? We tackle the challenge and propose an <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">A</b> daptive <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">M</b> ulti-channel <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">G</b> raph <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">C</b> onvolutional <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</b> etwork for semi-supervised classification ( <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">AM-GCN</b> ). The central idea is that we extract the specific and common embeddings from node features, topological structures, and their combinations simultaneously, and use the attention mechanism to learn adaptive importance weights of the embeddings. However, considering that the input topology and feature structure in AM-GCN are still predefined and fixed, once the properties of graph structures are not consistent with tasks, the fusion performance of AM-GCN will be hindered from the beginning. Therefore, we need to adjust the structure and further propose the <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">L</b> abel <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">P</b> ropagation guided <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">M</b> ulti-channel <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">G</b> raph <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">C</b> onvolutional <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</b> etwork ( <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">LPM-GCN</b> ). LPM-GCN introduces edge weights learning on both topology and feature spaces to improve structural homophily, which can better promote the fusion process of graph convolutional networks. Our extensive experiments on benchmark data sets clearly show that our proposed models extract the most correlated information from both node features and topological structures substantially, and improves the classification accuracy with a clear margin.

Real-Time Dynamic Gesture Recognition Algorithm Based on Adaptive Information Fusion and Multi-Scale Optimization Transformer

Gesture recognition is a popular technology in the field of computer vision and an important technical mean of achieving human-computer interaction. To address problems such as the limited long-range feature extraction capability of existing dynamic gesture recognition networks based on convolutional operators, we propose a dynamic gesture recognition algorithm based on spatial pyramid pooling Transformer and optical flow information fusion. We take advantage of Transformer’s large receptive field to reduce model computation while improving the model’s ability to extract features at different scales by embedding spatial pyramid pooling. We use the optical flow algorithm with the global motion aggregation module to obtain an optical flow map of hand motion, and to extract the key frames based on the similarity minimization principle. We also design an adaptive feature fusion method to fuse the spatial and temporal features of the dual channels. Finally, we demonstrate the effectiveness of model components on model recognition enhancement through ablation experiments. We conduct training and validation on the SCUT-DHGA dynamic gesture dataset and on a dataset we collected, and we perform real-time dynamic gesture recognition tests using the trained model. The results show that our algorithm achieves high accuracy even while keeping the parameters balanced. It also achieves fast and accurate recognition of dynamic gestures in real-time tests.

SFHG-YOLO: A Simple Real-Time Small-Object-Detection Method for Estimating Pineapple Yield from Unmanned Aerial Vehicles.

The counting of pineapple buds relies on target recognition in estimating pineapple yield using unmanned aerial vehicle (UAV) photography. This research proposes the SFHG-YOLO method, with YOLOv5s as the baseline, to address the practical needs of identifying small objects (pineapple buds) in UAV vision and the drawbacks of existing algorithms in terms of real-time performance and accuracy. Field pineapple buds are small objects that may be detected in high density using a lightweight network model. This model enhances spatial attention and adaptive context information fusion to increase detection accuracy and resilience. To construct the lightweight network model, the first step involves utilizing the coordinate attention module and MobileNetV3. Additionally, to fully leverage feature information across various levels and enhance perception skills for tiny objects, we developed both an enhanced spatial attention module and an adaptive context information fusion module. Experiments were conducted to validate the suggested algorithm's performance in detecting small objects. The SFHG-YOLO model exhibited significant gains in assessment measures, achieving mAP@0.5 and mAP@0.5:0.95 improvements of 7.4% and 31%, respectively, when compared to the baseline model YOLOv5s. Considering the model size and computational cost, the findings underscore the superior performance of the suggested technique in detecting high-density small items. This program offers a reliable detection approach for estimating pineapple yield by accurately identifying minute items.

A data-driven distributionally newsvendor problem for edge-cloud collaboration in intelligent manufacturing systems

In intelligent manufacturing systems, the industrial informatics has features of multi-source, multi-noise, and time series. It is difficult for small and medium enterprises (SMEs) to directly exploit the enormous amounts of data due to the limited budgets and computing capabilities. Edge intelligence is a key technique to power intelligent manufacturing systems and provide knowledge transferred from the cloud to SMEs at the edges. To address edge-cloud collaboration issue, we propose a refined data-driven distributionally robust newsvendor model based on φ-divergence measures and imprecise Dirichlet models (DRN-IDM). We construct new distributional uncertainty sets by effectively integrating local censored demand data and cloud knowledge, which helps SMEs to make intelligent production decisions and reduce significant decision deviations, even under a small censored data set. In particular, the novel demand uncertainty sets can depict the distance between distributions and probability intervals. Then, we transform the DRN-IDM model into a convex optimization model that is amenable to algorithmic implementation. Additionally, based on the coefficient of variation of limited historical data, we propose an adaptive demand information fusion procedure to achieve excellent synergy effect from cloud knowledge. We also validate the effectiveness of the DRN-IDM model and the practicability of adaptive procedure using extensive numerical studies with both simulated and real-life data. Furthermore, we measure the relative expected value of cloud knowledge and investigate the effect of censored demand samples. Our results verify the effectiveness condition of the DRN-IDM model and indicate that cloud knowledge can improve the precision and robustness of SMEs’ production decisions with small-scale censored data. Interestingly, the verified adaptive procedure can be applied in the learning criteria design of metaheuristics in intelligent manufacturing systems, and the reconstructed uncertainty set can narrow the search space to improve the convergence performance of algorithms.

Prior knowledge-embedded meta-transfer learning for few-shot fault diagnosis under variable operating conditions

In recent years, intelligent fault diagnosis based on deep learning has achieved vigorous development thanks to its powerful feature representation ability. However, scarcity of high-quality data, especially samples under severe fault states, and variable operating conditions have limited the industrial application of intelligent fault diagnosis. To alleviate this predicament, a novel prior knowledge-embedded meta-transfer learning (PKEMTL) is proposed for few-shot fault diagnosis with limited training data and scarce test data. The method focuses on the problem of few-shot fault diagnosis under variable operating conditions to improve adaptability. Different from traditional models, the PKEMTL employs a metric-based meta-learning framework and embeds prior knowledge to enable cross-task learning under variable operating conditions. Specifically, order tracking is firstly introduced as preliminary prior information for data augmentation, and then the augmented data are divided into a series of meta-tasks. Secondly, the meta-tasks are performed by lightweight multiscale feature encoding to obtain high-level feature representations. Next, the meta-learning module based on diagnostic knowledge embedding guides the model to acquire meta-knowledge of speed generalization by constructing the self-supervised task to embed additional prior knowledge into the meta-training process. The generalization performance of the model is further improved by adaptive information fusion learning as a comprehensive decision-making module. Two case studies under variable operating conditions are implemented to validate the effectiveness and superiority of the proposed few-shot fault diagnosis method.

Adaptive nonlinear information fusion preview control for autonomous surface vessels subject to measurement noises and unknown input saturations

AbstractThis study presents an adaptive nonlinear information fusion preview control (NIFPC) method for trajectory tracking of autonomous surface vessels (ASVs) subject to system uncertainty, measurement noise, and unknown input saturations. The NIFPC is developed based on the nonlinear information fusion estimation methodology, in which the system's future reference trajectory information, noise information, performance index requirements, and system dynamic model are all transformed into information equations related to control input, and then the current control action is obtained by fusing these previewed future information via the nonlinear information fusion optimal estimation. In order to avoid the unknown input saturation constraints, a fuzzy asymmetric saturated approximator (FASA) is designed and integrated into the controller, where the fuzzy logic system (FLS) is used to adaptively adjust the key boundary parameters of the approximator. As a result, the negative effects caused by system uncertainty and measurement noise can be effectively suppressed, while the completely unknown input saturation constraints in the system actuator are guaranteed not to be violated. The convergence of the tracking errors of the closed‐loop system is guaranteed via Lyapunov stability theory. Numerical simulation results have been provided to demonstrate the satisfactory performance of the proposed control scheme.

A New Adaptive Federated Cubature Kalman Filter Based on Chi-Square Test for SINS/GNSS/SRS/CNS Integration

As an emerging means of transportation for the intelligent transportation system (ITS) in aviation and aerospace, hypersonic cruise vehicles (HCVs) have received numerous research interests during the past several decades. However, the navigation and positioning strictly limit the progress and application of HCVs due to their special characteristics on dynamics and environments. To improve the stability of navigation in HCVs, a chi-square test-based adaptive federated cubature Kalman filter (CAFCKF) is proposed in this paper. In the proposed approach, the chi-square test is adopted for the estimation of the measurement noise statistics firstly. Subsequently, a new adaptive information fusion factor is designed for the federated filter to adjust the contribution of each subsystem. Finally, the information sharing factor, which is used for the amendment of the state covariance of each subsystem, is refined based on the judging index of the chi-square test accordingly. Simulation results show that the proposed CAFCKF can be used to improve the accuracy and stability of the navigation system.

Observability Analysis Based on QR Decomposition and Adaptive Information Fusion Mechanism for Low-Cost Multi-Rotor UAVs

This paper proposes a novel observability analysis method with less computation, which is based on QR decomposition and is suitable for low-cost devices with limited computing power. On this basis, an adaptive information fusion mechanism (AIFM) based on the criteria from the degree of observability (DOO) for the traditional extended Kalman filter (EKF) is designed to improve the tolerance of the integrated navigation system against time-varying observation conditions. Theoretical analysis and experimental results illustrate that the proposed observability analysis method can effectively evaluate each state variable’s observability, whose computational complexity is significantly reduced by 93.07% (the size of stripped observability matrix is <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$105\times15$ </tex-math></inline-formula> ) compared with the conventional method based on singular value decomposition (SVD). Moreover, the simulation and real-world experiment results show that the AIFM makes the navigation system more robust by automatically updating the state variables’ weights according to their observability, which also enhances the reliability of the horizontal attitude angle estimation with excellent efficiency since they are given top priority in our fusion mechanism. In conclusion, our method is practical to improve the integrated navigation system’s accuracy, robustness, and fault-tolerance ability in complex and dynamic environments, suggesting that this technique is viable for multi-rotor navigation and control applications.

Low-light image enhancement network with decomposition and adaptive information fusion

Low-light image enhancement network with decomposition and adaptive information fusion

Decision-Level Fusion with a Pluginable Importance Factor Generator for Remote Sensing Image Scene Classification

Remote sensing image scene classification acts as an important task in remote sensing image applications, which benefits from the pleasing performance brought by deep convolution neural networks (CNNs). When applying deep models in this task, the challenges are, on one hand, that the targets with highly different scales may exist in the image simultaneously and the small targets could be lost in the deep feature maps of CNNs; and on the other hand, the remote sensing image data exhibits the properties of high inter-class similarity and high intra-class variance. Both factors could limit the performance of the deep models, which motivates us to develop an adaptive decision-level information fusion framework that can incorporate with any CNN backbones. Specifically, given a CNN backbone that predicts multiple classification scores based on the feature maps of different layers, we develop a pluginable importance factor generator that aims at predicting a factor for each score. The factors measure how confident the scores in different layers are with respect to the final output. Formally, the final score is obtained by a class-wise and weighted summation based on the scores and the corresponding factors. To reduce the co-adaptation effect among the scores of different layers, we propose a stochastic decision-level fusion training strategy that enables each classification score to randomly participate in the decision-level fusion. Experiments on four popular datasets including the UC Merced Land-Use dataset, the RSSCN 7 dataset, the AID dataset, and the NWPU-RESISC 45 dataset demonstrate the superiority of the proposed method over other state-of-the-art methods.

An adaptive variational Bayesian filter for nonlinear multi-sensor systems with unknown noise statistics

An improved adaptive variational Bayesian cubature information fusion algorithm for nonlinear multi-sensor systems with uncertain noise statistics is proposed in this paper. Aiming to estimate uncertain process and measurement noise covariances in nonlinear systems, the variational Bayesian theory is combined with the inverse Wishart distribution. System states and uncertain noise covariances are jointly estimated for nonlinear systems by means of cubature sampling, deriving the variational Bayesian cubature Kalman filter (VBCKF-QR). In addition, a variational Bayesian Cubature Information filter (VBCIF-QR) is proposed, and a distributed information feedback fusion algorithm is also derived for multi-sensor systems with unknown noise statistics. Simulation results demonstrate that the proposed VBCKF-QR/VBCIF-QR outperform conventional cubature Kalman/information filters.

A Multiscale Self-Adaptive Attention Network for Remote Sensing Scene Classification

High-resolution optical remote sensing image classification is an important research direction in the field of computer vision. It is difficult to extract the rich semantic information from remote sensing images with many objects. In this paper, a multiscale self-adaptive attention network (MSAA-Net) is proposed for the optical remote sensing image classification, which includes multiscale feature extraction, adaptive information fusion, and classification. In the first part, two parallel convolution blocks with different receptive fields are adopted to capture multiscale features. Then, the squeeze process is used to obtain global information and the excitation process is used to learn the weights in different channels, which can adaptively select useful information from multiscale features. Furthermore, the high-level features are classified by many residual blocks with an attention mechanism and a fully connected layer. Experiments were conducted using the UC Merced, NWPU, and the Google SIRI-WHU datasets. Compared to the state-of-the-art methods, the MSAA-Net has great effect and robustness, with average accuracies of 94.52%, 95.01%, and 95.21% on the three widely used remote sensing datasets.

Fuzzy adaptive nonlinear information fusion model predictive attitude control of unmanned rotorcrafts

This study proposes a novel nonlinear information fusion model predictive control (NIFMPC) approach for the attitude control of unmanned rotorcraft subject to external wind disturbance. In the proposed control structure, the fusion estimation of control input is obtained by fusing the soft constraint information and the hard one via the nonlinear information fusion estimation method. The output error convergence of the closed-loop system is analyzed using the Lyapunov stability theory. In order to avoid input saturation, the fuzzy logic system (FLS) is adopted to design the adaptive fuzzy gain-scheduling (AFGS)-NIFMPC, in which the soft constraint information related to control action are adaptively scheduled according to fuzzy rules, with control input and its rate of change as FLS inputs and soft constraint information as FLS outputs. Finally, simulation results are presented to demonstrate the effectiveness and robustness of the proposed control strategies.

Siamese Visual Tracking With Deep Features and Robust Feature Fusion

Trackers based on fully-convolutional Siamese networks regard tracking as a process of learning a similarity function. By utilizing shallow networks and off-line training, Siamese trackers can achieve high tracking speed and perform well in some simple scenes. However, due to the less semantic information and the invariant template, Siamese trackers still have a gap compared with the state-of-the-art methods in complex scenes and other challenging problems (Occlusion, Deformation, etc.). In this paper, we propose a Siamese tracking algorithm with deep features and robust feature fusion (SiamDF). The improved ResNet-18 network is utilized to replace the traditional shallow network and extract the deep features with more semantic information. For eliminating the negative effect of padding and making better use of the deep network, the proposed algorithm adopts the spatial aware sampling strategy to overcome the strict translation invariance. Meanwhile, a final response map with high quality can be obtained by using the multi-layer feature fusion. Thus, the tracker can significantly reduce the impact of the distractors in complex scenes. In addition, an adaptive feature information fusion is adopted to update the template, so that the algorithm can adapt to various changes of the target appearance. Objective evaluation on the OTB100 dataset shows that the precision and the overlap success can reach 0.852 and 0.658 respectively. Moreover, the EAO value evaluated on the VOT2016 database can reach 0.336. These results demonstrate that our algorithm can effectively improve the tracking performance and perform favorably in both robustness and accuracy.