Complementarity Of Information Research Articles

3D Point Cloud Semantic Segmentation based on Multi-scale Dense Nested Networks

Aiming at the problem that the relationship between geometric features and semantic features is ignored in the point cloud data downsampling process, which leads to inaccurate segmentation of object boundaries and structural details, this paper proposes a 3D point cloud semantic segmentation network based on multi-scale dense nested type. Firstly, a dense nested network architecture is constructed by nesting multiple multi-scale feature fusion modules to fuse multi-scale features of different directions between encoder-decoder paths, so as to effectively propagate local ge-ometric context information and enhance the ability of cross-scale information interaction. Secondly, a local feature aggregation unit is constructed in the multi-scale feature fusion module, which strengthens the structural awareness within the local point set based on graph convolution and attention mechanism, and promotes the complementarity of local geometric features and abstract semantic information. Then, the cross-layer multi-loss supervision module is combined to further optimize the multi-scale feature propagation, which makes the network training more stable and improves the point cloud segmentation accuracy. Finally, this paper verified the proposed network on the S3DIS dataset. The experimental results show that the proposed network has the mean intersection over Union of 71.2% and the overall accuracy of 88.7%, which is 1.2 and 0.7 percentage points higher than RandLA-Net, respectively, which proves that the proposed network can effectively improve the accuracy of 3D point cloud semantic segmentation.

Complementarity of which-path information in induced and stimulated coherences via four-wave mixing process from warm Rb atomic ensemble: publisher’s note

This publisher’s note reports a correction made to Optica Quantum 2, 288 (2024)10.1364/OPTICAQ.528135.

Visible and near‐infrared image fusion based on information complementarity

AbstractImages with complementary spectral information can be recorded using image sensors that can identify visible and near‐infrared spectrum. The fusion of visible and near‐infrared (NIR) aims to enhance the quality of images acquired by video monitoring systems for the ease of user observation and data processing. Unfortunately, current fusion algorithms produce artefacts and colour distortion since they cannot make use of spectrum properties and are lacking in information complementarity. Therefore, an information complementarity fusion (ICF) model is designed based on physical signals. In order to separate high‐frequency noise from important information in distinct frequency layers, the authors first extracted texture‐scale and edge‐scale layers using a two‐scale filter. Second, the difference map between visible and near‐infrared was filtered using the extended‐DoG filter to produce the initial visible‐NIR complementary weight map. Then, to generate a guide map, the near‐infrared image with night adjustment was processed as well. The final complementarity weight map was subsequently derived via an arctanI function mapping using the guide map and the initial weight maps. Finally, fusion images were generated with the complementarity weight maps. The experimental results demonstrate that the proposed approach outperforms the state‐of‐the‐art in both avoiding artificial colours as well as effectively utilising information complementarity.

MRCFN: A multi-sensor residual convolutional fusion network for intelligent fault diagnosis of bearings in noisy and small sample scenarios

Bearing fault diagnosis is of great importance to ensure the safe and stable operation of mechanical equipment. The actual collected bearing fault signals are susceptible to strong noise interference and bearing samples for each fault state may be insufficient, which increases the difficulty of capturing effective features. Most of the existing diagnostic methods extract features from a single sensor signal for pattern recognition and fault diagnosis. The fault information provided by a single sensor is limited and incomplete, which is usually very difficult to meet the demand for accurate and reliable fault diagnosis in complex scenarios. To solve these problems, this paper proposes a multi-sensor residual convolutional fusion network (MRCFN) for intelligent fault diagnosis of bearings. Firstly, a convolutional pooling module (CPM) is coupled with the designed double ring residual module (DRRM) to rough feature extraction and deep feature mining, which not only captures the discriminative fault features from multi-sensor signal, but also avoids the performance degradation of network. Secondly, a spatial channel reconstruction module (SCRM) is further introduced to eliminate redundant information in the features and improve the network training efficiency. Finally, the presented global interactive perception fusion module (GIPFM) is connected with a classification block (CB) to globally fuse the features extracted from the acoustic and vibration signals and conduct automatic fault identification, which both can realize the complementarity and calibration of multi-sensor feature information and high precision diagnosis. The experiments and a series of comparisons are implemented on two datasets to efficaciously verify the superiority of the proposed method over five existing representative multi-sensor fusion diagnosis methods (i.e., FAC-CNN, MB-CNN, MsACNN, MRSDF, and IMSFDFL) under strong noise and small samples.

A facial expression recognition network based on attention double branch enhanced fusion.

The facial expression reflects a person's emotion, cognition, and even physiological or mental state to a large extent. It has important application value in medical treatment, business, criminal investigation, education, and human-computer interaction. Automatic facial expression recognition technology has become an important research topic in computer vision. To solve the problems of insufficient feature extraction, loss of local key information, and low accuracy in facial expression recognition, this article proposes a facial expression recognition network based on attention double branch enhanced fusion. Two parallel branches are used to capture global enhancement features and local attention semantics respectively, and the fusion and complementarity of global and local information is realized through decision-level fusion. The experimental results show that the features extracted by the network are made more complete by fusing and enhancing the global and local features. The proposed method achieves 89.41% and 88.84% expression recognition accuracy on the natural scene face expression datasets RAF-DB and FERPlus, respectively, which is an excellent performance compared with many current methods and demonstrates the effectiveness and superiority of the proposed network model.

Multi-modality hierarchical attention networks for defect identification in pipeline MFL detection

Abstract Magnetic flux leakage (MFL) testing is widely used for acquiring MFL signals to detect pipeline defects, and data-driven approaches have been effectively investigated for MFL defect identification. However, with the increasing complexity of pipeline defects, current methods are constrained by the incomplete information from single modal data, which fail to meet detection requirements. Moreover, the incorporation of multimodal MFL data results in feature redundancy. Therefore, the multi-modality hierarchical attention networks (MMHAN) are proposed for defect identification. Firstly, stacked residual blocks with cross-level attention module (CLAM) and multiscale 1D-CNNs with multiscale attention module are utilized to extract multiscale defect features. Secondly, the multi-modality feature enhancement attention module (MMFEAM) is developed to enhance critical defect features by leveraging correlations among multimodal features. Lastly, the multi-modality feature fusion attention module (MMFFAM) is designed to dynamically integrate multimodal features deeply, utilizing the consistency and complementarity of multimodal information. Extensive experiments were conducted on multimodal pipeline datasets to assess the proposed MMHAN. The experimental results demonstrate that MMHAN achieves a higher identification accuracy, validating its exceptional performance.

Fabric defect detection based on feature enhancement and complementary neighboring information

Abstract Fabric defect detection is a crucial aspect of quality control in the textile industry. Given the complexities of fabric backgrounds, the high similarity between patterned backgrounds and defects, and the variety of defect scales, we propose a fabric defect detection method based on feature enhancement and complementary neighboring information . The core of this method lies in two main components: the feature enhancement module and the neighboring information complementation strategy. The feature enhancement module includes two sub-modules: similarity feature enhancement(SFE) and edge detail feature enhancement(EDFE). The SFE aims to capture the similarities between features to strengthen the distinction between defects and complex backgrounds, thereby highlighting the correlations among defects and the differences between defects and the background. The EDFE focuses on improving the network's ability to capture the edge details of fabrics, preventing edge information from becoming blurred or lost due to deeper network layers. The neighboring information complementation strategy consists of shallow-level information complementation(SLIC) and top-down information fusion complementation(TDIFC). The SLIC integrates newly introduced shallow features with neighboring features that have a smaller semantic gap, injecting richer detail information into the network. The TDIFC adaptively guides the interaction of information between adjacent feature maps, effectively aggregating multi-scale features to ensure information complementarity between features of different scales. Additionally, to further optimize model performance, we introduced partial convolution(PConv) in the backbone of the feature extraction network. PConv reduces redundant computations and decreases the model's parameter count. Experimental results show that our proposed method achieved an mAP@50 of 82.4%, which is a 6.6% improvement over the baseline model YOLOv8s. The average inference frame rate reached 61.8 FPS, meeting the real-time detection requirements for fabric defects. Moreover, the model demonstrated good generalization capabilities, effectively adapting to detecting defects in different types and colors of fabrics.

A dynamic graph structural framework for implicit sentiment identification based on complementary semantic and structural information

Implicit sentiment identification has become the classic challenge in text mining due to its lack of sentiment words. Recently, graph neural network (GNN) has made great progress in natural language processing (NLP) because of its powerful feature capture ability, but there are still two problems with the current method. On the one hand, the graph structure constructed for implicit sentiment text is relatively single, without comprehensively considering the information of the text, and it is more difficult to understand the semantics. On the other hand, the constructed initial static graph structure is more dependent on human labor and domain expertise, and the introduced errors cannot be corrected. To solve these problems, we introduce a dynamic graph structure framework (SIF) based on the complementarity of semantic and structural information. Specifically, for the first problem, SIF integrates the semantic and structural information of the text, and constructs two graph structures, structural information graph and semantic information graph, respectively, based on specialized knowledge, which complements the information between the two graph structures, provides rich semantic features for the downstream identification task, and helps to understanding of the contextual information between implicit sentiment semantics. To deal with the second issue, SIF dynamically learns the initial static graph structure to eliminate the noise information in the graph structure, preventing noise accumulation that affects the performance of the downstream identification task. We compare SIF with mainstream natural language processing methods in three publicly available datasets, all of which outperform the benchmark model. The accuracy on the Puns of day dataset, SemEval-2021 task 7 dataset, and Reddit dataset reaches 95.73%, 85.37%, and 65.36%, respectively. The experimental results demonstrate a good application scenario for our proposed method on implicit sentiment identification tasks.

Complementarity of which-path information in induced and stimulated coherences via four-wave mixing process from warm Rb atomic ensemble.

Complementarity of which-path information in induced and stimulated coherences via four-wave mixing process from warm Rb atomic ensemble.

Difficult Airway Assessment Based on Multi-View Metric Learning.

The preoperative assessment of difficult airways is of great significance in the practice of anesthesia intubation. In recent years, although a large number of difficult airway recognition algorithms have been investigated, defects such as low recognition accuracy and poor recognition reliability still exist. In this paper, we propose a Dual-Path Multi-View Fusion Network (DMF-Net) based on multi-view metric learning, which aims to predict difficult airways through multi-view facial images of patients. DMF-Net adopts a dual-path structure to extract features by grouping the frontal and lateral images of the patients. Meanwhile, a Multi-Scale Feature Fusion Module and a Hybrid Co-Attention Module are designed to improve the feature representation ability of the model. Consistency loss and complementarity loss are utilized fully for the complementarity and consistency of information between multi-view data. Combined with Focal Loss, information bias is effectively avoided. Experimental validation illustrates the effectiveness of the proposed method, with the accuracy, specificity, sensitivity, and F1 score reaching 77.92%, 75.62%, 82.50%, and 71.35%, respectively. Compared with methods such as clinical bedside screening tests and existing artificial intelligence-based methods, our method is more accurate and reliable and can provide a reliable auxiliary tool for clinical healthcare personnel to effectively improve the accuracy and reliability of preoperative difficult airway assessments. The proposed network can help to identify and assess the risk of difficult airways in patients before surgery and reduce the incidence of postoperative complications.

Semi-Identical Twins Variational AutoEncoder for Few-Shot Learning.

Data augmentation is a popular way for few-shot learning (FSL). It generates more samples as supplements and then transforms the FSL task into a common supervised learning problem for a solution. However, most data-augmentation-based FSL approaches only consider the prior visual knowledge for feature generation, thereby leading to low diversity and poor quality of generated data. In this study, we attempt to address this issue by incorporating both prior visual and prior semantic knowledge to condition the feature generation process. Inspired by some genetic characteristics of semi-identical twins, a novel multimodal generative FSL approach was developed named semi-identical twins variational autoencoder (STVAE) to better exploit the complementarity of these modality information by considering the multimodal conditional feature generation process as a process that semi-identical twins are born and collaborate to simulate their father. STVAE conducts feature synthesis by pairing two conditional variational autoencoders (CVAEs) with the same seed but different modality conditions. Subsequently, the generated features of two CVAEs are considered as semi-identical twins and adaptively combined to yield the final feature, which is considered as their fake father. STVAE requires that the final feature can be converted back into its paired conditions while ensuring these conditions remain consistent with the original in both representation and function. Moreover, STVAE is able to work in the partial modality-absence case due to the adaptive linear feature combination strategy. STVAE essentially provides a novel idea to exploit the complementarity of different modality prior information inspired by genetics in FSL. Extensive experimental results demonstrate that our work achieves promising performances in comparison to the recent state-of-the-art approaches, as well as validate its effectiveness on FSL under various modality settings.

MSDE-Net: A Multi-Scale Dual-Encoding Network for Surgical Instrument Segmentation.

Minimally invasive surgery, which relies on surgical robots and microscopes, demands precise image segmentation to ensure safe and efficient procedures. Nevertheless, achieving accurate segmentation of surgical instruments remains challenging due to the complexity of the surgical environment. To tackle this issue, this paper introduces a novel multiscale dual-encoding segmentation network, termed MSDE-Net, designed to automatically and precisely segment surgical instruments. The proposed MSDE-Net leverages a dual-branch encoder comprising a convolutional neural network (CNN) branch and a transformer branch to effectively extract both local and global features. Moreover, an attention fusion block (AFB) is introduced to ensure effective information complementarity between the dual-branch encoding paths. Additionally, a multilayer context fusion block (MCF) is proposed to enhance the network's capacity to simultaneously extract global and local features. Finally, to extend the scope of global feature information under larger receptive fields, a multi-receptive field fusion (MRF) block is incorporated. Through comprehensive experimental evaluations on two publicly available datasets for surgical instrument segmentation, the proposed MSDE-Net demonstrates superior performance compared to existing methods.

Privacy preservation-based federated learning with uncertain data

Federated learning (FL) belongs to distributed machine learning. It allows data information sharing between users while protecting their data privacy at the same time. However, in many real-world scenarios, the data collected by client devices may be affected by noise in the working environment, leading to the decreased accuracy and confidence. Therefore, it is necessary to take measures to reduce data uncertainty in order to enhance the performance of FL algorithms. Traditional FL methods encounter challenges in handling uncertain data, which motivates the introduction of multi-view learning in this paper which designs an FL model suitable for highly variable data characteristics. We first achieve information complementarity among data views while ensuring the consistency in data representation. Furthermore, we quantify data uncertainty using the reachable region of data noise, thereby improving model robustness. To maintain data privacy between clients, we design an adaptive Kalman filter-based differential protection security protocol. Clients use the protocol to process local data and upload it to the master server, which returns the updated model parameters to the clients. The experimental results demonstrate the effectiveness of the federated learning model proposed in this paper.

Peer effect and funding success: Analyzing friendship networks in online credit markets

The challenge of limited information in online credit markets is well-recognized. This study examines the peer effect in microloan transactions using data from a leading Chinese lending platform. Results show that the likelihood of a borrower's funding success is higher when a higher proportion of online friends succeed. Both borrowers’ network centrality and digital footprints strengthen this effect, suggesting two mechanisms: social learning and informational complementarity. The peer effect is stronger among borrowers with larger networks, more loan experience, better repayment, and older age. The research provides new insights to mitigate information asymmetry in financial markets and guide investment decisions.

Multichannel Multimodal Emotion Analysis of Cross-Modal Feedback Interactions Based on Knowledge Graph

Multimodal sentiment analysis is a downstream branch task of sentiment analysis with high attention at present. Previous work in multimodal sentiment analysis have focused on the representation and fusion of modalities, capturing the underlying semantic relationships between modalities by considering contextual information. While this approach is feasible for simple contextual comments, more complex comments require the integration of external knowledge to obtain more accurate sentiment information. However, incorporating external knowledge into sentiment analysis to enhance information complementarity has not been thoroughly investigated. To address this, we propose a multichannel cross-modal feedback interaction model that incorporates the knowledge graph into multimodal sentiment analysis. Our proposed model consists of two main components: the cross-modal feedback recurrent interaction module and the external knowledge module for capturing latent information. The cross-modal interaction employs a self-feedback mechanism during network training, extracting feature representations of each modality and using these representations to mask sensory inputs, allowing the model to perform feedback-based feature masking. The external knowledge graph captures potential semantic information representations in the textual data through knowledge graph embedding. Finally, a global feature fusion module is employed for multichannel multimodal information integration. On two publicly available datasets, our method demonstrates good performance in terms of accuracy and F1 scores, compared to state-of-the-art models and several baselines.

Synergistic fusion network for landslide segmentation: the vital complementarity of geological information to remote sensing imagery

In the contemporary technological environment, access to a multitude of data sources, such as digital elevation models (DEM), slope data, and other geological information, has become essential for augmenting remote sensing imagery in landslide analysis. These datasets enhance landslide identification capabilities, but existing methods often fail to fully leverage their unique characteristics due to oversimplified fusion strategies. To overcome this challenge, we introduce the Synergistic Fusion Network (SFNet), which is specifically designed to utilize the complementary nature of geological data to significantly improve the accuracy and efficiency of landslide segmentation. SFNet incorporates Multi-Source Fusion Modules (MSFM) and Reverse Feature Refinement Modules(RFRM), through an innovative network architecture, enhancing remote sensing imagery with integrated geological information. In our experiments, SFNet outperforms existing mainstream networks, demonstrating superior F1 scores, thereby showcasing its ability to effectively segment landslide regions with high precision. This advancement highlights the importance of considering the unique contributions of each data source, especially the complementary effects of geological information.

Spatial-Temporal Characteristics of Brain Activity in Autism Spectrum Disorder Based on Hidden Markov Model and Dynamic Graph Theory: A Resting-State fMRI Study.

Autism spectrum disorder (ASD) is a common neurodevelopmental disorder. Functional magnetic resonance imaging (fMRI) can be used to measure the temporal correlation of blood-oxygen-level-dependent (BOLD) signals in the brain to assess the brain's intrinsic connectivity and capture dynamic changes in the brain. In this study, the hidden Markov model (HMM) and dynamic graph (DG) theory are used to study the spatial-temporal characteristics and dynamics of brain networks based on dynamic functional connectivity (DFC). By using HMM, we identified three typical brain states for ASD and healthy control (HC). Furthermore, we explored the correlation between HMM time-varying properties and clinical autism scale scores. Differences in brain topological characteristics and dynamics between ASD and HC were compared by DG analysis. The experimental results indicate that ASD is more inclined to enter a strongly connected HMM brain state, leading to the isolation of brain networks and alterations in the topological characteristics of brain networks, such as default mode network (DMN), ventral attention network (VAN), and visual network (VN). This work suggests that using different data-driven methods based on DFC to study brain network dynamics would have better information complementarity, which can provide a new direction for the extraction of neuro-biomarkers in the early diagnosis of ASD.

ADHD diagnosis guided by functional brain networks combined with domain knowledge

Utilizing functional magnetic resonance imaging (fMRI) to model functional brain networks (FBNs) is increasingly prominent in attention-deficit/hyperactivity disorder (ADHD) research, revealing neural impact and mechanisms through the exploration of activated brain regions. However, current FBNs-based methods face two major challenges. The primary challenge stems from the limitations of existing modeling methods in accurately capturing both regional correlations and long-distance dependencies (LDDs) within the dynamic brain, thereby affecting the diagnostic accuracy of FBNs as biomarkers. Additionally, limited sample size and class imbalance also pose a challenge to the learning performance of the model. To address the issues, we propose an automated diagnostic framework, which integrates modeling, multimodal fusion, and classification into a unified process. It aims to extract representative FBNs and efficiently incorporate domain knowledge to guide ADHD classification. Our work mainly includes three-fold: 1) A multi-head attention-based region-enhancement module (MAREM) is designed to simultaneously capture regional correlations and LDDs across the entire sequence of brain activity, which facilitates the construction of representative FBNs. 2) The multimodal supplementary learning module (MSLM) is proposed to integrate domain knowledge from phenotype data with FBNs from neuroimaging data, achieving information complementarity and alleviating the problems of insufficient medical data and unbalanced sample categories. 3) An ADHD automatic diagnosis framework guided by FBNs and domain knowledge (ADF-FAD) is proposed to help doctors make more accurate decisions, which is applied to the ADHD-200 dataset to confirm its effectiveness. The results indicate that the FBNs extracted by MAREM perform well in modeling and classification. After with MSLM, the model achieves accuracy of 92.4%, 74.4%, and 80% at NYU, PU, and KKI, respectively, demonstrating its ability to effectively capture crucial information related to ADHD diagnosis. Codes are available at https://github.com/zhuimengxuebao/ADF-FAD.

MFMENet: multi-scale features mutual enhancement network for change detection in remote sensing images

ABSTRACT Change detection, an important task in remote sensing image analysis, has been extensively studied in recent years. However, change detection still faces problems such as difficulty in detecting small targets and incomplete edge detection. Furthermore, pseudo changes such as seasonal changes can also lead to many false detections. In response to these challenges, we propose the Multi-scale Features Mutual Enhancement Network (MFMENet), a simple yet efficient network. MFMENet maximizes feature utilization through mutual guidance and supplementation, enhancing the detection capabilities for small targets and edges. First, we use a lightweight feature extraction network to extract features, which mitigates the information loss caused by continuous downsampling of an overly deep network structure. Then, we design a Context Adaptive Interaction Module (CAIM) to realize the complementarity of feature information at different levels. This facilitates shallow features in gaining more semantic information and deep features in acquiring more texture information, thereby enhancing the model’s capability to capture more comprehensive edge features while effectively mitigating interference from pseudo changes. Finally, we introduce a Feature Aggregation Comparison Module (FACM), which uses a combination of aggregation and comparison methods to refine and enhance features. FACM can not only highlight the changed features but also retain more details, improving the model’s detection ability of small targets and edge details. The full utilization of features and effective mutual enhancement of information ensure the improvement of MFMENet’s performance in small target and edge detection. Extensive experiments on three publicly available datasets (LEVIR, DSIFN, and CDD) demonstrate that our approach achieves superior performance with fewer parameters compared to state-of-the-art methods in recent years. In comparison to these baseline methods, our proposed approach achieves improvements of 0.98%, 12.24%, and 2.03% in the IOU metric on the LEVIR, DSIFN, and CDD datasets, respectively, while utilizing only 1.1 M parameters.

Pixel-wise crack defect segmentation with dual-encoder fusion network

Automatic crack defect detection plays an important role in early road maintenance. However, the crack defect detection accuracy is seriously affected by some challenging factors, such as complex background, class imbalance issue, poor texture, etc. In this paper, a deep crack defect segmentation network with dual-path encoding and hierarchical fusion schemes, named DEHF-Net, is presented to accurately and effectively segment the crack defects generated in crack images. In order to capture rich feature information of crack defects, a dual-path encoder unit is built to extract the spatial information and context information from crack images simultaneously. On the basis, to strengthen the features extracted from two encoding paths, an attention fusion (AF) block and feature enhancement (FE) block are proposed for effective feature learning to achieve information complementarity of dual-branch encoding paths. Meanwhile, to alleviate the problem of semantic gaps arising from large information differences between the encoding path and decoding path, a residual refinement unit (RRU) is also introduced to enable effective refinement of edges and details. Finally, to make the segmentation results more significant on multi-scale cracks, a weighted hierarchical fusion (WHF) block is introduced in the decoding stage to fuse the shallow texture information with the deep semantic information more effectively. Experimental results show the proposed model has obtained an excellent performance comparison with advanced methods.