Multi-domain Information Research Articles

A multi‐feature fusion graph attention network for decoding motor imagery intention in spinal cord injury patients

Objective.Electroencephalogram (EEG) signals exhibit temporal-frequency-spatial multi-domain feature, and due to the nonplanar nature of the brain surface, the electrode distributions follow non-Euclidean topology. To fully resolve the EEG signals, this study proposes a temporal-frequency-spatial multi-domain feature fusion graph attention network (GAT) for motor imagery (MI) intention recognition in spinal cord injury (SCI) patients.Approach.The proposed model uses phase-locked value (PLV) to extract spatial phase connectivity information between EEG channels and continuous wavelet transform to extract valid EEG information in the time-frequency domain. It then models as a graph data structure containing multi-domain information. The gated recurrent unit and GAT learn EEG's dynamic temporal-spatial information. Finally, the fully connected layer outputs the MI intention recognition results.Main results.After 10 times 10-fold cross-validation, the proposed model can achieve an average accuracy of 95.82%. Furthermore, this study analyses the event-related desynchronization/event-related synchronization and PLV brain network to explore the brain activity of SCI patients during MI.Significance.This study confirms the potential of the proposed model in terms of EEG decoding performance and provides a reference for the mechanism of neural activity in SCI patients.

PMCN: Parallax-motion collaboration network for stereo video dehazing

Despite progress in learning-based stereo dehazing, few studies have focused on stereo video dehazing (SVD). Existing methods may fall short in the SVD task by not fully leveraging multi-domain information. To address this gap, we propose a parallax-motion collaboration network (PMCN) that integrates parallax and motion information for efficient stereo video fog removal. We delicately design a parallax-motion collaboration block (PMCB) as the critical component of PMCN. Firstly, to capture binocular parallax correspondences more efficiently, we introduce a window-based parallax attention mechanism (W-PAM) in the parallax interaction module (PIM) of PMCB. By horizontally splitting the whole frame into multiple windows and extracting parallax relationships within each window, memory usage and runtime can be reduced. Meanwhile, we further conduct horizontal feature modulation to handle cross-window disparity variations. Secondly, a motion alignment module (MAM) based on deformable convolution explores the temporal correlation in the feature space for an independent view. Finally, we propose a fog-adaptive refinement module (FARM) to refine the features after interaction and alignment. FARM incorporates fog prior information and guides the network in dynamically generating processing kernels for dehazing to adapt to different fog scenarios. Quantitative and qualitative results demonstrate that the proposed PMCN outperforms state-of-the-art methods on both synthetic and real-world datasets. In addition, our PMCN also benefits the accuracy improvement for high-level vision tasks in fog scenes, e.g., object detection and stereo matching.

Depression recognition using high-order generalized multilayer brain functional network fused with EEG multi-domain information

Major Depressive Disorder (MDD) is a serious and highly heterogeneous psychological disorder. According to the network hypothesis, depression originates from abnormal neural network information processing, typically resulting in aberrant changes in the topological structure of the brain’s functional network. Recent evidence further reveals that depression involves dynamic changes related to both within- and cross-frequency coupling. Therefore, we utilize second-order tensor expansion to integrate frequency- and time-varying multilayer brain functional networks based on node sharing, thus propose a generalized multilayer brain functional network (GMBFN) incorporating multi-domain information. Concurrently, we derive global and local topological properties from both the frequency and temporal domains to characterize the novel network structure. To uncover more reliable biomarkers and explore various coupling features that can assess the interaction between signals from different perspectives, we conduct research in two datasets employing four sets of within- and cross-frequency coupling. Leveraging the novel multi-domain high-order GMBFNs, abnormalities of information integration abilities in patients with MDD are observed, particularly in the theta-band and overall temporal-domain. Through the fusion of topological properties across both domains with multiple classifiers, the alpha-band can serve as a potential biomarker for depression identification. More importantly, the combination of global topological properties from both domains, on average, enhances the classification performance for identifying patients with MDD by 5.18% compared to using just one domain. This study presents a systematic framework for comprehending the aberrant mechanisms of MDD from multiple perspectives, offering significant value for clinical applications aimed at assisting in depression diagnosis and intervention.

VTNet: A multi-domain information fusion model for long-term multi-variate time series forecasting with application in irrigation water level

Time series forecasting is intricately tied to production and life, garnering widespread attention over an extended period. Enhancing the performance of long-term multivariate time series forecasting (MTSF) poses a highly challenging task, as it requires mining complicated and obscure temporal patterns in many aspects. For this reason, this paper proposes a long-term forecasting model based on multi-domain fusion (VTNet) to adaptively capture and refine multi-scale intra- and inter-variate dependencies. In contrast to previous techniques, we devise a dual-stream learning architecture. Firstly, the fast Fourier Transform (FFT) is adopted to extract frequency domain information. The original sequences are then transformed into 2D visual features in the temporal-frequency domain, and a 2D-TBlock is designed for multi-scale dynamic learning. Secondly, a combination of convolution and recurrent networks continues to explore the local temporal features and preserve the global trend. Finally, multi-modal circulant fusion is applied to achieve a more comprehensive and enriched feature fusion representation, further promoting overall performance. Extensive experiments are conducted on 9 public benchmark datasets and the real-world irrigation water level to showcase VTNet’s promoted performance and generalization. Moreover, VTNet yields 46.93% and 25.36% relative improvements for water level forecasting, revealing its potential application value in water-saving planning and extreme event early warning.

Tools and Strategies to Integrate Multi-Domain Information for Personalized Decision-Making in Oncological Care Pathways: A Scoping Review.

There is a growing interest in personalized decision-making in oncology. According to the Integrated Oncological Decision-Making Model (IODM), decisions should be based on information from three domains: (1) medical technical information, (2) patients' general health status and (3) patients' preferences and goals. Little is known about what kind of tool/strategy is used to collect the information, by whom this is collected (nurse, clinician) when this is collected (moment in the care pathway), and how this information should be collected and integrated within decision-making in oncological care pathways, and what its impact is. We searched PUBMED, Embase and Web of Science in October 2023 for studies looking at tools to collect and integrate information from the three domains of the IODM. We extracted data on the content and implementation of these tools, and on decision and patient outcomes. The search yielded 2576 publications, of which only seven studies described collection of information from all three domains (inclusion criteria). In the seven included studies, information on the three domains was collected through dialogue, questionnaires, and assessments (what) by a nurse (2 out of 7 studies) or by other members of the Multi-Disciplinary Team (by whom) (5 out of 7 studies). Members of the Multi-Disciplinary Team subsequently integrated the information (5 out 7 studies) during their meeting (when), with patients and family attending this meeting in 2 studies (how). In terms of decision outcomes, 5 out of 7 studies compared the treatment recommendations before and after implementation of the tools, showing a modification of the treatment plan in 3% to 53% of cases. The limited data on patient outcomes suggest positive effects on well-being and fewer complications (3 out of 7 studies). The seven studies identified that integrated information from the three IODM domains into treatment decision-making lacked comprehensive information regarding the strategies, process, timing and individuals involved in implementing the tools. Nevertheless, the few studies that looked at patient outcomes showed promising findings.

DPSG: Dynamic Propagation Social Graphs for multi-modal fake news detection

Fake news can mislead public perception and have a negative impact on people’s lives. With the increase of multi-modal news content in social media, multi-modal fake news detection has received extensive attention from researchers. However, existing methods face difficulties in capturing temporal intensity and deep social graph representations. Considering the multi-modality of news content and the relationship between entities, this paper proposes a multi-modal fake news detection model based on dynamic propagation social graph (DPSG). Specifically, it first utilizes the temporal penetration fusion block to perform multi-modal dynamic fusion on various types of nodes in the propagation sequence, and then employs dynamic signed weighting to enhance the social graph features of the propagation graph. Finally, the Transformer captures news propagation patterns and fuses multi-domain information. Experimental results on three datasets demonstrate the effectiveness of the proposed model on the fake news detection task.

Cognitive radio network architecture for GEO and LEO satellites shared downlink spectrum

The fixed spectrum assignment policy in the space sector and large constellations of Low Earth Orbit (LEO) satellites left little or no spectrum available for future LEO satellite communications services. Cognitive Radio (CR) technology enables spectrum sharing between primary and secondary users without limiting the transmission power, and thus is of great interest to commercial and defense entities. A number of Radio Environment Map (REM) techniques have been making Cognitive Radio Networks (CRNs) practical by constructing a comprehensive map of the CRN by utilizing multi-domain information from geolocation databases, characteristics of spectrum use, geographical terrain models, propagation environment, and regulations. In this paper, we investigate spectrum sharing for a network comprised of a Geostationary Orbit (GEO) and a LEO satellite with a multibeam antenna array. A CRN architecture of GEO and LEO satellites shared downlink spectrum is proposed and details are provided covering its architecture, REM structure and channel utilization data aggregation, as well as a frequency slot assignment mechanism.

Cooperative Jamming Resource Allocation with Joint Multi-Domain Information Using Evolutionary Reinforcement Learning

Cooperative Jamming Resource Allocation with Joint Multi-Domain Information Using Evolutionary Reinforcement Learning

Fault transfer diagnosis of rolling bearings across different devices via multi-domain information fusion and multi-kernel maximum mean discrepancy

The current deep learning-based intelligent diagnosis algorithms depend on large amounts of well-labeled data, but they may not perform well in engineering practice where the fault data available from a single device is limited. The significant difference in data distribution between different devices makes it more difficult to transfer diagnosis across devices. Therefore, aiming at the problem that the great difference in the data distribution of source and target domains in cross-device transfer diagnosis leads to the low diagnosis accuracy, this paper investigates a deep transfer network model based on multi-domain information fusion and multi-kernel maximum mean discrepancy (MK-MMD) for fault diagnosis of rolling bearings across different devices. Firstly, to address the problem that single-domain information is difficult to adequately characterize rolling bearing health status information of different devices, a multi-domain information feature extractor based on multi-attention mechanism is constructed to capture the important features of vibration signals in different transform domains. Secondly, the extracted multi-domain features are input into the bidirectional gated recurrent unit (BiGRU) network for feature fusion, and MK-MMD is used to narrow the distribution distance between the source domain and target domain to obtain domain invariant features, and then achieve the transfer diagnosis of rolling bearing faults between different devices. Finally, the method investigated is verified by rolling bearing fault transfer diagnosis tests between different devices, and the results show that the suggested method can adapt to the feature distribution between different domains, and improve the transfer diagnosis accuracy between different devices.

Steganalysis of AMR Speech Stream Based on Multi-Domain Information Fusion

Steganalysis of AMR Speech Stream Based on Multi-Domain Information Fusion

Dual-Decoupling With Frequency-Spatial Domains for Image Manipulation Localization.

Leveraging trace-rich features within embedded spaces has been established as effective in image manipulation localization (IML). Nevertheless, the feature of manipulated traces frequently comprises substantial redundant information only loosely related to IML tasks. This complexity has hindered existing methods in fully comprehending the essence of trace features. In light of this challenge, we introduce a novel decoupling representation learning network (DRN) tailored for IML. The DRN excels at decoupling intricate multidomain information and transforming it into representations directly pertinent to IML objectives. This is achieved through a meticulously designed frequency decoupling representation learning module (FDM) and spatial decoupling representation learning module (SDM). Specifically, the FDM operates by acquiring distinct low and high-frequency components to effectively decouple redundant information. The decoupled high-frequency components are then harnessed as intricate trace complements, enhancing the overall aggregation process. In addition, the redundant information is expertly separated into authentic and manipulated representations through the use of channel activation maps in SDM. Through extensive experimentation on three public benchmarks including CASIA, NIST, and Coverage, our method consistently demonstrates superior performance and enhanced robustness compared with existing state-of-the-art methods.

SiamEFT: adaptive-time feature extraction hybrid network for RGBE multi-domain object tracking.

Integrating RGB and Event (RGBE) multi-domain information obtained by high-dynamic-range and temporal-resolution event cameras has been considered an effective scheme for robust object tracking. However, existing RGBE tracking methods have overlooked the unique spatio-temporal features over different domains, leading to object tracking failure and inefficiency, especally for objects against complex backgrounds. To address this problem, we propose a novel tracker based on adaptive-time feature extraction hybrid networks, namely Siamese Event Frame Tracker (SiamEFT), which focuses on the effective representation and utilization of the diverse spatio-temporal features of RGBE. We first design an adaptive-time attention module to aggregate event data into frames based on adaptive-time weights to enhance information representation. Subsequently, the SiamEF module and cross-network fusion module combining artificial neural networks and spiking neural networks hybrid network are designed to effectively extract and fuse the spatio-temporal features of RGBE. Extensive experiments on two RGBE datasets (VisEvent and COESOT) show that the SiamEFT achieves a success rate of 0.456 and 0.574, outperforming the state-of-the-art competing methods and exhibiting a 2.3-fold enhancement in efficiency. These results validate the superior accuracy and efficiency of SiamEFT in diverse and challenging scenes.

Engagement Recognition Using a Multi-Domain Feature Extraction Method Based on Correlation-Based Common Spatial Patterns

Engagement ability plays a fundamental role in allocating attentional resources and helps us perform daily tasks efficiently. Therefore, it is of great importance to recognize engagement level. Electroencephalography is frequently employed to recognize engagement for its objective and harmless nature. To fully exploit the information contained in EEG signals, an engagement recognition method integrating multi-domain information is proposed. The proposed method extracts frequency information by a filter bank. In order to utilize spatial information, the correlation-based common spatial patterns method is introduced and extended into three versions by replacing different correlation coefficients. In addition, the Hilbert transform helps to obtain both amplitude and phase information. Finally, features in three domains are combined and fed into a support vector machine to realize engagement recognition. The proposed method is experimentally validated on an open dataset composed of 29 subjects. In the comparison with six existing methods, it achieves the best accuracy of 87.74±5.98% in binary engagement recognition with an improvement of 4.03%, which proves its efficiency in the engagement recognition field.

Temporal-frequency-phase feature classification using 3D-convolutional neural networks for motor imagery and movement.

Recently, convolutional neural networks (CNNs) have been widely applied in brain-computer interface (BCI) based on electroencephalogram (EEG) signals. Due to the subject-specific nature of EEG signal patterns and the multi-dimensionality of EEG features, it is necessary to employ appropriate feature representation methods to enhance the decoding accuracy of EEG. In this study, we proposed a method for representing EEG temporal, frequency, and phase features, aiming to preserve the multi-domain information of EEG signals. Specifically, we generated EEG temporal segments using a sliding window strategy. Then, temporal, frequency, and phase features were extracted from different temporal segments and stacked into 3D feature maps, namely temporal-frequency-phase features (TFPF). Furthermore, we designed a compact 3D-CNN model to extract these multi-domain features efficiently. Considering the inter-individual variability in EEG data, we conducted individual testing for each subject. The proposed model achieved an average accuracy of 89.86, 78.85, and 63.55% for 2-class, 3-class, and 4-class motor imagery (MI) classification tasks, respectively, on the PhysioNet dataset. On the GigaDB dataset, the average accuracy for 2-class MI classification was 91.91%. For the comparison between MI and real movement (ME) tasks, the average accuracy for the 2-class were 87.66 and 80.13% on the PhysioNet and GigaDB datasets, respectively. Overall, the method presented in this paper have obtained good results in MI/ME tasks and have a good application prospect in the development of BCI systems based on MI/ME.

Physical model-based cascaded generative adversarial networks for accelerating quantitative multi-parametric magnetic resonance imaging

To explore the feasibility and interpretation of physical model- based cascaded generative adversarial networks for accelerating quantitative multi-echo multi-parametric magnetic resonance imaging using raw multi-echo multicoil k-space data. A physical model-based cascaded generative adversarial network is proposed to enhance image feature information to obtain high-quality reconstructed images using joint training of multi-domain information and learning of key parameters required for image reconstruction through a system matrix and adaptively optimizing the k-space generator and image generator structures. Raw multi-echo multi-coil k-space data are used to accelerate multi-contrast multi-parametric magnetic resonance imaging. A physically driven deep learning reconstruction method is used to increase the generalization capability and improve the model performance by building a system matrix function instead of direct end-to-end training of the model. In terms of overall image quality, the proposed model achieved significant improvements compared to other methods. On an 80- case test set, the average PSNR value of the reconstructed images was 34.13, SSIM was 0.965, and NRMSE was 0.114. In terms of multi-contrast multi-parametric image reconstruction, the model achieved PSNR values of 38.87 for PDW, 35.62 for T1W, and 34.38 for T2* Map, which were significantly better than those of other methods for quantitative evaluation. The model also produced clearer features of the brain gray matter, white matter, and cerebrospinal fluid. Furthermore, compared with the existing methods with a reconstruction time difference of less than 10%, the proposed method achieved the highest improvement of up to 20% in the metrics of PSNR, SSIM, and NRMSE. Compared with other existing methods, the physical model-based cascaded generative adversarial networks can reconstruct more image details and features, thus improving the quality and accuracy of the reconstructed images.

Generative adversarial networks driven by multi-domain information for improving the quality of generated samples in fault diagnosis

The performance of intelligent fault diagnosis models is often hindered by the lack of available samples, a common issue in both the few-shot learning and imbalanced learning problems. While data generation has been shown to be an effective strategy for addressing this issue, existing methods tend to focus solely on the similarity of the generated samples, overlooking their diversity. In this regard, a self-reasoning training strategy that enables the participation of highly reliable generated samples in the training process is proposed. By augmenting the training samples, the strategy provides the generation model with diverse input information, effectively enhancing the diversity of the samples generated by the model. To assess the reliability of the generated samples, Pearson correlation coefficient between the generated and real samples in the amplitude–frequency domain is utilized. To this end, an amplitude–frequency domain generation model is constructed. In order to avoid issues such as gradient disappearance and convergence degradation during the generation process in the amplitude–frequency domain, a phase-frequency domain generation model as well as a time domain discriminator model are developed. While the phase-frequency domain generation model enhances the diversity of the generated samples in that domain, the time domain discriminator model ensures sample similarity by correlating the amplitude–frequency domain generation model with the phase-frequency domain generation model. Through experiments, it is demonstrated that the proposed method can effectively address the overfitting problem of generative adversarial networks with few samples, improving the diversity of the generated samples. Moreover, the improvement in fault diagnosis performance achieved by the samples generated by the proposed method further underscores its potential and superiority in practical applications.

Gearbox Fault Diagnosis Method Based on Multidomain Information Fusion

Traditional methods of gearbox fault diagnosis rely heavily on manual experience. To address this problem, our study proposes a gearbox fault diagnosis method based on multidomain information fusion. An experimental platform consisting of a JZQ250 fixed-axis gearbox was built. An acceleration sensor was used to obtain the vibration signal of the gearbox. Singular value decomposition (SVD) was used to preprocess the signal in order to reduce noise, and the processed vibration signal was subjected to short-time Fourier transform to obtain a two-dimensional time–frequency map. A multidomain information fusion convolutional neural network (CNN) model was constructed. Channel 1 was a one-dimensional convolutional neural network (1DCNN) model that input a one-dimensional vibration signal, and channel 2 was a two-dimensional convolutional neural network (2DCNN) model that input short-time Fourier transform (STFT) time–frequency images. The feature vectors extracted using the two channels were then fused into feature vectors for input into the classification model. Finally, support vector machines (SVM) were used to identify and classify the fault types. The model training performance used multiple methods: training set, verification set, loss curve, accuracy curve and t-SNE visualization (t-SNE). Through experimental verification, the method proposed in this paper was compared with FFT-2DCNN, 1DCNN-SVM and 2DCNN-SVM in terms of gearbox fault recognition performance. The model proposed in this paper had the highest fault recognition accuracy (98.08%).

Intelligent identification of radar active jamming type based on multi-domain information fusion

Feature fusion is beneficial to improving the recognition rate of radar active jamming types, which usually costs a lot of computation and network complexity, however. Contrapose the disadvantage of low computational power and resource allocation of missile-borne radar, this paper introduces data enhancement theory into the feature fusion method and proposes an intelligent identification model of radar active jamming type based on multi-domain information fusion. The model first analyzes the time-frequency domain features of the signal and one-dimensional features such as its real part, imaginary part, frequency spectrum, and power spectrum, and then uses the Cutout&Patchup algorithm to fuse the one-dimensional and two-dimensional features into a new multi-domain information fusion matrix as the input of the recognition network. The simulation results show that this method greatly improves the recognition accuracy of active jamming types. Under the WideResNet28_2 classifier, the classification accuracy of active interference type reaches 88.06%, which is 0.79% higher than that before fusion.

Multi-domain ubiquitous digital twin model for information management of complex infrastructure systems

Digital twin is a comprehensive digital equivalent of an object or an activity, reflecting the semantic and geometric properties and behaviors through virtual models and data. Digital twin and related information technologies are widely used in the construction, operation and maintenance of infrastructure and facility. Transdisciplinary stakeholders are always involved in the long-term and cross-scene management of IoT and digital twin-enabled smart infrastructures and facilities. The intensive interactions among stakeholders often cause conflicts due to the variations in experience, knowledge, and interests. Moreover, with the change propagation of digital twins, cyber-physical resources can’t be efficiently and consistently established, connected, and utilized with multi-domain information through selective simplification and structured methods. This paper proposes a Ubiquitous Digital Twin model for the information management of complex infrastructure systems based on Domain-Driven Design. To achieve the unified and structured description, six domains are deployed in the proposed model with sequential or parallel tuples for shared understanding of overall system framework or specific functional modules. Three cases of one smart nuclear plant management scenario are hierarchically instantiated to evaluate the proposed model.

A multi-order Fractional Fourier domain feature union method for active sonar target classification

Different order domains in Fractional Fourier Transform (FrFT) reflect the features of different time–frequency planes. They contain multi-domain information about the target. The optimal order domain feature is used widely because of its anti-reverberation interference performance, but other order domains also have anti-reverberation ability, and even have more separable performance. It is of great significance to use effectively the multi-order domain feature for target classification and recognition. Therefore, a multi-order Fractional Fourier domain feature union method based on the information entropy is proposed in this paper. This method aims to find more distinguishable and separable target features than that of the optimal order domain by calculating the information entropies of the amplitude features and union them by determining the weights according to the value of the information entropies. The performance of the presented method is verified using the field data of four types of similar targets’ echo signals when the signal-to-reverberation ratios (SRRs) are respectively 4 dB, 0 dB, −3 dB, and −6 dB. Experimental results show that the method proposed in this paper can reflect the overall characteristics of the signals and has a more distinguishable ability and a good anti-reverberation ability.