Multi-source Feature Research Articles

Bearing fault diagnosis via fusing small samples and training multi-state Siamese neural networks

Recently, deep learning techniques have been widely applied to fault diagnosis due to their outstanding feature extraction abilities. The success of deep-learning-based fault diagnosis methods is highly dependent on the quantity and quality of the training data. In practical scenarios, it is challenging to obtain sufficient high-quality training data for fault diagnosis tasks due to complex environments, which would affect the effectiveness of the deep learning methods. In this paper, a new fault diagnosis method is proposed for motor bearing fault diagnosis under small samples. The Siamese neural networks (SNNs) are employed to extract the fault features. A multi-stage training strategy is proposed to train the SNNs with the aim to prevent the training stagnation problem and handle the small sample problem. A multi-source feature fusion network is developed to make full use of the multi-source sensor data by fusing the extracted fault features for further fault diagnosis. The proposed method is applied to motor bearing fault diagnosis on two real-world datasets. Experimental results demonstrate the effectiveness and feasibility of the introduced method for motor bearing fault diagnosis under small samples.

Cotton Verticillium wilt monitoring based on UAV multispectral-visible multi-source feature fusion

Verticillium wilt seriously jeopardizes cotton growth and restricts cotton yields. Therefore, it is important to accurately, rapidly, and non-destructively estimate the extent of cotton Verticillium wilt (CVW). The focus of this study was to explore the potential of combining the vegetation index (VI), color index (CI), and texture features to improve the accuracy of CVW disease severity estimation based on hexacopter Unmanned Aerial Vehicle (UAV) images. Simple Linear Regression (LR) and Multiple Linear Regression (MLR) methods were used to determine correlations between VI, CI, texture, and normalized difference texture index (NDTI) variables and cotton Verticillium wilt disease index (DI). The LR model based on VI, CI, and NDTI was constructed, VIs, CIs, and NDTIs were fused, and Grey Wolf Optimizer (GWO) Extreme Learning Machine (ELM) and Particle Swarm Optimization (PSO) Backpropagation Neural Network (BP) models were constructed to comparatively explore the estimation ability of each model on the degree of CVW disease. The results showed that VI was significantly correlated with DI, followed by NDTI, and CI. Compared with texture, NDTI containing both texture features was more significantly correlated with DI. The accuracy of the DI estimation using LR was highest for the one-factor VI model (R2 > 0.48, RRMSE < 51.48), followed by the NDTI model (R2 > 0.47, RRMS < 60.81) and the CI model (R2 > 0.33, RRMSE < 52.58). The PSO-BP and GWO-ELM were further used to model the DI estimation with different input variables. Regardless of the period, the fusion of three data sources (VIs + CIs + TIs) was preferable to a single data source or a combination of two data sources for different model inputs. In terms of different modeling algorithms, GWO-ELM combining VIs, CIs, and NDTIs had the highest estimation accuracy compared with SR and PSO-BP, with a validated R2 values of 0.65 (RRMSE = 42.96) at the flowering stage, 0.66 (RRMSE = 20.00) at the flower and boll stage, and 0.88 (RRMSE = 10.53) at the boll stage. This study demonstrated that the estimation accuracy of DI was significantly improved using collaborative modeling with multiple data sources. This study provides ideas and methods for monitoring crop disease conditions using low-altitude remote sensing.

Multisource Feature Embedding and Interaction Fusion Network for Coastal Wetland Classification With Hyperspectral and LiDAR Data

Multisource Feature Embedding and Interaction Fusion Network for Coastal Wetland Classification With Hyperspectral and LiDAR Data

Deep Citywide Multisource Data Fusion-Based Air Quality Estimation.

With the increasingly serious air pollution, people are paying more and more attention to air quality. However, air quality information is not available for all regions, as the number of air quality monitoring stations in a city is limited. Existing air quality estimation methods only consider the multisource data of partial regions and separately estimate the air qualities of all regions. In this article, we propose a deep citywide multisource data fusion-based air quality estimation (FAIRY) method. FAIRY considers the citywide multisource data and estimates the air qualities of all regions at a time. Specifically, FAIRY constructs images from the citywide multisource data (i.e., meteorology, traffic, factory air pollutant emission, point of interest, and air quality) and uses SegNet to learn the multiresolution features from these images. The features with the same resolution are fused by the self-attention mechanism to provide multisource feature interactions. To get a complete air quality image with high resolution, FAIRY refines low-resolution fused features by employing high-resolution fused features through residual connections. In addition, the Tobler's first law of geography is used to constrain the air qualities of adjacent regions, which can fully use the air quality relevance of nearby regions. Extensive experimental results demonstrate that FAIRY achieves the state-of-the-art performance on the Hangzhou city dataset, outperforming the best baseline by 15.7% on MAE.

MSFF-MA-DDI: Multi-Source Feature Fusion with Multiple Attention blocks for predicting Drug–Drug Interaction events

The interaction of multiple drugs could lead to severe events, which cause medical injuries and expenses. Accurate prediction of drug–drug interaction (DDI) events can help clinicians make effective decisions and establish appropriate therapy programs. However, there exist two issues worthy of further consideration. (i) The global features of drug molecules should be paid attention to, rather than just their local characteristics. (ii) The fusion of multi-source features should also be studied to capture the comprehensive features of the drug. This study designs a Multi-Source Feature Fusion framework with Multiple Attention blocks named MSFF-MA-DDI that utilizes multimodal data for DDI event prediction. MSFF-MA-DDI can (i) encode global correlations between long-distance atoms in drug molecular sequences by a self-attention layer based on a position embedding block and (ii) fuse drug sequence features and heterogeneous features (chemical substructure, target, and enzyme) through a multi-head attention block to better represent the features of drugs. Experiments on real-world datasets show that MSFF-MA-DDI can achieve performance that is close to or even better than state-of-the-art models. Especially in cold start scenarios, the model can achieve the best performance. The effectiveness of the model is also supported by the case study on nervous system drugs. The source codes and data are available at https://github.com/BioCenter-SHU/MSFF-MA-DDI.

Evaluating Feature Selection Methods and Machine Learning Algorithms for Mapping Mangrove Forests Using Optical and Synthetic Aperture Radar Data

Mangrove forests, mostly found in the intertidal zone, are among the highest-productivity ecosystems and have great ecological and economic value. The accurate mapping of mangrove forests is essential for the scientific management and restoration of mangrove ecosystems. However, it is still challenging to perform the rapid and accurate information mapping of mangrove forests due to the complexity of mangrove forests themselves and their environments. Utilizing multi-source remote sensing data is an effective approach to address this challenge. Feature extraction and selection, as well as the selection of classification models, are crucial for accurate mangrove mapping using multi-source remote sensing data. This study constructs multi-source feature sets based on optical (Sentinel-2) and SAR (synthetic aperture radar) (C-band: Sentinel-1; L-band: ALOS-2) remote sensing data, aiming to compare the impact of three feature selection methods (RFS, random forest; ERT, extremely randomized tree; MIC, maximal information coefficient) and four machine learning algorithms (DT, decision tree; RF, random forest; XGBoost, extreme gradient boosting; LightGBM, light gradient-boosting machine) on classification accuracy, identify sensitive feature variables that contribute to mangrove mapping, and formulate a classification framework for accurately recognizing mangrove forests. The experimental results demonstrated that using the feature combination selected via the ERT method could obtain higher accuracy with fewer features compared to other methods. Among the feature combinations, the visible bands, shortwave infrared bands, and the vegetation indices constructed from these bands contributed the greatest to the classification accuracy. The classification performance of optical data was significantly better than SAR data in terms of data sources. The combination of optical and SAR data could improve the accuracy of mangrove mapping to a certain extent (0.33% to 4.67%), which is essential for the research of mangrove mapping in a larger area. The XGBoost classification model performed optimally in mangrove mapping, with the highest overall accuracy of 95.00% among all the classification models. The results of the study show that combining optical and SAR remote sensing data with the ERT feature selection method and XGBoost classification model has great potential for accurate mangrove mapping at a regional scale, which is important for mangrove restoration and protection and provides a reliable database for mangrove scientific management.

MMFIL-Net: Multi-level and multi-source feature interactive lightweight network for polyp segmentation

Accuracy, generalized performance, and model size are critical considerations for designing a real-time polyp segmentation model. However, existing techniques primarily focus on accuracy and do not consider the other two vital characteristics. This paper proposes MMFIL-Net, a novel polyp segmentation technique. As part of MMFIL-Net, the Hierarchical Multi-source Feature Interaction Module (HMFIM) comprises Multi-source Feature Interaction Blocks (MFIB). MFIB manipulates multi-level and multi-sourced features to reduce the gap between low and high-level feature maps to achieve generalized performance. Additionally, the Multiple Receptive Field Feature Interaction Block (MRFFIB) targets the issues of segmenting polyps of different sizes. Finally, Dual Source Attention Fusion Block (DSAFB) is introduced to deal with hazy boundary information for early-stage polyps detection and segmentation. The proposed model outperformed existing lightweight models in conducted evaluation on different datasets. In addition to the achieved generalized performance and higher accuracy, the proposed model presents a significant reduction in the model size than existing approaches. The proposed model only contains 6.68 million parameters and has 4.32G MACs (Multiply–Accumulate Operations), which is better than the current approaches.

MS-FRAN: A Novel Multi-Source Domain Adaptation Method for EEG-Based Emotion Recognition.

Electroencephalogram (EEG), an important physiological signal, has been widely applied in many fields. Among them, EEG-based emotion recognition has gradually become a research hotspot. However, the large distribution differences and variations of EEG signals across subjects make the current research on emotion recognition stuck in a dilemma. In order to resolve this problem, we choose the domain adaptation methodology to explore the inspiration for solutions. In this paper, a Multi-Source Feature Representation and Alignment Network (MS-FRAN) is proposed, in which a new feature learning module named Wide Feature Extractor (WFE) has been designed. The MS-FRAN adopts and improves the strategy of existing multi-source domain adaptation, which possesses two main benefits. For one thing, it can align the distribution of each pair of source and target domains. For another, it can also reduce the distributional differences among the multiple source domains. To demonstrate the effectiveness of MS-FRAN, we have carried out cross-subject experiments on two public benchmark datasets, SEED and DEAP. Experimental results have shown that our method outperforms the related competitive approaches with greater performance for EEG-based emotion recognition.

MDformer: A transformer-based method for predicting miRNA-Disease associations using multi-source feature fusion and maximal meta-path instances encoding

There is a growing body of evidence suggesting that microRNAs (miRNAs), small biological molecules, play a crucial role in the diagnosis, treatment, and prognostic assessment of diseases. However, it is often inefficient to verify the association between miRNAs and diseases (MDA) through traditional experimental methods. Based on this situation, researchers have proposed various computational-based methods, but the existing methods often have many drawbacks in terms of predictive effectiveness and accuracy. Therefore, in order to improve the prediction performance of computational methods, we propose a transformer-based prediction model (MDformer) for multi-source feature information. Specifically, first, we consider multiple features of miRNAs and diseases from the molecular biology perspective and utilize them in a fusion. Then high-quality node feature embeddings were generated using a feature encoder based on the transformer architecture and meta-path instances. Finally, a deep neural network was built for MDA prediction. To evaluate the performance of our model, we performed multiple 5-fold cross-validations as well as comparison experiments on HMDD v3.2 and HMDD v2.0 databases, and the experimental results of the average ROC area under the curve (AUC) were higher than the comparative methods for both databases at 0.9506 and 0.9369. We conducted case studies on five highly lethal cancers (breast, lung, colorectal, gastric, and hepatocellular cancers), and the first 30 predictions for these five diseases achieved 97.3% accuracy. In conclusion, MDformer is a reliable and scientifically sound tool that can be used to accurately predict MDA. In addition, the source code is available at https://github.com/Linda908/MDformer.

TCUNet: A Lightweight Dual-Branch Parallel Network for Sea–Land Segmentation in Remote Sensing Images

Remote sensing techniques for shoreline extraction are crucial for monitoring changes in erosion rates, surface hydrology, and ecosystem structure. In recent years, Convolutional neural networks (CNNs) have developed as a cutting-edge deep learning technique that has been extensively used in shoreline extraction from remote sensing images, owing to their exceptional feature extraction capabilities. They are progressively replacing traditional methods in this field. However, most CNN models only focus on the features in local receptive fields, and overlook the consideration of global contextual information, which will hamper the model’s ability to perform a precise segmentation of boundaries and small objects, consequently leading to unsatisfactory segmentation results. To solve this problem, we propose a parallel semantic segmentation network (TCU-Net) combining CNN and Transformer, to extract shorelines from multispectral remote sensing images, and improve the extraction accuracy. Firstly, TCU-Net imports the Pyramid Vision Transformer V2 (PVT V2) network and ResNet, which serve as backbones for the Transformer branch and CNN branch, respectively, forming a parallel dual-encoder structure for the extraction of both global and local features. Furthermore, a feature interaction module is designed to achieve information exchange, and complementary advantages of features, between the two branches. Secondly, for the decoder part, we propose a cross-scale multi-source feature fusion module to replace the original UNet decoder block, to aggregate multi-scale semantic features more effectively. In addition, a sea–land segmentation dataset covering the Yellow Sea region (GF Dataset) is constructed through the processing of three scenes from Gaofen-6 remote sensing images. We perform a comprehensive experiment with the GF dataset to compare the proposed method with mainstream semantic segmentation models, and the results demonstrate that TCU-Net outperforms the competing models in all three evaluation indices: the PA (pixel accuracy), F1-score, and MIoU (mean intersection over union), while requiring significantly fewer parameters and computational resources compared to other models. These results indicate that the TCU-Net model proposed in this article can extract the shoreline from remote sensing images more effectively, with a shorter time, and lower computational overhead.

Automatic Identification of Thaw Slumps Based on Neural Network Methods and Thaw Slumping Susceptibility

Thaw slumping is a periglacial process that occurs on slopes in cold environments, where the ground becomes unstable and the surface slides downhill due to saturation with water during thawing. In this study, GaoFen-1 remote sensing and fused multi-source feature data were used to automatically map thaw slumping landforms in the Beilu River Basin of the Qinghai–Tibet Plateau. The bi-directional cascade network structure was used to extract edges at different scales, where an individual layer was supervised by labeled edges at its specific scale, rather than directly applying the same supervision to all convolutional neural network outputs. Additionally, we conducted a 5-year multi-scale feature analysis of small baseline subset interferometric synthetic aperture radar deformation, normalized difference vegetation index, and slope, among other features. Our study analyzed the performance and accuracy of three methods based on edge object supervised learning and three preconfigured neural networks, ResNet101, VGG16, and ResNet152. Through verification using site surveys and multi-data fusion results, we obtained the best ResNet101 model score of intersection over union of 0.85 (overall accuracy of 84.59%).The value of intersection over union of the VGG and ResNet152 are 0.569 and 0.773, respectively. This work provides a new insight for the potential feasibility of applying the designed edge detection method to map diverse thaw slumping landforms in larger areas with high-resolution images.

Multi-Task Segmentation and Classification Network for Artery/Vein Classification in Retina Fundus.

Automatic classification of arteries and veins (A/V) in fundus images has gained considerable attention from researchers due to its potential to detect vascular abnormalities and facilitate the diagnosis of some systemic diseases. However, the variability in vessel structures and the marginal distinction between arteries and veins poses challenges to accurate A/V classification. This paper proposes a novel Multi-task Segmentation and Classification Network (MSC-Net) that utilizes the vessel features extracted by a specific module to improve A/V classification and alleviate the aforementioned limitations. The proposed method introduces three modules to enhance the performance of A/V classification: a Multi-scale Vessel Extraction (MVE) module, which distinguishes between vessel pixels and background using semantics of vessels, a Multi-structure A/V Extraction (MAE) module that classifies arteries and veins by combining the original image with the vessel features produced by the MVE module, and a Multi-source Feature Integration (MFI) module that merges the outputs from the former two modules to obtain the final A/V classification results. Extensive empirical experiments verify the high performance of the proposed MSC-Net for retinal A/V classification over state-of-the-art methods on several public datasets.

Fatigue Detection for Ship OOWs Based on Input Data Features, From the Perspective of Comparison With Vehicle Drivers: A Review

Ninety percent of the world’s cargo is transported by sea, and the fatigue of ship officers of the watch (OOWs) contributes significantly to maritime accidents. The fatigue detection of ship OOWs is more difficult than that of vehicles drivers owing to an increase in the automation degree. In this study, research progress pertaining to fatigue detection in OOWs is comprehensively analysed based on a comparison with that in vehicle drivers. Fatigue detection techniques for OOWs are organised based on input sources, which include the physiological/behavioural features of OOWs, vehicle/ship features, and their comprehensive features. Prerequisites for detecting fatigue in OOWs are summarised. Subsequently, various input features applicable and existing applications to the fatigue detection of OOWs are proposed, and their limitations are analysed. The results show that the reliability of the acquired feature data is insufficient for detecting fatigue in OOWs, as well as a non-negligible invasive effect on OOWs. Hence, low-invasive physiological information pertaining to the OOWs, behaviour videos, and multisource feature data of ship characteristics should be used as inputs in future studies to realise quantitative, accurate, and real-time fatigue detections in OOWs on actual ships.

Exploring Multisource Feature Fusion and Stacking Ensemble Learning for Accurate Estimation of Maize Chlorophyll Content Using Unmanned Aerial Vehicle Remote Sensing

Crop chlorophyll content measuring plays a vital role in monitoring crop growth and optimizing agricultural inputs such as water and fertilizer. However, traditional methods for measuring chlorophyll content primarily rely on labor-intensive chemical analysis. These methods not only involve destructive sampling but also are time-consuming, often resulting in obtaining monitoring results after the optimal growth period of crops. Unmanned aerial vehicle (UAV) remote sensing technology offers the potential for rapidly acquiring chlorophyll content estimations over large areas. Currently, most studies only utilize single features from UAV data and employ traditional machine learning algorithms to estimate chlorophyll content, while the potential of multisource feature fusion and stacking ensemble learning in chlorophyll content estimation research remains largely unexplored. Therefore, this study collected UAV spectral features, thermal features, structural features, as well as chlorophyll content data during maize jointing, trumpet, and big trumpet stages, creating a multisource feature dataset. Subsequently, chlorophyll content estimation models were built based on four machine learning algorithms, namely, ridge regression (RR), light gradient boosting machine (LightGBM), random forest regression (RFR), and stacking ensemble learning. The research results demonstrate that (1) the multisource feature fusion approach achieves higher estimation accuracy compared to the single-feature method, with R2 ranging from 0.699 to 0.754 and rRMSE ranging from 8.36% to 9.47%; and (2) the stacking ensemble learning outperforms traditional machine learning algorithms in chlorophyll content estimation accuracy, particularly when combined with multisource feature fusion, resulting in the best estimation results. In summary, this study proves the effective improvement in chlorophyll content estimation accuracy through multisource feature fusion and stacking ensemble learning. The combination of these methods provides reliable estimation of chlorophyll content using UAV remote sensing technology and brings new insights to precision agriculture management in this field.

IMSFNet: integrated multi-source feature network for salient object detection

IMSFNet: integrated multi-source feature network for salient object detection

Learning a robust unified domain adaptation framework for cross-subject EEG-based emotion recognition

Over the last few years, unsupervised domain adaptation (UDA) based on deep learning has emerged as a solution to build cross-subject emotion recognition models from Electroencephalogram (EEG) signals, aligning the subject distributions within a latent feature space. However, most reported works have a common intrinsic limitation: the subject distribution alignment is coarse-grained, but not all of the feature space is shared between subjects. In this paper, we propose a robust unified domain adaptation framework, named Multi-source Feature Alignment and Label Rectification (MFA-LR), which performs a fine-grained domain alignment at subject and class levels, while inter-class separation and robustness against input perturbations are encouraged in coarse grain. As a complementary step, a pseudo-labeling correction procedure is used to rectify mislabeled target samples. Our proposal was assessed over two public datasets, SEED and SEED-IV, on each of the three available sessions, using leave-one-subject-out cross-validation. Experimental results show an accuracy performance of up to 89.11 ± 07.72% and 74.99 ± 12.10% for the best session on SEED and SEED-IV, as well as an average accuracy of 85.27% and 69.58% on all three sessions, outperforming state-of-the-art results.

A hybrid network capturing multisource feature correlations for tool remaining useful life prediction

A hybrid network capturing multisource feature correlations for tool remaining useful life prediction

A multi-source feature extraction network for salient object detection

A multi-source feature extraction network for salient object detection

Local Information Interaction Transformer for Hyperspectral and LiDAR Data Classification

The multisource remote sensing classification task has two main challenges. 1) How to capture hyperspectral image (HSI) and light detection and ranging (LiDAR) features cooperatively to fully mine the complementary information between data. 2) How to adaptively fuse multisource features, which should not only overcome the imbalance between HSI and LiDAR data but also avoid the generation of redundant information. The local information interaction transformer (LIIT) model proposed herein can effectively address these above issues. Specifically, multibranch feature embedding is first performed to help in the fine-grained serialization of multisource features; subsequently, a local-based multisource feature interactor (L-MSFI) is designed to explore HSI and LiDAR features together. This structure provides an information transmission environment for multibranch features and further alleviates the homogenization processing mode of the self-attention process. More importantly, a multisource feature selection module (MSTSM) is developed to dynamically fuse HSI and LiDAR features to solve the problem of insufficient fusion. Experiments were carried out on three multisource remote-sensing classification datasets, the results of which show that LIIT has more performance advantages than the state-of-the-art CNN and transformer methods.

Multi-Source Feature Mapping via Spectral Embedding for Heterogeneous Defect Prediction

Multi-Source Feature Mapping via Spectral Embedding for Heterogeneous Defect Prediction