Discriminant Feature Fusion Research Articles

Brain tumor segmentation with missing MRI modalities using edge aware discriminative feature fusion based transformer U-net

Brain tumor segmentation is an essential task for medical diagnosis and treatment planning. Multi-modal MRI provides complementary information that is essential for accurate segmentation of brain tumors, but missing modality images are a common problem in clinical practice. Existing segmentation methods often fail to generate accurate object boundaries and selectively fuse the tumor region, resulting in unreliable segmentation masks. In this work, we propose an Edge-aware Discriminative Feature Fusion Based Transformer U-Net (EA-DFFTU-Net) to segment the brain tumor effectively even in the absence of modalities. First, the MRI input data is pre-processed, and features are then extracted using a ResNet-50 encoder, which learns local spatial. We employ an Edge Feature Module (EFM) to acquire edge attention representations. These features are then transferred to the Discriminative Feature Fusion based Transformer U-Net (DFFTU-Net) that learns global contexts and captures multiscale features. We use a Discriminative Feature Fusion Module (DFFM) in the decoder of the DFFTU-Net to effectively fuse the multiscale features in order to obtain accurate segmentation. The performance of the proposed EA-DFFTU-Net segmentation method was determined by evaluating it and comparing the obtained results with those of existing brain tumor segmentation techniques.

Human centric attention with deep multiscale feature fusion framework for activity recognition in Internet of Medical Things

Recent advancements in the Internet of Medical Things (IoMT) have revolutionized the healthcare sector, making it an active research area in the academic and industrial sectors. Following these advances, an automatic Human Activity Recognition (HAR) is now integrated into the IoMT, facilitating remote patient monitoring systems for smart healthcare. However, implementing HAR via computer vision is intricate due to complex spatiotemporal patterns, single stream fusion, and clutter backgrounds. Mainstream approaches practice pre-trained CNN model, which extract non-salient features due to their generalized weight optimization and limited discriminative feature fusion. In addition, their sequential models have inadequate performance in complex scenarios due to the vanishing gradients encountered during backpropagation across multiple layers. In response to these challenges, we propose a multiscale feature fusion framework for both indoor and outdoor environments to enhance HAR in healthcare monitoring systems, which is mainly composed of two stages: First, the proposed Human Centric Attentional Fusion (HCAF) network is fused with the intermediate convolutional feature of lightweight MobileNetV3 backbone to enriches spatial learning capabilities for accurate HAR. Next, a Deep Multiscale Features Fusion (DMFF) network is proposed that enhanced the long-range temporal dependencies by redesigning the traditional bidirectional LSTM network into a residual fashion followed by Sequential Multihead Attention (SMA) to eliminate non-relevant information and optimized spatiotemporal feature vectors. The performance of the proposed fusion model is evaluated on benchmark healthcare and general activity datasets. In the healthcare, we used Multiple Camera Fall and UR Fall Detection datasets that achieved 99.941% and 100% accuracy. Despite this, our fusion strategy is rigorously evaluated over three challenging general HAR datasets, including HMDB51,UCF101, and UCF50, demonstrating 74.942%, 97.337%, and 96.156% superior performance compared to State-of-The-Art (SOTA) methods. The run time analysis shows that the proposed method is 2x times faster than the existing methods.

Discriminative feature fusion for RGB-D salient object detection

RGB-D salient object detection aims to separate salient object from an image aided by depth. While a number of effective approaches have been proposed, difficulties still exist, which is due to two challenges: (1) It is difficult to fully and effectively fuse RGB and depth features especially in challenging scenes; (2) How to enhance the semantic information of low-level feature and enrich the spatial information of high-level feature. Most of the existing approaches design separate modules to address them. In this paper, a unified discriminative feature fusion module is proposed to be used for both multimodal and multiscale feature fusion. The module can also increase the semantic information in low-level features and enrich the spatial information in high-level features. A multi-scale contextual perception module is embedded in the network to accurately localize objects at different scales. Unlike other methods, the depth branch in the network uses pure convolution for complementary feature extraction. This paper conducted a comparison with 14 state-of-the-art methods on 8 datasets, and the experimental results suggest that the proposed approach is more effective and superior.

Voxel-Wise GBM Recurrence Prediction Based on Sparse Attention Multi-Modal MR Image Fusion Coupling with Stem Cell Niches Proximity Estimation

<h3>Purpose/Objective(s)</h3> Nearly all glioblastoma multiforme (GBM) patients experience local recurrence despite aggressive therapies. The outcomes of patients with recurrence continue to be disappointing, emphasizing the importance of early and voxel-wise recurrence prediction for early treatment intervention. Emerging evidence has linked the tumorigenesis of recurrence to stem cell niches (SCN). For the first time, our group introduced a novel multidimensional support vector machine (SVM) method coupling with SCN proximity estimation to predict GBM recurrence on the voxel level. The current study aims to develop an L1-sparsity regularized attention fusion U-net (L1-AF U-net) to further distinguish the most salient and complementary features from multi-modal MR images, improving voxel-wise GBM recurrence prediction. <h3>Materials/Methods</h3> With IRB approval, fifty patients with pre-and post-surgery MR scans were retrospectively solicited, including T1, T2, T1_CE, ADC, and FLAIR. Post-surgery MR scans included two months before the clinical diagnosis of GBM recurrence and the day of the radiologically confirmed recurrence. The recurrences were manually annotated on the contrast-enhanced T1 MR, which were deformably registered to the 2-month pre-recurrence MR scans. The processing pipeline included a sparse attention learning module with L1 norm regularization for multi-modal feature fusion and a U-net for voxel-wise GBM recurrence prediction. The L1 sparsity regularized attention network embedded in the U-net architecture learns the coherent and mutual-complementary features across five MR modalities while eliminating the inherent redundancy and noisy artifacts. The high-risk recurrence (HRR) proximity map was determined by the weighted sum of inverse distances from two possible origins of recurrence: SCN and tumor cavity. The U-net was trained given the discriminative feature fusion in HRR for voxel-wise GBM recurrence prediction <h3>Results</h3> On the 2-month pre-recurrence testing MRs, the U-net segmentation with sparse multi-modal attention fusion achieved a recall of 0.83±0.15, a precision of 0.75±0.12, and an F1-score of 0.79±0.14. As a comparison, our previous GBM recurrence prediction using proximity estimation coupled SVM (SVM_PE) achieved a recall of 0.80±0.10, a precision of 0.69±0.14, and an F1-score of 0.74±0.10. A Student's t-test has also been performed to evaluate the significance of the proposed method over SVM_PE. The P-values are 0.0013 (for recall), 4.97 × 10⁻¹⁰ (for precision) and 7.94 × 10⁻⁸ (for F1-score), respectively. <h3>Conclusion</h3> We present an advanced U-net-based GBM recurrence prediction framework with sparse multi-modal MR fusion. This constrained attention fusion of the multi-modal MR images facilitated the U-net to learn more effective features from the multi-modal MR images and enhanced the prediction performance comparing to SVM method.

Exponential Multi-Modal Discriminant Feature Fusion for Small Sample Size

Multi-modal Canonical Correlation Analysis (MCCA) is an important information fusion method, and some discriminant variations of MCCA have been proposed. However, the variations suffer from the Small Sample Size (SSS) problem and the absence of cross-modal discriminant scatters. Thus we propose a novel exponential multi-modal discriminant feature fusion method for a small amount of training samples, i.e. exponential multi-modal discriminant correlation analysis. In the method, we construct a discriminative integration scatter of all the modalities by constraining the aggregation towards cross-modal discriminative centroids. Besides, the method gives a decomposition-based matrix exponential strategy. The strategy can solve the SSS problem and improve the robustness of noises, and we further provide corresponding theoretical proofs and some intuitive analysis. The method can learn correlation fusion features with well discriminative power from a small amount of samples. Encouraging experimental results show the effectiveness and robustness of our method.

Geometric discriminative deep features for traffic image analysis

Traffic image analysis is an important application in intelligent transportation. For local features’ robustness to image variances, such as scale changes and occlusions, they are widely used in image classification. However, how to integrate these local features for modeling traffic images optimally is still a crucial challenge. In this paper, a novel deep learning method, geometric discriminative feature fusion (GDFF), is proposed to tackle this problem. First, we use a variety of data sets to train the general convolutional neural network (CNN), which is used to extract the features of the training and test set after deep level. Deep architecture makes it possible for people to learn more abstract and internal features that are robust to changes in viewpoint and illumination. It can fuse image geometric related local features, such as local regions’ RGB histograms, into high level discriminative features, which can be used for better classifying complex scene images. Our framework’s central task is to build a structural kernel, called discriminative topological kernel. Firstly, we segment the traffic images into several regions and use a region connected graph (RCG) to model regions location relationships. We use frequent sub graph mining algorithm to mine all frequent sub structures (topologies) occurs in all training RCGs. And a selection algorithm is designed to select the k qualified topologies from the entire mined frequent topologies. We call these selected topologies geometric feature fusers, which are both high discriminative and low redundant structures in all training RCGs. Finally, given a pair of RCGs and to each geometric fuser, we extract all pairs of sub graphs sharing the same topology and calculate distance between them. All k distances are accumulated for the final kernel. The experimental result demonstrates the effectiveness of our method.

Independent Gabor Analysis of Discriminant Features Fusion for Face Recognition

A discriminant feature fusion model is proposed for face recognition with large variations of pose, expression, lighting, etc. Discriminant features are extracted by the wavelet transform-based method from two source images. One source image is a holistic gray value image and the other is an illumination invariant geometric image. Face sample is reconstructed by the adaptive fused discriminant feature. Then a bank of Gabor filters is built to extract Gabor representations of the reconstructed samples. Finally higher-order statistical relationships among variables of samples are extracted for classifier. According to experiments, the model outperforms conventional algorithms under complex conditions (large variations of lighting, expression, accessory, etc.).