Tri-SeizureDualNet: A novel multimodal brain seizure detection using triple stream skipped feature extraction module entrenched dual parallel attention transformer

Abstract

The timely and accurate detection of epileptic seizures is highly needed to enhance the quality of a patient’s life. The state-of-the-art works to design and utilize many deep learning techniques to detect and predict seizures using EEG, fMRI, and a combination of EEG and fMRI modalities respectively. Whereas the existing models are highly vulnerable to complexity, and overfitting issues due to improper feature analysis. To this end, we design a novel seizure detection model named Triple stream skipped feature extraction module and Dual parallel attention transformer Network (Tri-SeizureDualNet) using EEG and fMRI modalities. The major scope of this research is to enhance the detection accuracy of seizures with better performance. The data acquired for this research is from the CHB-MIT database for EEG and the UW Madison database for fMRI. The designed model tends to perform feature extraction, feature selection, feature engineering, and classification respectively with high feature extraction accuracy and less complexity. Beforehand, we perform pre-processing on both the EEG and fMRI data using ICA, PCA, and high pass filtering. Followed by we perform feature extraction in three streams with skip connections from both modalities to improve the extractor capability and feature extraction accuracy. The extracted features are selected in the feature selection module using the Humming Bird Optimization (HBO) algorithm based on the feature importance ratio. The selected features from both modalities were fused and provided to the Dual Parallel Attention Transformer (DPAT) for feature engineering and context learning respectively. The utilized DPAT is composed of two regularized attention modules with a gating mechanism to picture the feature similarity rate. At last, the softmax layer classifies the features into three classes such as focal onset, general onset, and unknown onset seizures respectively. The experimental results were conducted with two utilized datasets such as CHB-MIT and UW Madison datasets separately for EEG and fMRI data in which the proposed work achieves an overall accuracy rate of 99.67% and AUC rate of 0.989 respectively. The results reveal that the proposed research outperforms the existing works.