Visual image reconstruction based on EEG signals using a generative adversarial and deep fuzzy neural network

Abstract

The decoding of the brain stimulated by visual images has been discussed in this study in order to classify and reconstruct the stimulating images. The dataset utilized was given by Stanford University and included the EEG signal recorded from ten healthy people. In the first stage, the proposed approach combines the ability of the Long-Short-Term Memory (LSTM) network to extract sequence (EEG signals) features with the ability of fuzzy network to extract features in high-uncertainty contexts. The features extracted in the previous stage, which is a noise-free and compressed form of the EEG data, were employed as a conditional vector in the CGAN network in the following step. As a result, this network was used to reconstruct visual stimuli that evoke specific brain responses. When compared to earlier research, the evaluation results produced utilizing the suggested approach demonstrate a greater classification accuracy, precision, recall, and F1 score. Also, the similarity of produced and real images was measured quantitatively using two parameters, FID and SSIM; the SSIM results for 10 participants were on average 0.37, 0.35, 0.35, 0.34, 0.42, and 0.42 for groups (1) to (6), and the FID results were 0.44, 0.45, 0.51, 0.50, 0.39 and 0.39, respectively. Therefore, it can be stated that the method suggested in this study can reconstruct visual stimuli using brain response (EEG signal) with acceptable accuracy, so taking a significant step towards the objective of mind reading.