Imagery Recognition Research Articles

Decoding motor imagery loaded on steady-state somatosensory evoked potential based on complex task-related component analysis

Background and objectiveMotor Imagery (MI) recognition is one of the most critical decoding problems in brain- computer interface field. Combined with the steady-state somatosensory evoked potential (MI-SSSEP), this new paradigm can achieve higher recognition accuracy than the traditional MI paradigm. Typical algorithms do not fully consider the characteristics of MI-SSSEP signals. Developing an algorithm that fully captures the paradigm's characteristics to reduce false triggering rate is the new step in improving performance. MethodsThe idea to use complex signal task-related component analysis (cTRCA) algorithm for spatial filtering processing has been proposed in this paper according to the features of SSSEP signal. In this research, it's proved from the analysis of simulation signals that task-related component analysis (TRCA) as typical method is affected when the response between stimuli has reduced correlation and the proposed algorithm can effectively overcome this problem. The experimental data under the MI-SSSEP paradigm have been used to identify right-handed target tasks and three unique interference tasks are used to test the false triggering rate. cTRCA demonstrates superior performance as confirmed by the Wilcoxon signed-rank test. ResultsThe recognition algorithm of cTRCA combined with mutual information-based best individual feature (MIBIF) and minimum distance to mean (MDM) can obtain AUC value up to 0.89, which is much higher than traditional algorithm common spatial pattern (CSP) combined with support vector machine (SVM) (the average AUC value is 0.77, p < 0.05). Compared to CSP+SVM, this algorithm model reduced the false triggering rate from 38.69 % to 20.74 % (p < 0.001). ConclusionsThe research prove that TRCA is influenced by MI-SSSEP signals. The results further prove that the motor imagery task in the new paradigm MI-SSSEP causes the phase change in evoked potential. and the cTRCA algorithm based on such phase change is more suitable for this hybrid paradigm and more conducive to decoding the motor imagery task and reducing false triggering rate.

EEG-Based Motor Imagery Recognition Framework via Multisubject Dynamic Transfer and Iterative Self-Training.

A robust decoding model that can efficiently deal with the subject and period variation is urgently needed to apply the brain-computer interface (BCI) system. The performance of most electroencephalogram (EEG) decoding models depends on the characteristics of specific subjects and periods, which require calibration and training with annotated data prior to application. However, this situation will become unacceptable as it would be difficult for subjects to collect data for an extended period, especially in the rehabilitation process of disability based on motor imagery (MI). To address this issue, we propose an unsupervised domain adaptation framework called iterative self-training multisubject domain adaptation (ISMDA) that focuses on the offline MI task. First, the feature extractor is purposefully designed to map the EEG to a latent space of discriminative representations. Second, the attention module based on dynamic transfer matches the source domain and target domain samples with a higher coincidence degree in latent space. Then, an independent classifier oriented to the target domain is employed in the first stage of the iterative training process to cluster the samples of the target domain through similarity. Finally, a pseudolabel algorithm based on certainty and confidence is employed in the second stage of the iterative training process to adequately calibrate the error between prediction and empirical probabilities. To evaluate the effectiveness of the model, extensive testing has been performed on three publicly available MI datasets, the BCI IV IIa, the High gamma dataset, and Kwon et al. datasets. The proposed method achieved 69.51%, 82.38%, and 90.98% cross-subject classification accuracy on the three datasets, which outperforms the current state-of-the-art offline algorithms. Meanwhile, all results demonstrated that the proposed method could address the main challenges of the offline MI paradigm.

Authentication with a one-dimensional CNN model using EEG-based brain-computer interface

Brain-computer interface (BCI) technology uses electroencephalogram (EEG) signals to create a direct interaction between the human body and its surroundings. Motor imagery (MI) classification using EEG signals is an important application that can help a rehabilitated or motor-impaired stroke patient perform certain tasks. Robust classification of these signals is an important step toward making the use of EEG more practical in many applications and less dependent on trained professionals. Deep learning methods have produced impressive results in BCI in recent years, especially with the availability of large electroencephalography (EEG) data sets. Dealing with EEG-MI signals is difficult because noise and other signal sources can interfere with the electrical amplitude of the brain, and its generalization ability is limited, so it is difficult to improve EEG classifiers. To address these issues, this paper presents a methodology based on one-dimensional convolutional neural networks (1-D CNN) for motor imagery (MI) recognition for the right hand, left hand, feet, and sedentary task. The proposed model is a lightweight model with fewer parameters and has an accuracy of 91.75%. Then, in an innovative exploitation of the four output classes, there is an idea that allows people with disabilities who are deprived of security measures, such as entering a secret code, to use the output classification, such as password codes. It is also an idea for a unique authentication system that is more secure and less vulnerable to theft or the like for a healthy person at the same time.

EMPT: a sparsity Transformer for EEG-based motor imagery recognition.

Transformer network is widely emphasized and studied relying on its excellent performance. The self-attention mechanism finds a good solution for feature coding among multiple channels of electroencephalography (EEG) signals. However, using the self-attention mechanism to construct models on EEG data suffers from the problem of the large amount of data required and the complexity of the algorithm. We propose a Transformer neural network combined with the addition of Mixture of Experts (MoE) layer and ProbSparse Self-attention mechanism for decoding the time-frequency-spatial domain features from motor imagery (MI) EEG of spinal cord injury patients. The model is named as EEG MoE-Prob-Transformer (EMPT). The common spatial pattern and the modified s-transform method are employed for achieving the time-frequency-spatial features, which are used as feature embeddings to input the improved transformer neural network for feature reconstruction, and then rely on the expert model in the MoE layer for sparsity mapping, and finally output the results through the fully connected layer. EMPT achieves an accuracy of 95.24% on the MI EEG dataset for patients with spinal cord injury. EMPT has also achieved excellent results in comparative experiments with other state-of-the-art methods. The MoE layer and ProbSparse Self-attention inside the EMPT are subjected to visualisation experiments. The experiments prove that sparsity can be introduced to the Transformer neural network by introducing MoE and kullback-leibler divergence attention pooling mechanism, thereby enhancing its applicability on EEG datasets. A novel deep learning approach is presented for decoding EEG data based on MI.

Smoke Recognition in Satellite Imagery via an Attention Pyramid Network With Bidirectional Multilevel Multigranularity Feature Aggregation and Gated Fusion

Smoke Recognition in Satellite Imagery via an Attention Pyramid Network With Bidirectional Multilevel Multigranularity Feature Aggregation and Gated Fusion

СТРАТЕГІЇ ВИЗНАЧЕННЯ ОПТИМАЛЬНОЇ КІЛЬКОСТІ ВХІДНИХ ДАНИХ ДЛЯ РОЗПІЗНАВАННЯ ОБ’ЄКТІВ НА СУПУТНИКОВИХ ЗОБРАЖЕННЯХ

Satellite imagery recognition is a vital application of computer vision, with numerous potential use cases in fields such as disaster management, land use, and urban planning. This study aims to identify the optimal amount of input data required for training the neural network for satellite imagery recognition. To this end, a series of rigorous experiments are conducted to investigate the impact of input data quantity on several performance metrics, including accuracy, convergence, and model generalization. The findings reveal that an increase in input data quantity generally improves model performance up to a saturation point, beyond which further increases do not lead to significant improvements and may even lead to overfitting. The research proposes several strategies for identifying this saturation point and mitigating the effects of overfitting. The results gained from this research may help in designing more effective models for satellite imagery recognition.

Self-supervised contrastive learning for EEG-based cross-subject motor imagery recognition.

Objective. The extensive application of electroencephalography (EEG) in brain-computer interfaces (BCIs) can be attributed to its non-invasive nature and capability to offer high-resolution data. The acquisition of EEG signals is a straightforward process, but the datasets associated with these signals frequently exhibit data scarcity and require substantial resources for proper labeling. Furthermore, there is a significant limitation in the generalization performance of EEG models due to the substantial inter-individual variability observed in EEG signals.Approach. To address these issues, we propose a novel self-supervised contrastive learning framework for decoding motor imagery (MI) signals in cross-subject scenarios. Specifically, we design an encoder combining convolutional neural network and attention mechanism. In the contrastive learning training stage, the network undergoes training with the pretext task of data augmentation to minimize the distance between pairs of homologous transformations while simultaneously maximizing the distance between pairs of heterologous transformations. It enhances the amount of data utilized for training and improves the network's ability to extract deep features from original signals without relying on the true labels of the data.Main results. To evaluate our framework's efficacy, we conduct extensive experiments on three public MI datasets: BCI IV IIa, BCI IV IIb, and HGD datasets. The proposed method achieves cross-subject classification accuracies of 67.32%, 82.34%, and 81.13%on the three datasets, demonstrating superior performance compared to existing methods.Significance. Therefore, this method has great promise for improving the performance of cross-subject transfer learning in MI-based BCI systems.

Shallow Inception Domain Adaptation Network for EEG-Based Motor Imagery Classification

Electroencephalography (EEG) data across multiple individuals have a high variance. Directly using the data to train a deep learning model usually degrades the performance. To address this issue, we propose a shallow Inception domain adaptation framework to extract informative deep features from data of multiple subjects for accurate motor imagery (MI) recognition. To our best knowledge, the Inception architecture in deep learning is combined with domain adaptation (DA) scheme for the first time for the MI classification task. The approach contains two compact Inception blocks that decode temporal features in different scales. In addition, we jointly optimize a novel combined loss function to reduce both marginal and class conditional discrepancies caused by the multi-modal structure of EEG signals. The DA-based loss enables Inception blocks to take full advantage of their learning abilities to capture discriminative patterns of MI data from multiple subjects instead of relying on the target user only. To demonstrate the effectiveness of our approach, we conduct substantial experiments on two wellknown datasets, BCI competition IV-2a and IV-2b. Results show that our model achieves better performance than state-of-theart strategies. The proposed model is able to extract informative features from high-variant EEG data collected from different individuals and achieves accurate MI classifications.

Imagery Recognition and Semantic Analysis Techniques in Chinese Literary Texts

Abstract Chinese literary texts contain a sizeable vivid imagery vocabulary, which makes it difficult for average readers to judge the boundaries between words, and the current pre-trained language model is also difficult for them to learn its implicit knowledge effectively, which brings troubles to machine semantic analysis. The study uses CRF training to obtain a semantic analysis model of Chinese literary texts that recognizes the semantic relationship between two words. SVM is used to train classifiers for confusing categories, and the two semantic relations in the output of the CRF model are further recognized to determine the final semantic relations between word pairs. Finally, the LCQMC dataset is used as the experimental data, and the semantic analysis technique based on CRF and SVM is employed to obtain the participle, lexical, and dependent syntactic annotations. According to the results, the model’s correct rates on the LAS for paraphrase recognition and dependency analysis of Chinese literary texts are 74.83% and 92.05%, respectively. The study enhances the efficiency of semantic analysis of relevant Chinese texts and is crucial for the study on the semantic analysis of terms.

Target Recognition Using Pre-Trained Convolutional Neural Networks and Transfer Learning

In modern surveillance, automatic target recognition (ATR) is a critical challenge, necessitating rapid and precise object identification, especially in military and disaster response scenarios. This research presents a comprehensive framework for ground target classification, focusing on Synthetic Aperture Radar (SAR) imagery. Harnessing the Moving and Stationary Target Acquisition and Recognition (MSTAR) Mixed Targets dataset, this study integrates Convolutional Neural Networks (CNNs) into SAR image analysis. Transfer learning emerges as a key strategy, adapting insights from pre-trained CNNs to the intricacies of target recognition in SAR imagery. This approach outperforms traditional methods, enhancing efficiency amid dataset annotation challenges. Noteworthy CNN architectures including DarkNet-19, DenseNet-121, InceptionV4, ResNet-152, and VGG19 are explored. The ResNet-152 model demonstrates exceptional performance, emerging as a leading contender with a remarkable testing accuracy of 98.56% when trained from scratch. Furthermore, by employing transfer learning, the model’s accuracy may be further enhanced to reach 98.81% on the testing dataset. This research signifies a transformative SAR imagery path guided by pre-trained CNNs and transfer learning. It reshapes ATR’s accuracy and efficiency, pointing to a future where innovation surmounts challenges previously deemed insurmountable.

Discriminative Adversarial Network Based on Spatial-Temporal-Graph Fusion for Motor Imagery Recognition

Discriminative Adversarial Network Based on Spatial-Temporal-Graph Fusion for Motor Imagery Recognition

ADFCNN: Attention-Based Dual-Scale Fusion Convolutional Neural Network for Motor Imagery Brain-Computer Interface.

Convolutional neural networks (CNNs) have been successfully applied to motor imagery (MI)-based brain-computer interface (BCI). Nevertheless, single-scale CNN fail to extract abundant information over a wide spectrum from EEG signals, while typical multi-scale CNNs cannot effectively fuse information from different scales with concatenation-based methods. To overcome these challenges, we propose a new scheme equipped with attention-based dual-scale fusion convolutional neural network (ADFCNN), which jointly extracts and fuses EEG spectral and spatial information at different scales. This scheme also provides novel insight through self-attention for effective information fusion from different scales. Specifically, temporal convolutions with two different kernel sizes identify EEG μ and β rhythms, while spatial convolutions at two different scales generate global and detailed spatial information, respectively, and the self-attention mechanism performs feature fusion based on the internal similarity of the concatenated features extracted by the dual-scale CNN. The proposed scheme achieves the superior performance compared with state-of-the-art methods in subject-specific motor imagery recognition on BCI Competition IV dataset 2a, 2b and OpenBMI dataset, with the cross-session average classification accuracies of 79.39% and significant improvements of 9.14% on BCI-IV2a, 87.81% and 7.66% on BCI-IV2b, 65.26% and 7.2% on OpenBMI dataset, and the within-session average classification accuracies of 86.87% and significant improvements of 10.89% on BCI-IV2a, 87.26% and 8.07% on BCI-IV2b, 84.29% and 5.17% on OpenBMI dataset, respectively. What is more, ablation experiments are conducted to investigate the mechanism and demonstrate the effectiveness of the dual-scale joint temporal-spatial CNN and self-attention modules. Visualization is also used to reveal the learning process and feature distribution of the model.

Motor Imagery Recognition Based on GMM-JCSFE Model.

Features from EEG microstate models, such as time-domain statistical features and state transition probabilities, are typically manually selected based on experience. However, traditional microstate models assume abrupt transitions between states, and the classification features can vary among individuals due to personal differences. To date, both empirical and theoretical classification results of EEG microstate features have not been entirely satisfactory. Here, we introduce an enhanced feature extraction method that combines Joint label-Common and label-Specific Feature Exploration (JCSFE) with Gaussian Mixture Models (GMM) to explore microstate features. First, GMMs are employed to represent the smooth transitions of EEG spatiotemporal features within microstate models. Second, category-common and category-specific features are identified by applying regularization constraints to linear classifiers. Third, a graph regularizer is used to extract subject-invariant microstate features. Experimental results on publicly available datasets demonstrate that the proposed model effectively encodes microstate features and improves the accuracy of motor imagery recognition across subjects. The primary code is accessible for download from the website: https://github.com/liaoliao3450/GMM-JCSFE.

Graph Learning With Co-Teaching for EEG-Based Motor Imagery Recognition

Graph Learning With Co-Teaching for EEG-Based Motor Imagery Recognition

Optimal Deep Learning-Based Recognition Model for EEG Enabled Brain-Computer Interfaces Using Motor-Imagery

Abstract Brain-Computer Interfaces (BCIs) facilitate the translation of brain activity into actionable commands and act as a crucial link between the human brain and the external environment. Electroencephalography (EEG)-based BCIs, which focus on motor imagery, have emerged as an important area of study in this domain. They are used in neurorehabilitation, neuroprosthetics, and gaming, among other applications. Optimal Deep Learning-Based Recognition for EEG Signal Motor Imagery (ODLR-EEGSM) is a novel approach presented in this article that aims to improve the recognition of motor imagery from EEG signals. The proposed method includes several crucial stages to improve the precision and effectiveness of EEG-based motor imagery recognition. The pre-processing phase starts with the Variation Mode Decomposition (VMD) technique, which is used to improve EEG signals. The EEG signals are decomposed into different oscillatory modes by VMD, laying the groundwork for subsequent feature extraction. Feature extraction is a crucial component of the ODLR-EEGSM method. In this study, we use Stacked Sparse Auto Encoder (SSAE) models to identify significant patterns in the pre-processed EEG data. Our approach is based on the classification model using Deep Wavelet Neural Network (DWNN) optimized with Chaotic Dragonfly Algorithm (CDFA). CDFA optimizes the weight and bias values of the DWNN, significantly improving the classification accuracy of motor imagery. To evaluate the efficacy of the ODLR-EEGSM method, we use benchmark datasets to perform rigorous performance validation. The results show that our approach outperforms current methods in the classification of EEG motor imagery, confirming its promising performance. This study has the potential to make brain-computer interface applications in various fields more accurate and efficient, and pave the way for brain-controlled interactions with external systems and devices.

Machine learning for biomedical-focus on brain signals

There is a growing interest in machine learning (ML) in this decade. This growing interest is accelerated by cheaper computing power and low-cost memory. Thus, large amount of data can be stored, processed and analyzed efficiently. Machine learning has used in brain machine systems (BMS) system that converts brain impulses into messages or commands. In this paper we propose an EEG-based BMS with the focus on evoked potential. An average classification accuracy of 95% was attained among nine participants. With a rate of 4 flashes per second implemented, selecting one of four possibilities takes 5 s, resulting in an information transfer rate of 24 bits/min. Also, brain computer interfacing using oscillatory activity was measured. The results show that after around 5 h of co-adaptive training over many days, the average 3-class accuracy of the Linear Discriminant Analysis committee classifier reached about 80%, with a false positive rate for motor imagery recognition of around 17%.

Deep temporal networks for EEG-based motor imagery recognition.

The electroencephalogram (EEG) based motor imagery (MI) signal classification, also known as motion recognition, is a highly popular area of research due to its applications in robotics, gaming, and medical fields. However, the problem is ill-posed as these signals are non-stationary and noisy. Recently, a lot of efforts have been made to improve MI signal classification using a combination of signal decomposition and machine learning techniques but they fail to perform adequately on large multi-class datasets. Previously, researchers have implemented long short-term memory (LSTM), which is capable of learning the time-series information, on the MI-EEG dataset for motion recognition. However, it can not model very long-term dependencies present in the motion recognition data. With the advent of transformer networks in natural language processing (NLP), the long-term dependency issue has been widely addressed. Motivated by the success of transformer algorithms, in this article, we propose a transformer-based deep learning neural network architecture that performs motion recognition on the raw BCI competition III IVa and IV 2a datasets. The validation results show that the proposed method achieves superior performance than the existing state-of-the-art methods. The proposed method produces classification accuracy of 99.7% and 84% on the binary class and the multi-class datasets, respectively. Further, the performance of the proposed transformer-based model is also compared with LSTM.

DeepVision: Enhanced Drone Detection and Recognition in Visible Imagery through Deep Learning Networks.

Drones are increasingly capturing the world's attention, transcending mere hobbies to revolutionize areas such as engineering, disaster aid, logistics, and airport protection, among myriad other fascinating applications. However, there is growing concern about the risks that they pose to physical infrastructure, particularly at airports, due to potential misuse. In recent times, numerous incidents involving unauthorized drones at airports disrupting flights have been reported. To solve this issue, this article introduces an innovative deep learning method proposed to effectively distinguish between drones and birds. Evaluating the suggested approach with a carefully assembled image dataset demonstrates exceptional performance, surpassing established detection systems previously proposed in the literature. Since drones can appear extremely small compared to other aerial objects, we developed a robust image-tiling technique with overlaps, which showed improved performance in the presence of very small drones. Moreover, drones are frequently mistaken for birds due to their resemblances in appearance and movement patterns. Among the various models tested, including SqueezeNet, MobileNetV2, ResNet18, and ResNet50, the SqueezeNet model exhibited superior performance for medium area ratios, achieving higher average precision (AP) of 0.770. In addition, SqueezeNet's superior AP scores, faster detection times, and more stable precision-recall dynamics make it more suitable for real-time, accurate drone detection than the other existing CNN methods. The proposed approach has the ability to not only detect the presence or absence of drones in a particular area but also to accurately identify and differentiate between drones and birds. The dataset utilized in this research was obtained from a real-world dataset made available by a group of universities and research institutions as part of the 2020 Drone vs. Bird Detection Challenge. We have also tested the performance of the proposed model on an unseen dataset, further validating its better performance.

BrainWave-Scattering Net: a lightweight network for EEG-based motor imagery recognition

Objective. Brain–computer interfaces (BCIs) enable a direct communication of the brain with the external world, using one’s neural activity, measured by electroencephalography (EEG) signals. In recent years, convolutional neural networks (CNNs) have been widely used to perform automatic feature extraction and classification in various EEG-based tasks. However, their undeniable benefits are counterbalanced by the lack of interpretability properties as well as the inability to perform sufficiently when only limited amount of training data is available. Approach. In this work, we introduce a novel, lightweight, fully-learnable neural network architecture that relies on Gabor filters to delocalize EEG signal information into scattering decomposition paths along frequency and slow-varying temporal modulations. Main results. We utilize our network in two distinct modeling settings, for building either a generic (training across subjects) or a personalized (training within a subject) classifier. Significance. In both cases, using two different publicly available datasets and one in-house collected dataset, we demonstrate high performance for our model with considerably less number of trainable parameters as well as shorter training time compared to other state-of-the-art deep architectures. Moreover, our network demonstrates enhanced interpretability properties emerging at the level of the temporal filtering operation and enables us to train efficient personalized BCI models with limited amount of training data.

Effective Human Motor Imagery Recognition via Segment Pool Based on One-Dimensional Convolutional Neural Network with Bidirectional Recurrent Attention Unit Network

Brain–computer interface (BCI) technology enables humans to interact with computers by collecting and decoding electroencephalogram (EEG) from the brain. For practical BCIs based on EEG, accurate recognition is crucial. However, existing methods often struggle to achieve a balance between accuracy and complexity. To overcome these challenges, we propose 1D convolutional neural networks with bidirectional recurrent attention unit network (1DCNN-BiRAU) based on a random segment recombination strategy (segment pool, SegPool). It has three main contributions. First, SegPool is proposed to increase training data diversity and reduce the impact of a single splicing method on model performance across different tasks. Second, it employs multiple 1D CNNs, including local and global models, to extract channel information with simplicity and efficiency. Third, BiRAU is introduced to learn temporal information and identify key features in time-series data, using forward–backward networks and an attention gate in the RAU. The experiments show that our model is effective and robust, achieving accuracy of 99.47% and 91.21% in binary classification at the individual and group levels, and 90.90% and 92.18% in four-category classification. Our model demonstrates promising results for recognizing human motor imagery and has the potential to be applied in practical scenarios such as brain–computer interfaces and neurological disorder diagnosis.