Canonical Correlation Analysis Algorithm Research Articles

ClusterMatch aligns single-cell RNA-sequencing data at the multi-scale cluster level via stable matching.

Unsupervised clustering of single-cell RNA sequencing (scRNA-seq) data holds the promise of characterizing known and novel cell type in various biological and clinical contexts. However, intrinsic multi-scale clustering resolutions poses challenges to deal with multiple sources of variability in the high-dimensional and noisy data. We present ClusterMatch, a stable match optimization model to align scRNA-seq data at the cluster level. In one hand, ClusterMatch leverages the mutual correspondence by canonical correlation analysis (CCA) and multi-scale Louvain clustering algorithms to identify cluster with optimized resolutions. In the other hand it utilizes stable matching framework to align scRNA-seq data in the latent space while maintaining interpretability with overlapped marker gene set. Through extensive experiments, we demonstrate the efficacy of ClusterMatch in data integration, cell type annotation, and cross-species/timepoint alignment scenarios. Our results show ClusterMatch's ability to utilize both global and local information of scRNA-seq data, sets the appropriate resolution of multi-scale clustering, and offers interpretability by utilizing marker genes. The code of CusterMatch software is freely available at https://github.com/AMSSwanglab/ClusterMatch. Supplementary data are available at Bioinformatics online.

Abnormality in Peripheral and Brain Iron Contents and the Relationship with Grey Matter Volumes in Major Depressive Disorder.

Major depressive disorder (MDD) is a prevalent mental illness globally, yet its etiology remains largely elusive. Recent interest in the scientific community has focused on the correlation between the disruption of iron homeostasis and MDD. Prior studies have revealed anomalous levels of iron in both peripheral blood and the brain of MDD patients; however, these findings are not consistent. This study involved 95 MDD patients aged 18-35 and 66 sex- and age-matched healthy controls (HCs) who underwent 3D-T1 and quantitative susceptibility mapping (QSM) sequence scans to assess grey matter volume (GMV) and brain iron concentration, respectively. Plasma ferritin (pF) levels were measured in a subset of 49 MDD individuals and 41 HCs using the Enzyme-linked immunosorbent assay (ELISA), whose blood data were simultaneously collected. We hypothesize that morphological brain changes in MDD patients are related to abnormal regulation of iron levels in the brain and periphery. Multimodal canonical correlation analysis plus joint independent component analysis (MCCA+jICA) algorithm was mainly used to investigate the covariation patterns between the brain iron concentration and GMV. The results of "MCCA+jICA" showed that the QSM values in bilateral globus pallidus and caudate nucleus of MDD patients were lower than HCs. While in the bilateral thalamus and putamen, the QSM values in MDD patients were higher than in HCs. The GMV values of these brain regions showed a significant positive correlation with QSM. The GMV values of bilateral putamen were found to be increased in MDD patients compared with HCs. A small portion of the thalamus showed reduced GMV values in MDD patients compared to HCs. Furthermore, the region of interest (ROI)-based comparison results in the basal ganglia structures align with the outcomes obtained from the "MCCA+jICA" analysis. The ELISA results indicated that the levels of pF in MDD patients were higher than those in HCs. Correlation analysis revealed that the increase in pF was positively correlated with the iron content in the left thalamus. Finally, the covariation patterns obtained from "MCCA+jICA" analysis as classification features effectively differentiated MDD patients from HCs in the support vector machine (SVM) model. Our findings indicate that elevated peripheral ferritin in MDD patients may disrupt the normal metabolism of iron in the brain, leading to abnormal changes in brain iron levels and GMV.

Computerized diagnosis of knee osteoarthritis from x‐ray images using combined texture features: Data from the osteoarthritis initiative

AbstractThe prevalence of knee osteoarthritis (KOA) cases has witnessed a significant increase on a global scale in recent years, emphasizing the need for automated diagnostic computer systems to aid in early‐stage osteoarthritis (OA) diagnosis. The accurate characterization of knee KOA stages through feature extraction poses significant research challenges due to the complexity of identifying relevant attributes. In this study, the development of a KOA diagnostic system is presented, leveraging a combination of Gabor, and Tamura parameters using the Canonical Correlation Analysis algorithm. Two feature selection algorithms, namely Principal Component Analysis and Relief, were employed for KOA classification. Furthermore, various classifiers, including K‐Nearest Neighbors, AdaBoost, Bagging, and Random Forest, were used to assess the proposed feature extraction approach. The diagnostic system was assessed using a dataset comprising 688 x‐ray images sourced from the OA initiative (OAI) dataset, consisting of 344 images from healthy subjects (Grade 0) and 344 images from pathological patients (Grade 2). To mitigate overfitting, a 10‐fold cross‐validation method was utilized. The experimental results indicate that the combination of Tamura and Gabor parameters with the Random Forest classifier achieved remarkable performance in KOA diagnosis, yielding an accuracy of 94.59%, and an area under the curve of 98.3%. Notably, the combined Gabor and Tamura models exhibited superior performance compared to individual models, as well as existing techniques reported in the literature.

Research on SSVEP‐EEG feature enhancement algorithm based on fractional differentiation

AbstractSteady‐state visual evoked potentials (SSVEP), significant in brain‐computer interfaces (BCI) and medical diagnostics, benefit from enhanced signal processing for improved analysis and interpretation. This study introduces a novel enhancement algorithm for SSVEP electroencephalogram (EEG) signals, employing fractional‐order differentiation operators combined with image processing techniques. Utilizing fractional‐order differentiation within a Laplace pyramid framework, the algorithm achieves hierarchical signal enhancement, facilitating detailed feature extraction and emphasizing SSVEP signal characteristics. This innovative approach merges the precision of fractional calculus with the structural benefits of the Laplace pyramid, leading to enhanced signal clarity and feature discrimination. The efficacy of this method was validated using canonical correlation analysis (CCA), filter bank CCA (FBCCA), and task‐related component analysis (TRCA) on a public dataset. Compared to conventional methods, our algorithm not only mitigates trend components in SSVEP signals but also significantly boosts the recognition accuracy of CCA, FBCCA, and TRCA algorithms. Experimental results indicate a marked improvement in recognition precision, underscoring the algorithm's potential to advance SSVEP‐based BCI research.

Control of the robotic arm system with an SSVEP-based BCI

Recent studies on brain–computer interfaces (BCIs) implemented in robotic systems have shown that the system’s effectiveness in assisting individuals with movement disorders to enhance their human–computer interaction skills. However, achieving precise and rapid online completion of tasks remains a challenge for manipulators with multiple degrees of freedom (DOFs). In this paper, we explore a time-sharing control strategy for studying motion control of a robotic arm based on steady-state visual evoked potentials. The signals are generated by the joint frequency-phase modulation method, analyzed with the filter-bank canonical correlation analysis algorithm, and identified to control the six-DOF robotic arm for task execution. The shared control strategy not only reduces user’s cognitive fatigue but also enhances system in practical environments. The use of high-frequency stimuli significantly improves user comfort, and hybrid coding increases the universality of the BCI system. Additionally, by setting multiple locations and actions randomly, the robotic arm can adaptively program the optimal path. The online results showed that BCI instructions of the proposed system could be accurately chosen from six options within 6.45 s. Subjects used an average of 12 commands for the robotic arm to achieve the proposed task with an average accuracy of 98.21%. These findings validate the feasibility and effectiveness of applying the system to robotic control. The control strategy proposed in this study exhibits versatility in controlling robots to perform various complex tasks across different domains.

Weed density evaluation using KCCA-CFBLS based on fusion of visual and tactile features in special paddy field environment

Accurately evaluating weed density in paddy fields is crucial for achieving high-quality control of rice weeds, as it enables precise application of herbicides or matching of mechanical weeding intensity. However, visual methods alone may not provide ideal classification of weed density in specialized paddy field environments due to complex backgrounds, environmental interference, and visual occlusion. In this study, we propose an innovative method that combines vision and tactile features to achieve accurate weed density evaluation. We first obtained tactile data containing weed density information using a self-made tactile sensor, while simultaneously capturing corresponding visual images. We then utilized the kernel canonical correlation analysis (KCCA) algorithm to maximize the correlation of discriminating features extracted from the tactile data and visual images, and obtained fusion eigenvectors that characterize the weed density. These eigenvectors were used as input for a broad learning system (BLS), in which the random feature mapping was replaced by a cascade feature node to construct a KCCA-based cascade feature broad learning system (KCCA-CFBLS). Our method achieved accurate evaluation of weed density in low, medium, and high weed conditions using a novel weed dataset from a specialized paddy field environment. We analyzed the accuracy and running time, and found that KCCA-CFBLS outperformed methods based on SVM and YOLOv5-Lite, achieving 11.56% and 7.56% higher accuracy, respectively, with less time consumption. Experimental results demonstrated significant advantages of our method in terms of accuracy and real-time performance compared to visual methods, providing a decision-making basis for the implementation of intelligent chemical and mechanical weeding in specialized paddy field environments.

Hybrid Brain-Computer Interface Controlled Soft Robotic Glove for Stroke Rehabilitation.

Soft robotic glove controlled by a brain-computer interface (BCI) have demonstrated effectiveness in hand rehabilitation for stroke patients. Current systems mostly rely on static visual representations for patients to perform motor imagination (MI) tasks, resulting in lower BCI performance. Therefore, this study innovatively used MI and high-frequency steady-state visual evoked potential (SSVEP) to construct a friendly and natural hybrid BCI paradigm. Specifically, the stimulation interface sequentially presented decomposed action pictures of the left and right hands gripping a ball, with the pictures flashing at specific stimulation frequencies (left: 34 Hz, right: 35 Hz). Integrating soft robotic glove as feedback, we established a comprehensive "peripheral - central - peripheral" hand rehabilitation system to facilitate the hand rehabilitation of patients. Filter bank common spatial pattern (FBCSP) and filter bank canonical correlation analysis (FBCCA) algorithms were used to identify MI and SSVEP signals, respectively. Additionally, to fuse the features of these two signals, we proposed a novel fusion algorithm for improving the recognition accuracy of the system. The feasibility of the proposed system was validated through online experiments involving 12 healthy subjects and 9 stroke patients, achieving accuracy rates of 95.83 ± 6.83% and 63.33 ± 10.38%, respectively. The accuracy of MI and SSVEP in 12 healthy subjects reached 81.67 ± 15.63% and 95.14 ± 7.47%, both lower than the accuracy after fusion, these results confirmed the effectiveness of the proposed algorithm. The accuracy rate was more than 50% in both healthy subjects and patients, confirming the effectiveness of the proposed system.

IHBPs-VWDC: variable-length window-based dynamic connectivity approach for identifying hormone-binding proteins

Hormone-binding proteins (HBPs) are soluble carrier proteins that play a vital role in the growth and development of living organisms. Identifying HBPs accurately is crucial for understanding their functions. However, traditional wet lab experimental methods are labor intensive and cost ineffective. Therefore, there is a need for computational methods to efficiently identify HBPs. In this study, a machine learning method based on support vector machine (SVM) was proposed for the accurate and efficient identification of HBPs. The encoding of protein sequences involved using fifty different physicochemical (PC) properties. A variable-length window-based dynamic connectivity method was applied to capture the connection information between two different PC properties through two distinct strategies. The canonical correlation analysis algorithm was then used to fuse features obtained from these approaches. Feature selection was performed using the F-score approach to choose the most discriminative features. Finally, these selected features were fed into the SVM to discriminate between HBPs and non-HBPs. The proposed method achieved high classification accuracies of 99.19%, 96.77%, and 94.57% on the main dataset and two independent datasets, respectively, as demonstrated in the jackknife test. Comparative results showed that our proposed method outperforms existing approaches on the same datasets, indicating its potential as a useful tool for identifying HBPs. The Matlab codes and datasets used in the current study are freely available at https://figshare.com/articles/online_resource/iHBPs-VWDC/23559834. Communicated by Ramaswamy H. Sarma

Recognition of high-frequency steady-state visual evoked potential for brain-computer interface

Coding with high-frequency stimuli could alleviate the visual fatigue of users generated by the brain-computer interface (BCI) based on steady-state visual evoked potential (SSVEP). It would improve the comfort and safety of the system and has promising applications. However, most of the current advanced SSVEP decoding algorithms were compared and verified on low-frequency SSVEP datasets, and their recognition performance on high-frequency SSVEPs was still unknown. To address the aforementioned issue, electroencephalogram (EEG) data from 20 subjects were collected utilizing a high-frequency SSVEP paradigm. Then, the state-of-the-art SSVEP algorithms were compared, including 2 canonical correlation analysis algorithms, 3 task-related component analysis algorithms, and 1 task discriminant component analysis algorithm. The results indicated that they all could effectively decode high-frequency SSVEPs. Besides, there were differences in the classification performance and algorithms' speed under different conditions. This paper provides a basis for the selection of algorithms for high-frequency SSVEP-BCI, demonstrating its potential utility in developing user-friendly BCI.

Quantum canonical correlation analysis algorithm

Canonical correlation analysis (CCA) is a fundamental technique used to analyze data correlation in various fields, including video and medical data analysis. In this paper, we propose a quantum canonical correlation analysis (QCCA) algorithm. First, we introduce a combined density matrix representation method that transforms CCA into generalized eigenvalue decomposition. Moreover, to address the challenge of performing generalized eigenvalue decomposition in high-dimensional scenarios, we propose a quantum method for extracting the canonical principal axes. In this method, two sets of variables are transformed into a reduced density matrix, so that the product of variable matrices can be accelerated by phase estimation and controlled rotation. Complexity analysis shows that the QCCA algorithm achieves exponential acceleration in variable dimensions n, p and variable size m compared to classical algorithms. The QCCA algorithm serves as a foundation for the subsequent development of quantum algorithms for classification, regression, and other machine learning tasks.

A Methodology for Enhancing SSVEP Features Using Adaptive Filtering Based on the Spatial Distribution of EEG Signals.

In this paper, we propose a classification algorithm of EEG signal based on canonical correlation analysis (CCA) and integrated with adaptive filtering. It can enhance the detection of steady-state visual evoked potentials (SSVEPs) in a brain-computer interface (BCI) speller. An adaptive filter is employed in front of the CCA algorithm to improve the signal-to-noise ratio (SNR) of SSVEP signals by removing background electroencephalographic (EEG) activities. The ensemble method is developed to integrate recursive least squares (RLS) adaptive filter corresponding to multiple stimulation frequencies. The method is tested by the SSVEP signal recorded from six targets by actual experiment and the EEG in a public SSVEP dataset of 40 targets from Tsinghua University. The accuracy rates of the CCA method and the CCA-based integrated RLS filter algorithm (RLS-CCA method) are compared. Experiment results show that the proposed RLS-CCA-based method significantly improves the classification accuracy compared with the pure CCA method. Especially when the number of EEG leads is low (three occipital electrodes and five non occipital electrodes), its advantage is more significant, and accuracy reaches 91.23%, which is more suitable for wearable environments where high-density EEG is not easy to collect.

Effect of Local Network Characteristics on the Performance of the SSVEP Brain-Computer Interface

ObjectiveFor decades, a great deal of interest in investigating brain network functional connective features has arisen in brain-computer interfaces (BCIs) based on steady-state visual evoked potentials (SSVEPs). Traditional decoding algorithms, for example, canonical correlation analysis (CCA), only consider the inherent properties of each channel in terms of feature extraction for the single channel electroencephalogram (EEG) signal, with inadequate features that cannot fully utilize the information transmitted by the brain. Material and methodsThis paper proposes a fused feature extraction method, CCA-DTF, which combines CCA with a direct transfer function (DTF) to construct local brain network features with seven leads in the occipital region. First, the features extracted by the CCA algorithm were combined with these features extracted by DTF to analyze the EEG data from 20 subjects. Then, two methods, support vector machine (SVM) and random forest (RF), were used in constructing the classifiers for the four tasks classification of the SSVEP-BCI. ResultsThe experimental results showed that incorporating local network features (extracted from DTF or Pearson correlation coefficient) can effectively improve the classification average accuracy and the information transfer rate (ITR) of SSVEP, not only for SVM but also for the ensemble method RF. In particular, CCA-DTF plus SVM obtained a 94.52% classification average accuracy and a 49.23 bits/min ITR in a time window of 2 seconds. The performance was 5.57% and 8.01 bits/min higher than those of traditional CCA plus SVM, respectively. ConclusionThe proposed feature extraction method based on local network features is robust for improving the performance of SSVEP-BCI significantly, which has a perspective of being used in neural rehabilitation engineering field.

A Canonical Correlation Analysis-Based Transfer Learning Framework for Enhancing the Performance of SSVEP-Based BCIs.

A steady-state visual evoked potential (SSVEP)- based brain-computer interface (BCI) can either achieve high classification accuracy in the case of sufficient training data or suppress the training stage at the cost of low accuracy. Although some researches attempted to conquer the dilemma between performance and practicality, a highly effective approach has not yet been established. In this paper, we propose a canonical correlation analysis (CCA)-based transfer learning framework for improving the performance of an SSVEP BCI and reducing its calibration effort. Three spatial filters are optimized by a CCA algorithm with intra- and inter-subject EEG data (IISCCA), two template signals are estimated separately with the EEG data from the target subject and a set of source subjects and six coefficients are yielded by correlation analysis between a testing signal and each of the two templates after they are filtered by each of the three spatial filters. The feature signal used for classification is extracted by the sum of squared coefficients multiplied by their signs and the frequency of the testing signal is recognized by template matching. To reduce the individual discrepancy between subjects, an accuracy-based subject selection (ASS) algorithm is developed for screening those source subjects whose EEG data are more similar to those of the target subject. The proposed ASS-IISCCA integrates both subject-specific models and subject-independent information for the frequency recognition of SSVEP signals. The performance of ASS-IISCCA was evaluated on a benchmark data set with 35 subjects and compared with the state-of-the-art algorithm task-related component analysis (TRCA). The results show that ASS-IISCCA can significantly improve the performance of SSVEP BCIs with a small number of training trials from a new user, thus helping to facilitate their applications in real world.

Communication-Efficient Federated Linear and Deep Generalized Canonical Correlation Analysis

Classic and deep learning-based generalized canonical correlation analysis (GCCA) algorithms seek low-dimensional common representations of data entities from multiple “views” (e.g., audio and image) using linear transformations and neural networks, respectively. When the views are acquired and stored at different computing agents (e.g., organizations and edge devices) and data sharing is undesired due to privacy or communication cost considerations, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">federated learning</i> -based GCCA is well-motivated. In federated learning, the views are kept locally at the agents and only derived, limited information exchange with a central server is allowed. However, applying existing GCCA algorithms onto such federated learning setting may incur prohibitively high communication overhead. This work puts forth a communication-efficient federated learning framework for both linear and deep GCCA under the maximum variance (MAX-VAR) formulation. The overhead issue is addressed by aggressively compressing (via quantization) the exchanging information between the computing agents and a central controller. Compared to the unquantized version, our empirical study shows that the proposed algorithm enjoys a substantial reduction of communication overheads with virtually no loss in accuracy and convergence speed. Rigorous convergence analyses are also presented, which is a nontrivial effort. Generic federated optimization results do not cover the special problem structure of GCCA, which is a manifold constrained multi-block nonconvex eigen problem. Our result shows that the proposed algorithms for both linear and deep GCCA converge to critical points in a sublinear rate, even under heavy quantization and stochastic approximations. In addition, in the linear MAX-VAR case, the quantized algorithm approaches a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">global optimum</i> in a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">geometric</i> rate under reasonable conditions. Synthetic and real-data experiments are used to showcase the effectiveness of the proposed approach.

Compact Dynamic Inner Canonical Correlation Analysis for Nonstationary Dynamic Feature Extraction and Prediction

In this paper, a novel compact dynamic inner canonical correlation analysis (DiCCA) algorithm with autoregressive integrated moving average (ARIMA) inner models is proposed to better capture the latent dynamics of high dimensional time series. It can extract latent factors that capture the underlying dynamics of the data and model them using the ARIMA structure, which has fewer parameters and more flexibility than the potentially high-order AR structure used in the original DiCCA algorithm. The proposed algorithm can also handle nonstationary latent factors by explicitly modeling the unit roots. The algorithm integrates the extraction and the modeling of a latent factor in one step, resulting in a consistent inner and outer model. The algorithm is applied to an industrial dataset and shows better performance than the original DiCCA algorithm.

K-Means Clustering-Based Kernel Canonical Correlation Analysis for Multimodal Emotion Recognition in Human–Robot Interaction

In this article, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i> -meansclustering-based Kernel canonical correlation analysis algorithm is proposed for multimodal emotion recognition in human–robot interaction (HRI). The multimodal features (gray pixels; time and frequency domain) extracted from facial expression and speech are fused based on Kernel canonical correlation analysis. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i> -means clustering is used to select features from multiple modalities and reduce dimensionality. The proposed approach can improve the heterogenicity among different modalities and make multiple modalities complementary to promote multimodal emotion recognition. Experiments on two datasets, namely SAVEE and eNTERFACE‘05, are conducted to evaluate the accuracy of the proposed method. The results show that the proposed method produces good recognition rates that are higher than the ones produced by the methods without <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i> -means clustering; more specifically, they are 2.77% higher in SAVEE and 4.7% higher in eNTERFACE‘05.

Automated CCA-MWF Algorithm for Unsupervised Identification and Removal of EOG Artifacts From EEG.

Affective brain computer interface (ABCI) enables machines to perceive, understand, express and respond to people's emotions. Therefore, it is expected to play an important role in emotional care and mental disorder detection. EEG signals are most frequently adopted as the physiology measurement in ABCI applications. Eye blinking and movements introduce lots of artifacts into raw EEG data, which seriously affect the quality of EEG signal and the subsequent emotional EEG feature engineering and recognition. In this paper, we propose a fully automatic and unsupervised ocular artifact identification and removal algorithm named automated canonical correlation analysis (CCA)-multi-channel wiener filter (MWF) (ACCAMWF). Firstly, spatial distribution entropy (SDE) and spectral entropy (SE) are computed to automatically annotate artifact segments. Then, CCA algorithm is used to extract neural signal from artifact contaminated data to further supplement the clean EEG data. Finally, MWF is trained to remove ocular artifacts from multiple channel EEG data adaptively. Extensive experiments have been carried out on semi-simulated EEG/EOG dataset and real eye blinking-contaminated EEG dataset to verify the effectiveness of our method when compared to two state-of-the-art algorithms. The results clearly demonstrate that ACCAMWF is a promising solution for removing EOG artifacts from emotional EEG data.

Humanoid Robot Walking in Maze Controlled by SSVEP-BCI Based on Augmented Reality Stimulus.

The application study of robot control based brain-computer interface (BCI) not only helps to promote the practicality of BCI but also helps to promote the advancement of robot technology, which is of great significance. Among the many obstacles, the importability of the stimulator brings much inconvenience to the robot control task. In this study, augmented reality (AR) technology was employed as the visual stimulator of steady-state visual evoked potential (SSVEP)-BCI and the robot walking experiment in the maze was designed to testify the applicability of the AR-BCI system. The online experiment was designed to complete the robot maze walking task and the robot walking commands were sent out by BCI system, in which human intentions were decoded by Filter Bank Canonical Correlation Analysis (FBCCA) algorithm. The results showed that all the 12 subjects could complete the robot walking task in the maze, which verified the feasibility of the AR-SSVEP-NAO system. This study provided an application demonstration for the robot control base on brain–computer interface, and further provided a new method for the future portable BCI system.

Development of an Ultra Low-Cost SSVEP-based BCI Device for Real-Time On-Device Decoding.

This study details the development of a novel, approx. £20 electroencephalogram (EEG)-based brain-computer interface (BCI) intended to offer a financially and operationally accessible device that can be deployed on a mass scale to facilitate education and public engagement in the domain of EEG sensing and neurotechnologies. Real-time decoding of steady-state visual evoked potentials (SSVEPs) is achieved using variations of the widely-used canonical correlation analysis (CCA) algorithm: multi-set CCA and generalised CCA. All BCI functionality is executed on board an inexpensive ESP32 microcontroller. SSVEP decoding accuracy of 95.56 ± 3.74% with an ITR of 102 bits/min was achieved with modest calibration.

Activity Classification Based on Feature Fusion of FMCW Radar Human Motion Micro-Doppler Signatures

Fall is a challenging task that poses a great danger to the elderly person’s health as they carry out their daily routines and activities and could lead to serious injuries, long hospitalization, or even death. One of the key solutions to this important problem is the prompt and automatic detection of their fall motion, among other activities, so that immediate help can be rendered to avoid further complications. Cameras and other wearable sensors are the conventional means employed to monitor the daily activities performed by elderly people. However, recently radar sensing technology has been favored largely due to its short-range and velocity estimation. And most importantly, their ability to remotely capture the human activity motion electromagnetic wave signal without infringing on the user’s privacy. To this end, we present an elderly people fall detection system that integrates the radar signal micro-Doppler features extracted with the convolutional neural networks (CNNs). The features were extracted using Alex-Net, VGG-16-Net, and VGG-19-Net pre-trained models. We adapted the canonical correlation analysis (CCA) algorithm by proposing a channel attention network to fuse the extracted features effectively. The channel attention module employed a series of convolutional filters to determine the most discriminative features to focus on and discard the redundant ones. The fused features are classified using an SVM classifier. Our proposed method achieved the best performance against the state-of-the-art approaches. Specifically, our approach attained 99.77% test accuracy, which is about a 2% to 4 % increase compared to the recent state-of-the-art approaches.