Spectral Regression Discriminant Analysis Research Articles

Nonlinear Feature Extraction Methods Based on Dual-Tree Complex Wavelet Transform Subimages of Brain Magnetic Resonance Imaging for the Classification of Multiple Diseases.

It has been a long time since we use magnetic resonance imaging (MRI) to detect brain diseases and many useful techniques have been developed for this task. However, there is still a potential for further improvement of classification of brain diseases in order to be sure of the results. In this research we presented, for the first time, a non-linear feature extraction method from the MRI sub-images that are obtained from the three levels of the two-dimensional Dual tree complex wavelet transform (2D DT-CWT) in order to classify multiple brain disease. After extracting the non-linear features from the sub-images, we used the spectral regression discriminant analysis (SRDA) algorithm to reduce the classifying features. Instead of using the deep neural networks that are computationally expensive, we proposed the Hybrid RBF network that uses the k-means and recursive least squares (RLS) algorithm simultaneously in its structure for classification. To evaluate the performance of RBF networks with hybrid learning algorithms, we classify nine brain diseases based on MRI processing using these networks, and compare the results with the previously presented classifiers including, supporting vector machines (SVM) and K-nearest neighbour (KNN). Comprehensive comparisons are made with the recently proposed cases by extracting various types and numbers of features. Our aim in this paper is to reduce the complexity and improve the classifying results with the hybrid RBF classifier and the results showed 100 percent classification accuracy in both the two class and the multiple classification of brain diseases in 8 and 10 classes. In this paper, we provided a low computational and precise method for brain MRI disease classification. the results show that the proposed method is not only accurate but also computationally reasonable.

Hyperspectral Remote Sensing Images Feature Extraction Based on Higher-Order Spectra

Hyperspectral remote sensing images (HRSIs) contain rich spectral information, a HRSIs feature extraction method, i.e., higher-order spectra linear discriminant analysis (HOS-LDA) subspace method, is presented, which demonstrates that the calculation of higher-order spectra (HOS) is a kernel mapping with the inner product of HOS as the kernel function, and uses LDA to extract the HOS feature (HOSF) from the HOS of the spectral pixels of HRSIs according to a presented strategy of kernel space methods. To verify the efficiency of the deep feature of HOSF, a two-branch convolutional neural network (CNN), i.e., Two-CNN <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">HOSF-spa</sub> , is designed with each branch fed with HOSF and spatial feature, respectively. A 3-dimensionality (3D) CNN, i.e., spa-hos-spe 3D-CNN, is designed, which is fed with the spatial-higher order spectra-spectral (spa-hos-spe) feature to evaluate the deep joint spa-hos-spe feature. The experimental results of three measured HRSIs show that: the presented HOS-LDA method has improved the classification rates compared with the LDA, spectral regression discriminant analysis (SRDA), kernel LDA (KLDA) methods by using support vector machine, Bayesian, and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i> -nearest neighbor classifiers, which shows the efficiency of the presented HOSF; the presented Two-CNN <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">HOSF-spa</sub> outperforms the two-branch CNN fed with spectral and spatial features and the two-branch CNNs fed with LDA, SRDA, KLDA extracted feature and spatial feature, which verifies the effectiveness of the deep HOSF; the presented spa-hos-spe 3D-CNN outperforms the 3D-CNN fed with spatial-spectral (spa-spe) feature, which shows that the deep spa-hos-spe feature is better than the deep spa-spe feature.

Application of Min-Max Normalization on Subject-Invariant EMG Pattern Recognition

Surface electromyography (EMG) is one of the promising signals for the recognition of the intended hand movement of an amputee. Nevertheless, there are several barriers to its successful implementation in the advanced prosthetic hand. Subject-dependent EMG pattern recognition is one of them, which limits the use of a training model for a specific subject to others. So, this study aims to explore a subject invariant EMG pattern recognition method that is performed by extracting subject invariant features. To extract subject invariant features, we have created a feature space using a feature extraction method, and the dimensionality of the feature space is reduced by employing spectral regression discriminant analysis (SRDA). Finally, each SRDA feature is normalized using min-max normalization, which confines the scale of each SRDA feature from 0 to 1. The proposed subject invariant EMG pattern recognition method achieves the F1 score of 97.26%, 96.47%, 95.42%, and 93.71% with a linear discriminant analysis classifier (LDA) for an electrode array of 8×16, 8×8, 8×4, and 8×2, respectively. The achieved performances are almost equal to or sometimes better than those achieved in subject independent and subject-dependent EMG pattern recognition. Also, the proposed method is simple, classifier independent, time complexity free, and does not require any customization or fine-tuning of classifiers. So, the proposed subject invariant EMG pattern recognition method would be an option to overcome the training barrier for each subject without compromising the EMG pattern recognition performance.

Shoulder girdle recognition using electrophysiological and low frequency anatomical contraction signals for prosthesis control

Shoulder disarticulation amputees account for a small portion of upper-limb amputees, thus little emphasis has been devoted to developing functional prosthesis for this cohort of amputees. In this study, shoulder girdle recognition was investigated with acquired data from electrophysiological (electromyography [EMG]) and low frequency contraction (accelerometer [Acc]) signals from both amputee and non-amputee participants. The contribution of this study is based around the contrast of the classification accuracy (CA) for different sensor configurations using a unique set of signal features. It was seen that the fusion of the EMG-Acc produced an enhancement in the CA in the range of 10%–20%, depending on which windowing parameters were considered. From this, it was seen that the best combination of a windowing scheme and classifier would likely be for the 350 ms and spectral regression discriminant analysis, with a fusion of the EMG-Acc information. The results have thus provided evidence that the two sensors can be combined and used in practice for prosthesis control. Taking a holistic view on the study, the authors conclude by providing a framework on how the shoulder motion recognition could be combined with neuromuscular reprogramming to contribute towards easing the cognitive burden of amputees during the prosthesis control process.

Online detection of class-imbalanced error-related potentials evoked by motor imagery

Objective. Error-related potentials (ErrPs) are spontaneous electroencephalogram signals related to the awareness of erroneous responses within brain domain. ErrPs-based correction mechanisms can be applied to motor imagery-brain–computer interface (MI-BCI) to prevent incorrect actions and ultimately improve the performance of the hybrid BCI. Many studies on ErrPs detection are mostly conducted under offline conditions with poor classification accuracy and the error rates of ErrPs are preset in advance, which is too ideal to apply in realistic applications. In order to solve these problems, a novel method based on adaptive autoregressive (AAR) model and common spatial pattern (CSP) is proposed for ErrPs feature extraction. In addition, an adaptive threshold classification method based spectral regression discriminant analysis (SRDA) is suggested for class-unbalanced ErrPs data to reduce the false positives and false negatives. Approach. As for ErrPs feature extraction, the AAR coefficients in the temporal domain and CSP in the spatial domain are fused. Given that the performance of different subjects’ MI tasks is different but stable, and the samples of ErrPs are class-imbalanced, an adaptive threshold based SRDA is suggested for classification. Two datasets are used in this paper. The open public clinical neuroprosthetics and brain interaction (CNBI) dataset is used to validate the performance of the proposed feature extraction algorithm and the real-time data recorded in our self-designed system is used to validate the performance of the proposed classification algorithm under class-imbalanced situations. Different from the pseudo-random paradigm, the ErrPs signals collected in our experiments are all elicited by four-class of online MI-BCI tasks, and the sample distribution is more natural and suitable for practical tests. Main results. The experimental results on the CNBI dataset show that the average accuracy and false positive rate for ErrPs detection are 94.1% and 8.1%, which outperforms methods using features extracted from a single domain. What’s more, although the ErrPs induction rate is affected by the performance of subjects’ MI-BCI tasks, experimental results on data recorded in the self-designed system prove that the ErrPs classification algorithm based on an adaptive threshold is robust under different ErrPs data distributions. Compared with two other methods, the proposed algorithm has advantages in all three measures which are accuracy, F1-score and false positive rate. Finally, ErrPs detection results were used to prevent wrong actions in a MI-BCI experiment, and it leads to a reduction of the hybrid BCI error rate from 48.9% to 24.3% in online tests. Significance. Both the AAR-CSP fused feature extraction and the adaptive threshold based SRDA classification methods suggested in our work are efficient in improving the ErrPs detection accuracy and reducing the false positives. In addition, by introducing ErrPs to multi-class MI-BCIs, the MI decoding results can be corrected after ErrPs are detected to avoid executing wrong instructions, thereby improving the BCI accuracy and lays the foundation for using MI-BCIs in practical applications.

Performance Evaluation of SRELM on Bio-signal Pattern Recognition Using Two Electromyography Channels

The classification accuracy of pattern recognition is determined by the extracted features and the utilized classifiers. Many efforts have been conducted to obtain the best features either by introducing a new feature or proposing a new projection method to increase class separability. Recently, spectral regression extreme learning machine (SRELM) has been introduced to improve the class separability of the features. However, the evaluation of SRELM was only focused on the myoelectric or electromyography pattern recognition from many EMG channels. In practical application, the user is more convenient with less number of channels. Then, the problem is whether the SRELM would be able to work efficiently for less EMG channels. The objective of this paper is to examine the performance of SRELM for bio-signal pattern recognition using two EMG channels. The EMG electrodes were located on flexor policies lounges and flexor digitorium superficial muscles of ten healthy subjects. Various time domain features were involved with various sizes. SRELM will project these features to more recognize features before being feed to multiple classifiers. Those classifiers are randomized Variable Translation Wavelet Neural Networks (RVT-WNN), extreme learning machine(ELM), support vector machine (SVM), and linear discriminant analysis (LDA). The performance of SREM was compared to other feature methods, such as LDA, uncorrelated LDA (ULDA), orthogonal fuzzy neighborhood dimensionality reduction (OFNDA), and spectral regression discriminant analysis (SRDA). The experimental results show that SRELM performed well when dealing with different class numbers by classification accuracy of around 95.67% for ten class movements and performed better than SRDA.

Introducing the Hybrid "K-means, RLS" Learning for the RBF Network in Obstructive Apnea Disease Detection using Dual-tree Complex Wavelet Transform Based Features.

Apnea is one of the deadliest diseases that can be prevented and cured if it is detected in time. In this paper, we propose a precise method for early detection of the obstructive sleep apnea (OSA) disease using the latest feature selection and extraction methods. The feature selection in this paper is based on the Dual tree complex wavelet (DT-CWT) coefficients of the ECG signals of several patients. The feature extraction from these coefficients is done using frequency and time techniques. The Feature selection is done using the spectral regression discriminant analysis (SRDA) algorithm and the classification is performed using the hybrid RBF network. A hybrid RBF neural network is introduced in this paper for detecting apnea that is much less computationally demanding than the previously presented SVM networks. Our findings showed a 3 percent improvement in the detection and at least a 30 percent reduction in the computational complexity in comparison with methods that have been presented recently.

Classification of Marine Vessels with Multi-Feature Structure Fusion

The classification of marine vessels is one of the important problems of maritime traffic. To fully exploit the complementarity between different features and to more effectively identify marine vessels, a novel feature structure fusion method based on spectral regression discriminant analysis (SF-SRDA) was proposed. Firstly, we selected the different convolutional neural network features that better describe the characteristics of ships, and constructed the features based on graphs by the similarity metric. Then we weighed the concatenate multi-feature and fused their structures according to the linear relationship assumption. Finally, we constructed the optimization formula to solve the fusion features and structure by using spectral regression discriminant analyses. Experiments on the VAIS dataset show that the proposed SF-SRDA method can reduce the feature dimension from the original 102,400 dimensions to 5 dimensions, that the classification accuracy of visible images can reach 87.60%, and that that of the infrared image can reach 74.68% at daytime. The experimental results demonstrate that the proposed method can not only extract the optimal features from the original redundant feature space, but also greatly reduce the dimensions of the feature. Furthermore, the classification performance of SF-SRDA also gets a promising result.

Feature Selection for Motor Imagery EEG Classification Based on Firefly Algorithm and Learning Automata.

Motor Imagery (MI) electroencephalography (EEG) is widely studied for its non-invasiveness, easy availability, portability, and high temporal resolution. As for MI EEG signal processing, the high dimensions of features represent a research challenge. It is necessary to eliminate redundant features, which not only create an additional overhead of managing the space complexity, but also might include outliers, thereby reducing classification accuracy. The firefly algorithm (FA) can adaptively select the best subset of features, and improve classification accuracy. However, the FA is easily entrapped in a local optimum. To solve this problem, this paper proposes a method of combining the firefly algorithm and learning automata (LA) to optimize feature selection for motor imagery EEG. We employed a method of combining common spatial pattern (CSP) and local characteristic-scale decomposition (LCD) algorithms to obtain a high dimensional feature set, and classified it by using the spectral regression discriminant analysis (SRDA) classifier. Both the fourth brain–computer interface competition data and real-time data acquired in our designed experiments were used to verify the validation of the proposed method. Compared with genetic and adaptive weight particle swarm optimization algorithms, the experimental results show that our proposed method effectively eliminates redundant features, and improves the classification accuracy of MI EEG signals. In addition, a real-time brain–computer interface system was implemented to verify the feasibility of our proposed methods being applied in practical brain–computer interface systems.

Higher order spectral regression discriminant analysis (HOSRDA): A tensor feature reduction method for ERP detection

Tensors are valuable tools to represent Electroencephalogram (EEG) data. Tucker decomposition is the most used tensor decomposition in multidimensional discriminant analysis and tensor extension of Linear Discriminant Analysis (LDA), called Higher Order Discriminant Analysis (HODA), is a popular tensor discriminant method used for analyzing Event Related Potentials (ERP). In this paper, we introduce a new tensor-based feature reduction technique, named Higher Order Spectral Regression Discriminant Analysis (HOSRDA), for use in a classification framework for ERP detection. The proposed method (HOSRDA) is a tensor extension of Spectral Regression Discriminant Analysis (SRDA) and casts the eigenproblem of HODA to a regression problem. The formulation of HOSRDA can open a new framework for adding different regularization constraints in higher order feature reduction problem. Additionally, when the dimension and number of samples is very large, the regression problem can be solved via efficient iterative algorithms. We applied HOSRDA on data of a P300 speller from BCI competition III and reached average character detection accuracy of 96.5% for the two subjects. HOSRDA outperforms almost all of other reported methods on this dataset. Additionally, the results of our method are fairly comparable with those of other methods when 5 and 10 repetitions are used in the P300 speller paradigm.

A Decision Support System for Early-Stage Diabetic Retinopathy Lesions

Retina is a network layer containing light-sensitive cells. Diseases that occur in this layer, which performs the eye-sight, threaten our eye-sight directly. Diabetic Retinopathy is one of the main complications of diabetes mellitus and it is the most significant factor contributing to blindness in the later stages of the disease. Therefore, early diagnosis is of great importance to prevent the progress of this disease. For this purpose, in this study, an application based on image processing techniques and machine learning, which provides decision support to specialist, was developed for the detection of hard exudates, cotton spots, hemorrhage and microaneurysm lesions which appear in the early stages of the disease. The meaningful information was extracted from a set of samples obtained from the DIARETDB1 dataset during the system modeling process. In this process, Gabor and Discrete Fourier Transform attributes were utilized and dimension reduction was performed by using Spectral Regression Discriminant Analysis algorithm. Then, Random Forest and Logistic Regression and classifier algorithms’ performances were evaluated on each attribute dataset. Experimental results were obtained using the retinal fundus images provided from both DIARETDB1 dataset and the department of Ophthalmology, Ataturk Training and Research Hospital in Ankara.

Shape-based features for reliable action recognition using spectral regression discriminant analysis

This paper deals with the classification task of human activities in videos with learning using different image sequences as input methods. Image sequences of purely binary silhouette (or shape), that is Raw Silhouette Representation (RSR), distance transform (DT) images of RSR, edge images of RSR, Silhouette History Image (SHI) and Silhouette Energy Image (SEI), image sequences of Wavelet Transform (WT) are used for training and testing using spectral regression discriminant analysis. Hausdorff distance was used for similarity measures to match the embedded action trajectories. Then action classification is achieved in a K-nearest neighbour framework. Using these different input methods we achieved 100% for all the cases except WT cases. From the results, it is evident that SHI and SEI are effective input method in terms of time and space consumption.

Shape-based features for reliable action recognition using spectral regression discriminant analysis

This paper deals with the classification task of human activities in videos with learning using different image sequences as input methods. Image sequences of purely binary silhouette (or shape), that is Raw Silhouette Representation (RSR), distance transform (DT) images of RSR, edge images of RSR, Silhouette History Image (SHI) and Silhouette Energy Image (SEI), image sequences of Wavelet Transform (WT) are used for training and testing using spectral regression discriminant analysis. Hausdorff distance was used for similarity measures to match the embedded action trajectories. Then action classification is achieved in a K-nearest neighbour framework. Using these different input methods we achieved 100% for all the cases except WT cases. From the results, it is evident that SHI and SEI are effective input method in terms of time and space consumption.

Application of higher-order spectra for automated grading of diabetic maculopathy.

Diabetic macular edema (DME) is one of the most common causes of visual loss among diabetes mellitus patients. Early detection and successive treatment may improve the visual acuity. DME is mainly graded into non-clinically significant macular edema (NCSME) and clinically significant macular edema according to the location of hard exudates in the macula region. DME can be identified by manual examination of fundus images. It is laborious and resource intensive. Hence, in this work, automated grading of DME is proposed using higher-order spectra (HOS) of Radon transform projections of the fundus images. We have used third-order cumulants and bispectrum magnitude, in this work, as features, and compared their performance. They can capture subtle changes in the fundus image. Spectral regression discriminant analysis (SRDA) reduces feature dimension, and minimum redundancy maximum relevance method is used to rank the significant SRDA components. Ranked features are fed to various supervised classifiers, viz. Naive Bayes, AdaBoost and support vector machine, to discriminate No DME, NCSME and clinically significant macular edema classes. The performance of our system is evaluated using the publicly available MESSIDOR dataset (300 images) and also verified with a local dataset (300 images). Our results show that HOS cumulants and bispectrum magnitude obtained an average accuracy of 95.56 and 94.39% for MESSIDOR dataset and 95.93 and 93.33% for local dataset, respectively.

Hyperspectral remote sensing images terrain classification in DCT SRDA subspace

Hyperspectral remote sensing images terrain classification faces the problems of high data dimensionality and lack of labeled training data, resulting in unsatisfied terrain classification efficiency. The feature extraction is required before terrain classification for preserving discriminative information and reducing data dimensionality. A hyperspectral remote sensing images feature extraction method, i.e., discrete cosine transform (DCT) spectral regression discriminant analysis (SRDA) subspace method, was presented to solve the above problems. The proposed DCT SRDA subspace method firstly takes DCT in the original spectral space and gets the DCT coefficients of each pixel spectral curve; secondly performs SRDA in the DCT coefficients space and obtains the DCT SRDA subspace. Minimum distance classifier was designed in the resulting DCT SRDA subspace to evaluate the feature extraction performance. Experiments for two real airborne visible/infrared imaging spectrometer (AVIRIS) hyperspectral images show that, comparing with spectral LDA subspace method, the proposed DCT SRDA subspace method can improve terrain classification efficiency.

Evaluating Feature Extractors and Dimension Reduction Methods for Near Infrared Face Recognition Systems

This study evaluates the performance of global and local feature extractors as well as dimension reduction methods in NIR domain. Zernike moments (ZMs), Independent Component Analysis (ICA), Radon Transform + Discrete Cosine Transform (RDCT), Radon Transform + Discrete Wavelet Transform (RDWT) are employed as global feature extractors and Local Binary Pattern (LBP), Gabor Wavelets (GW), Discrete Wavelet Transform (DWT) and Undecimated Discrete Wavelet Transform (UDWT) are used as local feature extractors. For evaluation of dimension reduction methods Principal Component Analysis (PCA), Kernel Principal Component Analysis (KPCA), Linear Discriminant Analysis + Principal Component Analysis (Fisherface), Kernel Fisher Discriminant Analysis (KFD) and Spectral Regression Discriminant Analysis (SRDA) are used. Experiments conducted on CASIA NIR database and PolyU-NIRFD database indicate that ZMs as a global feature extractor, UDWT as a local feature extractor and SRDA as a dimension reduction method have superior overall performance compared to some other methods in the presence of facial expressions, eyeglasses, head rotation, image noise and misalignments.

Near infrared face recognition by combining Zernike moments and undecimated discrete wavelet transform

This study proposes a novel near infrared face recognition algorithm based on a combination of both local and global features. In this method local features are extracted from partitioned images by means of undecimated discrete wavelet transform (UDWT) and global features are extracted from the whole face image by means of Zernike moments (ZMs). Spectral regression discriminant analysis (SRDA) is then used to reduce the dimension of features. In order to make full use of global and local features and further improve the performance, a decision fusion technique is employed by using weighted sum rule. Experiments conducted on CASIA NIR database and PolyU-NIRFD database indicate that the proposed method has superior overall performance compared to some other methods in the presence of facial expressions, eyeglasses, head rotation, image noise and misalignments. Moreover its computational time is acceptable for on-line face recognition systems.

How to Estimate the Regularization Parameter for Spectral Regression Discriminant Analysis and its Kernel Version?

Spectral regression discriminant analysis (SRDA) has recently been proposed as an efficient solution to large-scale subspace learning problems. There is a tunable regularization parameter in SRDA, which is critical to algorithm performance. However, how to automatically set this parameter has not been well solved until now. So this regularization parameter was only set to be a constant in SRDA, which is obviously suboptimal. This paper proposes to automatically estimate the optimal regularization parameter of SRDA based on the perturbation linear discriminant analysis (PLDA). In addition, two parameter estimation methods for the kernel version of SRDA are also developed. One is derived from the method of optimal regularization parameter estimation for SRDA. The other is to utilize the kernel version of PLDA. Experiments on a number of publicly available databases demonstrate the effectiveness of the proposed methods for face recognition, spoken letter recognition, handwritten digit recognition, and text categorization.

Rotation and noise invariant near-infrared face recognition by means of Zernike moments and spectral regression discriminant analysis

Face recognition is a rapidly growing research area, which is based heavily on the methods of machine learning, computer vision, and image processing. We propose a rotation and noise invariant near-infrared face-recognition system using an orthogonal invariant moment, namely, Zernike moments (ZMs) as a feature extractor in the near-infrared domain and spectral regression discriminant analysis (SRDA) as an efficient algorithm to decrease the computational complexity of the system, enhance the discrimination power of features, and solve the “small sample size” problem simultaneously. Experimental results based on the CASIA NIR database show the noise robustness and rotation invariance of the proposed approach. Further analysis shows that SRDA as a sophisticated technique, improves the accuracy and time complexity of the system compared with other data reduction methods such as linear discriminant analysis.

SPECTRAL REGRESSION DISCRIMINANT ANALYSIS FOR HYPERSPECTRAL IMAGE CLASSIFICATION

Abstract. Dimensionality reduction algorithms, which aim to select a small set of efficient and discriminant features, have attracted great attention for Hyperspectral Image Classification. The manifold learning methods are popular for dimensionality reduction, such as Locally Linear Embedding, Isomap, and Laplacian Eigenmap. However, a disadvantage of many manifold learning methods is that their computations usually involve eigen-decomposition of dense matrices which is expensive in both time and memory. In this paper, we introduce a new dimensionality reduction method, called Spectral Regression Discriminant Analysis (SRDA). SRDA casts the problem of learning an embedding function into a regression framework, which avoids eigen-decomposition of dense matrices. Also, with the regression based framework, different kinds of regularizes can be naturally incorporated into our algorithm which makes it more flexible. It can make efficient use of data points to discover the intrinsic discriminant structure in the data. Experimental results on Washington DC Mall and AVIRIS Indian Pines hyperspectral data sets demonstrate the effectiveness of the proposed method.