Supervised Dimensionality Reduction Technique Research Articles

Distractibility and impulsivity neural states are distinct from selective attention and modulate the implementation of spatial attention

In the context of visual attention, it has been classically assumed that missing the response to a target or erroneously selecting a distractor occurs as a consequence of the (miss)allocation of attention in space. In the present paper, we challenge this view and provide evidence that, in addition to encoding spatial attention, prefrontal neurons also encode a distractibility-to-impulsivity state. Using supervised dimensionality reduction techniques in prefrontal neuronal recordings in monkeys, we identify two partially overlapping neuronal subpopulations associated either with the focus of attention or overt behaviour. The degree of overlap accounts for the behavioral gain associated with the good allocation of attention. We further describe the neural variability accounting for distractibility-to-impulsivity behaviour by a two dimensional state associated with optimality in task and responsiveness. Overall, we thus show that behavioral performance arises from the integration of task-specific neuronal processes and pre-existing neuronal states describing task-independent behavioral states.

Identifying Abnormal CFRP Holes Using Both Unsupervised and Supervised Learning Techniques on In-Process Force, Current, and Vibration Signals

This study aims to conduct abnormality detection by applying machine learning algorithms when drilling a carbon fiber reinforced plastic laminate. In-process signals including current, thrust force, and vibration were captured during the dry drilling experiments using a 6 mm physical vapor deposit diamond-coated drill at the consistent spindle speed of 6500 RPM and 0.05 mm/rev. Across measurements from out-of-process variables, including hole diameter, roundness, surface roughness, entry/exit delamination, and entry/exit uncut fiber area, in-process measurements were most able to find outliers with respect to diameter. Both Principal Component Analysis, an unsupervised dimensionality reduction technique, and Linear Discriminant Analysis, a supervised dimensionality reduction technique, could separate oversize or undersize holes from average-sized holes when using fast Fourier transformation data of in-process vibration. Predictive performance with k-Nearest Neighbors shows that our machine learning pipeline can predict oversized vs. non-oversized holes with over 85% accuracy in this dataset. Peak prediction performance is obtained when in-process measurement data is viewed from the frequency domain, and predictions are weighted based on the relative distances of the nearest neighbors.

A diagnostic unified classification model for classifying multi-sized and multi-modal brain graphs using graph alignment

BackgroundPresence of multimodal brain graphs derived from different neuroimaging modalities is inarguably one of the most critical challenges in building unified classification models that can be trained and tested on any brain graph regardless of its size and the modality it was derived from. Existing methodsOne solution is to learn a model for each modality independently, which is cumbersome and becomes more time-consuming as the number of modalities increases. Another traditional solution is to build a model inputting multimodal brain graphs for the target prediction task; however, this is only applicable to datasets where all samples have joint neuro-modalities. New methodIn this paper, we propose to build a unified brain graph classification model trained on unpaired multimodal brain graphs, which can classify any brain graph of any size. This is enabled by incorporating a graph alignment step where all multi-modal graphs of different sizes and heterogeneous distributions are mapped to a common template graph. Next, we design a graph alignment strategy to the target fixed-size template and further apply linear discriminant analysis (LDA) to the aligned graphs as a supervised dimensionality reduction technique for the target classification task. ResultsWe tested our method on unpaired autistic and healthy brain connectomes derived from functional and morphological MRI datasets (two modalities). ConclusionOur results showed that our unified model method not only has great promise in solving such a challenging problem but achieves comparable performance to models trained on each modality independently.

Supervised Multidimensional Scaling and its Application in MRI-Based Individual Age Predictions.

It has been a popular trend to decode individuals' demographic and cognitive variables based on MRI. Features extracted from MRI data are usually of high dimensionality, and dimensionality reduction (DR) is an effective way to deal with these high-dimensional features. Despite many supervised DR techniques for classification purposes, there is still a lack of supervised DR techniques for regression purposes. In this study, we advanced a novel supervised DR technique for regression purposes, namely, supervised multidimensional scaling (SMDS). The implementation of SMDS includes two steps: (1) evaluating pairwise distances among entities based on their labels and constructing a new space through a distance-preserving projection; (2) establishing an explicit linear relationship between the feature space and the new space. Based on this linear relationship, DR for test entities can be performed. We evaluated the performance of SMDS first on a synthetic dataset, and the results indicate that (1) SMDS is relatively robust to Gaussian noise existing in the features and labels; (2) the dimensionality of the new space exerts negligible influences upon SMDS; and (3) when the sample size is small, the performance of SMDS deteriorates with the increase of feature dimension. When applied to features extracted from resting state fMRI data for individual age predictions, SMDS was observed to outperform classic DR techniques, including principal component analysis, locally linear embedding and multidimensional scaling (MDS). Hopefully, SMDS can be widely used in studies on MRI-based predictions. Furthermore, novel supervised DR techniques for regression purposes can easily be developed by replacing MDS with other nonlinear DR techniques.

Discriminative low-rank projection for robust subspace learning

The robustness to outliers, noises, and corruptions has been paid more attention recently to increase the performance in linear feature extraction and image classification. As one of the most effective subspace learning methods, low-rank representation (LRR) can improve the robustness of an algorithm by exploring the global representative structure information among the samples. However, the traditional LRR cannot project the training samples into low-dimensional subspace with supervised information. Thus, in this paper, we integrate the properties of LRR with supervised dimensionality reduction techniques to obtain optimal low-rank subspace and discriminative projection at the same time. To achieve this goal, we proposed a novel model named Discriminative Low-Rank Projection (DLRP). Furthermore, DLRP can break the limitation of the small class problem which means the number of projections is bound by the number of classes. Our model can be solved by alternatively linearized alternating direction method with adaptive penalty and the singular value decomposition. Besides, the analyses of differences between DLRP and previous related models are shown. Extensive experiments conducted on various contaminated databases have confirmed the superiority of the proposed method.

Discriminant component analysis via distance correlation maximization

In the following study, an innovative supervised dimensionality reduction technique is proposed. dCor-based Dimensionality Reduction or dDR technique is based on distance correlation; a powerful correlation measure which is applicable to arbitrary-dimensional random variables. By projecting the samples to a lower dimensional space, dDR maximizes the correlation between explanatory and response variables. The proposed dDR algorithm can be easily implemented and it is computationally efficient. Moreover, it has a closed-form and a simple solution which makes it significantly effective in many different applications. In order to apply the proposed technique on non-linear problems, the kernel version of the dDR is also derived. Extensive analyses and empirical experiments across various visualization, classification, and regression tasks indicate that our algorithm is the method of choice; as it offers statistically superior results in comparison with other state-of-the-art approaches in the literature.

A General Framework for Dimensionality Reduction of K-Means Clustering

Dimensionality reduction plays an important role in many machine learning and pattern recognition applications. Linear discriminant analysis (LDA) is the most popular supervised dimensionality reduction technique which searches for the projection matrix that makes the data points of different classes to be far from each other while requiring data points of the same class to be close to each other. In this paper, trace ratio LDA is combined with K-means clustering into a unified framework, in which K-means clustering is employed to generate class labels for unlabeled data and LDA is used to investigate low-dimensional representation of data. Therefore, by combining the subspace clustering with dimensionality reduction together, the optimal subspace can be obtained. Differing from other existing dimensionality reduction methods, our novel framework is suitable for different scenarios: supervised, semi-supervised, and unsupervised dimensionality reduction cases. Experimental results on benchmark datasets validate the effectiveness and superiority of our algorithm compared with other relevant techniques.

Fault diagnosis for analog circuits utilizing time-frequency features and improved VVRKFA

This paper proposes a novel scheme for analog circuit fault diagnosis utilizing features extracted from the time-frequency representations of signals and an improved vector-valued regularized kernel function approximation (VVRKFA). First, the cross-wavelet transform is employed to yield the energy-phase distribution of the fault signals over the time and frequency domain. Since the distribution is high-dimensional, a supervised dimensionality reduction technique—the bilateral 2D linear discriminant analysis—is applied to build a concise feature set from the distributions. Finally, VVRKFA is utilized to locate the fault. In order to improve the classification performance, the quantum-behaved particle swarm optimization technique is employed to gradually tune the learning parameter of the VVRKFA classifier. The experimental results for the analog circuit faults classification have demonstrated that the proposed diagnosis scheme has an advantage over other approaches.

Block-Diagonal Constrained Low-Rank and Sparse Graph for Discriminant Analysis of Image Data.

Recently, low-rank and sparse model-based dimensionality reduction (DR) methods have aroused lots of interest. In this paper, we propose an effective supervised DR technique named block-diagonal constrained low-rank and sparse-based embedding (BLSE). BLSE has two steps, i.e., block-diagonal constrained low-rank and sparse representation (BLSR) and block-diagonal constrained low-rank and sparse graph embedding (BLSGE). Firstly, the BLSR model is developed to reveal the intrinsic intra-class and inter-class adjacent relationships as well as the local neighborhood relations and global structure of data. Particularly, there are mainly three items considered in BLSR. First, a sparse constraint is required to discover the local data structure. Second, a low-rank criterion is incorporated to capture the global structure in data. Third, a block-diagonal regularization is imposed on the representation to promote discrimination between different classes. Based on BLSR, informative and discriminative intra-class and inter-class graphs are constructed. With the graphs, BLSGE seeks a low-dimensional embedding subspace by simultaneously minimizing the intra-class scatter and maximizing the inter-class scatter. Experiments on public benchmark face and object image datasets demonstrate the effectiveness of the proposed approach.

Bearing remaining useful life estimation based on time–frequency representation and supervised dimensionality reduction

The extraction of ideal age feature is a challenging task in vibration-based bearing remaining useful life (RUL) estimation. Aiming at this problem, a new approach is proposed on the basis of time–frequency representation (TFR) and supervised dimensionality reduction. Firstly, S transform and Gaussian pyramid are employed to obtain TFRs at multiple scales. Textural features of TFRs are used as the high-dimensional features. Then, a two-step supervised dimensionality reduction technique, i.e. principal component analysis (PCA) plus linear discriminant analysis, is employed to reduce the dimensionality, in which the target dimension and number of classes are taken as variable parameters. Finally, the simple multiple linear regression model is utilized to estimate the RUL. Experimental results indicate that the proposed approach outperforms the methods using traditional statistical features and/or PCA. Additionally, variable conditions of load and speed should be considered in the future to further improve the proposed approach.

Learning regularized LDA by clustering.

As a supervised dimensionality reduction technique, linear discriminant analysis has a serious overfitting problem when the number of training samples per class is small. The main reason is that the between- and within-class scatter matrices computed from the limited number of training samples deviate greatly from the underlying ones. To overcome the problem without increasing the number of training samples, we propose making use of the structure of the given training data to regularize the between- and within-class scatter matrices by between- and within-cluster scatter matrices, respectively, and simultaneously. The within- and between-cluster matrices are computed from unsupervised clustered data. The within-cluster scatter matrix contributes to encoding the possible variations in intraclasses and the between-cluster scatter matrix is useful for separating extra classes. The contributions are inversely proportional to the number of training samples per class. The advantages of the proposed method become more remarkable as the number of training samples per class decreases. Experimental results on the AR and Feret face databases demonstrate the effectiveness of the proposed method.

Multi-label classification of chronically ill patients with bag of words and supervised dimensionality reduction algorithms

ObjectiveThis research is motivated by the issue of classifying illnesses of chronically ill patients for decision support in clinical settings. Our main objective is to propose multi-label classification of multivariate time series contained in medical records of chronically ill patients, by means of quantization methods, such as bag of words (BoW), and multi-label classification algorithms. Our second objective is to compare supervised dimensionality reduction techniques to state-of-the-art multi-label classification algorithms. The hypothesis is that kernel methods and locality preserving projections make such algorithms good candidates to study multi-label medical time series. MethodsWe combine BoW and supervised dimensionality reduction algorithms to perform multi-label classification on health records of chronically ill patients. The considered algorithms are compared with state-of-the-art multi-label classifiers in two real world datasets. Portavita dataset contains 525 diabetes type 2 (DT2) patients, with co-morbidities of DT2 such as hypertension, dyslipidemia, and microvascular or macrovascular issues. MIMIC II dataset contains 2635 patients affected by thyroid disease, diabetes mellitus, lipoid metabolism disease, fluid electrolyte disease, hypertensive disease, thrombosis, hypotension, chronic obstructive pulmonary disease (COPD), liver disease and kidney disease. The algorithms are evaluated using multi-label evaluation metrics such as hamming loss, one error, coverage, ranking loss, and average precision. ResultsNon-linear dimensionality reduction approaches behave well on medical time series quantized using the BoW algorithm, with results comparable to state-of-the-art multi-label classification algorithms. Chaining the projected features has a positive impact on the performance of the algorithm with respect to pure binary relevance approaches. ConclusionsThe evaluation highlights the feasibility of representing medical health records using the BoW for multi-label classification tasks. The study also highlights that dimensionality reduction algorithms based on kernel methods, locality preserving projections or both are good candidates to deal with multi-label classification tasks in medical time series with many missing values and high label density.

Supervised block sparse dictionary learning for simultaneous clustering and classification in computational anatomy.

An important prerequisite for computational neuroanatomy is the spatial normalization of the data. Despite its importance for the success of the subsequent statistical analysis, image alignment is dealt with from the perspective of image matching, while its influence on the group analysis is neglected. The choice of the template, the registration algorithm as well as the registration parameters, all confound group differences and impact the outcome of the analysis. In order to limit their influence, we perform multiple registrations by varying these parameters, resulting in multiple instances for each sample. In order to harness the high dimensionality of the data and emphasize the group differences, we propose a supervised dimensionality reduction technique that takes into account the organization of the data. This is achieved by solving a supervised dictionary learning problem for block-sparse signals. Structured sparsity allows the grouping of instances across different independent samples, while label supervision allows for discriminative dictionaries. The block structure of dictionaries allows constructing multiple classifiers that treat each dictionary block as a basis of a subspace that spans a separate band of information. We formulate this problem as a convex optimization problem with a geometric programming (GP) component. Promising results that demonstrate the potential of the proposed approach are shown for an MR image dataset of Autism subjects.

On improving robustness of LDA and SRDA by using tangent vectors

In the area of pattern recognition, it is common for few training samples to be available with respect to the dimensionality of the representation space; this is known as the curse of dimensionality. This problem can be alleviated by using a dimensionality reduction approach, which overcomes the curse relatively well. Moreover, supervised dimensionality reduction techniques generally provide better recognition performance; however, several of these tend to suffer from the curse when applied directly to high-dimensional spaces. We propose to overcome this problem by incorporating additional information to supervised subspace learning techniques using what is known as tangent vectors. This additional information accounts for the possible differences that the sample data can suffer. In fact, this can be seen as a way to model the unseen data and make better use of the scarce training samples. In this paper, methods for incorporating tangent vector information are described for one classical technique (LDA) and one state-of-the-art technique (SRDA). Experimental results confirm that this additional information improves performance and robustness to known transformations.

Face recognition using discriminant sparsity neighborhood preserving embedding

In this paper, we propose an effective supervised dimensionality reduction technique, namely discriminant sparsity neighborhood preserving embedding (DSNPE), for face recognition. DSNPE constructs graph and corresponding edge weights simultaneously through sparse representation (SR). DSNPE explicitly takes into account the within-neighboring information and between-neighboring information. Further, by taking the advantage of the maximum margin criterion (MMC), the discriminating power of DSNPE is further boosted. Experiments on the ORL, Yale, AR and FERET face databases show the effectiveness of the proposed DSNPE.

Relevance learning in generative topographic mapping

Generative topographic mapping (GTM) provides a flexible statistical model for unsupervised data inspection and topographic mapping. Since it yields to an explicit mapping of a low-dimensional latent space to the observation space and an explicit formula for a constrained Gaussian mixture model induced thereof, it offers diverse functionalities including clustering, dimensionality reduction, topographic mapping, and the like. However, it shares the property of most unsupervised tools that noise in the data cannot be recognized as such and, in consequence, is visualized in the map. The framework of visualization based on auxiliary information and, more specifically, the framework of learning metrics as introduced in [14,21] constitutes an elegant way to shape the metric according to auxiliary information at hand such that only those aspects are displayed in distance-based approaches which are relevant for a given classification task. Here we introduce the concept of relevance learning into GTM such that the metric is shaped according to auxiliary class labels. Relying on the prototype-based nature of GTM, efficient realizations of this paradigm are developed and compared on a couple of benchmarks to state-of-the-art supervised dimensionality reduction techniques.

Dimensionality reduction by minimizing nearest-neighbor classification error

There is a great interest in dimensionality reduction techniques for tackling the problem of high-dimensional pattern classification. This paper addresses the topic of supervised learning of a linear dimension reduction mapping suitable for classification problems. The proposed optimization procedure is based on minimizing an estimation of the nearest neighbor classifier error probability, and it learns a linear projection and a small set of prototypes that support the class boundaries. The learned classifier has the property of being very computationally efficient, making the classification much faster than state-of-the-art classifiers, such as SVMs, while having competitive recognition accuracy. The approach has been assessed through a series of experiments, showing a uniformly good behavior, and competitive compared with some recently proposed supervised dimensionality reduction techniques.

Face recognition using Intrinsicfaces

In this paper, we propose a novel face model, called intrinsic face model. Under this model, each face image is divided into three components, i.e., facial commonness difference, individuality difference and intrapersonal difference, to characterize some certain differences conveyed by this image. Then, a new supervised dimensionality reduction technique coined Intrinsic Discriminant Analysis (IDA) is developed. Intrinsic Discriminant Analysis tries to best classify different face images by maximizing the individuality difference, while minimizing the intrapersonal difference. By using perturbation technique to tackle the singularity problem of IDA which occurs frequently in face recognition, we obtain a new appearance-based face recognition method called Intrinsicfaces. A series of experiments to compare our proposed approach with other dimensionality reduction methods are tested on three well-known face databases. Experimental results demonstrate the efficacy of the proposed Intrinsicfaces approach in face recognition.