Problem Of Dimensionality Reduction Research Articles

Binary Ant Colony Algorithm with Particle Optimization Feature for Attribute Reduction Problem

Binary Ant Colony Algorithm with Particle Optimization Feature for Attribute Reduction Problem

A geometric viewpoint of manifold learning

In many data analysis tasks, one is often confronted with very high dimensional data. The manifold assumption, which states that the data is sampled from a submanifold embedded in much higher dimensional Euclidean space, has been widely adopted by many researchers. In the last 15 years, a large number of manifold learning algorithms have been proposed. Many of them rely on the evaluation of the geometrical and topological of the data manifold. In this paper, we present a review of these methods on a novel geometric perspective. We categorize these methods by three main groups: Laplacian-based, Hessian-based, and parallel field-based methods. We show the connection and difference between these three groups on their continuous and discrete counterparts. The discussion is focused on the problem of dimensionality reduction and semi-supervised learning.

Interpretation of FTIR spectra of polymers and Raman spectra of car paints by means of likelihood ratio approach supported by wavelet transform for reducing data dimensionality.

The problem of interpretation of common provenance of the samples within the infrared spectra database of polypropylene samples from car body parts and plastic containers as well as Raman spectra databases of blue solid and metallic automotive paints was under investigation. The research involved statistical tools such as likelihood ratio (LR) approach for expressing the evidential value of observed similarities and differences in the recorded spectra. Since the LR models can be easily proposed for databases described by a few variables, research focused on the problem of spectra dimensionality reduction characterised by more than a thousand variables. The objective of the studies was to combine the chemometric tools easily dealing with multidimensionality with an LR approach. The final variables used for LR models' construction were derived from the discrete wavelet transform (DWT) as a data dimensionality reduction technique supported by methods for variance analysis and corresponded with chemical information, i.e. typical absorption bands for polypropylene and peaks associated with pigments present in the car paints. Univariate and multivariate LR models were proposed, aiming at obtaining more information about the chemical structure of the samples. Their performance was controlled by estimating the levels of false positive and false negative answers and using the empirical cross entropy approach. The results for most of the LR models were satisfactory and enabled solving the stated comparison problems. The results prove that the variables generated from DWT preserve signal characteristic, being a sparse representation of the original signal by keeping its shape and relevant chemical information.

The logic transformations for reducing the complexity of the discernibility function-based attribute reduction problem

The basic solution for locating an optimal reduct is to generate all possible reducts and select the one that best meets the given criterion. Since this problem is NP-hard, most attribute reduction algorithms use heuristics to find a single reduct with the risk to overlook for the best ones. There is a discernibility function (DF)-based approach that generates all reducts but may fail due to memory overflows even for datasets with dimensionality much below the medium. In this study, we show that the main shortcoming of this approach is its excessively high space complexity. To overcome this, we first represent a DF of $$n$$n attributes by a bit-matrix (BM). Second, we partition the BM into no more than $$n-1$$n-1 sub-BMs (SBMs). Third, we convert each SBM into a subset of reducts by preventing the generation of redundant products, and finally, we unite the subsets into a complete set of reducts. Among the SBMs of a BM, the most complex one is the first SBM with a space complexity not greater than the square root of that of the original BM. The proposed algorithm converts such a SBM with $$n$$n attributes into the subset of reducts with the worst case space complexity of $$\left( _{n/2}^n \right) /2$$n/2n/2.

Dimension reduction for anisotropic Bose–Einstein condensates in the strong interaction regime

We study the problem of dimension reduction for the three dimensional Gross–Pitaevskii equation (GPE) describing a Bose–Einstein condensate confined in a strongly anisotropic harmonic trap. Since the gas is assumed to be in a strong interaction regime, we have to analyse two combined singular limits: a semi-classical limit in the transport direction and the strong partial confinement limit in the transversal direction. We prove that both limits commute together and we provide convergence rates. The by-products of this work are approximated models in reduced dimension for the GPE, with a priori estimates of the approximation errors.

Worst case linear discriminant analysis as scalable semidefinite feasibility problems.

In this paper, we propose an efficient semidefinite programming (SDP) approach to worst case linear discriminant analysis (WLDA). Compared with the traditional LDA, WLDA considers the dimensionality reduction problem from the worst case viewpoint, which is in general more robust for classification. However, the original problem of WLDA is non-convex and difficult to optimize. In this paper, we reformulate the optimization problem of WLDA into a sequence of semidefinite feasibility problems. To efficiently solve the semidefinite feasibility problems, we design a new scalable optimization method with a quasi-Newton method and eigen-decomposition being the core components. The proposed method is orders of magnitude faster than standard interior-point SDP solvers. Experiments on a variety of classification problems demonstrate that our approach achieves better performance than standard LDA. Our method is also much faster and more scalable than standard interior-point SDP solvers-based WLDA. The computational complexity for an SDP with m constraints and matrices of size d by d is roughly reduced from O(m(3)+md(3)+m(2)d(2)) to O(d(3)) (m>d in our case).

On Finding All Reducts of Consistent Decision Tables

Abstract The problem of finding reducts plays an important role in processing information on decision tables. The objective of the attribute reduction problem is to reject a redundant attribute in order to find a core attribute for data processing. The attribute reduction in decision tables is the process of finding a minimal subset of conditional attributes which preserve the classification ability of decision tables. In this paper we present the time complexity of the problem of finding all reducts of a consistent decision table. We prove that this time complexity is exponential with respect to the number of attributes of the decision tables. Our proof is performed in two steps. The first step is to show that there exists an exponential algorithm which finds all reducts. The other step is to prove that the time complexity of the problem of finding all reducts of a decision table is not less than exponential.

The constitution of visual perceptual units in the functional architecture of V1.

In this paper we show that the emergence of perceptual units in V1 can be explained in terms of a physical mechanism of simmetry breaking of the mean field neural equation. We consider a mean field neural model which takes into account the functional architecture of the visual cortex modeled as a group of rotations and translations equipped with a degenerate metric. The model generalizes well known results of Bressloff and Cowan which, in absence of input, accounts for hallucination patterns. The main result of our study consists in showing that in presence of a visual input, the stable eigenmodes of the linearized operator represent perceptual units of the visual stimulus. The result is strictly related to dimensionality reduction and clustering problems.

Evaluation of projections, obtained by dimensionality reduction techniques

In this paper, the projection evaluation measures such as stress function, Spearman’s rho, Konig’s topology preservation, silhouette and Renyi entropy have been analyzed. The principal component analysis (PCA) and part–linear multidimensional projection (PLMP) techniques are used to reduce the dimensionality of the initial data set. The experiments have been carried out with seven real and artificial datasets. The experimental investigation has shown that several quality evaluationmeasures have to be used when dimension reduction problem is solved.

Research on Locality Preserving Discriminant Projection Algorithm Based on Gabor for Face Expression Recognition

For the problem of features extraction and dimensionality reduction of expression recognition, the paper proposes Gabor Locality Preserving Discriminant Projection (GLPDP) algorithm, which is based on Gabor Wavelet. Firstly, we use Gabor wavelet transform to have an expression feature extraction. Secondly, we improved the locality preserving projection (LPP) algorithm, introducing scatter difference in the LPP objective function to increase divergence constraints among the sample classes and extracts more discriminated features while having the dimensionality reduction. Finally, we use the nearest neighbor classifier to have a classification for expression category. The effectiveness of the proposed methods is validated through the experimental results on JAFFE and Cohn-Kanade Facial expression databases.

Discrete Hessian Eigenmaps method for dimensionality reduction

For a given set of data points lying on a low-dimensional manifold embedded in a high-dimensional space, the dimensionality reduction is to recover a low-dimensional parametrization from the data set. The recently developed Hessian Eigenmaps method is a mathematically rigorous method that also sets a theoretical framework for the nonlinear dimensionality reduction problem. In this paper, we develop a discrete version of the Hessian Eigenmaps method and present an analysis, giving conditions under which the method works as intended. As an application, a procedure to modify the standard constructions of k-nearest neighborhoods is presented to ensure that Hessian LLE can recover the original coordinates up to an affine transformation.

Multi-objective cost-sensitive attribute reduction on data with error ranges

In current supervised machine learning research spectrum, there are several attribute reduction methodologies to acquire reducts with low test cost. They can deal with symbolic data, or numeric data with error ranges. In many cases, they consider the situation with only one type of cost; therefore the problem is single-objective. This paper addresses the attribute reduction problem on data with multi-type-costs and error ranges. First, we define the multi-objective attribute reduction problem where multi-type-costs are involved. Second, we propose three metrics to evaluate the quality of a reduct set. Third, we design a backtrack algorithm to compute the Pareto optimal set, and a heuristic algorithm to find a sub-optimal reduct set. Finally, we compare these algorithms on seven UCI (University of California-Irvine) datasets. Experimental results indicate that our heuristic algorithm has good capability of tackling the proposed problem.

Feature selection for high-dimensional class-imbalanced data sets using Support Vector Machines

Feature selection and classification of imbalanced data sets are two of the most interesting machine learning challenges, attracting a growing attention from both, industry and academia. Feature selection addresses the dimensionality reduction problem by determining a subset of available features to build a good model for classification or prediction, while the class-imbalance problem arises when the class distribution is too skewed. Both issues have been independently studied in the literature, and a plethora of methods to address high dimensionality as well as class-imbalance has been proposed. The aim of this work is to simultaneously explore both issues, proposing a family of methods that select those attributes that are relevant for the identification of the target class in binary classification. We propose a backward elimination approach based on successive holdout steps, whose contribution measure is based on a balanced loss function obtained on an independent subset. Our experiments are based on six highly imbalanced microarray data sets, comparing our methods with well-known feature selection techniques, and obtaining a better prediction with consistently fewer relevant features.

Dynamic Functional Principal Components

SummaryWe address the problem of dimension reduction for time series of functional data (Xt:t∈Z). Such functional time series frequently arise, for example, when a continuous time process is segmented into some smaller natural units, such as days. Then each X t represents one intraday curve. We argue that functional principal component analysis, though a key technique in the field and a benchmark for any competitor, does not provide an adequate dimension reduction in a time series setting. Functional principal component analysis indeed is a static procedure which ignores the essential information that is provided by the serial dependence structure of the functional data under study. Therefore, inspired by Brillinger's theory of dynamic principal components, we propose a dynamic version of functional principal component analysis which is based on a frequency domain approach. By means of a simulation study and an empirical illustration, we show the considerable improvement that the dynamic approach entails when compared with the usual static procedure.

A new approach to minimum attribute reduction based on discrete artificial bee colony

The minimum attribute reduction problem in the context of rough set theory is an NP-hard nonlinearly constrained combinatorial optimization problem. In this paper, we propose an efficient and competitive combinatorial artificial bee colony algorithm for solving the minimum attribute reduction problem. In the proposed algorithm, a new multidimensional binary local search scheme for bee colonies based on velocity computation is presented; an employed bee and its recruited onlooker bees use different local search strategies so as to get a possibly more diversified neighboring search around a current food source; the information of the so-far best solution is exploited in various ways by employed bees, onlookers and scouts, respectively; the monotonicity property of classification quality of attribute subsets from the theory of rough sets is employed to avoid possibly invalid local searches. Performance comparisons with some best performing population-based metaheuristic algorithms for the minimum attribute reduction problem were carried out on a number of UCI data sets. The experimental results show that the proposed algorithm overall outperforms all the other algorithms in terms of solution quality and is therefore promising for solving the minimum attribute reduction problem.

A critical feature extraction by kernel PCA in stock trading model

This paper presents a kernel-based principal component analysis (kernel PCA) to extract critical features for improving the performance of a stock trading model. The feature extraction method is one of the techniques to solve dimensionality reduction problems (DRP). The kernel PCA is a feature extraction approach which has been applied to data transformation from known variables to capture critical information. The kernel PCA is a kernel-based data mapping tool that has characteristics of both principal component analysis and non-linear mapping. The feature selection method is another DRP technique that selects only a small set of features from known variables, but these features still indicate possible collinearity problems that fail to reflect clear information. However, most feature extraction methods use a variable mapping application to eliminate noisy and collinear variables. In this research, we use the kernel-PCA method in a stock trading model to transform stock technical indices (TI) which allows features of smaller dimension to be formed. The kernel-PCA method has been applied to various stocks and sliding window testing methods using both half-year and 1-year testing strategies. The experimental results show that the proposed method generates more profits than other DRP methods on the America stock market. This stock trading model is very practical for real-world application, and it can be implemented in a real-time environment.

Robust Principal Component Analysis Algorithm Based on Nonparametric Correlation Coefficient Matrix

Principal component analysis algorithm is widely applied in solving a large number of dimension reduction problems for high-dimensional data in fields such as image identification and comprehensive evaluation due to its clear conception and simple application. The PCA based on covariance matrix and correlation coefficient matrix is very sensitive to abnormal values so that the data structure is distorted. As a result, there exist great deviations in the analysis results and it is hard to explain. The traditional PCA is remade in this paper based on non-parametric statistical ideas, nonparametric correlation coefficient and gray relational coefficient, in order to define three kinds of robustness evaluation criteria and objectively evaluate the robustness of the algorithm through numerical experiments. The results of the numerical experiments show that the new algorithm is of good robustness to noise pollution and the algorithm is of good applicability.

State dimension reduction and analysis of quantized estimation systems

The problem of state dimension reduction and quantizer design under communication constraints is discussed for state estimation in quantized linear systems. Subject to the limited signal power, number and bandwidth of the parallel channels, a differential pulse code modulation (DPCM)-like structure is adopted to generate the quantized innovations as the transmitted signals, and the multi-level quantized Kalman filter (MLQ-KF) is used to serve as the pre- and post-filters. The dimension reduction matrix and quantizer are designed jointly under the MMSE criterion of estimation at the channel receiver. To demonstrate the validity of state estimation under the adopted framework, the state estimability based on quantized innovations is analyzed by using information theoretic method. This leads to a sufficient and necessary condition of a certain estimability Gramian matrix having full rank. The quantized Gramian is proved to converge to that of the original unquantized system when the quantization intervals turn to zero. Our work also provides an auxiliary analytic support for the estimation under 1-bit quantization. Simulations show that under communication constraints, the estimation performance is satisfactory when the designed dimension reduction method and quantizer are applied. The analytic conclusion of estimability is also verified by illustrative simulations.

Signed Laplacian Embedding for Supervised Dimension Reduction

Manifold learning is a powerful tool for solving nonlinear dimension reduction problems. By assuming that the high-dimensional data usually lie on a low-dimensional manifold, many algorithms have been proposed. However, most algorithms simply adopt the traditional graph Laplacian to encode the data locality, so the discriminative ability is limited and the embedding results are not always suitable for the subsequent classification. Instead, this paper deploys the signed graph Laplacian and proposes Signed Laplacian Embedding (SLE) for supervised dimension reduction. By exploring the label information, SLE comprehensively transfers the discrimination carried by the original data to the embedded low-dimensional space. Without perturbing the discrimination structure, SLE also retains the locality.Theoretically, we prove the immersion property by computing the rank of projection, and relate SLE to existing algorithms in the frame of patch alignment. Thorough empirical studies on synthetic and real datasets demonstrate the effectiveness of SLE.

Semi-Supervised Hyperspectral Subspace Learning Based on a Generalized Eigenvalue Problem for Regression and Dimensionality Reduction

Manifold learning for the hyperspectral data structure of intra-class variation provides useful information for investigating the intrinsic coordinates corresponding to the quantitative properties inherent in the class. However, in the high-dimensional feature space, it is unfeasible to acquire a statistically sufficient number of labeled data to estimate the coordinates. In this paper, we propose semi-supervised regression and dimensionality reduction methods for hyperspectral subspace learning that utilize abundant unlabeled data and a small number of labeled data. The quantitative target variables for regression and the order constraints for dimensionality reduction are embedded in matrices representing data relations, i.e., a set of between-class scatter matrices, within-class scatter matrices, and supervised local attraction matrices. The optimal projection matrices are estimated by generalized eigenvalue problems based on the matrices. The proposed methods are applied to synthetic linear regression problems and dimensionality reduction problems based on a time-series of hyperspectral data for a deciduous broad-leaved forest to extract local coordinates related to phenological changes. The order consistency of the projections is assessed by evaluating an index based on the Mann-Kendall test statistics. The proposed methods demonstrate much better performances in terms of both regression and dimensionality reduction than the alternative supervised and unsupervised methods.