Dimensionality Reduction Tool Research Articles

Random Projection Estimation of Discrete-Choice Models with Large Choice Sets

We introduce sparse random projection, an important dimension-reduction tool from machine learning, for the estimation of discrete-choice models with high-dimensional choice sets. Initially, high-dimensional data are compressed into a lower-dimensional Euclidean space using random projections. Subsequently, estimation proceeds using cyclic monotonicity moment inequalities implied by the multinomial choice model; the estimation procedure is semi-parametric and does not require explicit distributional assumptions to be made regarding the random utility errors. The random projection procedure is justified via the Johnson-Lindenstrauss Lemma -- the pairwise distances between data points are preserved during data compression, which we exploit to show convergence of our estimator. The estimator works well in simulations and in an application to a supermarket scanner dataset.

Tensor Canonical Correlation Analysis for Multi-View Dimension Reduction

Canonical correlation analysis (CCA) has proven an effective tool for two-view dimension reduction due to its profound theoretical foundation and success in practical applications. In respect of multi-view learning, however, it is limited by its capability of only handling data represented by two-view features, while in many real-world applications, the number of views is frequently many more. Although the ad hoc way of simultaneously exploring all possible pairs of features can numerically deal with multi-view data, it ignores the high order statistics (correlation information) which can only be discovered by simultaneously exploring all features. Therefore, in this work, we develop tensor CCA (TCCA) which straightforwardly yet naturally generalizes CCA to handle the data of an arbitrary number of views by analyzing the covariance tensor of the different views. TCCA aims to directly maximize the canonical correlation of multiple (more than two) views. Crucially, we prove that the multi-view canonical correlation maximization problem is equivalent to finding the best rank-1 approximation of the data covariance tensor, which can be solved efficiently using the well-known alternating least squares (ALS) algorithm. As a consequence, the high order correlation information contained in the different views is explored and thus a more reliable common subspace shared by all features can be obtained. In addition, a non-linear extension of TCCA is presented. Experiments on various challenge tasks, including large scale biometric structure prediction, internet advertisement classification and web image annotation, demonstrate the effectiveness of the proposed method.

DNA Microarray Gene Expression Analysis for Diagnosis of Oral Dysplasia and Squamous-Cell Carcinoma

A tremendous amount of research has investigated carcinoma and looked into treatments to lower cancer mortality. Nevertheless, oral cancer is still a considerable health issue worldwide and mortality and incidence rates are rising. Oral cancer is ranked as the sixth most common carcinoma and is a prime health dilemma globally. Thus, it is increasingly important to have tools that allow for discrimination between oral dysplasia and squamous-cell carcinoma. Although conventional methods such as medical imaging can be helpful in prediction and diagnosis of oral cancer, these methods continue to have limitations. More recently, machine learning has been used as a complementary approach in biomedical research and it now plays a leading role in the emerging domains of computational biology and bioinformatics. Specifically, DNA chip gene expression technology is currently helping researchers to distinguish oral dysplasia and squamous-cell carcinoma thru analysis of gene expression in tissue samples. Nonetheless, one of the prime challenges in the computational process of microarray data is the curse of dimensionality due to the crushing number of features. Therefore, we applied a random projection (RP) feature construction technique to tackle the problem of high-dimensional gene expression data and to increase the efficiency of our proposed model. In addition, we combined a RP technique with a support vector machine (SVM) that employs a sequential minimal optimization training algorithm (SMO) in order to efficiently differentiate squamous-cell carcinoma and oral dysplasia. The highest classification accuracy recorded by our proposed model was 95.6332%. We show in this study that using a SMO machine-learning classifier with a RP dimensionality reduction tool can be effective for classifying oral cancer.

Online control of a humanoid robot through hand movement imagination using CSP and ECoG based features.

Intention recognition through decoding brain activity could lead to a powerful and independent Brain-Computer-Interface (BCI) allowing for intuitive control of devices like robots. A common strategy for realizing such a system is the motor imagery (MI) BCI using electroencephalography (EEG). Changing to invasive recordings like electrocorticography (ECoG) allows extracting very robust features and easy introduction of an idle state, which might simplify the mental task and allow the subject to focus on the environment. Especially for multi-channel recordings like ECoG, common spatial patterns (CSP) provide a powerful tool for feature optimization and dimensionality reduction. This work focuses on an invasive and independent MI BCI that allows triggering from an idle state, and therefore facilitates tele-operation of a humanoid robot. The task was to lift a can with the robot's hand. One subject participated and reached 95.4 % mean online accuracy after six runs of 40 trials. To our knowledge, this is the first online experiment with a MI BCI using CSPs from ECoG signals.

Fold-change threshold screening: a robust algorithm to unmask hidden gene expression patterns in noisy aggregated transcriptome data.

Transcriptomics is often used to investigate changes in an organism's genetic response to environmental contamination. Data noise can mask the effects of contaminants making it difficult to detect responding genes. Because the number of genes which are found differentially expressed in transcriptome data is often very large, algorithms are needed to reduce the number down to a few robust discriminative genes. We present an algorithm for aggregated analysis of transcriptome data which uses multiple fold-change thresholds (threshold screening) and p values from Bayesian generalized linear model in order to assess the robustness of a gene as a potential indicator for the treatments tested. The algorithm provides a robustness indicator (ROBI) as well as a significance profile, which can be used to assess the statistical significance of a given gene for different fold-change thresholds. Using ROBI, eight discriminative genes were identified from an exemplary dataset (Danio rerio FET treated with chlorpyrifos, methylmercury, and PCB) which could be potential indicators for a given substance. Significance profiles uncovered genetic effects and revealed appropriate fold-change thresholds for single genes or gene clusters. Fold-change threshold screening is a powerful tool for dimensionality reduction and feature selection in transcriptome data, as it effectively reduces the number of detected genes suitable for environmental monitoring. In addition, it is able to unmask patterns in altered genetic expression hidden by data noise and reduces the chance of type II errors, e.g., in environmental screening.

Genome-wide gene-gene interaction analysis for next-generation sequencing.

The critical barrier in interaction analysis for next-generation sequencing (NGS) data is that the traditional pairwise interaction analysis that is suitable for common variants is difficult to apply to rare variants because of their prohibitive computational time, large number of tests and low power. The great challenges for successful detection of interactions with NGS data are (1) the demands in the paradigm of changes in interaction analysis; (2) severe multiple testing; and (3) heavy computations. To meet these challenges, we shift the paradigm of interaction analysis between two SNPs to interaction analysis between two genomic regions. In other words, we take a gene as a unit of analysis and use functional data analysis techniques as dimensional reduction tools to develop a novel statistic to collectively test interaction between all possible pairs of SNPs within two genome regions. By intensive simulations, we demonstrate that the functional logistic regression for interaction analysis has the correct type 1 error rates and higher power to detect interaction than the currently used methods. The proposed method was applied to a coronary artery disease dataset from the Wellcome Trust Case Control Consortium (WTCCC) study and the Framingham Heart Study (FHS) dataset, and the early-onset myocardial infarction (EOMI) exome sequence datasets with European origin from the NHLBI's Exome Sequencing Project. We discovered that 6 of 27 pairs of significantly interacted genes in the FHS were replicated in the independent WTCCC study and 24 pairs of significantly interacted genes after applying Bonferroni correction in the EOMI study.

Unsupervised learning for signal mapping in dynamic photon emission

Dynamic photon emission is an efficient tool for timing analysis of various areas. However, advances in transistors integration bring more complex test patterns and more objects to investigate. As a consequence, understanding the analyzed area and finding nodes of interest can be difficult. In this paper, a method for drawing synthesis of the various signals met inside an area is reported. It is based on unsupervised learning tool for dimension reduction and clustering. The process is applied to real data to show its efficiency and its quality is evaluated.

CNMA: a framework of encounter complex-based normal mode analysis to model conformational changes in protein interactions.

Motivation: It remains both a fundamental and practical challenge to understand and anticipate motions and conformational changes of proteins during their associations. Conventional normal mode analysis (NMA) based on anisotropic network model (ANM) addresses the challenge by generating normal modes reflecting intrinsic flexibility of proteins, which follows a conformational selection model for protein–protein interactions. But earlier studies have also found cases where conformational selection alone could not adequately explain conformational changes and other models have been proposed. Moreover, there is a pressing demand of constructing a much reduced but still relevant subset of protein conformational space to improve computational efficiency and accuracy in protein docking, especially for the difficult cases with significant conformational changes.Method and results: With both conformational selection and induced fit models considered, we extend ANM to include concurrent but differentiated intra- and inter-molecular interactions and develop an encounter complex-based NMA (cNMA) framework. Theoretical analysis and empirical results over a large data set of significant conformational changes indicate that cNMA is capable of generating conformational vectors considerably better at approximating conformational changes with contributions from both intrinsic flexibility and inter-molecular interactions than conventional NMA only considering intrinsic flexibility does. The empirical results also indicate that a straightforward application of conventional NMA to an encounter complex often does not improve upon NMA for an individual protein under study and intra- and inter-molecular interactions need to be differentiated properly. Moreover, in addition to induced motions of a protein under study, the induced motions of its binding partner and the coupling between the two sets of protein motions present in a near-native encounter complex lead to the improved performance. A study to isolate and assess the sole contribution of intermolecular interactions toward improvements against conventional NMA further validates the additional benefit from induced-fit effects. Taken together, these results provide new insights into molecular mechanisms underlying protein interactions and new tools for dimensionality reduction for flexible protein docking.Availability and implementation: Source codes are available upon request.Contact: yshen@tamu.edu

A Nonlinear Approach to Dimension Reduction

The $l_2$ flattening lemma of Johnson and Lindenstrauss [JL84] is a powerful tool for dimension reduction. It has been conjectured that the target dimension bounds can be refined and bounded in terms of the intrinsic dimensionality of the data set (for example, the doubling dimension). One such problem was proposed by Lang and Plaut [LP01] (see also [GKL03,MatousekProblems07,ABN08,CGT10]), and is still open. We prove another result in this line of work: The snowflake metric $d^{1/2}$ of a doubling set $S \subset l_2$ embeds with constant distortion into $l_2^D$, for dimension $D$ that depends solely on the doubling constant of the metric. In fact, the distortion can be made arbitrarily close to 1, and the target dimension is polylogarithmic in the doubling constant. Our techniques are robust and extend to the more difficult spaces $l_1$ and $l_\infty$, although the dimension bounds here are quantitatively inferior than those for $l_2$.

Unsupervised neighborhood component analysis for clustering

In this paper, we propose a novel unsupervised distance metric learning algorithm. The proposed algorithm aims to maximize a stochastic variant of the leave-one-out K-nearest neighbor (KNN) score on unlabeled data, which performs distance metric learning and clustering simultaneously. We show that the joint distance metric learning and clustering problem is formulated as a trace optimization problem, and can be solved efficiently by an iterative algorithm. Moreover, the proposed approach can also learn a low dimensional projection of high dimensional data, thus it can serve as an unsupervised dimensionality reduction tool, which is capable of performing joint dimensionality reduction and clustering. We validate our method on a number of benchmark datasets, and the results demonstrate the effectiveness of the proposed algorithm.

A Robust GA/KNN Based Hypothesis Verification System for Vehicle Detection

Vehicle detection is an important issue in driver assistance systems and self-guided vehicles that includes two stages of hypothesis generation and verification. In the first stage, potential vehicles are hypothesized and in the second stage, all hypothesis are verified. The focus of this work is on the second stage. We extract Pyramid Histograms of Oriented Gradients (PHOG) features from a traffic image as candidates of feature vectors to detect vehicles. Principle Component Analysis (PCA) and Linear Discriminant Analysis (LDA) are applied to these PHOG feature vectors as dimension reduction and feature selection tools parallelly. After feature fusion, we use Genetic Algorithm (GA) and cosine similarity-based K Nearest Neighbor (KNN) classification to improve the performance and generalization of the features. Our tests show good classification accuracy of more than 97% correct classification on realistic on-road vehicle images.

Risk-based fault detection using Self-Organizing Map

The complexity of modern systems is increasing rapidly and the dominating relationships among system variables have become highly non-linear. This results in difficulty in the identification of a system׳s operating states. In turn, this difficulty affects the sensitivity of fault detection and imposes a challenge on ensuring the safety of operation. In recent years, Self-Organizing Maps has gained popularity in system monitoring as a robust non-linear dimensionality reduction tool. Self-Organizing Map is able to capture non-linear variations of the system. Therefore, it is sensitive to the change of a system׳s states leading to early detection of fault. In this paper, a new approach based on Self-Organizing Map is proposed to detect and assess the risk of fault. In addition, probabilistic analysis is applied to characterize the risk of fault into different levels according to the hazard potential to enable a refined monitoring of the system. The proposed approach is applied on two experimental systems. The results from both systems have shown high sensitivity of the proposed approach in detecting and identifying the root cause of faults. The refined monitoring facilitates the determination of the risk of fault and early deployment of remedial actions and safety measures to minimize the potential impact of fault.

Sparse optimal discriminant clustering

In this manuscript, we reinvestigate an existing clustering procedure, optimal discriminant clustering (ODC; Zhang and Dai in Adv Neural Inf Process Syst 23(12):2241---2249, 2009), and propose to use cross-validation to select the tuning parameter. Furthermore, because in high-dimensional data many of the features may be non-informative for clustering, we develop a variation of ODC, sparse optimal discriminant clustering (SODC), by adding a group-lasso type of penalty to ODC. We also demonstrate that both ODC and SDOC can be used as a dimension reduction tool for data visualization in cluster analysis.

Fast orthogonal linear discriminant analysis with application to image classification

Compared to linear discriminant analysis (LDA), its orthogonalized version is a more effective statistical learning tool for dimension reduction, which devotes to better separating the data points from different classes in the lower-dimensional subspace. However, existing orthogonalized LDA techniques suffer from various drawbacks, including the requirement for expensive computing time. This paper develops an efficient orthogonal dimension reduction approach, referred to as fast orthogonal linear discriminant analysis (FOLDA), which is based on existing orthogonal linear discriminant analysis (OLDA) algorithms. However, different from previous efforts, the new approach applies the QR decomposition and the regression to solve for a new orthogonal projection vector at each iteration, leading to the by far cheaper computational cost. FOLDA achieves comparable recognition rate to existing OLDA algorithms due to the incorporation of the idea and spirit behind the latter ones. Experimental results on image databases, such as MINST, COIL20, MEPG-7 and OUTEX, show the effectiveness and efficiency of our algorithm.

A novel dynamic rough subspace based selective ensemble

Ensemble learning has been a hot topic in machine learning due to its successful utilization in many applications. Rough set theory has been proved to be an excellent mathematical tool for dimension reduction. In this paper, based on rough set, a novel framework for ensemble is proposed. In our proposed framework, the relationship among attributes in rough subspace is first considered, and the maximum dependency degree of attribute is first employed to effectively reduce the searching space of reducts and augment the diversity of selected reducts. In addition, in order to choose an appropriate reduct from the dynamic reduct searching space, an assessment function which can balance the accuracy and diversity is utilized. At last, a new method, i.e., Dynamic Rough Subspace based Selective Ensemble (DRSSE), which is derived from our framework is given. By repeatedly changing the searching space of reducts and selecting the next reduct from the changed searching space, DRSSE finally trains an ensemble system with these selected reducts. Compared with several available ensemble methods, experimental results with several datasets demonstrate that DRSSE can lead to a comparative or even better performance.

Novel multivariate methods for integration of genomics and proteomics data: applications in a kidney transplant rejection study.

Multi-omics research is a key ingredient of data-intensive life sciences research, permitting measurement of biological molecules at different functional levels in the same individual. For a complete picture at the biological systems level, appropriate statistical techniques must however be developed to integrate different 'omics' data sets (e.g., genomics and proteomics). We report here multivariate projection-based analyses approaches to genomics and proteomics data sets, using the case study of and applications to observations in kidney transplant patients who experienced an acute rejection event (n=20) versus non-rejecting controls (n=20). In this data sets, we show how these novel methodologies might serve as promising tools for dimension reduction and selection of relevant features for different analytical frameworks. Unsupervised analyses highlighted the importance of post transplant time-of-rejection, while supervised analyses identified gene and protein signatures that together predicted rejection status with little time effect. The selected genes are part of biological pathways that are representative of immune responses. Gene enrichment profiles revealed increases in innate immune responses and neutrophil activities and a depletion of T lymphocyte related processes in rejection samples as compared to controls. In all, this article offers candidate biomarkers for future detection and monitoring of acute kidney transplant rejection, as well as ways forward for methodological advances to better harness multi-omics data sets.

Estimation and inference in functional single-index models

We propose a functional single-index model (FSiM) to study the link between a scalar response variable and multiple functional predictors, in which the mean of the response is related to the linear predictors via an unknown link function. The FSiM serves as a good tool for dimension reduction in regression with multiple predictors and it is more flexible than functional linear models. Assuming that the functional predictors are observed at discrete points, we use B-spline basis functions to estimate the slope functions and the link function based on the least-squares criterion, and propose an iterative estimating procedure. Moreover, we provide uniform convergence rates of the proposed spline estimators in the FSiM, and construct asymptotic simultaneous confidence bands for the slope functions for inference. Our proposed method is illustrated by simulation studies and by an analysis of a diffusion tensor imaging data application.

A hedonic price analysis for the Italian wine in the domestic market

Price formation in the domestic market has not been widely studied in spite of the importance of the Italian wine market in terms of sales. We use a unique dataset to estimate the hedonic price function for Italian wine sold on the Italian market in the period 2005–2011. For each bottle considered, the dataset records several characteristics such as the price by retail channel (on the mass market and in wine shops), label characteristics, chemical analysis, sensory evaluations and experts’ opinions. The objective of the analysis is to examine price setting on the mass market and in wine shops and to explore the differences in price formation for red and white wines. Our results have been obtained using an innovative technique that consists of combining hedonic price techniques with dimensionality reduction tools.

Single trial detection of hand poses in human ECoG using CSP based feature extraction.

Decoding brain activity of corresponding highlevel tasks may lead to an independent and intuitively controlled Brain-Computer Interface (BCI). Most of today's BCI research focuses on analyzing the electroencephalogram (EEG) which provides only limited spatial and temporal resolution. Derived electrocorticographic (ECoG) signals allow the investigation of spatially highly focused task-related activation within the high-gamma frequency band, making the discrimination of individual finger movements or complex grasping tasks possible. Common spatial patterns (CSP) are commonly used for BCI systems and provide a powerful tool for feature optimization and dimensionality reduction. This work focused on the discrimination of (i) three complex hand movements, as well as (ii) hand movement and idle state. Two subjects S1 and S2 performed single `open', `peace' and `fist' hand poses in multiple trials. Signals in the high-gamma frequency range between 100 and 500 Hz were spatially filtered based on a CSP algorithm for (i) and (ii). Additionally, a manual feature selection approach was tested for (i). A multi-class linear discriminant analysis (LDA) showed for (i) an error rate of 13.89 % / 7.22 % and 18.42 % / 1.17 % for S1 and S2 using manually / CSP selected features, where for (ii) a two class LDA lead to a classification error of 13.39 % and 2.33 % for S1 and S2, respectively.

A Big Data Mining in Petroleum Exploration and Development

We take a well log in petroleum exploration and development as an example of the big data mining, and adopt three regression and two classification algorithms: the multiple regression analysis (MRA), the error back-propagation neural network (BPNN), the regression of support vector machine ( R -SVM), the classification of support vector machine ( C -SVM), and the Bayesian successive discrimination (BAYSD). It is well known that MRA, BPNN and R -SVM are regression algorithms while C -SVM and BAYSD are classification algorithms, and only MRA is linear algorithm whereas the other four algorithms are nonlinear algorithms. From this case study, we can draw the following five major conclusions: a) Since C -SVM is the best classifier, it is employed as a data cleaning tool. b) Since MRA is a linear algorithm, its total mean absolute relative residual R * (%) can express the nonlinearity of studied problem. For this case study, R * (%)=52.14 showing the nonlinearity of the studied problem is strong. c) Since both MRA and BAYD can establish the order of dependence between a dependent variable and independent variables, each of MRA and BAYD could serve as a pioneering dimension-reduction tool in data mining. In the case study, since the nonlinearity of the studied problem is strong, the nonlinear algorithm BAYSD can serve as a pioneering dimension-reduction tool, but the linear algorithm MRA cannot. d) Since the nonlinearity of the case study is strong, BPNN and R -SVM are not applicable though they are nonlinear algorithms, whereas other two nonlinear algorithms C -SVM and BAYSD are applicable, indicating the nonlinear ability of C -SVM and BAYSD is higher than that of BPNN and R -SVM. e) Comparing the two applicable algorithms C -SVM and BAYSD for this case study, it is seen that R * (%) of C -SVM is less than that of BAYSD; BAYSD can serve as a pioneering dimension-reduction tool, but C -SVM cannot; it is easy to code the BAYSD program whereas it is very complicated to code the C -SVM program, so BAYSD is a good software for this case study when C -SVM is not available. Key words : Big data mining; Well log; Data cleaning; Dimension-reduction; Regression; Classification