Combining Dimension Reduction Research Articles

The perfect marriage and much more: Combining dimension reduction, distance measures and covariance

We develop a novel methodology based on the marriage between the Bhattacharyya distance, a measure of similarity across distributions of random variables, and the Johnson–Lindenstrauss Lemma, a technique for dimension reduction. The resulting technique is a simple yet powerful tool that allows comparisons between data-sets representing any two distributions. The degree to which different entities, (markets, universities, hospitals, cities, groups of securities, etc.), have different distance measures of their corresponding distributions tells us the extent to which they are different, aiding participants looking for diversification or looking for more of the same thing. We demonstrate a relationship between covariance and distance measures based on a generic extension of Stein’s Lemma. We consider an asset pricing application and then briefly discuss how this methodology lends itself to numerous market–structure studies and even applications outside the realm of finance / social sciences by illustrating a biological application. We provide numerical illustrations using security prices, volumes and volatilities of both these variables from six different countries.

RetinaMatch: Efficient Template Matching of Retina Images for Teleophthalmology.

Retinal template matching and registration is an important challenge in teleophthalmology with low-cost imaging devices. However, the images from such devices generally have a small field of view (FOV) and image quality degradations, making matching difficult. In this paper, we develop an efficient and accurate retinal matching technique that combines dimension reduction and mutual information (MI), called RetinaMatch. The dimension reduction initializes the MI optimization as a coarse localization process, which narrows the optimization domain and avoids local optima. The effectiveness of RetinaMatch is demonstrated on the open fundus image database STARE with simulated reduced FOV and anticipated degradations, and on retinal images acquired by adapter-based optics attached to a smartphone. RetinaMatch achieves a success rate over 94% on human retinal images with the matched target registration errors below 2 pixels on average, excluding the observer variability, outperforming standard template matching solutions. In the application of measuring vessel diameter repeatedly, single pixel errors are expected. In addition, our method can be used in the process of image mosaicking with area-based registration, providing a robust approach when feature-based methods fail. To the best of our knowledge, this is the first template matching algorithm for retina images with small template images from unconstrained retinal areas. In the context of the emerging mixed reality market, we envision automated retinal image matching and registration methods as transformative for advanced teleophthalmology and long-term retinal monitoring.

Beyond Tandem Analysis: Joint Dimension Reduction and Clustering in R

We present the R package clustrd which implements a class of methods that combine dimension reduction and clustering of continuous or categorical data. In particular, for continuous data, the package contains implementations of factorial K-means and reduced K-means; both methods combine principal component analysis with K-means clustering. For categorical data, the package provides MCA K-means, i-FCB and cluster correspondence analysis, which combine multiple correspondence analysis with K-means. Two examples on real data sets are provided to illustrate the usage of the main functions.

Control of Self-Assembly with Dynamic Programming

Abstract This conference paper updates a previously-reported methodology for establishing feedback control of self-assembly (Griffin et al. (2016b)). The methodology combines dimension reduction, supervised learning, and dynamic programming to obtain an optimal feedback control policy for reaching a desired assembled state. The strategy is further demonstrated, with both simulation and experimental results, for two applications: control of colloidal assembly (to produce perfect colloidal crystals) and control of crystallization from solution (to produce crystals of desired average size).

Accounting for spatial dependency in multivariate spectroscopic data

We examine a hybrid multivariate regression technique to account for the spatial dependency in spectroscopic data due to adjacent measurement locations in the same joint by combining dimension reduction methods and linear mixed effects (LME) modeling. Spatial correlation is a common limitation (assumption of independence) encountered in diagnostic applications involving adjacent measurement locations, such as mapping of tissue properties, and can impede tissue evaluations. Near-infrared spectra were collected from equine joints (n = 5) and corresponding biomechanical (n = 202), compositional (n = 530), and structural (n = 530) properties of cartilage tissue were measured. Subsequently, hybrid regression models for estimating tissue properties from the spectral data were developed in combination with principal component analysis (PCA-LME) scores and least absolute shrinkage and selection operator (LASSO-LME). Performance comparison of PCA-LME and principal component regression, and LASSO-LME and LASSO regression was conducted to evaluate the effects of spatial dependency. A systematic improvement in calibration models’ correlation coefficients and a decrease in cross validation errors were observed when accounting for spatial dependency. Our results indicate that accounting for spatial dependency using a LME-based approach leads to more accurate prediction models.

Assessing Satisfaction with Public Transport Service by Ordered Multiple Correspondence Analysis

This paper provides a composite indicator for comparing the perceived service of satisfaction of public transport by residents of a southern Italian city across three time periods spanning 2008–2012. Data were collected from 400 respondents that rated their agreement with 15 attribute-related statements regarding local public transport services. This study identifies passenger satisfaction in terms of the various quality aspects of public transport services using features of ordered multiple correspondence analysis. Such a method combines dimension reduction and cluster analysis for categorical data by objectively assigning individuals to clusters and identifying optimal scaling values to each of the categories. The main findings of our study indicate that there are differences in how public transport is perceived during the period of time studied.

Biomaker patterns of fatty acids and other fat-soluble biocompounds in blood to indicate nuts intake in the European Prospective Investigation into Cancer and Nutrition (EPIC) study

Summary Introduction Nuts are nutrient dense foods, rich in unsaturated fatty acids and other fat-soluble biocompounds. Epidemiological and intervention studies have shown that a high intake of nuts is associated with a reduced risk of coronary heart disease; however, inaccurate estimation of nut intake using self-reported methods might limit detection of other health benefits such as reduced risk of cancer, where potential strengths of associations are likely smaller. This study aimed to identify biomarker patterns including fatty acids (FAs) and other fat-soluble compounds (carotenoids and tocopherols) in blood to indicate nut intake by applying a statistical algorithm combining dimension reduction and variable selection methods. Methods The study included 2324 subjects (44% men) aged 35–70 years from 8 European countries in EPIC cross-sectional study. Recent and habitual nut intakes were assessed with 24-h dietary recalls (24-HDR) and dietary questionnaire (DQ), respectively. Potential biomarkers related to nut intake included 22 plasma FAs and serum fat-soluble compounds (7 carotenoids and 2 tocopherols). They were measured by gas chromatography and high-performance liquid chromatography, respectively. In order to identify patterns of these biomarkers, maximizing the explained variability in nut intake, reduced rank regression (RRR) models were used with optimal subsets of biomarkers selected by two different variable selection methods; RRR-based variable importance in projection (RRR-VIP) and least absolute shrinkage and selection operator (LASSO). Prior to the main analysis, nut intake and biomarker levels were log-transformed and adjusted for energy intake (nut intake only) and country taking residuals in linear models. The performance of RRR models was evaluated by Pearson correlation coefficients of biomarker pattern scores with recent and habitual nut intakes through internal two-fold cross-validation. Results Higher performance was observed in biomarker patterns on habitual nut intake reported in the DQ. The selected subsets of biomarkers slightly differed depending on which selection method was used, but arachidic acid, linoleic acid and tocopherols, known as abundant compounds in nuts, were commonly selected as biomarkers to explain habitual nut intake in both methods. Pearson correlation coefficients of biomarker pattern scores with habitual nut intake (r = 0.27) were two to three times stronger than those of single biomarkers (linoleic acid r = 0.08; arachidic acid r = 0.11). Neither biomarker patterns nor single biomarkers performed well in explaining recent nut intake reported in the 24-HDR. Conclusions Biomarker patterns consisting of fatty acids or fat-soluble biocompounds better indicated nut intakes compared to any of these single biomarkers evaluated, especially for habitual nut intake. Whether these identified biomarker patterns improve assessment of nut intake in diet-disease studies needs to be evaluated in future studies.

A learning-based data-driven forecast approach for predicting future reservoir performance

Quantification of the predictive uncertainty of subsurface models has long been investigated. The traditional workflow is to calibrate prior models to match observed data, and then use the posterior models to simulate future system performance. Not only are these procedures computationally expensive, but they also have issues in maintaining geological model constraints during the calibration step. Data space inversion (DSI) was introduced recently to predict future system performance without the iterative history matching or model calibration step. In general, DSI approaches seek to establish a statistical relationship between the observed and forecast variables, as well as to quantify the predictive uncertainty of the forecast variables, by using an ensemble of uncalibrated prior models. Existing DSI approaches all require a number of complex transformation and mapping operations, which may deter their widespread use. In this study, we introduce a new and simpler DSI approach, the learning-based, data-driven forecast approach (LDFA), by combining dimension reduction and machine learning techniques to quickly provide accurate forecast results and reliably quantify corresponding uncertainty in the results. Our LDFA framework is demonstrated using two supervised learning algorithms, artificial neural network (ANN) and support vector regression (SVR), on two representative examples from reservoir engineering and geological carbon storage. Results suggest that our approach provides accurate forecast results (e.g., future oil production rate or cumulative injected CO2) and reasonable predictive uncertainty intervals. Our framework is generic and may be applied to other surface and subsurface problems.

Dimension Reduction of Multi-Spectral Satellite Image Time Series to Improve Deforestation Monitoring

In recent years, sequential tests for detecting structural changes in time series have been adapted for deforestation monitoring using satellite data. The input time series of such sequential tests is typically a vegetation index (e.g., NDVI), which uses two or three bands and ignores all other bands. Being limited to a vegetation index will not benefit from the richer spectral information provided by newly launched satellites and will bring two bottle-necks for deforestation monitoring. Firstly, it is hard to select a suitable vegetation index a priori. Secondly, a single vegetation index is typically affected by seasonal signals, noise and other natural dynamics, which decrease its power for deforestation detection. A novel multispectral time series change monitoring method that combines dimension reduction methods with a sequential hypothesis test is proposed to address these limitations. For each location, the proposed method automatically chooses a “suitable” index for deforestation monitoring. To demonstrate our approach, we implemented it in two study areas: a dry tropical forest in Bolivia (time series length: 444) with strong seasonality and a moist tropical forest in Brazil (time series length: 225) with almost no seasonality. Our method significantly improves accuracy in the presence of strong seasonality, in particular the temporal lag between disturbance and its detection.

Methods for Real-Time Prediction of the Mode of Travel Using Smartphone-Based GPS and Accelerometer Data.

We propose and compare combinations of several methods for classifying transportation activity data from smartphone GPS and accelerometer sensors. We have two main objectives. First, we aim to classify our data as accurately as possible. Second, we aim to reduce the dimensionality of the data as much as possible in order to reduce the computational burden of the classification. We combine dimension reduction and classification algorithms and compare them with a metric that balances accuracy and dimensionality. In doing so, we develop a classification algorithm that accurately classifies five different modes of transportation (i.e., walking, biking, car, bus and rail) while being computationally simple enough to run on a typical smartphone. Further, we use data that required no behavioral changes from the smartphone users to collect. Our best classification model uses the random forest algorithm to achieve 96.8% accuracy.

Towards a Systematic Combination of Dimension Reduction and Clustering in Visual Analytics.

Dimension reduction algorithms and clustering algorithms are both frequently used techniques in visual analytics. Both families of algorithms assist analysts in performing related tasks regarding the similarity of observations and finding groups in datasets. Though initially used independently, recent works have incorporated algorithms from each family into the same visualization systems. However, these algorithmic combinations are often ad hoc or disconnected, working independently and in parallel rather than integrating some degree of interdependence. A number of design decisions must be addressed when employing dimension reduction and clustering algorithms concurrently in a visualization system, including the selection of each algorithm, the order in which they are processed, and how to present and interact with the resulting projection. This paper contributes an overview of combining dimension reduction and clustering into a visualization system, discussing the challenges inherent in developing a visualization system that makes use of both families of algorithms.

Feature selection and extraction for class prediction in dysphonia measures analysis:A case study on Parkinson's disease speech rehabilitation.

Speech disorders such as dysphonia and dysarthria represent an early and common manifestation of Parkinson's disease. Class prediction is an essential task in automatic speech treatment, particularly in the Parkinson's disease case. Many classification experiments have been performed which focus on the automatic detection of Parkinson's disease patients from healthy speakers but results are still not optimistic. A major problem in accomplishing this task is high dimensionality of speech data. In this work, the potential of Principal Component Analysis (PCA) based modeling in dimensionality reduction is taken into consideration as the data smoothening tool with multiclass target expression data. On the basis of suggested PCA-based modeling, the power of class prediction using logistic regression (LR) and C5.0in numeric data is investigated in publicly available Parkinson's disease dataset Silverman voice treatment (LSVT) to develop an advanced classification model. The main advantage of our model is the effective reduction of the number of factors from p= 309 to k= 32 for LSVT Voice Rehabilitation dataset, with a fine classification accuracy of 100% and 99.92% for PCA-LR and PCA-C5.0 respectively. In addition, using only 9 dysphonia features, classification accuracy was (99.20%) and (99.11%) for PCA-LR, and PCA-C5.0 respectively. Our combined dimension reduction and data smoothening approaches have significant potential to minimize the number of features and increase the classification accuracy and then automatically classify subjects into Parkinson's disease patients or healthy speakers.

System Diagnosis of Coronary Heart Disease using A Combination of Dimensional Reduction and Data Mining Techniques: A Review

Coronary heart disease is a disease with the highest mortality rates in the world. This makes the development of the diagnostic system as a very interesting topic in the field of biomedical informatics, aiming to detect whether a heart is normal or not. In the literature there are diagnostic system models by combining dimension reduction and data mining techniques. Unfortunately, there are no review papers that discuss and analyze the themes to date. This study reviews articles within the period 2009-2016, with a focus on dimension reduction methods and data mining techniques, validated using a dataset of UCI repository. Methods of dimension reduction use feature selection and feature extraction techniques, while data mining techniques include classification, prediction, clustering, and association rules.

Identification of Urinary Polyphenol Metabolite Patterns Associated with Polyphenol-Rich Food Intake in Adults from Four European Countries.

We identified urinary polyphenol metabolite patterns by a novel algorithm that combines dimension reduction and variable selection methods to explain polyphenol-rich food intake, and compared their respective performance with that of single biomarkers in the European Prospective Investigation into Cancer and Nutrition (EPIC) study. The study included 475 adults from four European countries (Germany, France, Italy, and Greece). Dietary intakes were assessed with 24-h dietary recalls (24-HDR) and dietary questionnaires (DQ). Thirty-four polyphenols were measured by ultra-performance liquid chromatography–electrospray ionization-tandem mass spectrometry (UPLC-ESI-MS-MS) in 24-h urine. Reduced rank regression-based variable importance in projection (RRR-VIP) and least absolute shrinkage and selection operator (LASSO) methods were used to select polyphenol metabolites. Reduced rank regression (RRR) was then used to identify patterns in these metabolites, maximizing the explained variability in intake of pre-selected polyphenol-rich foods. The performance of RRR models was evaluated using internal cross-validation to control for over-optimistic findings from over-fitting. High performance was observed for explaining recent intake (24-HDR) of red wine (r = 0.65; AUC = 89.1%), coffee (r = 0.51; AUC = 89.1%), and olives (r = 0.35; AUC = 82.2%). These metabolite patterns performed better or equally well compared to single polyphenol biomarkers. Neither metabolite patterns nor single biomarkers performed well in explaining habitual intake (as reported in the DQ) of polyphenol-rich foods. This proposed strategy of biomarker pattern identification has the potential of expanding the currently still limited list of available dietary intake biomarkers.

Combining Dimension Reduction, Distance Measures and Covariance (Presentation Slides)

We consider a measure of similarity, the Bhattacharyya distance, across distributions of random variables. We develop a novel methodology based on the marriage between the Bhattacharyya distance and the Johnson Lindenstrauss Lemma, a technique for dimension reduction, providing us with a simple yet powerful tool that allows comparisons between data-sets representing any two distributions. We demonstrate a relationship between covariance and distance measures based on a generic extension of Stein’s Lemma. The degree to which different markets or sub groups of securities have different measures of their corresponding distributions tells us the extent to which they are different. This can aid investors looking for diversification or looking for more of the same thing. We consider an asset pricing application and then briefly discuss how this methodology lends itself to numerous Marketstructure studies and even applications outside the realm of finance / social sciences by illustrating a biological application.

Cluster Correspondence Analysis.

A method is proposed that combines dimension reduction and cluster analysis for categorical data by simultaneously assigning individuals to clusters and optimal scaling values to categories in such a way that a single between variance maximization objective is achieved. In a unified framework, a brief review of alternative methods is provided and we show that the proposed method is equivalent to GROUPALS applied to categorical data. Performance of the methods is appraised by means of a simulation study. The results of the joint dimension reduction and clustering methods are compared with the so-called tandem approach, a sequential analysis of dimension reduction followed by cluster analysis. The tandem approach is conjectured to perform worse when variables are added that are unrelated to the cluster structure. Our simulation study confirms this conjecture. Moreover, the results of the simulation study indicate that the proposed method also consistently outperforms alternative joint dimension reduction and clustering methods.

Fast Robot Localization Approach Based on Manifold Regularization with Sparse Area Features

Robot localization can be considered as a cognition process that takes place during a robot estimating metric coordinates with vision. It provides a natural method for revealing the true autonomy of robots. In this paper, a kernel principal component analysis (PCA)-regularized least-square algorithm for robot localization with uncalibrated monocular visual information is presented. Our system is the first to use a manifold regularization strategy in robot localization, which achieves real-time localization using a harmonic function. The core idea is to incorporate labelled and unlabelled observation data in offline training to generate a regression model smoothed by the intrinsic manifold embedded in area feature vectors. The harmonic function is employed to solve the online localization of new observations. Our key contributions include semi-supervised learning techniques for robot localization, the discovery and use of the visual manifold learned by kernel PCA and some solutions for simultaneous parameter selection. This simultaneous localization and mapping (SLAM) system combines dimension reduction methods, manifold regularization techniques and parameter selection to provide a paradigm of SLAM having self-contained theoretical foundations. In extensive experiments, we evaluate the localization errors from the perspective of reducing implementation and application difficulties in feature selection and magnitude ratio determination of labelled and unlabelled data. Then, a nonlinear optimization algorithm is adopted for simultaneous parameter selection. Our online localization algorithm outperformed the state-of-the-art appearance-based SLAM algorithms at a processing rate of 30 Hz for new data on a standard PC with a camera.

Locally linear embedding and neighborhood rough set-based gene selection for gene expression data classification.

Cancer subtype recognition and feature selection are important problems in the diagnosis and treatment of tumors. Here, we propose a novel gene selection approach applied to gene expression data classification. First, two classical feature reduction methods including locally linear embedding (LLE) and rough set (RS) are summarized. The advantages and disadvantages of these algorithms were analyzed and an optimized model for tumor gene selection was developed based on LLE and neighborhood RS (NRS). Bhattacharyya distance was introduced to delete irrelevant genes, pair-wise redundant analysis was performed to remove strongly correlated genes, and the wavelet soft threshold was determined to eliminate noise in the gene datasets. Next, prior optimized search processing was carried out. A new approach combining dimension reduction of LLE and feature reduction of NRS (LLE-NRS) was developed for selecting gene subsets, and then an open source software Weka was applied to distinguish different tumor types and verify the cross-validation classification accuracy of our proposed method. The experimental results demonstrated that the classification performance of the proposed LLE-NRS for selecting gene subset outperforms those of other related models in terms of accuracy, and our proposed approach is feasible and effective in the field of high-dimensional tumor classification.

Application of dimensionality reduction technique to improve geophysical log data classification performance in crystalline rocks

Abstract In this study, dimensionality reduction technique to improve geophysical log data classification performance in crystalline rocks is presented. In fact, in complex geological situations such as the study area in context, more complex nonlinear functional behaviors exist for well log classification purpose; thus posing challenges in accurate identification of log curves for this purpose. Dimensionality reduction (DR) using Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA) were used here to reduce the dimensionality of the original log set of Chinese Continental Scientific Drilling Main Hole to a convenient size, and then feed the reduced-log set into the classifiers. Three classifiers were addressed, namely, Support vector Machines, Feed forward Back Propagation and Radial Basis Function Neural Networks in the classification of metamorphic rocks. The strategy of combining dimensionality reduction methods and classifiers was demonstrated and discussed. The results showed that the reduced log sets found from DR can separate the metamorphic rocks types better or almost as well as the original log set. Therefore LDA and PCA can be suitable to be performed before geophysical well log data classification in the context of crystalline rocks.

Effects of combined dimension reduction and tabulation on the simulations of a turbulent premixed flame using a large-eddy simulation/probability density function method

A turbulent lean-premixed propane–air flame stabilised by a triangular cylinder as a flame-holder is simulated to assess the accuracy and computational efficiency of combined dimension reduction and tabulation of chemistry. The computational condition matches the Volvo rig experiments. For the reactive simulation, the Lagrangian Large-Eddy Simulation/Probability Density Function (LES/PDF) formulation is used. A novel two-way coupling approach between LES and PDF is applied to obtain resolved density to reduce its statistical fluctuations. Composition mixing is evaluated by the modified Interaction-by-Exchange with the Mean (IEM) model. A baseline case uses In Situ Adaptive Tabulation (ISAT) to calculate chemical reactions efficiently. Its results demonstrate good agreement with the experimental measurements in turbulence statistics, temperature, and minor species mass fractions. For dimension reduction, 11 and 16 represented species are chosen and a variant of Rate Controlled Constrained Equilibrium (RCCE) is applied in conjunction with ISAT to each case. All the quantities in the comparison are indistinguishable from the baseline results using ISAT only. The combined use of RCCE/ISAT reduces the computational time for chemical reaction by more than 50%. However, for the current turbulent premixed flame, chemical reaction takes only a minor portion of the overall computational cost, in contrast to non-premixed flame simulations using LES/PDF, presumably due to the restricted manifold of purely premixed flame in the composition space. Instead, composition mixing is the major contributor to cost reduction since the mean-drift term, which is computationally expensive, is computed for the reduced representation. Overall, a reduction of more than 15% in the computational cost is obtained.