Supervised Dimensionality Reduction Method Research Articles

Country-level energy-related uncertainties and stock market returns: Insights from the U.S. and China

Scholars have widely acknowledged the influence of energy on stock market returns, yet the role of energy-related uncertainty in the stock market remains to be explored. This study employs scaled principal component analysis (sPCA), partial least squares (PLS), and principal component analysis (PCA) techniques to investigate the impact of country-level energy-related uncertainty indices (EUIs) on stock market returns. The results indicate that an EUI contains prediction information for U.S. and Chinese stock market returns. Furthermore, the diffusion index derived from all EUIs utilizing supervised dimensionality reduction methods (sPCA and PLS) facilitates predicting stock market returns better and more robustly than country-level EUIs and traditional macroeconomic predictors. The results of the economic value assessment also indicate that models considering the EUI diffusion index tend to yield greater economic value than models considering individual EUIs or traditional predictive factors. Our results emphasize the significance of the EUI diffusion index in predicting stock market returns, providing valuable guidance for portfolio management and decision-making for market stakeholders.

Toward Robust Discriminative Projections Learning Against Adversarial Patch Attacks.

As one of the most popular supervised dimensionality reduction methods, linear discriminant analysis (LDA) has been widely studied in machine learning community and applied to many scientific applications. Traditional LDA minimizes the ratio of squared l2 norms, which is vulnerable to the adversarial examples. In recent studies, many l1 -norm-based robust dimensionality reduction methods are proposed to improve the robustness of model. However, due to the difficulty of l1 -norm ratio optimization and weakness on defending a large number of adversarial examples, so far, scarce works have been proposed to utilize sparsity-inducing norms for LDA objective. In this article, we propose a novel robust discriminative projections learning (rDPL) method based on the l1,2 -norm trace-ratio minimization optimization algorithm. Minimizing the l1,2 -norm ratio problem directly is a much more challenging problem than the traditional methods, and there is no existing optimization algorithm to solve such nonsmooth terms ratio problem. We derive a new efficient algorithm to solve this challenging problem and provide a theoretical analysis on the convergence of our algorithm. The proposed algorithm is easy to implement and converges fast in practice. Extensive experiments on both synthetic data and several real benchmark datasets show the effectiveness of the proposed method on defending the adversarial patch attack by comparison with many state-of-the-art robust dimensionality reduction methods.

Global subclass discriminant analysis

Linear discriminant analysis (LDA) is a powerful supervised dimensionality reduction method for analysing high-dimensional data. However, LDA cannot use locality information in data, which makes LDA degrade dramatically in performance on multimodal data. A number of LDA variants have been proposed to exploit locality information in data, including subclass-based LDAs. We discover a problem with these variants, which is that subclasses are selected on a within-class basis without considering other classes. This causes the loss of important information at class boundaries. In this paper, we present a novel variant of subclass-based LDA, Global Subclass Discriminant Analysis (GSDA). Unlike other subclass-based LDAs, GSDA selects subclasses from global clusters that may cross class boundaries, thus utilising within-class information and between-class information. More specifically, GSDA applies an effective clustering algorithm to the whole data to construct global clusters. It then utilises the local structure refining strategy on these global clusters to construct subclasses. Finally, GSDA learns a representative data subspace by maximising inter-subclass distance and minimising intra-subclass distance simultaneously. GSDA is extensively evaluated on a wide range of public datasets through comparison with the state-of-the-art LDA algorithms. Experimental results demonstrate its superiority in terms of accuracy and run times.

Fisher's Linear Discriminant Analysis With Space-Folding Operations.

Fisher's linear discriminant analysis (LDA) is an easy-to-use supervised dimensionality reduction method. However, LDA may be ineffective against complicated class distributions. It is well-known that deep feedforward neural networks with rectified linear units as activation functions can map many input neighborhoods to similar outputs by a succession of space-folding operations. This short paper shows that the space-folding operation can reveal to LDA classification information in the subspace where LDA cannot find any. A composition of LDA with the space-folding operation can find classification information more than LDA can do. End-to-end fine-tuning can improve that composition further. Experimental results on artificial and open data sets have shown the feasibility of the proposed approach.

Morphological relationships between vertebrate claw unguals and sheaths and the functional morphology of these structures.

The link between claw morphology and function has been historically difficult to quantify, analyze, and interpret. A confounding factor is the ambiguous morphological relationship between the ungual and the sheath and whether one structure or the other is more useful for inferring function from morphology. In this study, the functional morphology of vertebrate claws is analyzed using sheath and ungual measurements taken from modern claw specimens spanning birds and mammals. Claw measurements were chosen for their potential biomechanical significance and a revised, expanded categorization of claw function is used. When corresponding claw measurements from the ungual and sheath are compared independently, some features are highly correlated whereas others are not. A principal component analysis of the claw measurements reveals that some of the morphological disparity is related to functional differences; however, different functional categories are not clearly separated based solely on morphology. A linear discriminant analysis incorporating a supervised dimensionality reduction method (J-function) successfully classifies 94.52% of the claw specimens to their documented functional categories. When the posterior probabilities of each classification are examined, and the next highest probabilities are considered, the analysis can successfully classify 98.63% of the claw specimens. Sheath measurements perform better than ungual measurements but combining measurements from both structures perform better than considering either structure individually. Both structures contribute valuable morphological information when it comes to inferring claw function from morphology.

Discriminative Projection Learning With Adaptive Reversed Graph Embedding for Supervised and Semi-Supervised Dimensionality Reduction

The graph embedding based dimensionality reduction (DR) methods have been widely used to overcome the curse of dimensionality problem in high-dimensional data. Especially in pattern recognition and data analysis, they are frequently used to extract features. However, there are two problems in the graph embedding based supervised DR methods: 1) the original data will inevitably be corrupted by noise or outliers, which makes the fixed graph constructed from these data unreliable; and 2) determining how to model the projection for enhancing the discrimination of projected samples. To overcome the above two challenges, this paper proposes a new supervised DR method called discriminative projection learning with adaptive reversed graph embedding (DP-ARGE). Specifically, DP-ARGE learns the graph dynamically in a low-dimensional subspace by using the low-dimensional embedding rather than projected samples for improving its robustness. In addition, a novel discriminative regularization term is designed to enhance the discrimination of the projection samples by using a few pairwise marginal heterogeneous samples rather than the all. To tackle the problem of estimating the neighborhood size parameter of the graph, a simple yet effective automatic parameter estimation strategy based on the density and similarity of data set is proposed. Furthermore, for better handling the data set with a few labels, DP-ARGE is extend to a semi-supervised method (SDP-ARGE) by a neighborhood preserving regularization term. SDP-ARGE can be used to preserve the local manifold structure of the data in the semi-supervised scenario. Comprehensive experiments show the superiority of the proposed methods in several real-world data sets.

Linear discriminant analysis with generalized kernel constraint for robust image classification

Linear discriminant analysis (LDA) as a classical supervised dimensionality reduction method has shown powerful capability in various image classification tasks. The purpose of LDA seeks an optimal linear transformation that maps the original data to a low-dimensional space. Inspired by the fact that the kernel trick can capture the nonlinear similarity of features, we propose a novel generalized distance constraint dubbed intra-class and inter-class kernel constraint (IIKC). The proposed IIKC explicitly models the category kernel distance and focuses on helping the original LDA capture more discriminant features in order to further improve the separability and magnitude difference between nearby data points. Our proposed method with IIKC aims to achieve maximum category separability by minimizing the intra-class kernel distances as well as maximizing the inter-class kernel distance, simultaneously. Extensive experimental results on six publicly available benchmark databases illustrate that the LDA-based methods embedded with the proposed IIKC significantly improve the discrimination ability and achieve a better classification performance than the original and state-of-the-art LDA algorithms.

A Funnel Fukunaga–Koontz Transform for Robust Indoor–Outdoor Detection Using Channel-State Information in 5G IoT Context

The massive machine-type communication will be at the core of ambient connectivity, requiring for energy-efficient systems. Earlier studies highlighted the efficiency of positioning approaches based on channel-state information (CSI) in different environments. Many works limited the solution assessment to a single room in a fully indoor testbed. This article extends the application of CSI for indoor–outdoor detection on an unprecedented large area and considers mMTC-oriented long-term evolution and fifth-generation Internet of Things in the sub-GHz frequency band. Hinged on a novel long-term evolution protocol dedicated for machine-type communications, the results focus on a unique packet exchange with a single access point to save battery life and simplify deployment. The study evaluates different input features and investigates the target positioning accuracy for multiple unsupervised and supervised dimensionality reduction methods. We present a new dimension reduction scheme consisting of an unsupervised funnel on top of a supervised dimension reduction approach. Results show that the introduced Funnel Fukunaga–Koontz transform outperforms other dimension reduction approaches, regardless of the input features and the number of locations.

Data science through the lens of systems immunology

Glass, a post-doctoral researcher, and Amouzgar, a PhD student, in Bendall lab proposed a supervised dimensionality reduction method to explore and analyze single-cell data. Their Patterns paper highlights the advantages of supervised learning in single-cell datasets with class labels. They talk about the essential role of data science in this project and in their lives.

A class-driven approach to dimension embedding

The preservation of the inbuilt structures of data sets and the more decomposition of the classes are a significant interest in dimension embedding. In this respect, the dimensionality reduction methods use novel techniques to better ascertain the fundamental structure of the manifold on which the data lies. However, both conventional and state-of-art supervised dimensionality reduction methods cannot benefit from class information good enough. Therefore, their generalization performances on the test data are weak. A new non-linear supervised algorithm, which we call Class-driven Dimension Embedding (CDE), is proposed for utilizing class information. CDE performs three outstanding characteristics: (i) preserving the intrinsic relationship between the data points and classes; (ii) producing wide margins between classes; (iii) enhancing the generalization performance on the test data. The proposed method embeds a d-dimensional data set into the c-dimensional space (c designates the number of classes) through the corresponding values to classes of each point by exploiting a neighborhood graph and a feature weighting function. The experimental results on forty-eight data sets demonstrate that CDE is comparable to or better than twenty-four dimensionality reduction algorithms in terms of classification accuracy and visualization. The source code of CDE can be found in https://doi.org/10.24433/CO.0967299.v1 for computational reproducibility.

Improving the regional Y-STR haplotype resolution utilizing haplogroup-determining Y-SNPs and the application of machine learning in Y-SNP haplogroup prediction in a forensic Y-STR database: A pilot study on male Chinese Yunnan Zhaoyang Han population

Improving the resolution of the current widely used Y-chromosomal short tandem repeat (Y-STR) dataset is of great importance for forensic investigators, and the current approach is limited, except for the addition of more Y-STR loci. In this research, a regional Y-DNA database was investigated to improve the Y-STR haplotype resolution utilizing a Y-SNP Pedigree Tagging System that includes 24 Y-chromosomal single nucleotide polymorphism (Y-SNP) loci. This pilot study was conducted in the Chinese Yunnan Zhaoyang Han population, and 3473 unrelated male individuals were enrolled. Based on data on the male haplogroups under different panels, the matched or near-matching (NM) Y-STR haplotype pairs from different haplogroups indicated the critical roles of haplogroups in improving the regional Y-STR haplotype resolution. A classic median-joining network analysis was performed using Y-STR or Y-STR/Y-SNP data to reconstruct population substructures, which revealed the ability of Y-SNPs to correct misclassifications from Y-STRs. Additionally, population substructures were reconstructed using multiple unsupervised or supervised dimensionality reduction methods, which indicated the potential of Y-STR haplotypes in predicting Y-SNP haplogroups. Haplogroup prediction models were built based on nine publicly accessible machine-learning (ML) approaches. The results showed that the best prediction accuracy score could reach 99.71% for major haplogroups and 98.54% for detailed haplogroups. Potential influences on prediction accuracy were assessed by adjusting the Y-STR locus numbers, selecting Y-STR loci with various mutabilities, and performing data processing. ML-based predictors generally presented a better prediction accuracy than two available predictors (Nevgen and EA-YPredictor). Three tree models were developed based on the Yfiler Plus panel with unprocessed input data, which showed their strong generalization ability in classifying various Chinese Han subgroups (validation dataset). In conclusion, this study revealed the significance and application prospects of Y-SNP haplogroups in improving regional Y-STR databases. Y-SNP haplogroups can be used to discriminate NM Y-STR haplotype pairs, and it is important for forensic Y-STR databases to develop haplogroup prediction tools to improve the accuracy of biogeographic ancestry inferences.

Least squares regression principal component analysis: A supervised dimensionality reduction method

AbstractDimensionality reduction is an important technique in surrogate modeling and machine learning. In this article, we propose a supervised dimensionality reduction method, “least squares regression principal component analysis” (LSR‐PCA), applicable to both classification and regression problems. To show the efficacy of this method, we present different examples in visualization, classification, and regression problems, comparing it with several state‐of‐the‐art dimensionality reduction methods. Finally, we present a kernel version of LSR‐PCA for problems where the inputs are correlated nonlinearly. The examples demonstrate that LSR‐PCA can be a competitive dimensionality reduction method.

Data-driven fault diagnosis for wind turbines using modified multiscale fluctuation dispersion entropy and cosine pairwise-constrained supervised manifold mapping

Condition monitoring and fault diagnosis of wind turbines is an attractive yet challenging task. This paper presents a novel data-driven fault diagnosis scheme for wind turbines, using refined time-shift multiscale fluctuation-based dispersion entropy (RTSMFDE) and cosine pairwise-constrained supervised manifold mapping (CPCSMM). The developed RTSMFDE can improve multiscale dispersion entropy in complexity measurement of time series, by employing the refined time-shift analysis approach. Moreover, the proposed CPCSMM combines cosine distance metric, supervised theory and sparse global manifold structure, to obtain the most important features, resulting in a low dimensional representation. In this fault diagnosis scheme, a high-dimensional fault feature set is first constructed using RTSMFDE. Then, sensitive features are extracted using CPCSMM. Finally, the low-dimensional feature set is fed to a beetle antennae search-based support vector machine for fault identification. Simulation experiments and wind turbine fault diagnosis experiments were used, to show that the proposed RTSMFDE can comprehensively mine the wind turbine features and outperforms the existing state-of-the-art entropy methods. The visualization results and classification effects of the proposed CPCSMM approach are better than existing unsupervised and supervised dimensionality reduction methods. The proposed data-driven fault diagnosis method can effectively and accurately identify multi-faults and outperforms existing fault diagnosis techniques.

Robust two-dimensional capped [formula omitted]-norm linear discriminant analysis with regularization and its applications on image recognition

Two-dimensional linear discriminant analysis (2DLDA) is an effective matrix-based supervised dimensionality reduction method that expresses 2D data directly. However, 2DLDA magnifies the influence of outliers and noise since the construction of 2DLDA is based on squared Frobenius norm. To overcome its sensitivity, this paper investigates a two-dimensional capped l2,1-norm linear discriminant analysis, called 2DCLDA. The application of capped l2,1-norm makes 2DCLDA identify outliers and suppress the effect of noise effectively. To avoid singularity and give a stable performance, a regularization term is also considered in the between-class scatter. 2DCLDA is solved through a series of generalized eigenvalue problems, with each subproblem can be deemed as a weighted 2DLDA with regularization. It is proved that the proposed iteration algorithm monotonously decreases the objective of 2DCLDA. At last, 2DCLDA with its related approaches on several face image databases is compared, and the experimental results demonstrate the superiority of 2DCLDA, especially for noise data.

Supervised dimensionality reduction for big data

To solve key biomedical problems, experimentalists now routinely measure millions or billions of features (dimensions) per sample, with the hope that data science techniques will be able to build accurate data-driven inferences. Because sample sizes are typically orders of magnitude smaller than the dimensionality of these data, valid inferences require finding a low-dimensional representation that preserves the discriminating information (e.g., whether the individual suffers from a particular disease). There is a lack of interpretable supervised dimensionality reduction methods that scale to millions of dimensions with strong statistical theoretical guarantees. We introduce an approach to extending principal components analysis by incorporating class-conditional moment estimates into the low-dimensional projection. The simplest version, Linear Optimal Low-rank projection, incorporates the class-conditional means. We prove, and substantiate with both synthetic and real data benchmarks, that Linear Optimal Low-Rank Projection and its generalizations lead to improved data representations for subsequent classification, while maintaining computational efficiency and scalability. Using multiple brain imaging datasets consisting of more than 150 million features, and several genomics datasets with more than 500,000 features, Linear Optimal Low-Rank Projection outperforms other scalable linear dimensionality reduction techniques in terms of accuracy, while only requiring a few minutes on a standard desktop computer.

Towards Robust Discriminative Projections Learning via Non-Greedy l2,1-Norm MinMax.

Linear Discriminant Analysis (LDA) is one of the most successful supervised dimensionality reduction methods and has been widely used in many real-world applications. However, l2-norm is employed as the distance metric in the objective of LDA, which is sensitive to outliers. Many previous works improve the robustness of LDA by using l1-norm distance. However, the robustness against outliers is limited and the solver of l1-norm is mostly based on the greedy search strategy, which is time-consuming and easy to get stuck in a local optimum. In this paper, we propose a novel robust LDA measured by l2,1-norm to learn robust discriminative projections. The proposed model is challenging to solve since it needs to minimize and maximize (minmax) l2,1-norm terms simultaneously. As a result, we first systematically derive an efficient iterative optimization algorithm to solve a general ratio minimization problem, and then rigorously prove its convergence. More importantly, an alternately non-greedy iterative re-weighted optimization algorithm is developed based on the preceding approach for solving proposed l2,1-norm minmax problem. Besides, an optimal weighted mean mechanism is driven according to the designed objective and solver, which can be applied to other approaches for robustness improvement. Experimental results on several real-world datasets show the effectiveness of proposed method.

Locality Adaptive Discriminant Analysis Framework.

Linear discriminant analysis (LDA) is a well-known technique for supervised dimensionality reduction and has been extensively applied in many real-world applications. LDA assumes that the samples are Gaussian distributed, and the local data distribution is consistent with the global distribution. However, real-world data seldom satisfy this assumption. To handle the data with complex distributions, some methods emphasize the local geometrical structure and perform discriminant analysis between neighbors. But the neighboring relationship tends to be affected by the noise in the input space. In this research, we propose a new supervised dimensionality reduction method, namely, locality adaptive discriminant analysis (LADA). In order to directly process the data with matrix representation, such as images, the 2-D LADA (2DLADA) is also developed. The proposed methods have the following salient properties: 1) they find the principle projection directions without imposing any assumption on the data distribution; 2) they explore the data relationship in the desired subspace, which contains less noise; and 3) they find the local data relationship automatically without the efforts for tuning parameters. The performance of dimensionality reduction shows the superiorities of the proposed methods over the state of the art.

Interactive visual clustering and classification based on dimensionality reduction mappings: A case study for analyzing patients with dermatologic conditions

Multidimensional data sets are becoming more frequent in practically all research fields, and require complex data analysis techniques in order to extract knowledge from them. In this paper, we propose an interactive visualization tool for performing exploratory data analysis. The tool combines unsupervised and supervised dimensionality reduction methods, such as linear discriminant analysis, or t-SNE, with clustering and classification techniques. Analysts can use several machine learning methods for extracting data structure, and can group data into clusters interactively or through clustering algorithms. In addition they can visualize projections of the data to evaluate the quality of obtained clusters, and to analyze the performance of classification methods. We have applied this tool to analyze a clinical data set related to patients with dermatologic conditions that are under photodynamic therapy. The analysis allowed medical doctors to identify several clinically interesting patient groups. In addition, clinicians discovered a greater efficacy in the treatment of patients with the photosensitizer 5-aminolaevulinic acid nanoemulsion gel compared to those treated with methyl-5-aminolaevulinate cream.

A new formation of supervised dimensionality reduction method for moving vehicle classification

Analyzing a large number of features set for the classification process entails cost and complexity. To reduce this burden, dimensionality reduction has been applied to the extracted set of features as a preprocessing step. Among dimensionality reduction algorithms, many methods fail to handle high-dimensional data and they increase information loss and are sensitive to outliers. Therefore, this research proposes a new supervised dimensionality reduction method developed using an improved formation of linear discriminant analysis with diagonal eigenvalues (LDA-DE) that simultaneously preserves the information and addresses the issues of the classification process. The proposed framework focuses on reducing the dimension of extracted features set by computing the scattered matrices from the class labels and the diagonal eigenvalue matrix. Methods to eliminate duplicate rows and columns, to avoid feature overwriting, and to remove outliers are included in the newly developed LDA-DE method. The new LDA-DE method implemented with a fuzzy random forest classifier is tested on two datasets—MIO-TCD and BIT-Vehicle—to classify the moving vehicles. The performance of our LDA-DE method is compared with five state-of-the-art dimensionality reduction methods. The experimental confusion matrix results show that the LDA-DE method generates the reduced feature vector of the objects to a maximum extent. Further, the newly developed LDA-DE method achieves the best reduction results with optimal performance parameter values (lowest mean and standard deviation and highest f-measure and accuracy) and minimal data processing time than the state-of-the-art methods, promising its application for a fast and effective dimensionality reduction for moving vehicle classification.

Multiclass spectral feature scaling method for dimensionality reduction

Irregular features disrupt the desired classification. In this paper, we consider aggressively modifying scales of features in the original space according to the label information to form well-separated clusters in low-dimensional space. The proposed method exploits spectral clustering to derive scaling factors that are used to modify the features. Specifically, we reformulate the Laplacian eigenproblem of the spectral clustering as an eigenproblem of a linear matrix pencil whose eigenvector has the scaling factors. Numerical experiments show that the proposed method outperforms well-established supervised dimensionality reduction methods for toy problems with more samples than features and real-world problems with more features than samples.