Original Data Space Research Articles

Correntropy-Based Autoencoder-Like NMF With Total Variation for Hyperspectral Unmixing

In order to unmix the hyperspectral imagery (HSI) with better performance, this letter proposes a correntropy-based autoencoder-like nonnegative matrix factorization (NMF) (CANMF) with total variation (CANMF-TV) method. NMF is extensively applied to unmix the mixed pixels. However, it only reconstructs the original data from the abundances in endmember space. To directly project the original data space into the endmember space, and then achieve the abundance matrix, we first exploit an autoencoder-like NMF for hyperspectral unmixing, which integrates both <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">decoder</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">encoder</i> . Considering that HSI is typically degraded by noise, the correntropy-induced metric (CIM) is introduced to construct a CANMF model. In addition, TV regularizer is imposed into the CANMF model so as to preserve the spatial-contextual information by promoting the piecewise smoothness of abundances. Finally, a series of experiments are conducted on both synthetic and real data sets, demonstrating the effectiveness of the proposed CANMF-TV method over comparison.

Rank-Aware Generative Adversarial Network for Hyperspectral Band Selection

Traditional clustering-based band selection (BS) methods treat each band as individuals, and selection is conducted by enlarging the difference between clusters, which leads to the loss of band interaction and information saliency evaluation. In this article, we propose a BS method named rank-aware generative adversarial network (R-GAN) to address these problems. First, centralized reference feature extraction (FE) with GAN aids R-GAN to combine interpretability and interband relevance. Then, the reference feature is refined with the saliency estimation provided by the rank-aware strategy. According to data characteristics, there are two versions of rank computation including tensor and matrix. Finally, the structural similarity index measurement (SSIM) maps the saliency to the original data space to obtain the final BS result. Extensive comparison experiments with popular existing BS approaches on five hyperspectral images (HSIs) datasets show that the proposed R-GAN can address spectral saliency effectively and select more informative band subsets, which outperforms other competitors for both detection and classification tasks. For example, on the SD-1 dataset, the ten bands selected by R-GAN achieve 0.982 ± 0.003 with an improvement of 13.7% in the area under the curve (AUC) value of anomaly detection performance. The peaked accuracy surpasses the baseline by 0.46% for the classification on the PaviaU dataset.

Remote Sensing Cross-Modal Retrieval by Deep Image-Voice Hashing

Remote sensing image retrieval aims at searching remote sensing images of interest among immense volumes of remote sensing data, which is an enormous challenge. Direct use of voice for human-computer interaction is more convenient and intelligent. In this paper, a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Deep Image-Voice Hashing</i> (DIVH) method is proposed for remote sensing image-voice retrieval. First, the whole framework is composed of the image and the voice feature learning subnetwork. Then, the hash code learning procedure will be leveraged in remote sensing image-voice retrieval to further improve retrieval efficiency and reduce memory footprint. Hash code learning maps the deep features of images and voices into a common Hamming space. Finally, image-voice pairwise loss is proposed, which considers the similarity preservation and balance of hash codes. The similarity preserving and the balance controlling term of the loss function improve the similarity preservation from original data space to Hamming space and the discrimination of binary code, respectively. This unified cross-modal feature and hash code learning framework significantly reduce the semantic gap between the two modal data. Experiments demonstrate that the proposed DIVH method can achieve superior retrieval performance than other state-of-the-art remote sensing image-voice retrieval methods.

Symmetric positive definite manifold learning and its application in fault diagnosis

Locally linear embedding (LLE) is an effective tool to extract the significant features from a dataset. However, most of the relevant existing algorithms assume that the original dataset resides on a Euclidean space, unfortunately nearly all the original data space is non-Euclidean. In addition, the original LLE does not use the discriminant information of the dataset, which will degrade its performance in feature extraction. To address these problems raised in the conventional LLE, we first employ the original dataset to construct a symmetric positive definite manifold, and then estimate the tangent space of this manifold. Furthermore, the local and global discriminant information are integrated into the LLE, and the improved LLE is operated in the tangent space to extract the important features. We introduce Iris dataset to analyze the capability of the proposed method to extract features. Finally, several experiments are performed on five machinery datasets, and experimental results indicate that our proposed method can extract the excellent low-dimensional representations of the original dataset. Compared with the state-of-the-art methods, the proposed algorithm shows a strong capability for fault diagnosis.

Non-negative multi-label feature selection with dynamic graph constraints

Feature selection can combat dimension disasters and improve the performance of classification algorithms, so multi-label feature selection is an essential part of multi-label learning and has attracted widespread attention. Many existing multi-label feature selection methods either do not consider the correlation between labels or directly use logical labels to guide the feature selection process, which leads to the loss of label information. This paper proposes a non-negative multi-label feature selection (NMDG) with dynamic graph constraints to address this issue. In the NMDG model, the original data space is projected into a low-dimensional manifold space by linear regression to construct the pseudo label matrix. The pseudo label matrix has the same topological structure as the original data by combining the non-negative constraints and the label graph matrix. Then, the robust low-dimensional space of the pseudo label matrix is used to construct the dynamic graph matrix, which is combined with the feature manifold to guide the learning of the feature weight matrix. Finally, we design an iterative algorithm based on alternating optimization to solve the proposed method and give convergence proof. Experimental results on ten real multi-label data sets compared with seven representative methods show the effectiveness of the proposed method.

Fixing Bias in Reconstruction-based Anomaly Detection with Lipschitz Discriminators

Anomaly detection is of great interest in fields where abnormalities need to be identified and corrected (e.g., medicine and finance). Deep learning methods for this task often rely on autoencoder reconstruction error, sometimes in conjunction with other penalties. We show that this approach exhibits intrinsic biases that lead to undesirable results. Reconstruction-based methods can sometimes show low error on simple-to-reconstruct points that are not part of the training data, for example the all black image. Instead, we introduce a new unsupervised Lipschitz anomaly discriminator (LAD) that does not suffer from these biases. Our anomaly discriminator is trained, similar to the discriminator of a GAN, to detect the difference between the training data and corruptions of the training data. We show that this procedure successfully detects unseen anomalies with guarantees on those that have a certain Wasserstein distance from the data or corrupted training set. These additions allow us to show improved performance on MNIST, CIFAR10, and health record data. Further, LAD does not require decoding back to the original data space, which makes anomaly detection possible in domains where it is difficult to define a decoder, such as in irregular graph structured data. Empirically, we show this framework leads to improved performance on image, health record, and graph data.

Noise Robustness Low-Rank Learning Algorithm for Electroencephalogram Signal Classification.

Electroencephalogram (EEG) is often used in clinical epilepsy treatment to monitor electrical signal changes in the brain of patients with epilepsy. With the development of signal processing and artificial intelligence technology, artificial intelligence classification method plays an important role in the automatic recognition of epilepsy EEG signals. However, traditional classifiers are easily affected by impurities and noise in epileptic EEG signals. To solve this problem, this paper develops a noise robustness low-rank learning (NRLRL) algorithm for EEG signal classification. NRLRL establishes a low-rank subspace to connect the original data space and label space. Making full use of supervision information, it considers the local information preservation of samples to ensure the low-rank representation of within-class compactness and between-classes dispersion. The asymmetric least squares support vector machine (aLS-SVM) is embedded into the objective function of NRLRL. The aLS-SVM finds the maximum quantile distance between the two classes of samples based on the pinball loss function, which further improves the noise robustness of the model. Several classification experiments with different noise intensity are designed on the Bonn data set, and the experiment results verify the effectiveness of the NRLRL algorithm.

Semiconductor Bearing Fault Recognition

Semiconductor manufacturing is consid-ered to be one of the most technologically complicated manufacturing processes. Bearing, being a critical part of the rotating machinery used in the process, plays an essential role as it supports the mechanical rotating body and decreases the friction coefficient. However, extensive use makes this element a target of health degradation, which indirectly causes machine failure. A defective bearing causes approximately 50% of failures in electrical machines. Hence, there arises a dire need for effective fault detection and diagnosis methods to recog-nise fault patterns and help take preventive measures. This paper carries out a comprehensive comparative study of the pre-existing machine learning and deep learning techniques used for diagnosing bearing faults and further devises a novel framework for bearing fault diagnosis based on the results. Unlike the conventional Fault Detection Classifiers (FDC) that operate in the original data space, this algorithm explores the scope for feature extraction and transferability empowered by the deep learning models used.

Gene selection for microarray data classification via dual latent representation learning

With the rapid development of genetic sequencing and DNA microarray technologies, a large number of gene expression data has been generated, which provides an important reference for tumor diagnosis. However, it is challenging to classify these gene expression data due to the high-dimensionality and small number of data samples. In this work, we propose an effective method to select the most discriminative genes from high-dimensional microarray data for benefiting tumor classification. In detail, each gene is regarded as a feature dimension and we build a novel computational model based on dual latent feature representation learning, referred as DLRL briefly, which can capture both the internal association of data samples and the relationship between different genes. Instead of measuring the importance of genes in original data space, we perform gene selection in the learned latent representation space which is more robust to noisy and redundant information. We first construct the affinity matrices for both samples and genes, which can represent the correlation information between data samples and genes, respectively. Then the dual latent representation learning is modelled via non-negative matrix factorization of the two affinity matrices. The low-dimensional latent representation matrix of sample space is treated as a pseudo-label matrix to guide the latent space projection of original data. Meanwhile, the sample projection matrix is unified with the latent representation matrix of gene space. An alternating algorithm is carefully designed to solve the resultant optimization problem. Extensive experiments on six commonly used publicly microarray datasets are conducted to demonstrate that the proposed method can steadily outperform other state-of-the-art methods in terms of microarray data classification. In addition, we also test the proposed model on a face image dataset as well as a digit image dataset to validate its efficacy for general unsupervised feature selection.

A novel AdaBoost ensemble model based on the reconstruction of local tangent space alignment and its application to multiple faults recognition

In order to recognize the coupling faults of complex industrial processes effectively, this paper proposed an AdaBoost ensemble (AdBE) model based on the reconstruction of local tangent space alignment (RLTSA). First, to obtain the low-dimensional manifold structure embedded in original data space, RLTSA algorithm is designed by constructing the tangent space in the neighborhood of each data point to represent the local geometry and then aligning them to obtain the embedding coordinates. Secondly, to solve the loss of global feature information, an affine matrix is used to inversely map the low-dimensional coordinates to restore the global structure information. Thirdly, based on the above reconstruction of local tangent space alignment, an AdaBoost ensemble (AdBE) classifier is constructed for multiple faults recognition in which the AdaBoost algorithm is used to improve the performance of Decision Tree (DT), and One vs. Rest (OvR) ensemble strategy is introduced to establish the RLTSA-AdBE model. Case studies are conducted using a three-dimensional S_curve data set and the Tennessee Eastman process (TEP) to respectively verify the performance of the RLTSA algorithm and the proposed RLTSA-AdBE model. The simulation results indicate that the proposed method guarantees high diagnosis accuracy and macro_F1 Score of coupling faults recognition.

Multi-label feature selection via manifold regularization and dependence maximization

Feature selection is able to select more discriminative features for classification and plays an important role in multi-label learning to alleviate the effect of the curse of dimensionality. Recently, the multi-label feature selection methods based on the sparse regression model have received increasing attentions. However, most of these methods directly project original data space to label space in the regression model, which is inappropriate because the linear assumption between data space and label space doesn't hold in most cases. In the paper, we propose a feature selection method named multi-label feature selection via manifold regularization and dependence maximization (MRDM). In the regression model of MRDM, the original data space is projected to a low-dimensional manifold space, which not only has the same topological structure with the original data, but also has a strong dependence with the class labels. Then, an objective function involving l2,1-norm regularization is formulated, and an alternating optimization-based iterative algorithm is designed to obtain the sparse coefficients for multi-label feature selection. Extensive experiments on various multi-label data sets demonstrate the superiority of the proposed method compared with some state-of-the-art multi-label feature selection methods.

One-Layer vs. Two-Layer SOM in the Context of Outlier Identification: A Simulation Study

In this work, we applied a stochastic simulation methodology to quantify the power of the detection of outlying mixture components of a stochastic model, when applying a reduced-dimension clustering technique such as Self-Organizing Maps (SOMs). The essential feature of SOMs, besides dimensional reduction into a discrete map, is the conservation of topology. In SOMs, two forms of learning are applied: competitive, by sequential allocation of sample observations to a winning node in the map, and cooperative, by the update of the weights of the winning node and its neighbors. By means of cooperative learning, the conservation of topology from the original data space to the reduced (typically 2D) map is achieved. Here, we compared the performance of one- and two-layer SOMs in the outlier representation task. The same stratified sampling was applied for both the one-layer and two-layer SOMs; although, stratification would only be relevant for the two-layer setting—to estimate the outlying mixture component detection power. Two distance measures between points in the map were defined to quantify the conservation of topology. The results of the experiment showed that the two-layer setting was more efficient in outlier detection while maintaining the basic properties of the SOM, which included adequately representing distances from the outlier component to the remaining ones.

Detecting genetic associations with brain imaging phenotypes in Alzheimer's disease via a novel structured KCCA approach.

Neuroimaging genetics has become an important research topic since it can reveal complex associations between genetic variants (i.e. single nucleotide polymorphisms (SNPs) and the structures or functions of the human brain. However, existing kernel mapping is difficult to directly use the sparse representation method in the kernel feature space, which makes it difficult for most existing sparse canonical correlation analysis (SCCA) methods to be directly promoted in the kernel feature space. To bridge this gap, we adopt a novel alternating projected gradient approach, gradient KCCA (gradKCCA) model to develop a powerful model for exploring the intrinsic associations among genetic markers, imaging quantitative traits (QTs) of interest. Specifically, this model solves kernel canonical correlation (KCCA) with an additional constraint that projection directions have pre-images in the original data space, a sparsity-inducing variant of the model is achieved through controlling the [Formula: see text]-norm of the preimages of the projection directions. We evaluate this model using Alzheimer's disease Neuroimaging Initiative (ADNI) cohort to discover the relationships among SNPs from Alzheimer's disease (AD) risk gene APOE, imaging QTs extracted from structural magnetic resonance imaging (MRI) scans. Our results show that the algorithm not only outperforms the traditional KCCA method in terms of Root Mean Square Error (RMSE) and Correlation Coefficient (CC) but also identify the meaningful and relevant biomarkers of SNPs (e.g. rs157594 and rs405697), which are positively related to right Postcentral and right SupraMarginal brain regions in this study. Empirical results indicate its promising capability in revealing biologically meaningful neuroimaging genetics associations and improving the disease-related mechanistic understanding of AD.

Joint Adaptive Dual Graph and Feature Selection for Domain Adaptation

Domain adaptation aims to exploit domain-invariant features by aligning the cross-domain distributions in the manifold subspace for applying the classifier trained on the source domain to the target domain. However, two limitations may still deteriorate their performances: (1) the influences of noisy or irrelevant features in the original feature space are ignored, which may unexpectedly hurt the classification of target samples; (2) the graph constructed directly in the original data space cannot accurately capture the inherent local manifold structures of high-dimensional data due to the curse of dimensionality, which may seriously mislead the transferable features learning. In this paper, we propose a novel approach to address these problems, referred to as joint Adaptive Dual Graph and Feature Selection for domain adaptation (ADGFS). Specifically, feature selection can characterize the relative importance of different features through a scaling factor, which enables ADGFS to not only reduce the impacts of noisy or irrelevant features on knowledge transfer but also learn informative domain-invariant features. Meanwhile, ADGFS adaptively optimizes the dual graph by learning the similarity matrices of both instance-level and feature-level graphs in the projected low-dimensional manifold subspace rather than the original high-dimensional space, such that the intrinsic local manifold structures of data can be captured precisely. Moreover, ADGFS simultaneously aligns the marginal and conditional probability distributions in the nonnegative matrix factorization framework to narrow the distribution discrepancies between the two different domains, which can adequately transfer knowledge from the source domain to the target domain. Comprehensive experiments on four benchmark datasets can demonstrate that the effectiveness of the proposed approach in cross-domain image classification.

Detecting the damage of bridges under changing environmental conditions using the characteristics of the nonlinear narrow dimension of damage features

Changes in environmental conditions have a considerable influence on the results of damage detection of bridges, and these changing conditions may cause nonlinear behavior in the damage features used for detecting the damage of bridges. Some approaches have been developed to perform damage detection under the linear condition of damage features; however, few studies have considered damage features possessing nonlinear performance caused by changing environmental conditions. To address this concern, the characteristics of the nonlinear narrow dimension (CNND) of damage features is proposed to analyze the nonlinear characteristics of the damage features. For the damage features with strong CNND, a method based on the k-segments algorithm of the principal curve is proposed to equivalently solve the nonlinear issue using piecewise linearization. Compared with the kernel principal component analysis (KPCA)-based method, a popular method to solve the nonlinear issue of damage features in unknown high-dimensional space, the proposed method has a better performance for damage detection because it eliminates the environmental effects in the original data space to prevent excessive elimination of information related to the structural damage. Moreover, it also avoids the disadvantage of some studies in which the data partition is based on observations of the data structure. Finally, numerical examples and monitoring data from an actual bridge are used to validate the effectiveness and applicable conditions of the proposed method.

Multi-region symbolic regression: combining functions under a multi-objective approach

This paper introduces Multi-Region Symbolic Regression (MR-SR), a general framework that divides the original input data space of symbolic regression problems into subspaces (regions), generates different solutions to fit these regions and then combines them. MR-SR has three main components: (1) a strategy for finding the different regions of the data input space; (2) a method for generating the functions for each region; and (3) a strategy for combining the models found by (2). The main contribution of this paper is on how we generate the functions for each region. We model the function generation problem following a multi-objective approach, where each objective corresponds to the quality of the evolved function in a region, and the number of objectives is equal to the number of regions of the data input space. We test MR-SR in two scenarios with different objectives. In the first, we used the new approach to solve the symbolic regression problem with standard GP, with the main objective of reducing error rate. In the second, we took advantage of this method for a different purpose: to reduce the dimensionality of the semantic space of a variation of GP, namely Geometric Semantic Genetic Programming (GSGP). Results in 10 datasets showed that the method using clustering k-means and a model switching strategy—which makes predictions using the best evolved function for the region of interest—obtained better results in 5 out of 10 datasets for GP with 2 regions. For GSGP the framework was less effective due to the lack of diversity of the solutions evolved.

A Robust Cost-Sensitive Feature Selection Via Self-Paced Learning Regularization

Feature selection is a useful and important process, which has a widely use in high-dimensional data processing and artificial intelligence. Its goal is to select a relatively small and representative subset of data from the original data space so that can obtain a better learning performance. Many existing feature selection algorithms simply pursue high accuracy and ignore the cost of feature acquisition and error classification. In this paper, a novel cost-sensitive feature selection method via self-paced learning is proposed. The $$\sigma $$ -norm is introduced to constrain the model and enhance its robustness. Then, we combine the self-paced learning framework. It can control the number of samples involved in training process, which can reduce the impact of noise points. The proposed method can obtain a better classification accuracy while maintaining the lowest total cost. It also has better interpretability and practicability than the traditional feature selection algorithm due to the consideration of the collection and misclassification cost among various features. Extensive experiments have been conducted on eight data sets with six comparison algorithms. The proposed method achieves a good performance.

Joint cortical surface and structural connectivity analysis of Alzheimer's Disease.

Prior neuroimaging studies have demonstrated isolated structural and connectivity changes in the brain due to Alzheimer's Disease (AD). However, how these changes relate to each other is not well understood. We present a preliminary study to begin to fill this gap by leveraging joint independent component analysis (jICA). We explore how jICA performs in an analysis of T1 and diffusion weighted MRI by characterizing the joint changes of complex cortical surface and structural connectivity metrics in AD in subjects from the Baltimore Longitudinal Study of Aging. We calculate 588 region-based cortical metrics and 4,753 fractional anisotropy-based connectivity metrics and project them into a low-dimensional manifold with principal component analysis. We perform jICA on the manifold and subsequently backproject the independent components to the original data space. We demonstrate component stability with 3-fold cross validation and find differential component loadings between 776 cognitively unimpaired control subjects and 23 with AD that generalizes across folds. In addition, we perform the same analysis on the surface and connectivity metrics separately and find that the joint approach identifies both novel and similar components to the separate approaches. To illustrate the joint approach's primary utility, we provide an example hypothesis for how surface and connectivity components may vary together with AD. These preliminary results suggest jointly varying independent cortical surface and structural connectivity components can be consistently extracted from MRI data and provide a data-driven way for generating novel hypotheses about AD that may not be captured by separate analyses.

A Spatio-Temporal Analysis of the Health Situation in Poland Based on Functional Discriminant Coordinates.

The aim of this study was to investigate if the provinces of Poland are homogeneous in terms of the observed spatio-temporal data characterizing the health situation of their inhabitants. The health situation is understood as a set of selected factors influencing inhabitants’ health and the healthcare system in their area of residence. So far, studies concerning the health situation of selected territorial units have been based on data relating to a specific year rather than longer periods. The task of assessing province homogeneity was carried out in two stages. In stage one, the original spatio-temporal data space (space of multivariate time series) was transformed into a functional discriminant coordinates space. The resulting functional discriminant coordinates are synthetic measures of the health situation of inhabitants of particular provinces. These measures contain complete information regarding 8 diagnostic variables examined over a period of 6 years. In the second stage, the Ward method, commonly used in cluster analysis, was applied in order to identify groups of homogeneous provinces in the space of functional discriminant coordinates. Sixteen provinces were divided into four clusters. The homogeneity of the clusters was confirmed by the multivariate functional coefficient of variation.

Forward Stepwise Deep Autoencoder-Based Monotone Nonlinear Dimensionality Reduction Methods

Dimensionality reduction is an unsupervised learning task aimed at creating a low-dimensional summary and/or extracting the most salient features of a dataset. Principal component analysis is a linear dimensionality reduction method in the sense that each principal component is a linear combination of the input variables. To allow features that are nonlinear functions of the input variables, many nonlinear dimensionality reduction (NLDR) methods have been proposed. In this article, we propose novel NLDR methods based on bottleneck deep autoencoders. Our contributions are 2-fold: (1) We introduce a monotonicity constraint into bottleneck deep autoencoders for estimating a single nonlinear component and propose two methods for fitting the model. (2) We propose a new, forward stepwise deep learning architecture for estimating multiple nonlinear components. The former helps extract interpretable, monotone components when the assumption of monotonicity holds, and the latter helps evaluate reconstruction errors in the original data space for a range of components. We conduct numerical studies to compare different model fitting methods and use two real data examples from the studies of human immune responses to HIV to illustrate the proposed methods. Supplementary materials for this article are available online.