Structure Information Of Data Research Articles

Mesh-Conv: Convolution operator with mesh resolution independence for flow field modeling

Convolutional neural network (CNN) is a useful model in deep-learning and has become a promising alternative for flow field modeling due to its capability of spatial information representation. However, the standard convolution operator is only suitable for uniform Cartesian mesh, rather than non-uniform structured or unstructured mesh. In this work, a new convolution operator named Mesh-Conv is designed to allow CNN to model flow field without requiring pixelation preprocessing. This is achieved by decomposing standard convolution elements, combining the unique data type of the flow field, and introducing the concept of local weights to incorporate data structure information. The differences are clarified among the new operator, the standard convolution operator, and the graph convolution operator. In addition, new geometric features and loss functions are designed to further improve the model performance for flow field modeling. Numerical experiments show that the Mesh-Conv operator can obtain higher modeling accuracy than the standard convolution operator. The correlation coefficients of the flow fields predicted by the model are all above 0.999, and there is no checkerboard phenomenon. The Mesh-Conv operator is independent of mesh resolution to a certain extent, so its training and testing losses to the dataset with mixed mesh resolution are reduced by 63% and 55% respectively, compared with the standard convolution operator, which greatly relaxes the restriction on the data structure. Moreover, the Mesh-Conv operator is easy to be extended to most existing CNN architectures.

High-accuracy gearbox health state recognition based on graph sparse random vector functional link network

Health state recognition technology is of great significance for the maintenance decision-making and safety assessment of gearboxes. Traditional random vector functional link network (RVFLN) cannot fully leverage the label sparsity of gearbox health states and the manifold structure information of raw data is neglected. To overcome these drawbacks, a high-accuracy gearbox health state recognition model is proposed in this paper, namely graph sparse RVFLN (GSRVFLN). Firstly, a sparse constraint term is introduced into GSRVFLN to force the predicted label to be similar to the zero-one true label as much as possible, so as to fully exploit the sparsity of the output label. Secondly, a discriminative adjacency graph is designed for GSRVFLN with the label information to capture the inherent geometry structure and discriminative information of data. Finally, we derive an effective solution for GSRVFLN with the alternating direction method of multipliers (ADMM) framework, and this solution has great convergence. The applicability of GSRVFLN for health state recognition is validated with two gearbox datasets, and comparative results show that GSRVFLN achieves the excellent performance of gearbox health state recognition, winning other state-of-the-art models.

A topology-preserving dimensionality reduction method for single-cell RNA-seq data using graph autoencoder

Dimensionality reduction is crucial for the visualization and interpretation of the high-dimensional single-cell RNA sequencing (scRNA-seq) data. However, preserving topological structure among cells to low dimensional space remains a challenge. Here, we present the single-cell graph autoencoder (scGAE), a dimensionality reduction method that preserves topological structure in scRNA-seq data. scGAE builds a cell graph and uses a multitask-oriented graph autoencoder to preserve topological structure information and feature information in scRNA-seq data simultaneously. We further extended scGAE for scRNA-seq data visualization, clustering, and trajectory inference. Analyses of simulated data showed that scGAE accurately reconstructs developmental trajectory and separates discrete cell clusters under different scenarios, outperforming recently developed deep learning methods. Furthermore, implementation of scGAE on empirical data showed scGAE provided novel insights into cell developmental lineages and preserved inter-cluster distances.

Semi-supervised label distribution learning via projection graph embedding

Label distribution learning (LDL) is a new machine learning paradigm that addresses label ambiguity by emphasizing the relevance of each label to a particular instance. In supervised learning, many LDL algorithms have been proposed, which often require a large amount of well-annotated training data to achieve good performance. However, annotating a label distribution is more complicated and expensive than annotating a single label or multiple labels with logical values of 0 and 1. Thus, we propose a projection graph embedding algorithm for semi-supervised label distribution learning (PGE-SLDL). Specifically, we seek a potential space by orthogonal neighborhood preserving projections, named capture space. This capture space is used to select more valuable features and construct a graph that contains more accurate data structure information. We utilize the sample correlation information contained between graph nodes to recover the unknown label distribution. In addition, compared with fixed graphs in traditional semi-supervised learning, we carry out projection and graph construction simultaneously to obtain a self-updating projection graph, which is more helpful to learn label distribution. The experimental results validate the effectiveness of the proposed algorithm.

MeSIN: Multilevel selective and interactive network for medication recommendation

Recommending medications for patients using electronic health records (EHRs) is a crucial data mining task for an intelligent healthcare system. EHRs data, which comprise multiple temporal sequences (e.g., lab, diagnosis, and treatment sequences), encompass multilevel structure information, including flat structure information (e.g., sequential medical codes and their relationships) in each sequence and hierarchical structure information between sequences (e.g., information flow from lab sequence to diagnosis sequence or hierarchical codes relationships), which implicitly reflects a physician’s decision process. However, existing studies focus on mining flat structure information, which is incomplete and hence biased. Therefore, to realize more accurate medication recommendations, this paper proposes a multilevel selective and interactive network (MeSIN) that exploits complete structure information of EHR data; in particular, at the core of our framework is an interactive long short-term memory network (InLSTM), which is developed to model the temporal dependencies in each flat-structured sequence via a recurrent mechanism and capture the hierarchical relationships of different sequences via a reinforced interactive network. Moreover, to reduce the impact of irrelevant information on the multilevel interaction process, an attentional selective module (ASM) is introduced to filter out noisy information and thereby focus on important sequence information at each timestamp. Finally, a global selective fusion module (GSFM) is proposed to infuse multisourced information embeddings into multilevel structural relationship-aware patient representation. Experimental results on a practical clinical dataset show that our framework consistently outperformed all baseline models in the medication recommendation task.

Dual-Channel Heterogeneous Graph Network for Author Name Disambiguation

Name disambiguation has long been a significant issue in many fields, such as literature management and social analysis. In recent years, methods based on graph networks have performed well in name disambiguation, but these works have rarely used heterogeneous graphs to capture relationships between nodes. Heterogeneous graphs can extract more comprehensive relationship information so that more accurate node embedding can be learned. Therefore, a Dual-Channel Heterogeneous Graph Network is proposed to solve the name disambiguation problem. We use the heterogeneous graph network to capture various node information to ensure that our method can learn more accurate data structure information. In addition, we use fastText to extract the semantic information of the data. Then, a clustering method based on DBSCAN is used to classify academic papers by different authors into different clusters. In many experiments based on real datasets, our method achieved high accuracy, which proves its effectiveness.

Comparative study of different dimensionality reduction methods in hyperspectral image classification

Considering the high-dimensional characteristics of hyperspectral image (HSI) data, researchers generally adopt the dimensionality reduction (DR) methods to reduce the complexity and computing time of subsequent classification or regression tasks while preserving the intrinsic structure information of the data. At present, the research on DR of HSIs data mostly focuses on the application performance of a single method in specific tasks and few studies have been conducted on the adaptability of different DR methods to HSIs data. From the perspective of spectral domain and spatial domain, this paper makes a comparative study on the performance of various linear and nonlinear DR methods in the task of HSI classification of aerial HSI of Matiwan Village in Xiongan New Area. Specifically, it includes principal component analysis, independent component analysis, isometric mapping, Laplacian eigenmaps, autoencoder, etc. The results show that the intrinsic structure in Xiongan HSI is mainly linear structure. Compared with the nonlinear DR methods, the linear DR methods can better preserve the intrinsic structure information of data at a lower time-consuming cost. For the subsequent classification tasks, the linear DR methods have better classification performance and are more suitable for HSI data.

Aircraft Type Recognition in Remote Sensing Images: Bilinear Discriminative Extreme Learning Machine Framework

Small inter-class and massive intra-class changes are important challenges in aircraft model recognition in the field of remote sensing. Although the aircraft model recognition algorithm based on the convolutional neural network (CNN) has excellent recognition performance, it is limited by sample sets and computing resources. To solve the above problems, we propose the bilinear discriminative extreme learning machine (ELM) network (BD-ELMNet), which integrates the advantages of the CNN, autoencoder (AE), and ELM. Specifically, the BD-ELMNet first executes the convolution and pooling operations to form a convolutional ELM (ELMConvNet) to extract shallow features. Furthermore, the manifold regularized ELM-AE (MRELM-AE), which can simultaneously consider the geometrical structure and discriminative information of aircraft data, is developed to extract discriminative features. The bilinear pooling model uses the feature association information for feature fusion to enhance the substantial distinction of features. Compared with the backpropagation (BP) optimization method, BD-ELMNet adopts a layer-by-layer training method without repeated adjustments to effectively learn discriminant features. Experiments involving the application of several methods, including the proposed method, to the MTARSI benchmark demonstrate that the proposed aircraft type recognition method outperforms the state-of-the-art methods.

Robust supervised multi-view feature selection with weighted shared loss and maximum margin criterion

Supervised multi-view feature selection has recently been proven to be a valid way for reducing the dimensionality of data. This paper proposes a robust supervised multi-view feature selection method based on weighted shared loss and maximum margin criterion. Specifically, we fuse all weighted views to establish a shared loss term to maintain the complementary information of views, where the weight of each view can be automatically adjusted according to its contribution to the final task. Moreover, we adopt the capped l2-norm rather than the widely used l2-norm to measure the loss and remove the potential outliers. Further, we introduce the maximum margin criterion (MMC) into the feature selection and design a weighted view-based MMC such that it is a proper regularization associated with the proposed model. In this way, the inter-class and intra-class structure information of data is well exploited, which makes the margins of the samples belonging to the different classes increase and the margins of the samples belonging the same class decrease. Then the discriminative features of all views can be selected. The corresponding problem can be solved by the alternating direction method of multipliers (ADMM), which can realize the view-block calculation and reduce the computational complexity. Comprehensive experiments on various benchmark datasets and the application on the hydraulic system confirm the validity of our method.

Deep learning with nonlocal and local structure preserving stacked autoencoder for soft sensor in industrial processes

Deep learning-based soft sensor has been widely used for quality prediction in modern industry. Traditional deep learning like stacked autoencoder (SAE) only captures the feature representations by minimizing the global reconstruction errors, which causes a loss of the intrinsic geometric structure embedded in the raw data. To address this problem, a nonlocal and local structure preserving stacked autoencoder (NLSP-SAE) is proposed for soft sensor. Different from the original SAE, NLSP-SAE aims to extract the meaningful structure-relevant features by establishing a new objective function with a regularizer of the nonlocal and local data structure information. For local structure preserving, NLSP-SAE enforces two adjacent data points to be near each other in the reconstructed space. While for nonlocal structure preserving, NLSP-SAE constrains two nonadjacent data points to be far apart from each other. The application on an industrial hydrocracking process demonstrates that NLSP-SAE can improve the prediction accuracy for quality variables.

HTRPCA: Hypergraph Regularized Tensor Robust Principal Component Analysis for Sample Clustering in Tumor Omics Data.

In recent years, clustering analysis of cancer genomics data has gained widespread attention. However, limited by the dimensions of the matrix, the traditional methods cannot fully mine the underlying geometric structure information in the data. Besides, noise and outliers inevitably exist in the data. To solve the above two problems, we come up with a new method which uses tensor to represent cancer omics data and applies hypergraph to save the geometric structure information in original data. This model is called hypergraph regularized tensor robust principal component analysis (HTRPCA). The data processed by HTRPCA becomes two parts, one of which is a low-rank component that contains pure underlying structure information between samples, and the other is some sparse interference points. So we can use the low-rank component for clustering. This model can retain complex geometric information between more sample points due to the addition of the hypergraph regularization. Through clustering, we can demonstrate the effectiveness of HTRPCA, and the experimental results on TCGA datasets demonstrate that HTRPCA precedes other advanced methods. This paper proposes a new method of using tensors to represent cancer omics data and introduces hypergraph items to save the geometric structure information of the original data. At the same time, the model decomposes the original tensor into low-order tensors and sparse tensors. The low-rank tensor was used to cluster cancer samples to verify the effectiveness of the method.

Prediction of African Swine Fever Virus Inhibitors by Molecular Docking-Driven Machine Learning Models.

African swine fever virus (ASFV) causes a highly contagious and severe hemorrhagic viral disease with high mortality in domestic pigs of all ages. Although the virus is harmless to humans, the ongoing ASFV epidemic could have severe economic consequences for global food security. Recent studies have found a few antiviral agents that can inhibit ASFV infections. However, currently, there are no vaccines or antiviral drugs. Hence, there is an urgent need to identify new drugs to treat ASFV. Based on the structural information data on the targets of ASFV, we used molecular docking and machine learning models to identify novel antiviral agents. We confirmed that compounds with high affinity present in the region of interest belonged to subsets in the chemical space using principal component analysis and k-means clustering in molecular docking studies of FDA-approved drugs. These methods predicted pentagastrin as a potential antiviral drug against ASFVs. Finally, it was also observed that the compound had an inhibitory effect on AsfvPolX activity. Results from the present study suggest that molecular docking and machine learning models can play an important role in identifying potential antiviral drugs against ASFVs.

Deep Multiple Auto-Encoder-Based Multi-view Clustering

Multi-view clustering (MVC), which aims to explore the underlying structure of data by leveraging heterogeneous information of different views, has brought along a growth of attention. Multi-view clustering algorithms based on different theories have been proposed and extended in various applications. However, most existing MVC algorithms are shallow models, which learn structure information of multi-view data by mapping multi-view data to low-dimensional representation space directly, ignoring the nonlinear structure information hidden in each view, and thus, the performance of multi-view clustering is weakened to a certain extent. In this paper, we propose a deep multi-view clustering algorithm based on multiple auto-encoder, termed MVC-MAE, to cluster multi-view data. MVC-MAE adopts auto-encoder to capture the nonlinear structure information of each view in a layer-wise manner and incorporate the local invariance within each view and consistent as well as complementary information between any two views together. Besides, we integrate the representation learning and clustering into a unified framework, such that two tasks can be jointly optimized. Extensive experiments on six real-world datasets demonstrate the promising performance of our algorithm compared with 15 baseline algorithms in terms of two evaluation metrics.

Discriminative Invariant Alignment for Unsupervised Domain Adaptation

As one of the most prevalent branches of transfer learning, domain adaptation is dedicated to generalizing the knowledge of a source domain to a target domain to perform machine learning tasks. In domain adaptation, the key strategy is to overcome the shift between different domains and learn shared features with domain invariance. However, most existing methods focus on extracting the common features of the source and target domains, and do not consider the shift problem of class center in the target domain caused by this process. Specifically, when we align the domain distributions, we often ignore the inherent feature attributes of the data, or under the guidance of false pseudo-labels, cause the target domain data to be far away from the class center after projection. This is not conducive to classification task. To address these problems, in this study, we propose a novel domain adaptation method, referred to as discriminative invariant alignment (DIA), for image representation. DIA enriches the knowledge matrix by combining the class discriminative information of the source domain and local data structure information of the target domain into a new framework. By introducing the maximum margin criterion of the source domain, the classification boundaries are expanded. To verify the performance of the proposed method, we compared DIA with several state-of-the-art methods on five benchmark databases. The experimental results show that DIA is superior to the state-of-the-art methods.

Nonparametric manifold learning approach for improved process monitoring

AbstractA novel nonparametric method based on manifold learning is proposed for industrial process monitoring. In conventional algorithms, to preserve the global and local structure information of data, heat kernels containing two auxiliary parameters are introduced to define the global and local weight matrices, respectively. However, it is difficult to identify and choose these two parameters empirically. The inadequate selection of parameters can lead to one‐sided and inappropriate global and local feature extractions, resulting in an inadequate fault detection performance. To resolve the above problems, a nonparametric strategy is used in this study to generate two nonparametric weight matrices to replace the heat kernel‐based weight matrices. Consequently, the proposed method requires no auxiliary parameters in defining the weight matrices, making it more practical. Moreover, it automatically determines a good trade‐off between global and local feature extractions. A process monitoring model based on the proposed method was developed. The feasibility and effectiveness of the new nonparametric method are evaluated using a synthetic example and the Tennessee Eastman chemical process.

Auto-weighted robust low-rank tensor completion via tensor-train

Nowadays, multi-dimensional data (tensor data) have shown their capability of preserving multilinear structures. Due to the measuring error or other non-human factors, these data often suffer signal corruptions or missing values, or even both. To address these issues simultaneously, this paper studies the Robust Tensor Completion (RTC) problem, a mixed problem of the known Low-Rank Tensor Completion (LRTC) and Robust Principal Component Analysis (RPCA). Based on Tensor-Train rank (TT rank), the proposed model is able to capture the latent structure information of tensor data by recovering the low-rank component and separating the sparse component from the partial observations. To make TT rank more effective, an auto-weighted mechanism is utilized to balance the importance of different matricizations from the same tensor. We also propose a more flexible tensor augmentation approach called Tree Ket Augmentation (Tree-KA) to obtain a higher-order tensor from a lower one with it a new general explanation. Alternating direction method of multipliers (ADMM) is employed to solve the resulting model and extensive numerical experiments have verified the effectiveness of the proposed model compared with other state-of-the-art methods.

Semi-supervised classification by graph p-Laplacian convolutional networks

The graph convolutional networks (GCN) generalizes convolution neural networks into the graph with an arbitrary topology structure. Since the geodesic function in the null space of the graph Laplacian matrix is constant, graph Laplacian fails to preserve the local topology structure information between samples properly. GCN thus cannot learn better representative sample features by the convolution operation of the graph Laplacian based structure information and input sample information. To address this issue, this paper exploits the manifold structure information of data by the graph p-Laplacian matrix and proposes the graph p-Laplacian convolutional networks (GpLCN). As the graph p-Laplacian matrix is a generalization of the graph Laplacian matrix, GpLCN can extract more abundant sample features and improves the classification performance utilizing graph p-Laplacian to preserve the rich intrinsic data manifold structure information. Moreover, after simplifying and deducing the formula of the one-order spectral graph p-Laplacian convolution, we introduce a new layer-wise propagation rule based on the one-order approximation. Extensive experiment results on the Citeseer, Cora and Pubmed database demonstrate that our GpLCN outperforms GCN.

The Use of 3D Convolutional Autoencoder in Fault and Fracture Network Characterization

Conventional pattern recognition methods directly use 1D poststack data or 2D prestack data for the statistical pattern recognition of fault and fracture network, thereby ignoring the spatial structure information in 3D seismic data. As a result, the generated fault and fracture network is not distinguishable and has poor continuity. In this paper, a fault and fracture network characterization method based on 3D convolutional autoencoder is proposed. First, in the autoencoder training frame, 3D prestack data are used as input, and the 3D convolution operation is used to mine the spatial structure information to the maximum and gradually reduce the spatial dimension of the input. Then, the residual network is used to recover the input’s details and the corresponding spatial dimension. Lastly, the hidden features extracted by the encoders are recognized via k -means, SOM, and two-step clustering analysis. The validity of the method is verified by testing the seismic simulation data and applying real seismic data. The 3D convolution can directly process the seismic data and maximize the prestack texture attributes and spatial structure information provided by 3D seismic data without dimensionality reduction and other preprocessing operations. The interleaving convolution layer and residual block overcome low learning and accuracy rates due to the deepening of networks.

Graph Regularized Deep Sparse Representation for Unsupervised Anomaly Detection.

Anomaly detection (AD) aims to distinguish the data points that are inconsistent with the overall pattern of the data. Recently, unsupervised anomaly detection methods have aroused huge attention. Among these methods, feature representation (FR) plays an important role, which can directly affect the performance of anomaly detection. Sparse representation (SR) can be regarded as one of matrix factorization (MF) methods, which is a powerful tool for FR. However, there are some limitations in the original SR. On the one hand, it just learns the shallow feature representations, which leads to the poor performance for anomaly detection. On the other hand, the local geometry structure information of data is ignored. To address these shortcomings, a graph regularized deep sparse representation (GRDSR) approach is proposed for unsupervised anomaly detection in this work. In GRDSR, a deep representation framework is first designed by extending the single layer MF to a multilayer MF for extracting hierarchical structure from the original data. Next, a graph regularization term is introduced to capture the intrinsic local geometric structure information of the original data during the process of FR, making the deep features preserve the neighborhood relationship well. Then, a L1-norm-based sparsity constraint is added to enhance the discriminant ability of the deep features. Finally, a reconstruction error is applied to distinguish anomalies. In order to demonstrate the effectiveness of the proposed approach, we conduct extensive experiments on ten datasets. Compared with the state-of-the-art methods, the proposed approach can achieve the best performance.

Deep Nonlinear Dynamic Feature Extraction for Quality Prediction Based on Spatiotemporal Neighborhood Preserving SAE

Complex industrial process data often exhibit nonlinear static and dynamic characteristics. Traditional deep learning methods such as stacked autoencoder (SAE) have excellent nonlinear static feature learning capabilities, but they ignore the dynamic correlation existing in process data. Feature learning based on manifold learning using neighborhood structure preserving has been widely used in industrial dynamic process monitoring. However, most of the manifold learning methods extract linear features, and complex nonlinearities in process data are ignored. Therefore, a novel spatiotemporal neighborhood preserving stack autoencoder (STNP-SAE) is proposed to simultaneously learn deep nonlinear static and dynamic features of process data in this article. By constructing the spatial and temporal adjacent graphs, STNP-SAE can capture the spatiotemporal neighborhood structure information of process data during the feature learning process. Then, STNP-SAE is used to construct a soft sensor framework for quality prediction. The prediction performance of the proposed method is validated on a practical industrial process.