Low-dimensional Feature Representation Research Articles

Uncertainty-aware dynamic integration for multi-omics classification of tumors.

Omics data are crucial for medical diagnosis as it contains intrinsic biomedical information. Multi-omics integrated analysis has become a new direction for scientists to explore life mechanisms. Nevertheless, the quality of complex omics data often varies greatly due to different samples or even different omics types, it is challenging to dynamically capture the uncertainty for different kinds of omics data. This paper proposes a uncertainty-aware dynamic integration framework for multi-omics classification. The framework consists of three modules: deep embedding, confidence prediction, and downstream tasks. The deep embedding module extract key information from multi-omics data to obtain a low-dimensional feature representation which is used to train downstream tasks. Combined with the deep embedding module, the confidence prediction module is used to dynamically capture the uncertainty of the data. We introduce "confidNet" to assign a confidence value for each type of omics data, which is used for dynamic integration between multi-omics. Compared with other integration methods, the proposed method can contain more crucial biomedical information in the obtained low-dimensional representation. Our framework realizes reliable integration among multiple omics, and it can still achieve high accuracy on small sample data sets. We have verified the effectiveness of the model in a large number of experiments. Our framework can be widely applied to high-dimensional omics data and has great potential to facilitate medical decision-making and biological analysis.

Efficient and Highly Accurate Diagnosis of Malignant Hematological Diseases Based on Whole-Slide Images Using Deep Learning.

Hematopoietic disorders are serious diseases that threaten human health, and the diagnosis of these diseases is essential for treatment. However, traditional diagnosis methods rely on manual operation, which is time consuming and laborious, and examining entire slide is challenging. In this study, we developed a weakly supervised deep learning method for diagnosing malignant hematological diseases requiring only slide-level labels. The method improves efficiency by converting whole-slide image (WSI) patches into low-dimensional feature representations. Then the patch-level features of each WSI are aggregated into slide-level representations by an attention-based network. The model provides final diagnostic predictions based on these slide-level representations. By applying the proposed model to our collection of bone marrow WSIs at different magnifications, we found that an area under the receiver operating characteristic curve of 0.966 on an independent test set can be obtained at 10× magnification. Moreover, the performance on microscopy images can achieve an average accuracy of 94.2% on two publicly available datasets. In conclusion, we have developed a novel method that can achieve fast and accurate diagnosis in different scenarios of hematological disorders.

Prediction of Drug-Drug Interaction Using an Attention-Based Graph Neural Network on Drug Molecular Graphs.

The treatment of complex diseases by using multiple drugs has become popular. However, drug-drug interactions (DDI) may give rise to the risk of unanticipated adverse effects and even unknown toxicity. Therefore, for polypharmacy safety it is crucial to identify DDIs and explore their underlying mechanisms. The detection of DDI in the wet lab is expensive and time-consuming, due to the need for experimental research over a large volume of drug combinations. Although many computational methods have been developed to predict DDIs, most of these are incapable of predicting potential DDIs between drugs within the DDI network and new drugs from outside the DDI network. In addition, they are not designed to explore the underlying mechanisms of DDIs and lack interpretative capacity. Thus, here we propose a novel method of GNN-DDI to predict potential DDIs by constructing a five-layer graph attention network to identify k-hops low-dimensional feature representations for each drug from its chemical molecular graph, concatenating all identified features of each drug pair, and inputting them into a MLP predictor to obtain the final DDI prediction score. The experimental results demonstrate that our GNN-DDI is suitable for each of two DDI predicting scenarios, namely the potential DDIs among known drugs in the DDI network and those between drugs within the DDI network and new drugs from outside DDI network. The case study indicates that our method can explore the specific drug substructures that lead to the potential DDIs, which helps to improve interpretability and discover the underlying interaction mechanisms of drug pairs.

Identification of miRNA–disease associations via deep forest ensemble learning based on autoencoder

Increasing evidences show that the occurrence of human complex diseases is closely related to microRNA (miRNA) variation and imbalance. For this reason, predicting disease-related miRNAs is essential for the diagnosis and treatment of complex human diseases. Although some current computational methods can effectively predict potential disease-related miRNAs, the accuracy of prediction should be further improved. In our study, a new computational method via deep forest ensemble learning based on autoencoder (DFELMDA) is proposed to predict miRNA-disease associations. Specifically, a new feature representation strategy is proposed to obtain different types of feature representations (from miRNA and disease) for each miRNA-disease association. Then, two types of low-dimensional feature representations are extracted by two deep autoencoders for predicting miRNA-disease associations. Finally, two prediction scores of the miRNA-disease associations are obtained by the deep random forest and combined to determine the final results. DFELMDA is compared with several classical methods on the The Human microRNA Disease Database (HMDD) dataset. Results reveal that the performance of this method is superior. The area under receiver operating characteristic curve (AUC) values obtained by DFELMDA through 5-fold and 10-fold cross-validation are 0.9552 and 0.9560, respectively. In addition, case studies on colon, breast and lung tumors of different disease types further demonstrate the excellent ability of DFELMDA to predict disease-associated miRNA-disease. Performance analysis shows that DFELMDA can be used as an effective computational tool for predicting miRNA-disease associations.

Deep self-supervised clustering with embedding adjacent graph features

Deep clustering uses neural networks to learn the low-dimensional feature representations suitable for clustering tasks. Numerous studies have shown that learning embedded features and defining the clustering loss properly contribute to better performance. However, most of the existing studies focus on the deep local features and ignore the global spatial characteristics of the original data space. To address this issue, this paper proposes deep self-supervised clustering with embedding adjacent graph features (DSSC-EAGF). The significance of our efforts is three-fold: 1) To obtain the deep representation of the potential global spatial structure, a dedicated adjacent graph matrix is designed and used to train the autoencoder in the original data space; 2) In the deep encoding feature space, the KNN algorithm is used to obtain the virtual clusters for devising a self-supervised learning loss. Then, the reconstruction loss, clustering loss, and self-supervised loss are integrated, and a novel overall loss measurement is proposed for DSSC-EAGF. 3) An inverse-Y-shaped network model is designed to well learn the features of both the local and the global structures of the original data, which greatly improves the clustering performance. The experimental studies prove the superiority of the proposed DSSC-EAGF against a few state-of-the-art deep clustering methods.

A multi-network integration approach for measuring disease similarity based on ncRNA regulation and heterogeneous information

BackgroundMeasuring similarity between complex diseases has significant implications for revealing the pathogenesis of diseases and development in the domain of biomedicine. It has been consentaneous that functional associations between disease-related genes and semantic associations can be applied to calculate disease similarity. Currently, more and more studies have demonstrated the profound involvement of non-coding RNA in the regulation of genome organization and gene expression. Thus, taking ncRNA into account can be useful in measuring disease similarities. However, existing methods ignore the regulation functions of ncRNA in biological process. In this study, we proposed a novel deep-learning method to deduce disease similarity.ResultsIn this article, we proposed a novel method, ImpAESim, a framework integrating multiple networks embedding to learn compact feature representations and disease similarity calculation. We first utilize three different disease-related information networks to build up a heterogeneous network, after a network diffusion process, RWR, a compact feature learning model composed of classic Auto Encoder (AE) and improved AE model is proposed to extract constraints and low-dimensional feature representations. We finally obtain an accurate and low-dimensional feature representation of diseases, then we employed the cosine distance as the measurement of disease similarity.ConclusionImpAESim focuses on extracting a low-dimensional vector representation of features based on ncRNA regulation, and gene–gene interaction network. Our method can significantly reduce the calculation bias resulted from the sparse disease associations which are derived from semantic associations.

Unsupervised Denoising Feature Learning for Classification of Corrupted Images

Numerous corrupted or noisy images are generated owing to the problems in sampling and transmission. Facing the category of high-dimensional image data, directly operating like classification is inefficient and the accuracy is normally not ideal. Consequently, dimensionality reduction is a critical process. After dimensionality reduction by classical algorithms, the classification efficiency can be improved. However, the problem of low accuracy still exists. In this paper, an unsupervised denoising feature learning model is proposed to learn a discriminative feature representation from corrupted images. First, the relationship between initial data and low-dimensional feature representation is explored. We introduce several classical dimensionality reduction algorithms. Second, the working mechanism of denoising feature learning model is proved. Due to its structural characteristics, denoising model can extract robust features from partially destroyed data to represent the original data. It demonstrates proposed model can extract essential information from noisy image to decrease reconstruction error. Finally, extensive experiments demonstrate that the features representation obtained by the proposed method outperforms those obtained by the existing classical dimensionality reduction algorithms.

Graph Representation-Based Deep Multi-View Semantic Similarity Learning Model for Recommendation

With the rapid development of Internet technology, how to mine and analyze massive amounts of network information to provide users with accurate and fast recommendation information has become a hot and difficult topic of joint research in industry and academia in recent years. One of the most widely used social network recommendation methods is collaborative filtering. However, traditional social network-based collaborative filtering algorithms will encounter problems such as low recommendation performance and cold start due to high data sparsity and uneven distribution. In addition, these collaborative filtering algorithms do not effectively consider the implicit trust relationship between users. To this end, this paper proposes a collaborative filtering recommendation algorithm based on graphsage (GraphSAGE-CF). The algorithm first uses graphsage to learn low-dimensional feature representations of the global and local structures of user nodes in social networks and then calculates the implicit trust relationship between users through the feature representations learned by graphsage. Finally, the comprehensive evaluation shows the scores of users and implicit users on related items and predicts the scores of users on target items. Experimental results on four open standard datasets show that our proposed graphsage-cf algorithm is superior to existing algorithms in RMSE and MAE.

Graph-Cut-Based Collaborative Node Embeddings for Hyperspectral Images Classification

Node embedding (NE) is conducive to aggregating correlations and relieving the influence of the Hughes phenomenon when processing high-dimensional data. Although some graph neural networks can capture correlations during achieving NE, the application of NE still faces two rigorous challenges: numerous model parameters and poor generalization. In this letter, we propose a new approach for hyperspectral image (HSI) classification, called the graph-cut-based collaborative NEs (GCCNE). Specifically, we develop a graph-cut-based NE (GCNE) to achieve low-dimensional feature representation, which avoids numerous model parameters when using a graph structure. Considering that the graph-cut in a low-dimensional space does not need to set anchors to decrease the calculation amount, we adopt an ensemble framework based on random subspaces (RSs) to implement the GCNE to obtain the collaborative feature sets, enhancing the generalization of feature representation. Afterward, the collaborative feature sets are input in several kernel-based extreme learning machines (KELMs), respectively, classifying pixels. The number of RSs is the same as the number of KELMs. Finally, we acquire an ensemble result associated with each class. The effectiveness and competitiveness of the proposed method are evaluated by using real HSI datasets.

Automated Dysarthria Severity Classification: A Study on Acoustic Features and Deep Learning Techniques.

Assessing the severity level of dysarthria can provide an insight into the patient's improvement, assist pathologists to plan therapy, and aid automatic dysarthric speech recognition systems. In this article, we present a comparative study on the classification of dysarthria severity levels using different deep learning techniques and acoustic features. First, we evaluate the basic architectural choices such as deep neural network (DNN), convolutional neural network, gated recurrent units and long short-term memory network using the basic speech features, namely, Mel-frequency cepstral coefficients (MFCCs) and constant-Q cepstral coefficients. Next, speech-disorder specific features computed from prosody, articulation, phonation and glottal functioning are evaluated on DNN models. Finally, we explore the utility of low-dimensional feature representation using subspace modeling to give i-vectors, which are then classified using DNN models. Evaluation is done using the standard UA-Speech and TORGO databases. By giving an accuracy of 93.97% under the speaker-dependent scenario and 49.22% under the speaker-independent scenario for the UA-Speech database, the DNN classifier using MFCC-based i-vectors outperforms other systems.

RTIDS: A Robust Transformer-Based Approach for Intrusion Detection System

Due to the rapid growth in network traffic and increasing security threats, Intrusion Detection Systems (IDS) have become increasingly critical in the field of cyber security for providing secure communications against cyber adversaries. However, there exist many challenges for designing a robust, efficient and accurate IDS, especially when dealing with high-dimensional anomaly data with unforeseen and unpredictable attacks. In this paper, we propose a Robust Transformer-based Intrusion Detection System (RTIDS) reconstructing feature representations to make a trade-off between dimensionality reduction and feature retention in imbalanced datasets. The proposed method utilizes positional embedding technique to associate sequential information between features, then a variant stacked encoder-decoder neural network is used to learn low-dimensional feature representations from high-dimensional raw data. Furthermore, we apply self-attention mechanism to facilitate network traffic type classifications. Extensive experiments reveal the effectiveness of the proposed RTIDS on two publicly available real traffic intrusion detection datasets named CICIDS2017 and CIC-DDoS2019 with F1-Score of 99.17% and 98.48% respectively. A comparative study with classical machine learning algorithm support vector machine (SVM) and deep learning algorithms that include recurrent neural network (RNN), fuzzy neural network (FNN), and long short-term memory network (LSTM) is conducted to demonstrate the validity of the proposed method.

Intelligent fault diagnosis under small sample size conditions via Bidirectional InfoMax GAN with unsupervised representation learning

The abnormal detection of rotating machinery under small sample size conditions is of prime importance in the field of fault diagnosis. In this work, we proposed an unsupervised representation learning method called Bidirectional InfoMax GAN (BIMGAN), which can perform fast and effective feature extraction and fault recognition with few samples. First, we obtain the low-dimensional feature representation by a prior normalized encoder and reconstruction of the sample via the generator. Second, the mapping relationship between the sample and its corresponding feature representation is learned by maximizing mutual information estimation with the constraint of the feature matching (FM) strategy. Different from the general GANs, we are aiming at learning a good feature mapping of an encoder to capture the feature representation instead of reconstructing realistic samples. And then, a supervised pattern recognition task based on the feature representation is conducted for fault diagnosis. Finally, the inverse mapping learned by the encoder is visualized and the effectiveness is demonstrated. And the performance of the proposed method outperforms several advanced unsupervised methods on two case studies of rolling bearings fault recognition with some standard architectures, where the average accuracy can achieve 99.73% and 98.36% respectively.

Blind image quality assessment with channel attention based deep residual network and extended LargeVis dimensionality reduction

Image Quality Assessment (IQA) is one of the fundamental problems in the fields of image processing, image/video coding and transmission, and so on. In this paper, a Blind Image Quality Assessment (BIQA) approach with channel attention based deep Residual Network (ResNet)and extended LargeVis dimensionality reduction is proposed. Firstly, ResNet50 with channel attention mechanism is used as the backbone network to extract the deep features from the image. In order to reduce the dimensionality of the deep features, LargeVis, which is originally designed for the visualization of large scale high-dimensional data, is extended by using Support Vector Regression (SVR) to perform on a single feature vector data. The extended LargeVis can remove the redundant information of the deep features so as to obtain a low-dimensional and discriminative feature representation. Finally, the quality prediction model is established by using SVR as the fitting method. The low-dimensional feature representation and quality score of the image form the pair-wise data samples to train the fitting model. Experimental results on authentic distortions datasets and synthetic distortions datasets show that our proposed method can achieve superior performance compared with the state-of-the-art methods.

Quantum-assisted associative adversarial network: applying quantum annealing in deep learning

Generative models have the capacity to model and generate new examples from a dataset and have an increasingly diverse set of applications driven by commercial and academic interest. In this work, we present an algorithm for learning a latent variable generative model via generative adversarial learning where the canonical uniform noise input is replaced by samples from a graphical model. This graphical model is learned by a Boltzmann machine which learns low-dimensional feature representation of data extracted by the discriminator. A quantum processor can be used to sample from the model to train the Boltzmann machine. This novel hybrid quantum-classical algorithm joins a growing family of algorithms that use a quantum processor sampling subroutine in deep learning, and provides a scalable framework to test the advantages of quantum-assisted learning. For the latent space model, fully connected, symmetric bipartite and Chimera graph topologies are compared on a reduced stochastically binarized MNIST dataset, for both classical and quantum sampling methods. The quantum-assisted associative adversarial network successfully learns a generative model of the MNIST dataset for all topologies. Evaluated using the Fréchet inception distance and inception score, the quantum and classical versions of the algorithm are found to have equivalent performance for learning an implicit generative model of the MNIST dataset. Classical sampling is used to demonstrate the algorithm on the LSUN bedrooms dataset, indicating scalability to larger and color datasets. Though the quantum processor used here is a quantum annealer, the algorithm is general enough such that any quantum processor, such as gate model quantum computers, may be substituted as a sampler.

Integrating multi-scale neighbouring topologies and cross-modal similarities for drug-protein interaction prediction.

Identifying the proteins that interact with drugs can reduce the cost and time of drug development. Existing computerized methods focus on integrating drug-related and protein-related data from multiple sources to predict candidate drug-target interactions (DTIs). However, multi-scale neighboring node sequences and various kinds of drug and protein similarities are neither fully explored nor considered in decision making. We propose a drug-target interaction prediction method, DTIP, to encode and integrate multi-scale neighbouring topologies, multiple kinds of similarities, associations, interactions related to drugs and proteins. We firstly construct a three-layer heterogeneous network to represent interactions and associations across drug, protein, and disease nodes. Then a learning framework based on fully-connected autoencoder is proposed to learn the nodes' low-dimensional feature representations within the heterogeneous network. Secondly, multi-scale neighbouring sequences of drug and protein nodes are formulated by random walks. A module based on bidirectional gated recurrent unit is designed to learn the neighbouring sequential information and integrate the low-dimensional features of nodes. Finally, we propose attention mechanisms at feature level, neighbouring topological level and similarity level to learn more informative features, topologies and similarities. The prediction results are obtained by integrating neighbouring topologies, similarities and feature attributes using a multiple layer CNN. Comprehensive experimental results over public dataset demonstrated the effectiveness of our innovative features and modules. Comparison with other state-of-the-art methods and case studies of five drugs further validated DTIP's ability in discovering the potential candidate drug-related proteins.

An Inductive Logistic Matrix Factorization Model for Predicting Drug-Metabolite Association With Vicus Regularization.

Metabolites are closely related to human disease. The interaction between metabolites and drugs has drawn increasing attention in the field of pharmacomicrobiomics. However, only a small portion of the drug-metabolite interactions were experimentally observed due to the fact that experimental validation is labor-intensive, costly, and time-consuming. Although a few computational approaches have been proposed to predict latent associations for various bipartite networks, such as miRNA-disease, drug-target interaction networks, and so on, to our best knowledge the associations between drugs and metabolites have not been reported on a large scale. In this study, we propose a novel algorithm, namely inductive logistic matrix factorization (ILMF) to predict the latent associations between drugs and metabolites. Specifically, the proposed ILMF integrates drug–drug interaction, metabolite–metabolite interaction, and drug-metabolite interaction into this framework, to model the probability that a drug would interact with a metabolite. Moreover, we exploit inductive matrix completion to guide the learning of projection matrices U and V that depend on the low-dimensional feature representation matrices of drugs and metabolites: Fm and Fd. These two matrices can be obtained by fusing multiple data sources. Thus, FdU and FmV can be viewed as drug-specific and metabolite-specific latent representations, different from classical LMF. Furthermore, we utilize the Vicus spectral matrix that reveals the refined local geometrical structure inherent in the original data to encode the relationships between drugs and metabolites. Extensive experiments are conducted on a manually curated “DrugMetaboliteAtlas” dataset. The experimental results show that ILMF can achieve competitive performance compared with other state-of-the-art approaches, which demonstrates its effectiveness in predicting potential drug-metabolite associations.

Variational LSTM Enhanced Anomaly Detection for Industrial Big Data

With the increasing population of Industry 4.0, industrial big data (IBD) has become a hotly discussed topic in digital and intelligent industry field. The security problem existing in the signal processing on large scale of data stream is still a challenge issue in industrial internet of things, especially when dealing with the high-dimensional anomaly detection for intelligent industrial application. In this article, to mitigate the inconsistency between dimensionality reduction and feature retention in imbalanced IBD, we propose a variational long short-term memory (VLSTM) learning model for intelligent anomaly detection based on reconstructed feature representation. An encoder-decoder neural network associated with a variational reparameterization scheme is designed to learn the low-dimensional feature representation from high-dimensional raw data. Three loss functions are defined and quantified to constrain the reconstructed hidden variable into a more explicit and meaningful form. A lightweight estimation network is then fed with the refined feature representation to identify anomalies in IBD. Experiments using a public IBD dataset named UNSW-NB15 demonstrate that the proposed VLSTM model can efficiently cope with imbalance and high-dimensional issues, and significantly improve the accuracy and reduce the false rate in anomaly detection for IBD according to F1, area under curve (AUC), and false alarm rate (FAR).

An end-to-end heterogeneous graph representation learning-based framework for drug-target interaction prediction.

Accurately identifying potential drug-target interactions (DTIs) is a key step in drug discovery. Although many related experimental studies have been carried out for identifying DTIs in the past few decades, the biological experiment-based DTI identification is still timeconsuming and expensive. Therefore, it is of great significance to develop effective computational methods for identifying DTIs. In this paper, we develop a novel 'end-to-end' learning-based framework based on heterogeneous 'graph' convolutional networks for 'DTI' prediction called end-to-end graph (EEG)-DTI. Given a heterogeneous network containing multiple types of biological entities (i.e. drug, protein, disease, side-effect), EEG-DTI learns the low-dimensional feature representation of drugs and targets using a graph convolutional networks-based model and predicts DTIs based on the learned features. During the training process, EEG-DTI learns the feature representation of nodes in an end-to-end mode. The evaluation test shows that EEG-DTI performs better than existing state-of-art methods. The data and source code are available at: https://github.com/MedicineBiology-AI/EEG-DTI.

Social Network Embedding Method Basing Loop-free Path

Network embedding’s goal is to learn low-dimensional feature representations of nodes in a network and use the learned features in various analysis tasks of the network, such as node classification, link prediction, community discovery and recommendation. Existing network embedding methods do not make sufficient use of higher-order structural information in social networks and are not ideal for application in social networks. A Loop-free Path Combined with Attributes Network Embedding (LFNE) method is proposed to solve these problems. The algorithm first calculates the node higher-order structure similarity based on the loop-free paths between nodes, eliminates the influence of loop paths and large degree nodes on node structure similarity, and enables the network embedding method to better fuse the higher-order structure information of social networks. The node structural similarity is finally applied to the low-dimensional feature representation of the learned nodes in the stacked denoising autoencoder. Experimental comparisons with representative algorithms in recent years on three social network datasets show that the LFNE algorithm can achieve relatively significant results in node classification and link prediction experiments, with better network embedding performance.

A Random Fourier Feature Method for Emulating Computer Models With Gradient Information

Computer models with gradient information are increasingly used in engineering and science. The gradient-enhanced Gaussian process emulator can be used for emulating such models. Because the size of the covariance matrix increases proportionally with the dimension of inputs and the sample size, it is computationally challenging to fit such an emulator for large datasets. We propose a random Fourier feature method to mitigate this difficulty. The key idea of the proposed method is to employ random Fourier features to obtain an easily computable, low-dimensional feature representation for shift-invariant kernels involving gradients. The effectiveness of the proposed method is illustrated by several examples.