Low-dimensional Feature Representation Research Articles

Structural topology optimization based on deep learning

In the mechanical design of structures, traditional topology optimization methods involve numerous finite element iterative analyses, leading to a significant expenditure of computational resources. Therefore, the improved multi-scale gradient generative adversarial networks topology optimization technique is proposed. The topology optimization condition parameters are compressed into a low-dimensional latent space feature representation using the encoder, allowing the model to better extract features from these parameters. To speed up model training, the generator and discriminator networks use lightweight residual convolutional blocks. The hybrid attention mechanism extracts prominent region features from the topology optimization structure map. The model training process is guided by a multi-dimensional fusion loss function to enhance the quality of generated model samples. Finally, transferring the parameters of the low-resolution topology optimization model to the high-resolution model enables complete training on a limited amount of high-resolution topology optimization datasets. The experimental data on the low- and high-resolution topology optimization datasets demonstrate that, when compared to alternative methods, this method produces better-quality topology optimization structure maps. Additionally, it can generate high-resolution topology optimization structure maps in minimal time, enabling real-time topology optimization.

Graph based recurrent network for context specific synthetic lethality prediction.

The concept of synthetic lethality (SL) has been successfully used for targeted therapies. To further explore SL for cancer therapy, identifying more SL interactions with therapeutic potential are essential. Recently, graph neural network-based deep learning methods have been proposed for SL prediction, which reduce the SL search space of wet-lab based methods. However, these methods ignore that most SL interactions depend strongly on genetic context, which limits the application of the predicted results. In this study, we proposed a graph recurrent network-based model for specific context-dependent SL prediction (SLGRN). In particular, we introduced a Graph Recurrent Network-based encoder to acquire a context-specific, low-dimensional feature representation for each node, facilitating the prediction of novel SL. SLGRN leveraged gate recurrent unit (GRU) and it incorporated a context-dependent-level state to effectively integrate information from all nodes. As a result, SLGRN outperforms the state-of-the-arts models for SL prediction. We subsequently validate novel SL interactions under different contexts based on combination therapy or patient survival analysis. Through in vitro experiments and retrospective clinical analysis, we emphasize the potential clinical significance of this context-specific SL prediction model.

PiRNA-disease association prediction based on multi-channel graph variational autoencoder.

Piwi-interacting RNA (piRNA) is a type of non-coding small RNA that is highly expressed in mammalian testis. PiRNA has been implicated in various human diseases, but the experimental validation of piRNA-disease associations is costly and time-consuming. In this article, a novel computational method for predicting piRNA-disease associations using a multi-channel graph variational autoencoder (MC-GVAE) is proposed. This method integrates four types of similarity networks for piRNAs and diseases, which are derived from piRNA sequences, disease semantics, piRNA Gaussian Interaction Profile (GIP) kernel, and disease GIP kernel, respectively. These networks are modeled by a graph VAE framework, which can learn low-dimensional and informative feature representations for piRNAs and diseases. Then, a multi-channel method is used to fuse the feature representations from different networks. Finally, a three-layer neural network classifier is applied to predict the potential associations between piRNAs and diseases. The method was evaluated on a benchmark dataset containing 5,002 experimentally validated associations with 4,350 piRNAs and 21 diseases, constructed from the piRDisease v1.0 database. It achieved state-of-the-art performance, with an average AUC value of 0.9310 and an AUPR value of 0.9247 under five-fold cross-validation. This demonstrates the method's effectiveness and superiority in piRNA-disease association prediction.

Cube is a good form: Hyperspectral band selection via multi-dimensional and high-order structure preserved clustering

As an effective strategy for reducing the noisy and redundant information for hyperspectral imagery (HSI), hyperspectral band selection intends to select a subset of original hyperspectral bands, which boosts the subsequent different tasks. In this paper, we introduce a multi-dimensional high-order structure preserved clustering method for hyperspectral band selection, referred to as MHSPC briefly. By regarding original hyperspectral images as a tensor cube, we apply the tensor CP (CANDECOMP/PARAFAC) decomposition on it to exploit the multi-dimensional structural information as well as generate a low-dimensional latent feature representation. In order to capture the local geometrical structure along the spectral dimension, a graph regularizer is imposed on the new feature representation in the lower dimensional space. In addition, since the low rankness of HSIs is an important global property, we utilize a nuclear norm constraint on the latent feature representation matrix to capture the global data structure information. Different to most of previous clustering based hyperspectral band selection methods which vectorize each band as a vector without considering the 2-D spatial information, the proposed MHSPC can effectively capture the spatial structure as well as the spectral correlation of original hyperspectral cube in both local and global perspectives. An efficient alternatively updating algorithm with theoretical convergence guarantee is designed to solve the resultant optimization problem, and extensive experimental results on four benchmark datasets validate the effectiveness of the proposed MHSPC over other state-of-the-arts.

Combining EEG Features and Convolutional Autoencoder for Neonatal Seizure Detection.

Neonatal epilepsy is a common emergency phenomenon in neonatal intensive care units (NICUs), which requires timely attention, early identification, and treatment. Traditional detection methods mostly use supervised learning with enormous labeled data. Hence, this study offers a semi-supervised hybrid architecture for detecting seizures, which combines the extracted electroencephalogram (EEG) feature dataset and convolutional autoencoder, called Fd-CAE. First, various features in the time domain and entropy domain are extracted to characterize the EEG signal, which helps distinguish epileptic seizures subsequently. Then, the unlabeled EEG features are fed into the convolutional autoencoder (CAE) for training, which effectively represents EEG features by optimizing the loss between the input and output features. This unsupervised feature learning process can better combine and optimize EEG features from unlabeled data. After that, the pre-trained encoder part of the model is used for further feature learning of labeled data to obtain its low-dimensional feature representation and achieve classification. This model is performed on the neonatal EEG dataset collected at the University of Helsinki Hospital, which has a high discriminative ability to detect seizures, with an accuracy of 92.34%, precision of 93.61%, recall rate of 98.74%, and F1-score of 95.77%, respectively. The results show that unsupervised learning by CAE is beneficial to the characterization of EEG signals, and the proposed Fd-CAE method significantly improves classification performance.

A small-sample time-series signal augmentation and analysis method for quantitative assessment of bradykinesia in Parkinson's disease

Patients with Parkinson's disease (PD) usually have varying degrees of bradykinesia, and the current clinical assessment is mainly based on the Movement Disorder Society Unified PD Rating Scale, which can hardly meet the needs of objectivity and accuracy. Therefore, this paper proposed a small-sample time series classification method (DTW-TapNet) based on dynamic time warping (DTW) data augmentation and attentional prototype network. Firstly, for the problem of small sample sizes of clinical data, a DTW-based data merge method is used to achieve data augmentation. Then, the time series are dimensionally reorganized using random grouping, and convolutional operations are performed to learn features from multivariate time series. Further, attention mechanism and prototype learning are introduced to optimize the distance of the class prototype to which each time series belongs to train a low-dimensional feature representation of the time series, thus reducing the dependency on data volume. Clinical experiments were conducted to collect motion capture data of upper and lower limb movements from 36 patients with PD and eight healthy controls. For the upper limb movement data, the proposed method improved the classification accuracy, weighted precision, and kappa coefficient by 8.89%-15.56%, 9.22%-16.37%, and 0.13-0.23, respectively, compared with support vector machines, long short-term memory, and convolutional prototype network. For the lower limb movement data, the proposed method improved the classification accuracy, weighted precision, and kappa coefficient by 8.16%-20.41%, 10.01%-23.73%, and 0.12-0.28, respectively. The experiments and results show that the proposed method can objectively and accurately assess upper and lower limb bradykinesia in PD.

A deep learning based holistic diagnosis system for immunohistochemistry interpretation and molecular subtyping

BackgroundBreast cancer in different molecular subtypes, which is determined by the overexpression rates of human epidermal growth factor receptor 2 (HER2), estrogen receptor (ER), progesterone receptor (PR), and Ki67, exhibit distinct symptom characteristics and sensitivity to different treatment. The immunohistochemical method, one of the most common detecting tools for tumour markers, is heavily relied on artificial judgment and in clinical practice, with an inherent limitation in interpreting stability and operating efficiency. Here, a holistic intelligent breast tumour diagnosis system has been developed for tumour-markeromic analysis, combining the automatic interpretation and clinical suggestion. MethodsThe holistic intelligent breast tumour diagnosis system included two main modules. The interpreting modules were constructed based on convolutional neural network, for comprehensively extracting and analyzing the multi-features of immunostaining. Referring to the clinical classification criteria, the interpreting results were encoded in a low-dimensional feature representation in the subtyping module, to efficiently output a holistic detecting result of the critical tumour-markeromic with diagnosis suggestions on molecular subtypes. ResultsThe overexpression rates of HER2, ER, PR, and Ki67, as well as an effective determination of molecular subtypes were successfully obtained by this diagnosis system, with an average sensitivity of 97.6 % and an average specificity of 96.1 %, among those, the sensitivity and specificity for interpreting HER2 were up to 99.8 % and 96.9 %. ConclusionThe holistic intelligent breast tumour diagnosis system shows improved performance in the interpretation of immunohistochemical images over pathologist-level, which can be expected to overcome the limitations of conventional manual interpretation in efficiency, precision, and repeatability.

A novel microbe-drug association prediction model based on graph attention networks and bilayer random forest

BackgroundIn recent years, the extensive use of drugs and antibiotics has led to increasing microbial resistance. Therefore, it becomes crucial to explore deep connections between drugs and microbes. However, traditional biological experiments are very expensive and time-consuming. Therefore, it is meaningful to develop efficient computational models to forecast potential microbe-drug associations.ResultsIn this manuscript, we proposed a novel prediction model called GARFMDA by combining graph attention networks and bilayer random forest to infer probable microbe-drug correlations. In GARFMDA, through integrating different microbe-drug-disease correlation indices, we constructed two different microbe-drug networks first. And then, based on multiple measures of similarity, we constructed a unique feature matrix for drugs and microbes respectively. Next, we fed these newly-obtained microbe-drug networks together with feature matrices into the graph attention network to extract the low-dimensional feature representations for drugs and microbes separately. Thereafter, these low-dimensional feature representations, along with the feature matrices, would be further inputted into the first layer of the Bilayer random forest model to obtain the contribution values of all features. And then, after removing features with low contribution values, these contribution values would be fed into the second layer of the Bilayer random forest to detect potential links between microbes and drugs.ConclusionsExperimental results and case studies show that GARFMDA can achieve better prediction performance than state-of-the-art approaches, which means that GARFMDA may be a useful tool in the field of microbe-drug association prediction in the future. Besides, the source code of GARFMDA is available at https://github.com/KuangHaiYue/GARFMDA.git

Hyperspectral image band selection method combining the variation degree of salient features

ABSTRACTBand selection (BS) work for hyperspectral remote sensing images (HRSIs) has attracted increasing attention from researchers. In the HRSI, the salient features of each band always include the more important discriminative features. However, most existing BS methods always have difficulty in simultaneously performing local-global consistency analysis on salient features in bands of HRSIs. In addition, these methods also ignore the consideration of the diversity of band information to some extent when selecting a band subset. To tackle these problems, a novel unsupervised hyperspectral BS method combining the variation degree of salient features (UBSF) is proposed in this article. UBSF first promotes the singular value decomposition method to analyse the salient features in bands and develops a new low-dimensional local-global salient feature representation (NLR) method for discriminative feature representations of the bands. The NLR method combines the variation degree of the features of the significant representative patches of each band in their corresponding significant feature subspaces to achieve a local-global consistency analysis of salient features of each band of HRSIs. Next, UBSF proposes a new two-stage strategy (TSS) to preserve the diversity of BS results as much as possible. In TSS, we propose a novel similarity criterion, which can make the highly similar band subsets with near information amount in the overall bands adaptively compressed according to the obtained similarity results of bands further. Then, the band subset with the most information is selected as the final BS results in the obtained reduced-dimensional HRSI. Finally, the validity of the UBSF method is verified based on three measured HRSI datasets. The experimental results show that the proposed UBSF is superior to other state-of-the-art BS methods.

Drug-target interactions prediction based on network topology feature representation embedded deep forest

Identifying drug-target interactions (DTIs) is instructive in drug design and disease treatment. Existing studies typically used the properties of nodes (drug chemical structure and protein sequence) to construct drug and target features while ignoring the influence of network topology information on the prediction of DTIs. In this study, a hybrid computation model is proposed to predict DTIs based on the network topological feature representation embedded the deep forest model (NTFRDF). The main idea is to capture the topological differences by learning the low-dimensional feature representation of drugs and targets from the heterogeneous network. In addition, the multi-similarity fusion strategy is proposed to mine hidden useful information in the known DTIs from multi-view to enrich network features of the heterogeneous network. Based on the deep forest framework, the performance of the proposed method is examined on four benchmark datasets. Our experimental results verify that the proposed method is competitive compared with some existing DTIs prediction models.

Conditional Independence Induced Unsupervised Domain Adaptation

Learning domain-adaptive features is important to tackle the dataset bias problem, where data distributions in the labeled source domain and the unlabeled target domain can be different. The critical issue is to identify and then reduce the redundant information including class-irrelevant and domain-specific features. In this paper, a conditional independence induced unsupervised domain adaptation (CIDA) method is proposed to tackle the challenges. It aims to find the low-dimensional and transferable feature representation of each observation, namely the latent variable in the domain-adaptive subspace. Technically, two mutual information terms are optimized at the same time. One is the mutual information between the latent variable and the class label, and the other is the mutual information between the latent variable and the domain label. Note that the key module can be approximately reformulated as a conditional independence/dependence based optimization problem, and thus, it has a probabilistic interpretation with the Gaussian process. Temporary labels of the target samples and the model parameters are alternatively optimized. The objective function can be incorporated with deep network architectures, and the algorithm is implemented iteratively in an end-to-end manner. Extensive experiments are conducted on several benchmark datasets, and the results show effectiveness of CIDA.

Improving the Accuracy of Coal-Rock Dynamic Hazard Anomaly Detection Using a Dynamic Threshold and a Depth Auto-Coding Gaussian Hybrid Model

A coal-rock dynamic disaster is a rapid instability and failure process with dynamic effects and huge disastrous consequences that occurs in coal-rock mass under mining disturbance. Disasters lead to catastrophic consequences, such as mine collapse, equipment damage, and casualties. Early detection can prevent the occurrence of disasters. However, due to the low accuracy of anomaly detection, disasters still occur frequently. In order to improve the accuracy of anomaly detection for coal-rock dynamic disasters, this paper proposes an anomaly detection method based on a dynamic threshold and a deep self-encoded Gaussian mixture model. First, pre-mining data were used as the initial threshold, and the subsequent continuously arriving flow data were used to dynamically update the threshold to solve the impact of artificially setting the threshold on the detection accuracy. Second, feature dimensionality reduction and reorganization of the data were carried out, and low-dimensional feature representation and feature reconstruction error modeling were used to solve the difficulty of extracting features from high-dimensional data in real time. Finally, through the end-to-end optimization calculation of the energy probability distribution between different categories for anomaly detection, the problem that key abnormal information may be lost due to dimensionality reduction was solved and accurate detection of monitoring data was realized. Experimental results showed that this method has better performance than other methods.

GACNNMDA: a computational model for predicting potential human microbe-drug associations based on graph attention network and CNN-based classifier

As new drug targets, human microbes are proven to be closely related to human health. Effective computational methods for inferring potential microbe-drug associations can provide a useful complement to conventional experimental methods and will facilitate drug research and development. However, it is still a challenging work to predict potential interactions for new microbes or new drugs, since the number of known microbe-drug associations is very limited at present. In this manuscript, we first constructed two heterogeneous microbe-drug networks based on multiple measures of similarity of microbes and drugs, and known microbe-drug associations or known microbe-disease-drug associations, respectively. And then, we established two feature matrices for microbes and drugs through concatenating various attributes of microbes and drugs. Thereafter, after taking these two feature matrices and two heterogeneous microbe-drug networks as inputs of a two-layer graph attention network, we obtained low dimensional feature representations for microbes and drugs separately. Finally, through integrating low dimensional feature representations with two feature matrices to form the inputs of a convolutional neural network respectively, a novel computational model named GACNNMDA was designed to predict possible scores of microbe-drug pairs. Experimental results show that the predictive performance of GACNNMDA is superior to existing advanced methods. Furthermore, case studies on well-known microbes and drugs demonstrate the effectiveness of GACNNMDA as well. Source codes and supplementary materials are available at: https://github.com/tyqGitHub/TYQ/tree/master/GACNNMDA

Deeply Supervised Subspace Learning for Cross-Modal Material Perception of Known and Unknown Objects

In order to help robots understand and perceive an object's properties during noncontact robot-object interaction, this article proposes a deeply supervised subspace learning method. In contrast to previous work, it takes the advantages of low noise and fast response of noncontact sensors and extracts novel contactless feature information to retrieve cross-modal information, so as to estimate and infer material properties of known as well as unknown objects. Specifically, a depth-supervised subspace cross-modal material retrieval model is trained to learn a common low-dimensional feature representation to capture the clustering structure among different modal features of the same class of objects. Meanwhile, all of unknown objects are accurately perceived by an energy-based model, which forces an unlabeled novel object's features to be mapped beyond the common low-dimensional features. The experimental results show that our approach is effective in comparison with other advanced methods.

Detecting corporate financial fraud via two-stage mapping in joint temporal and financial feature domain

Corporate financial fraud (CFF) has great harmful influences on firms, stakeholders, and capital markets. For CFF detection, firm-year or firm-quarter data are frequently utilized to construct financial feature variables and they have been proved to be effective. However, the firm-year or firm-quarter data are sampled at a single time step only, and the information in the temporal domain has not been fully exploited. As such, we model the corporate financial data into a three-dimensional (3-D) data cube that incorporates both temporal and financial feature domain information. This information can be jointly processed by a two-stage mapping scheme. The first stage maps the joint domain data into discrete sparsity patterns, and the second stage learn some high-level low-dimensional feature representation from the first stage output. The proposed two-stage mapping scheme is tested with real financial data obtained from China Stock Market and Accounting Research (CSMAR) database, and is found to work well. We find that the detection performance is dramatically improved when the accumulated number of quarters increased, and a 3-year, i.e., 12-quarter processing interval will be enough for CFF detection. This finding is important because it is intuitive and similar joint processing interval can be found on other data set. We also find out that joint temporal and financial feature domain processing is helpful for CFF detection. Furthermore, the proposed two-stage mapping scheme for joint processing in temporal and financial feature domain is intuitive for developing intelligent CFF detection systems.

Toward Robust Fault Identification of Complex Industrial Processes Using Stacked Sparse-Denoising Autoencoder With Softmax Classifier.

This article proposes a robust end-to-end deep learning-induced fault recognition scheme by stacking multiple sparse-denoising autoencoders with a Softmax classifier, called stacked spare-denoising autoencoder (SSDAE)-Softmax, for the fault identification of complex industrial processes (CIPs). Specifically, sparse denoising autoencoder (SDAE) is established by integrating a sparse AE (SAE) with a denoising AE (DAE) for the low-dimensional but intrinsic feature representation of the CIP monitoring data (CIPMD) with possible noise contamination. SSDAE-Softmax is established by stacking multiple SDAEs with a layerwise pretraining procedure, and a Softmax classifier with a global fine-tuning strategy. Furthermore, SSDAE-Softmax hyperparameters are optimized by a relatively new global optimization algorithm, referred to as the state transition algorithm (STA). Benefiting from the deep learning-based feature representation scheme with the STA-based hyperparameter optimization, the underlying intrinsic characteristics of CIPMD can be learned automatically and adaptively for accurate fault identification. A numeric simulation system, the benchmark Tennessee Eastman process (TEP), and a real industrial process, that is, the continuous casting process (CCP) from a top steel plant of China, are used to validate the performance of the proposed method. Experimental results show that the proposed SSDAE-Softmax model can effectively identify various process faults, and has stronger robustness and adaptability against the noise interference in CIPMD for the process monitoring of CIPs.

Deep Embedded Clustering Framework for Mixed Data

Deep embedded clustering (DEC) is a representative clustering algorithm that leverages deep-learning frameworks. DEC jointly learns low-dimensional feature representations and optimizes the clustering goals but only works with numerical data. However, in practice, the real-world data to be clustered includes not only numerical features but also categorical features that DEC cannot handle. In addition, if the difference between the soft assignment and target values is large, DEC applications may suffer from convergence problems. In this study, to overcome these limitations, we propose a deep embedded clustering framework that can utilize mixed data to increase the convergence stability using soft-target updates; a concept that is borrowed from an improved deep Q learning algorithm used in reinforcement learning. To evaluate the performance of the framework, we utilized various benchmark datasets composed of mixed data and empirically demonstrated that our approach outperformed existing clustering algorithms in most standard metrics. To the best of our knowledge, we state that our work achieved state-of-the-art performance among its contemporaries in this field.

Drug repositioning based on heterogeneous networks and variational graph autoencoders.

Predicting new therapeutic effects (drug repositioning) of existing drugs plays an important role in drug development. However, traditional wet experimental prediction methods are usually time-consuming and costly. The emergence of more and more artificial intelligence-based drug repositioning methods in the past 2years has facilitated drug development. In this study we propose a drug repositioning method, VGAEDR, based on a heterogeneous network of multiple drug attributes and a variational graph autoencoder. First, a drug-disease heterogeneous network is established based on three drug attributes, disease semantic information, and known drug-disease associations. Second, low-dimensional feature representations for heterogeneous networks are learned through a variational graph autoencoder module and a multi-layer convolutional module. Finally, the feature representation is fed to a fully connected layer and a Softmax layer to predict new drug-disease associations. Comparative experiments with other baseline methods on three datasets demonstrate the excellent performance of VGAEDR. In the case study, we predicted the top 10 possible anti-COVID-19 drugs on the existing drug and disease data, and six of them were verified by other literatures.

An Exploration: Alzheimer’s Disease Classification Based on Spectral Matching of Shape Features

In this paper, we use structural deformations to classify dementia patients. Firstly, surface meshes are recovered from MRI segmented hippocampal, and node-to-node interactions between all the surface meshes are constructed using a spectral matching approach. Then, to learn the low-dimensional feature representation, an enhanced version of the variational auto-encoder (VAE) is given to the vertex coordinates of the surface meshes. We describe a new strategy for increasing variational autoencoder performance (VAE). We designed a generative adversarial training (GAN) technique to train the VAE to generate realistic medical images and apply the deep feature consistency principle, ensuring that the VAE output and its related input images have identical features. A discriminator with a SoftMax layer is concurrently trained to distinguish people with Alzheimer's from healthy people. Studies on the ADNI dataset show that the proposed method can distinguish normal people from early AD/NC and AD/EMCI classes with low computational time and higher accuracy that outperforms the support vector machine (SVM) baseline approach. All the simulation results are carried out with the Anaconda tool.

Semantic-aware network embedding via optimized random walk and paragaraph2vec

The Deepwalk-based network embedding algorithm, which was inspired by the word2vec model, can quickly and efficiently learn a low-dimensional vector representation for each node in a complex network. However, the success of word2vec in natural language environments relies heavily on ubiquitous contextual semantics, and the Deepwalk model is less accurate than other deep models due to ignoring contextual semantics in complex networks. To solve this challenge, a semantic-aware network embedding algorithm is proposed in this paper, simply called SANE, via optimized random walk and paragaraph2vec. In the SANE, a semantic-aware random walk strategy is designed to unsupervised generate many semantic-bearing node sequences from a complex network as a training set. Then, based on the training set, the paragraph2vec model is optimized by considering both global semantics and local semantics for learning more accurate low-dimensional node feature representations. Experiments on multiple tasks and multiple networks show the advantages of our SANE algorithm.