Deep Autoencoder Neural Network Research Articles

Prediction of drug-induced cell viability by SAE-XGBoost algorithm based on LINCS-L1000 perturbation signal

Different cell lines have different perturbation signals in response to specific compounds, and it is important to predict cell viability based on these perturbation signals and to uncover the drug sensitivity hidden underneath the phenotype. We developed an SAE-XGBoost cell viability prediction algorithm based on the LINCS-L1000 perturbation signal. By matching and screening three major dataset, LINCS-L1000, CTRP and Achilles, a stacked autoencoder deep neural network was used to extract the gene information. These information were combined with the RW-XGBoost algorithm to predict the cell viability under drug induction, and then to complete drug sensitivity inference on the NCI60 and CCLE datasets. The model achieved good results compared to other methods with a Pearson correlation coefficient of 0.85. It was further validated on an independent dataset, corresponding to a Pearson correlation coefficient of 0.68. The results indicate that the proposed method can help discover novel and effective anti-cancer drugs for precision medicine.

Building Machine Learning systems for multi-atoms structures: CH3NH3PbI3 perovskite nanoparticles

In this study, we built a variety of Machine Learning (ML) systems over 23 different sizes of CH3NH3PbI3 perovskite nanoparticles (NPs) to predict the atoms in the NPs from their geometric locations. Our findings show that a specific type of ML algorithms, tree-based models which are Random Forest (RF), Extreme Gradient Boosting (XGBoost), Decision Trees (DT), can perfectly learn CH3NH3PbI3 perovskite NPs. Surprisingly, some popular ML algorithms such as Naive Bayes (NB), Support Vector Machines (SVM), Partial Least Squares (PLS), Regularized Logistic Regression (LR), Neural Networks (NN), Stacked Auto-Encoder Deep Neural Network (DNN), K-Nearest Neighbor (KNN) fail to learn CH3NH3PbI3 perovskite NPs.

Breath analysis based early gastric cancer classification from deep stacked sparse autoencoder neural network

Deep learning is an emerging tool, which is regularly used for disease diagnosis in the medical field. A new research direction has been developed for the detection of early-stage gastric cancer. The computer-aided diagnosis (CAD) systems reduce the mortality rate due to their effectiveness. In this study, we proposed a new method for feature extraction using a stacked sparse autoencoder to extract the discriminative features from the unlabeled data of breath samples. A Softmax classifier was then integrated to the proposed method of feature extraction, to classify gastric cancer from the breath samples. Precisely, we identified fifty peaks in each spectrum to distinguish the EGC, AGC, and healthy persons. This CAD system reduces the distance between the input and output by learning the features and preserve the structure of the input data set of breath samples. The features were extracted from the unlabeled data of the breath samples. After the completion of unsupervised training, autoencoders with Softmax classifier were cascaded to develop a deep stacked sparse autoencoder neural network. In last, fine-tuning of the developed neural network was carried out with labeled training data to make the model more reliable and repeatable. The proposed deep stacked sparse autoencoder neural network architecture exhibits excellent results, with an overall accuracy of 98.7% for advanced gastric cancer classification and 97.3% for early gastric cancer detection using breath analysis. Moreover, the developed model produces an excellent result for recall, precision, and f score value, making it suitable for clinical application.

Coronavirus disease 2019 (COVID-19): survival analysis using deep learning and Cox regression model.

Coronavirus (COVID-19) is one of the most serious problems that has caused stopping the wheel of life all over the world. It is widely spread to the extent that hospital places are not available for all patients. Therefore, most hospitals accept patients whose recovery rate is high. Machine learning techniques and artificial intelligence have been deployed for computing infection risks, performing survival analysis and classification. Survival analysis (time-to-event analysis) is widely used in many areas such as engineering and medicine. This paper presents two systems, Cox_COVID_19 and Deep_ Cox_COVID_19 that are based on Cox regression to study the survival analysis for COVID-19 and help hospitals to choose patients with better chances of survival and predict the most important symptoms (features) affecting survival probability. Cox_COVID_19 is based on Cox regression and Deep_Cox_COVID_19 is a combination of autoencoder deep neural network and Cox regression to enhance prediction accuracy. A clinical dataset for COVID-19 patients is used. This dataset consists of 1085 patients. The results show that applying an autoencoder on the data to reconstruct features, before applying Cox regression algorithm, would improve the results by increasing concordance, accuracy and precision. For Deep_ Cox_COVID_19 system, it has a concordance of 0.983 for training and 0.999 for testing, but for Cox_COVID_19 system, it has a concordance of 0.923 for training and 0.896 for testing. The most important features affecting mortality are, age, muscle pain, pneumonia and throat pain. Both Cox_COVID_19 and Deep_ Cox_COVID_19 prediction systems can predict the survival probability and present significant symptoms (features) that differentiate severe cases and death cases. But the accuracy of Deep_Cox_Covid_19 outperforms that of Cox_Covid_19. Both systems can provide definite information for doctors about detection and intervention to be taken, which can reduce mortality.

A network security situation assessment method based on adversarial deep learning

Aiming at the poor performance and flexibility of traditional assessment methods of network security in dealing with a large number of network attack data, this paper proposes a network security situation assessment method based on adversarial deep learning. Firstly, establish the Deep Autoencoder-Deep Neural Network (AEDNN) model based on Deep Autoencoder (DAE) and Deep Neural Network (DNN). Conduct feature learning on the DAE network and apply the DNN network as a network attacks classifier. In the training process, for considering the results of feature learning in DAE, this paper builds an adversarial training process by changing the training weights. Besides, to increase the performance of the model on the minority network attacks, the Under–Over Sampling Weighted (UOSW) algorithm is designed; Finally, conduct model testing and calculate the network security situation value. The compared results of other models show the proposed model is more accurate for identifying network attacks and can evaluate the network situation more comprehensively and flexibly.

Noninvasive blood glucose sensing by near-infrared spectroscopy based on PLSR combines SAE deep neural network approach

Near-infrared spectroscopy has been considered as one of the most effective methods for noninvasive blood glucose sensing. Due to the strong scattering of human tissues and the differences among individuals, the relationship between spectral data and blood glucose concentration is nonlinear. Therefore, the linear prediction model has limitations when modeling multiple human samples. The present paper proposes a hybrid model in order to improve the prediction accuracy and versatility of the method, which was based on integrated linear partial least square regression (PLSR) with the nonlinear stacked auto-encoder (SAE) deep neural network. In this work, the diffuse reflectance spectrum of the palm was measured at six different wavelengths in 19 healthy subjects. The prediction results of multiple samples demonstrated that the correlation coefficients of the PLSR-SAE model is improved from 0.3021 to 0.9216 on average, which significantly optimizes the prediction effect compared with the traditional PLSR model. In addition, the prediction accuracy of Support Vector Regression (SVR) model and PLSR-SAE model are 0.8243 and 0.9216 respectively. Furthermore, in Clarke error grid analysis, the PLSR-SAE model could achieve 97.96% of points in A region, which has demonstrated that the prediction accuracy of this noninvasive blood glucose detection method might meets the precision range of clinical laboratory standards. Furthermore, it shows the potential of combining linear and nonlinear regression models for noninvasive prediction of other blood components.

Deep Learning Algorithms in EEG Signal Decoding Application: A Review

In recent years, deep learning algorithms have been developed rapidly, and they are becoming a powerful tool in biomedical engineering. Especially, there has been an increasing focus on the use of deep learning algorithms for decoding physiological or pathological status of the brain from electroencephalographic (EEG). This paper overviews current application of deep learning algorithms in various EEG decoding tasks, and introduces commonly used algorithms, typical application scenarios, important progresses and existing problems. Firstly, the basic principles of deep learning algorithms used in EEG decoding is briefly described, including convolutional neural network, deep belief network, auto-encoder and recurrent neural network. In this paper, existing applications of deep learning on EEG is discussed, including brain-computer interfaces, cognitive neuroscience and diagnosis of brain disorders. Finally, this paper outlines some key problems that will be addressed in future applications of deep learning for EEG decoding, such as parameter selection, computational complexity, and the capability of generalization.

Synthesis and Analysis-By-Synthesis of Modulated Diplophonic Glottal Area Waveforms

Diplophonia is a type of disordered voice in which two simultaneous pitches are perceived. Most commonly in diplophonic voices, the vocal folds are divided into two parts that vibrate at different frequencies. The glottal area is the projected area of the space between the vocal folds. The glottal area in time is referred to as the glottal area waveform (GAW). The GAW is modeled for diplophonic voice by superimposing two partial GAWs (pGAWs) that are trains of single-peak pulses with different pulse frequencies, i.e., fundamental frequencies ( $f_o$ s). In current kinematic models of diplophonic vocal fold vibration, the pGAWs are assumed to be quasiperiodic. This assumption is mitigated here by modulating pulse-to-pulse cycle length and amplitude. Both random and deterministic modulations are considered. Deterministic modulations depend on the difference of the pGAWs’ instantaneous phases. Model GAWs are fitted to input GAWs using an analysis-by-synthesis approach which we refer to as ‘modulated pulse trains decomposition’ (MPD). MPD is shown to be applicable to diplophonic as well as to nondiplophonic types of dysphonia, which include multi-pulse patterns, random timing behaviours, and chaos. It is mostly robust against modulations but degraded by large random modulations. MPD is compared to a deep autoencoder neural network, and the WaveGlow neural network. In terms of time-domain fitting errors, MPD outperforms the other two approaches unless random modulations are large. MPD outperforms the best of the other two approaches by up to approximately 5 dB. For large random modulations, the deep autoencoder network achieves the smallest fitting errors. In terms of magnitude spectrum fitting errors, WaveGlow is superior except for natural input GAWs containing only nondiplophonic types of dysphonia. Also pulse timing errors are shown to be advantageous for MPD.

DeepClean: Self-Supervised Artefact Rejection for Intensive Care Waveform Data Using Deep Generative Learning.

Waveform physiological data are important in the treatment of critically ill patients in the intensive care unit. Such recordings are susceptible to artefacts, which must be removed before the data can be reused for alerting or reprocessed for other clinical or research purposes. Accurate removal of artefacts reduces bias and uncertainty in clinical assessment, as well as the false positive rate of ICU alarms, and is therefore a key component in providing optimal clinical care. In this work, we present DeepClean, a prototype self-supervised artefact detection system using a convolutional variational autoencoder deep neural network that avoids costly and painstaking manual annotation, requiring only easily obtained 'good' data for training. For a test case with invasive arterial blood pressure, we demonstrate that our algorithm can detect the presence of an artefact within a 10s sample of data with sensitivity and specificity around 90%. Furthermore, DeepClean was able to identify regions of artefacts within such samples with high accuracy, and we show that it significantly outperforms a baseline principal component analysis approach in both signal reconstruction and artefact detection. DeepClean learns a generative model and therefore may also be used for imputation of missing data.

Robot skill learning in latent space of a deep autoencoder neural network

Just like humans, robots can improve their performance by practicing, i.e. by performing the desired behavior many times and updating the underlying skill representation using the newly gathered data. In this paper, we propose to implement robot practicing by applying statistical and reinforcement learning (RL) in a latent space of the selected skill representation. The latent space is computed by a deep autoencoder neural network, with the data to train the network generated in simulation. However, we show that the resulting latent space representation is useful also for learning on a real robot.Our simulation and real-world results demonstrate that by exploiting the latent space of the underlying motor skill representation, a significant reduction of the amount of data needed for effective learning by Gaussian Process Regression (GPR) can be achieved. Similarly, the number of RL epochs can be significantly reduced. Finally, it is evident from our results that an autoencoder-based latent space is more effective for these purposes than a latent space computed by principal component analysis.

Network traffic classification using deep convolutional recurrent autoencoder neural networks for spatial–temporal features extraction

The right choice of features to be extracted from individual or aggregated observations is an extremely critical factor for the success of modern network traffic classification approaches based on machine learning. Such activity, usually in charge of the designers of the classification scheme is strongly related to their experience and skills, and definitely characterizes the whole approach, implementation strategy as well as its performance. The main aim of this work is supporting this process by mining new and more expressive, meaningful and discriminating features from the basic ones without human intervention. For this purpose, a novel autoencoder-based deep neural network architecture is proposed where multiple autoencoders are embedded with convolutional and recurrent neural networks to elicit relevant knowledge about the relations existing among the basic features (spatial-features) and their evolution over time (temporal-features). Such knowledge, consisting in new properties that are not immediately evident and better represent the most hidden and representative traffic dynamics can be successfully exploited by machine learning-based classifiers. Different network combinations are analyzed both from a theoretical perspective, and through specific performance evaluation experiments on a real network traffic dataset. We show that the traffic classifier obtained by stacking the autoencoder with a fully-connected neural network, achieves up to a 28% improvement in average accuracy over state-of-the-art machine learning-based approaches, up to a 10% over pure convolutional and recurrent stacked neural networks, and 18% over pure feed-forward networks. It is also able to maintain high accuracy even in the presence of unbalanced training datasets.

Application of supervised learning to validation of damage detection

Unsupervised learning methods are effective and suitable tools for damage detection. The main reason for the popularity of these methods in structural health monitoring originates from the fact that the process of learning can be implemented by information of the only normal condition called training data. In contrast, supervised learning methods require information of both normal and current conditions for the process of interest. Because civil engineering structures are expensive and complex, it is not reasonable and economical to impose intentional damage on providing training data. Hence, it is not simple to directly exploit supervised learning techniques in structural health monitoring. To deal with this limitation, this article proposes a novel two-level strategy including three algorithms for using the concepts of both unsupervised learning and supervised learning. The major contribution of this strategy is to consider supervised learning as a validation tool for damage detection. First, the results of damage detection are obtained from two unsupervised learning methods developed by Mahalanobis squared distance and a deep autoencoder neural network in the first two algorithms of the proposed strategy. The main objective is to separate accurate and confusing results of damage detection based on Type I and Type II errors. Second, the confusing results are fed into the third algorithm to train a classifier and compute their classification margins for making the final decision and validating damage detection. The effectiveness and applicability of the proposed strategy are assessed by a numerical concrete beam and an experimental laboratory frame. Results show that this strategy with the aid of the Naive Bayes classifier enables the unsupervised learning methods to make accurate decisions.

Diagnosis of heart diseases by a secure Internet of Health Things system based on Autoencoder Deep Neural Network

Objective of this study is to introduce a secure IoHT system, which acts as a clinical decision support system with the diagnosis of cardiovascular diseases. In this sense, it was emphasized that the accuracy rate of diagnosis (classification) can be improved via deep learning algorithms, by needing no hybrid-complex models, and a secure data processing can be achieved with a multi-authentication and Tangle based approach. In detail, heart sounds were classified with Autoencoder Neural Networks (AEN) and the IoHT system was built for supporting doctors in real-time. For developing the diagnosis infrastructure by the AEN, PASCAL B-Training and Physiobank-PhysioNet A-Training heart sound datasets were used accordingly. For the PASCAL dataset, the AEN provided a diagnosis-classification performance with the accuracy of 100%, sensitivity of 100%, and the specificity of 100% whereas the rates were respectively 99.8%, 99.65%, and 99.13% for the PhysioNet dataset. It was seen that the findings by the developed AEN based solution were better than the alternative solutions from the literature. Additionally, usability of the whole IoHT system was found positive by the doctors, and according to the 479 real-case applications, the system was able to achieve accuracy rates of 96.03% for normal heart sounds, 91.91% for extrasystole, and 90.11% for murmur. In terms of security approach, the system was also robust against several attacking methods including synthetic data impute as well as trying to penetrating to the system via central system or mobile devices.

Semisupervised Generative Autoencoder for Single-Cell Data.

Single-cell transcriptomics offers a tool to study the diversity of cell phenotypes through snapshots of the abundance of mRNA in individual cells. Often there is additional information available besides the single-cell gene expression counts, such as bulk transcriptome data from the same tissue, or quantification of surface protein levels from the same cells. In this study, we propose models based on the Bayesian deep learning approach, where protein quantification, available as CITE-seq counts, from the same cells is used to constrain the learning process, thus forming a SemI-SUpervised generative Autoencoder (SISUA) model. The generative model is based on the deep variational autoencoder (VAE) neural network architecture.

Anomaly detection via blockchained deep learning smart contracts in industry 4.0

The complexity of threats in the ever-changing environment of modern industry is constantly increasing. At the same time, traditional security systems fail to detect serious threats of increasing depth and duration. Therefore, alternative, intelligent solutions should be used to detect anomalies in the operating parameters of the infrastructures concerned, while ensuring the anonymity and confidentiality of industrial information. Blockchain is an encrypted, distributed archiving system designed to allow for the creation of real-time log files that are unequivocally linked. This ensures the security and transparency of transactions. This research presents, for the first time in the literature, an innovative Blockchain Security Architecture that aims to ensure network communication between traded Industrial Internet of Things devices, following the Industry 4.0 standard and based on Deep Learning Smart Contracts. The proposed smart contracts are implementing (via computer programming) a bilateral traffic control agreement to detect anomalies based on a trained Deep Autoencoder Neural Network. This architecture enables the creation of a secure distributed platform that can control and complete associated transactions in critical infrastructure networks, without the intervention of a single central authority. It is a novel approach that fuses artificial intelligence in the Blockchain, not as a supportive framework that enhances the capabilities of the network, but as an active structural element, indispensable and necessary for its completion.

Seismic random noise suppression using deep convolutional autoencoder neural network

Due to human or environmental factors, random noise will inevitably be introduced during seismic data acquisition. Contaminated seismic data seriously affect subsequent seismic data processing and imaging. In this paper, we propose a deep convolutional autoencoder neural network for denoising, which consists of encoding and decoding frameworks. The encoding framework consisting of convolution layers and pooling layers captures features of seismic data while eliminating noise. The decoding framework consisting of convolution layers and upsampling layers is capable of enlarging the feature map and recovering the small scale features of the seismic data. In order to represent the characteristics of seismic data optimally, we introduce L1 regularization to constrain the weights. In order to learn multi-scale features of seismic data, we use the multiple filters with different sizes in both encoding and decoding frameworks. The proposed method is tested on synthetic and field cases. The experimental results show that the proposed denoising method can suppress random noise effectively.

Granular space, knowledge-encoded deep learning architecture and remote sensing image classification

Hand-crafted features of remotely sensed (RS) image require the involvement of expensive human experts for classification. This factor motivates for designing the classification model with representative feature learning-based deep architecture to automate the feature extraction process and improve the generalization capability of the model. With this reasoning, we propose a deep auto-encoder neural network (NN) architecture with knowledge-encoded granular space for the classification of RS images. The network works with wavelet-rough granulated spaces and its architecture is designed with the encoded domain knowledge that strategically initializes the network parameters. Mostly, the learning time and performance of deep auto-encoders are persuaded by randomly selected weights and thus, we aim here to minimize these efforts with the domain knowledge. Neighborhood rough sets (NRS) are used to encode the domain knowledge and explore the contextual information for improved decision. We perform the knowledge-encoding operation for all stages of the auto-encoder. The proposed model thus exploits the mutual merits of deep network, wavelet-rough granular space and knowledge-encoding method. Comparative experimental results with multispectral and hyperspectral RS images demonstrate the superiority of our model to the related advanced methods.

SAEROF: an ensemble approach for large-scale drug-disease association prediction by incorporating rotation forest and sparse autoencoder deep neural network

Drug-disease association is an important piece of information which participates in all stages of drug repositioning. Although the number of drug-disease associations identified by high-throughput technologies is increasing, the experimental methods are time consuming and expensive. As supplement to them, many computational methods have been developed for an accurate in silico prediction for new drug-disease associations. In this work, we present a novel computational model combining sparse auto-encoder and rotation forest (SAEROF) to predict drug-disease association. Gaussian interaction profile kernel similarity, drug structure similarity and disease semantic similarity were extracted for exploring the association among drugs and diseases. On this basis, a rotation forest classifier based on sparse auto-encoder is proposed to predict the association between drugs and diseases. In order to evaluate the performance of the proposed model, we used it to implement 10-fold cross validation on two golden standard datasets, Fdataset and Cdataset. As a result, the proposed model achieved AUCs (Area Under the ROC Curve) of Fdataset and Cdataset are 0.9092 and 0.9323, respectively. For performance evaluation, we compared SAEROF with the state-of-the-art support vector machine (SVM) classifier and some existing computational models. Three human diseases (Obesity, Stomach Neoplasms and Lung Neoplasms) were explored in case studies. As a result, more than half of the top 20 drugs predicted were successfully confirmed by the Comparative Toxicogenomics Database(CTD database). This model is a feasible and effective method to predict drug-disease correlation, and its performance is significantly improved compared with existing methods.

Detection of glucose concentration in a turbid medium using a stacked auto-encoder deep neural network

In order to detect the components of a turbid medium, this paper proposes a glucose concentration reconstruction method based on a stacked auto-encoder (SAE) deep neural network. Analysis of different optical properties is performed using multiple diffused reflection spectroscopy and multiple source-detector separation (SDS). In this experiment, a 20% intralipid solution was used to prepare 4%, 5% and 10% intralipid solutions as the research object. At a certain glucose concentration, thirty source-detector separations of diffused reflection spectral signals within the 0.47–4.095 mm range (0.125 mm interval) from the incident position were detected, and a SAE deep neural network was used for modeling and predicting glucose concentration under multiple spectra. The root mean square error of prediction (RMSEP) of the SAE deep neural network using the 4% and 5% intralipid solutions decreased by approximately 26.42% compared with the results using only the 4% intralipid solutions by partial least squares regression (PLSR) method. Moreover, the RMSEP of the SAE deep neural network using the 4% and 5% intralipid solutions decreased by approximately 34.25% compared with the PLSR method using the 5% intralipid solution sample. These results show that the reconstruction accuracy of the SAE deep neural network is higher than the traditional PLSR method, which proves that the SAE neural network is highly suitable for prediction of turbid medium concentration.

Pioneer dataset and automatic recognition of Urdu handwritten characters using a deep autoencoder and convolutional neural network

Automatic recognition of Urdu handwritten digits and characters, is a challenging task. It has applications in postal address reading, bank's cheque processing, and digitization and preservation of handwritten manuscripts from old ages. While there exists a significant work for automatic recognition of handwritten English characters and other major languages of the world, the work done for Urdu lan-guage is extremely insufficient. This paper has two goals. Firstly, we introduce a pioneer dataset for handwritten digits and characters of Urdu, containing samples from more than 900 individuals. Secondly, we report results for automatic recog-nition of handwritten digits and characters as achieved by using deep auto-encoder network and convolutional neural network. More specifically, we use a two-layer and a three-layer deep autoencoder network and convolutional neural network and evaluate the two frameworks in terms of recognition accuracy. The proposed framework of deep autoencoder can successfully recognize digits and characters with an accuracy of 97% for digits only, 81% for characters only and 82% for both digits and characters simultaneously. In comparison, the framework of convolutional neural network has accuracy of 96.7% for digits only, 86.5% for characters only and 82.7% for both digits and characters simultaneously. These frameworks can serve as baselines for future research on Urdu handwritten text.