Deep Convolutional Autoencoders Research Articles

Deep feature learning of in-cylinder flow fields to analyze cycle-to-cycle variations in an SI engine

Machine learning (ML) models based on a large data set of in-cylinder flow fields of an IC engine obtained by high-speed particle image velocimetry allow the identification of relevant flow structures underlying cycle-to-cycle variations of engine performance. To this end, deep feature learning is employed to train ML models that predict cycles of high and low in-cylinder maximum pressure. Deep convolutional autoencoders are self-supervised-trained to encode flow field features in low dimensional latent space. Without the limitations ascribable to manual feature engineering, ML models based on these learned features are able to classify high energy cycles already from the flow field during late intake and the compression stroke as early as 290 crank angle degrees before top dead center ([Formula: see text]) with a mean accuracy above chance level. The prediction accuracy from [Formula: see text] to [Formula: see text] is comparable to baseline ML approaches utilizing an extensive set of engineered features. Relevant flow structures in the compression stroke are revealed by feature analysis of ML models and are interpreted using conditional averaged flow quantities. This analysis unveils the importance of the horizontal velocity component of in-cylinder flows in predicting engine performance. Combining deep learning and conventional flow analysis techniques promises to be a powerful tool for ultimately revealing high-level flow features relevant to the prediction of cycle-to-cycle variations and further engine optimization.

Deep learning-based reduced order models in cardiac electrophysiology.

Predicting the electrical behavior of the heart, from the cellular scale to the tissue level, relies on the numerical approximation of coupled nonlinear dynamical systems. These systems describe the cardiac action potential, that is the polarization/depolarization cycle occurring at every heart beat that models the time evolution of the electrical potential across the cell membrane, as well as a set of ionic variables. Multiple solutions of these systems, corresponding to different model inputs, are required to evaluate outputs of clinical interest, such as activation maps and action potential duration. More importantly, these models feature coherent structures that propagate over time, such as wavefronts. These systems can hardly be reduced to lower dimensional problems by conventional reduced order models (ROMs) such as, e.g., the reduced basis method. This is primarily due to the low regularity of the solution manifold (with respect to the problem parameters), as well as to the nonlinear nature of the input-output maps that we intend to reconstruct numerically. To overcome this difficulty, in this paper we propose a new, nonlinear approach relying on deep learning (DL) algorithms—such as deep feedforward neural networks and convolutional autoencoders—to obtain accurate and efficient ROMs, whose dimensionality matches the number of system parameters. We show that the proposed DL-ROM framework can efficiently provide solutions to parametrized electrophysiology problems, thus enabling multi-scenario analysis in pathological cases. We investigate four challenging test cases in cardiac electrophysiology, thus demonstrating that DL-ROM outperforms classical projection-based ROMs.

Performance Analysis of Deep Convolutional Autoencoders with Different Patch Sizes for Change Detection from Burnt Areas

Fire is one of the primary sources of damages to natural environments globally. Estimates show that approximately 4 million km2 of land burns yearly. Studies have shown that such estimates often underestimate the real extent of burnt land, which highlights the need to find better, state-of-the-art methods to detect and classify these areas. This study aimed to analyze the use of deep convolutional Autoencoders in the classification of burnt areas, considering different sample patch sizes. A simple Autoencoder and the U-Net and ResUnet architectures were evaluated. We collected Landsat 8 OLI+ data from three scenes in four consecutive dates to detect the changes specifically in the form of burnt land. The data were sampled according to four different sampling strategies to evaluate possible performance changes related to sampling window sizes. The training stage used two scenes, while the validation stage used the remaining scene. The ground truth change mask was created using the Normalized Burn Ratio (NBR) spectral index through a thresholding approach. The classifications were evaluated according to the F1 index, Kappa index, and mean Intersection over Union (mIoU) value. Results have shown that the U-Net and ResUnet architectures offered the best classifications with average F1, Kappa, and mIoU values of approximately 0.96, representing excellent classification results. We have also verified that a sampling window size of 256 by 256 pixels offered the best results.

A Deep Learning Approach for Identifying User Communities Based on Geographical Preferences and Its Applications to Urban and Environmental Planning

Understanding human mobility plays a vital role in urban and environmental planning as cities continue to grow. Ubiquitous geo-location, localization technology, and availability of big-data-ready computing infrastructure have enabled the development of more sophisticated models to characterize human mobility in urban areas. In this work, our main goal is to extract spatio-temporal features that characterize user mobility and, based on the similarity of these features, identify user communities . To this end, we propose a novel approach that leverages image processing techniques to represent user geographical preferences as images and then apply deep convolutional autoencoders to extract latent spatio-temporal mobility features from these images. These features are then fed to a clustering algorithm that identifies the underlying community structures. We use a diverse urban mobility dataset to validate the proposed framework. Our results show that the proposed framework is able to significantly increase the similarity between intra-community nodes (by up to 107%) as well as dissimilarity between inter-community nodes (up to 54%) when compared against no pre-processing of the datasets, i.e without pre-processing the datasets through any feature fusion method. Moreover, it was also able to reach up to 100% improvement when compared against community identification using Principal Component Analysis (PCA). Our results also show that the proposed approach yields significant increase in contact time amongst users belonging to the same community, by up to 80% when compared to the average contact time when not considering community structures, and by up to 150% when compared to the baseline. To the best of our knowledge, our proposal is the first to consider deep convolutional autoencoding to perform automatic extraction of non-linear spatio-temporal mobility features characterizing individual users from raw mobility datasets with the goal of identifying user communities.

Hyperspectral Unmixing Using Deep Convolutional Autoencoders in a Supervised Scenario

Hyperspectral unmixing (HSU) is an essential technique that aims to address the mixed pixels problem in hyperspectral imagery via estimating the abundance of each endmember at every pixel given the endmembers. This article introduces two approaches intending to solve the challenge of the mixed pixels using deep convolutional autoencoders (DCAEs), namely pixel-based DCAE, and cube-based DCAE. The former estimates abundances with the help of only spectral information, while the latter utilizes both spectral and spatial information which results in better unmixing performance. In the proposed frameworks, the weights of the decoder are set equal to the endmembers in order to address the issue in a supervised scenario. The proposed frameworks are also adapted to the VGG-Net that proved increasing depth with small convolution filters ( $\text{3}\times \text{3}$ ) leads to a considerable improvement. In other words, inspired by this idea, we utilize small and fixed kernels of size 3 in all layers of both proposed frameworks. The network is trained via the spectral information divergence objective function, and the dropout and regularization techniques are utilized to prevent overfitting. The superiority of the proposed frameworks is proven via conducting some experiments on both synthetic and real hyperspectral datasets and drawing a comparison with state-of-the-art methods. Moreover, the quantitative and visual evaluation of the proposed frameworks indicate the necessity of integrating spatial information into the HSU.

DeepFall: Non-Invasive Fall Detection with Deep Spatio-Temporal Convolutional Autoencoders.

Human falls rarely occur; however, detecting falls is very important from the health and safety perspective. Due to the rarity of falls, it is difficult to employ supervised classification techniques to detect them. Moreover, in these highly skewed situations, it is also difficult to extract domain-specific features to identify falls. In this paper, we present a novel framework, DeepFall, which formulates the fall detection problem as an anomaly detection problem. The DeepFall framework presents the novel use of deep spatio-temporal convolutional autoencoders to learn spatial and temporal features from normal activities using non-invasive sensing modalities. We also present a new anomaly scoring method that combines the reconstruction score of frames across a temporal window to detect unseen falls. We tested the DeepFall framework on three publicly available datasets collected through non-invasive sensing modalities, thermal camera and depth cameras, and show superior results in comparison with traditional autoencoder methods to identify unseen falls.

Superpixel Embedding Network.

Superpixel segmentation is a fundamental computer vision technique that finds application in a multitude of high level computer vision tasks. Most state-of-the-art superpixel segmentation methods are unsupervised in nature and thus cannot fully utilize frequently occurring texture patterns or incorporate multiscale context. In this paper, we show that superpixel segmentation can be improved by leveraging the superior modeling power of deep convolutional autoencoders in a fully unsupervised manner. We pose the superpixel segmentation problem as one of manifold learning where pixels that belong to similar texture patterns are assigned near identical embedding vectors. The proposed deep network is able to learn image-wide and dataset-wide feature patterns and the relationships between them. This knowledge is used to segment and group pixels in a way that is consistent with a more global definition of pattern coherence. Experiments demonstrate that the superpixels obtained from the embeddings learned by the proposed method outperform the state-of-theart superpixel segmentation methods for boundary precision and recall values. Additionally, we find that semantic edges obtained from the superpixel embeddings to be significantly better than the contemporary unsupervised approaches.

Bagging deep convolutional autoencoders trained with a mixture of real data and GAN-generated data

While deep neural networks have achieved remarkable performance in representation learning, a huge amount of labeled training data are usually required by supervised deep models such as convolutional neural networks. In this paper, we propose a new representation learning method, namely generative adversarial networks (GAN) based bagging deep convolutional autoencoders (GAN-BDCAE), which can map data to diverse hierarchical representations in an unsupervised fashion. To boost the size of training data, to train deep model and to aggregate diverse learning machines are the three principal avenues towards increasing the capabilities of representation learning of neural networks. We focus on combining those three techniques. To this aim, we adopt GAN for realistic unlabeled sample generation and bagging deep convolutional autoencoders (BDCAE) for robust feature learning. The proposed method improves the discriminative ability of learned feature embedding for solving subsequent pattern recognition problems. We evaluate our approach on three standard benchmarks and demonstrate the superiority of the proposed method compared to traditional unsupervised learning methods.

Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders

Nearly all model-reduction techniques project the governing equations onto a linear subspace of the original state space. Such subspaces are typically computed using methods such as balanced truncation, rational interpolation, the reduced-basis method, and (balanced) proper orthogonal decomposition (POD). Unfortunately, restricting the state to evolve in a linear subspace imposes a fundamental limitation to the accuracy of the resulting reduced-order model (ROM). In particular, linear-subspace ROMs can be expected to produce low-dimensional models with high accuracy only if the problem admits a fast decaying Kolmogorov n-width (e.g., diffusion-dominated problems). Unfortunately, many problems of interest exhibit a slowly decaying Kolmogorov n-width (e.g., advection-dominated problems). To address this, we propose a novel framework for projecting dynamical systems onto nonlinear manifolds using minimum-residual formulations at the time-continuous and time-discrete levels; the former leads to manifold Galerkin projection, while the latter leads to manifold least-squares Petrov–Galerkin (LSPG) projection. We perform analyses that provide insight into the relationship between these proposed approaches and classical linear-subspace reduced-order models; we also derive a posteriori discrete-time error bounds for the proposed approaches. In addition, we propose a computationally practical approach for computing the nonlinear manifold, which is based on convolutional autoencoders from deep learning. Finally, we demonstrate the ability of the method to significantly outperform even the optimal linear-subspace ROM on benchmark advection-dominated problems, thereby demonstrating the method's ability to overcome the intrinsic n-width limitations of linear subspaces.

Studying the Manifold Structure of Alzheimer's Disease: A Deep Learning Approach Using Convolutional Autoencoders.

Many classical machine learning techniques have been used to explore Alzheimer's disease (AD), evolving from image decomposition techniques such as principal component analysis toward higher complexity, non-linear decomposition algorithms. With the arrival of the deep learning paradigm, it has become possible to extract high-level abstract features directly from MRI images that internally describe the distribution of data in low-dimensional manifolds. In this work, we try a new exploratory data analysis of AD based on deep convolutional autoencoders. We aim at finding links between cognitive symptoms and the underlying neurodegeneration process by fusing the information of neuropsychological test outcomes, diagnoses, and other clinical data with the imaging features extracted solely via a data-driven decomposition of MRI. The distribution of the extracted features in different combinations is then analyzed and visualized using regression and classification analysis, and the influence of each coordinate of the autoencoder manifold over the brain is estimated. The imaging-derived markers could then predict clinical variables with correlations above 0.6 in the case of neuropsychological evaluation variables such as the MMSE or the ADAS11 scores, achieving a classification accuracy over 80% for the diagnosis of AD.

A Deep Feature Extraction Method for HEp-2 Cell Image Classification

The automated and accurate classification of the images portraying the Human Epithelial cells of type 2 (HEp-2) represents one of the most important steps in the diagnosis procedure of many autoimmune diseases. The extreme intra-class variations of the HEp-2 cell images datasets drastically complicates the classification task. We propose in this work a classification framework that, unlike most of the state-of-the-art methods, uses a deep learning-based feature extraction method in a strictly unsupervised way. We propose a deep learning-based hybrid feature learning with two levels of deep convolutional autoencoders. The first level takes the original cell images as the inputs and learns to reconstruct them, in order to capture the features related to the global shape of the cells, and the second network takes the gradients of the images, in order to encode the localized changes in intensity (gray variations) that characterize each cell type. A final feature vector is constructed by combining the latent representations extracted from the two networks, giving a highly discriminative feature representation. The created features will be fed to a nonlinear classifier whose output will represent the type of the cell image. We have tested the discriminability of the proposed features on two of the most popular HEp-2 cell classification datasets, the SNPHEp-2 and ICPR 2016 datasets. The results show that the proposed features manage to capture the distinctive characteristics of the different cell types while performing at least as well as the actual deep learning-based state-of-the-art methods in terms of discrimination.

Object-based deep convolutional autoencoders for high-resolution remote sensing image classification

Deep learning has been applied to many fields due to its capability for unsupervised feature learning. However, it is unsuitable for remote sensing image classification of deep convolutional networks due to the regular and fixed shape of receptive fields in current methods, which cannot freely adapt for ground objects with various shapes and scales. To tackle this problem, we propose object-based deep convolutional autoencoders (ODCAEs) to encode high-resolution remote sensing image features automatically. The receptive fields of deep convolutional autoencoders were designed based on a fractal net evolution approach to adapt for various ground objects in images, retaining category purity while providing adequate information. We collected 109,333 samples of seven classes from high-resolution satellite images over various locations to train the ODCAE, which encoded high level features automatically, coupled with a support vector machine for classification. We assessed the classification results using two WorldView-II image locations. The proposed ODCAE approach achieved higher accuracy than three manual design feature systems. Thus, the proposed ODCAE approach is useful and efficient for feature learning problems for high-resolution remote sensing images.

Deep learning models to remix music for cochlear implant users.

The severe hearing loss problems that some people suffer can be treated by providing them with a surgically implanted electrical device called cochlear implant (CI). CI users struggle to perceive complex audio signals such as music; however, previous studies show that CI recipients find music more enjoyable when the vocals are enhanced with respect to the background music. In this manuscript source separation (SS) algorithms are used to remix pop songs by applying gain to the lead singing voice. This work uses deep convolutional auto-encoders, a deep recurrent neural network, a multilayer perceptron (MLP), and non-negative matrix factorization to be evaluated objectively and subjectively through two different perceptual experiments which involve normal hearing subjects and CI recipients. The evaluation assesses the relevance of the artifacts introduced by the SS algorithms considering their computation time, as this study aims at proposing one of the algorithms for real-time implementation. Results show that the MLP performs in a robust way throughout the tested data while providing levels of distortions and artifacts which are not perceived by CI users. Thus, an MLP is proposed to be implemented for real-time monaural audio SS to remix music for CI users.

Sea Clutter and Target Detection with Deep Neural Networks

In this paper, we investigate the feasibility of applying a deep learning approach to sea clutter suppression and target detection in an inhomogeneous oceanic environment. The employed method consists of deep convolutional auto-encoders (DCAEs) to filter sea clutter and a logistic regression classifier to achieve the detection of target. The sea clutter suppression processing using DCAEs automatically removes complex patterns like superimposed clutter from a target, rather than simple patterns like echoes missing at random. Compared with conventional methods for sea clutter suppression, the algorithm does not need to estimate the covariance matrix of clutter so as to have better flexibility. The results show that reliable suppression performance and higher detection accuracy can be achieved from our experiments, whose data include the measured data and simulation data.

Unsupervised Deep Learning for Gear Health Monitoring

Deep learning has revolutionized many fields in recent years by replacing expert-designed, handcrafted features with learned representations. Gear health monitoring is a field where expert-designed features are heavily used for predictive modeling. This paper investigates how unsuperviseddeep learning can be applied to gear health monitoring to make predictions on low frequency scales using high frequency data given small, sparsely labeled data sets. Deep convolutional autoencoders are trained and used to generate learned features. The learned features are compared with relevant handcrafted features via their performance in training machine learning models to predict discrete gear fatigue states. The learned features performed poorly against the handcrafted features, however models trained on feature sets tended to outperform those exclusively trained on handcrafted features. The top performing model was a multi-layer perceptron trained on both feature sets that leveraged the ability of the condition indicators to represent healthy and failure states and the ability of the learned features to represent the intermediate worn state. This work demonstrates that unsupervised deep learning techniques can be used to bolster the performance of handcrafted features in small, sparsely labeled data sets in gear health monitoring.

A study of deep convolutional auto-encoders for anomaly detection in videos

The detection of anomalous behaviors in automated video surveillance is a recurrent topic in recent computer vision research. Depending on the application field, anomalies can present different characteristics and challenges. Convolutional Neural Networks have achieved the state-of-the-art performance for object recognition in recent years, since they learn features automatically during the training process. From the anomaly detection perspective, the Convolutional Autoencoder (CAE) is an interesting choice, since it captures the 2D structure in image sequences during the learning process. This work uses a CAE in the anomaly detection context, by applying the reconstruction error of each frame as an anomaly score. By exploring the CAE architecture, we also propose a method for aggregating high-level spatial and temporal features with the input frames and investigate how they affect the CAE performance. An easy-to-use measure of video spatial complexity was devised and correlated with the classification performance of the CAE. The proposed methods were evaluated by means of several experiments with public-domain datasets. The promising results support further research in this area.

Deep convolutional autoencoders: Stereo matching for three-dimensional model reconstruction of low-textured objects

Deep convolutional autoencoders: Stereo matching for three-dimensional model reconstruction of low-textured objects

MO-G-201-03: Deep-Learning Based Prediction of Achievable Dose for Personalizing Inverse Treatment Planning

Purpose: Inverse planning starts with prescribed dose or DVHs. The time/effort spent on obtaining a clinically acceptable plan and, more importantly, the quality of the resultant treatment plan depends critically on the prescription. The present work aims to facilitate the treatment planning workflow by predicting the achievable dose distribution based on a dataset of quality plans designed for past patients. Methods: A novel learning-empowered approach is developed that predicts achievable 3D dose volume for a new patient based on the geometrical features of the contoured anatomical-scan. Two major steps are involved: (1) Feature extraction: To extract a rich set of features resilient to patient and tumor-shape variations, multi-layer convolutional auto-encoders (CAE) are applied separately to structural contours and datasets of isodose distribution. (2) Learning the relation map: From the historical dose and contour features, a predictive model based on multi-task regression is trained, where dose features are linearly correlated with a subset of contour features selected via sparsity regularization. IMRT plans for 115 prostate patients were used to train the models and the final method is tested on additional 10 prostate plans. Dose and contour volumes are rearranged to form a 3D polar grid with (r,ϕ,z) coordinates. Results: A deep-learning based dose prediction model for VMAT/IMRT treatment is established. Our preliminary results using a two-layer linear auto-encoder achieve more than %70 overlap volume between the actual and predicted isodose surfaces associated with V60%, V75%, and V90. Analysis of dose and contour data reveal nonlinear behaviors which we further deploy enhance prediction accuracy. Conclusion: The present work puts forth a novel predictive model for dose volume based on anatomical scan. Deep CAEs extract a subset of representative features that are subsequently used to derive a contour-to-dose relation map. The resultant dose map serves as a prior to facilitate treatment planning.