Variational Autoencoder Network Research Articles

Unsupervised learning for cuboid shape abstraction via joint segmentation from point clouds

Representing complex 3D objects as simple geometric primitives, known as shape abstraction, is important for geometric modeling, structural analysis, and shape synthesis. In this paper, we propose an unsupervised shape abstraction method to map a point cloud into a compact cuboid representation. We jointly predict cuboid allocation as part segmentation and cuboid shapes and enforce the consistency between the segmentation and shape abstraction for self-learning. For the cuboid abstraction task, we transform the input point cloud into a set of parametric cuboids using a variational auto-encoder network. The segmentation network allocates each point into a cuboid considering the point-cuboid affinity. Without manual annotations of parts in point clouds, we design four novel losses to jointly supervise the two branches in terms of geometric similarity and cuboid compactness. We evaluate our method on multiple shape collections and demonstrate its superiority over existing shape abstraction methods. Moreover, based on our network architecture and learned representations, our approach supports various applications including structured shape generation, shape interpolation, and structural shape clustering.

Latent space visualization of half face and full face by generative model

Generally, most face detection and recognition tasks are based on the training of intact facial images and their corresponding labels. The training image is supposed to contain as much facial area as possible, and sometimes expanding the training image area to the upper body may also enhance the learning ability. However, we noticed that both the three-dimensional structure and two-dimensional appearance from the frontal view of human faces are bilaterally symmetrical. Few research makes use of this characteristics to simplify the learning process. We have proposed a flipping strategy to apply the facial symmetrical characteristic to transfer learning and proved training with half faces can also achieve equivalent performance in face recognition for a small group of individuals. This paper extend the transfer learning of cropped half face images for face recognition rather than flipping the half face. The facial symmetrical characteristics is utilized to improve face recognition through transfer learning of only a half of the common human face image. We also investigate and explain the reason why the half face area is enough to accurately classify small groups of individuals. A variational autoencoder network is utilized to impose the probability distribution on the facial latent space. Finally, the dimensions of the facial latent space are reduced to visualize the distributed perceptual manifold for face identity.

Augmenting deviation of faults from the normal using fault assistant Gaussian mixture prior variational autoencoder

In this new era of Industry 4.0, manufacturers tend to store process data from the entire production, regardless of whether they are “normal” or “faulty” for further data analysis. However, almost all the existing monitoring models are constructed based on normal data instead of abnormal data. In fact, the “normal” and some of the “faulty” data originate from the same production line. Thus, not only the normal data but also the abnormal ones can be used to improve the monitoring performance of conventional monitoring performance by simultaneously sharing and extracting common knowledge. In this paper, a fault assistant Gaussian mixture prior variational autoencoder (FA-GMPVAE) is proposed to perform information sharing and enhance the statistic model for the normal operating region. Unlike an ordinary variational autoencoder (VAE) and an ordinary Gaussian mixture prior variational autoencoder (GMPVAE), the structure of FA-GMPVAE is a combination of a “normal” based VAE network and a “normal-relevant” based GMPVAE (NR-GMPVAE) network. FA-GMPVAE can make the shared information non-negative to prevent information loss because only the normal-relevant common information is shared by the one-step transfer learning procedure. In addition, fault diagnosis of NR-GMPVAE can be flexibly updated with the new type of fault. Correspondingly, the probability density estimates of latent variables and residuals instead of point estimates are then given so that distribution-based monitoring indices of the normal data can be designed and the fault detection decisions can be made opportunely. To show the effectiveness of the proposed method, a numerical and a real industrial example are presented.

Deep Learning Based Downlink Channel Covariance Estimation for FDD Massive MIMO Systems

Obtaining the downlink channel state information in frequency division duplexing (FDD) massive multi-input multi-output (MIMO) systems is challenging due to the overwhelming training and feedback overhead. In this letter, motivated by the existence of mapping characteristics between uplink and downlink, we propose a covariance variational auto-encoder network (CVENet) to approximate the mapping function. Different from normal auto-encoder, the CVENet extracts the uplink channel covariance to a latent distribution space and then predicts the downlink channel covariance by the sample of the space. Simulation results demonstrate that the CVENet performs better than the conventional dictionary pairs algorithm. And the CVENet still achieves robustness in a circumstance where the channel environment of the training stage is different from the deployment stage, which shows its practical applicability.

4-Class MI-EEG Signal Generation and Recognition with CVAE-GAN

As the capability of an electroencephalogram’s (EEG) measurement of the real-time electrodynamics of the human brain is known to all, signal processing techniques, particularly deep learning, could either provide a novel solution for learning but also optimize robust representations from EEG signals. Considering the limited data collection and inadequate concentration of during subjects testing, it becomes essential to obtain sufficient training data and useful features with a potential end-user of a brain–computer interface (BCI) system. In this paper, we combined a conditional variational auto-encoder network (CVAE) with a generative adversarial network (GAN) for learning latent representations from EEG brain signals. By updating the fine-tuned parameter fed into the resulting generative model, we could synthetize the EEG signal under a specific category. We employed an encoder network to obtain the distributed samples of the EEG signal, and applied an adversarial learning mechanism to continuous optimization of the parameters of the generator, discriminator and classifier. The CVAE was adopted to adjust the synthetics more approximately to the real sample class. Finally, we demonstrated our approach take advantages of both statistic and feature matching to make the training process converge faster and more stable and address the problem of small-scale datasets in deep learning applications for motor imagery tasks through data augmentation. The augmented training datasets produced by our proposed CVAE-GAN method significantly enhance the performance of MI-EEG recognition.

A Robust Image Watermarking Approach Using Cycle Variational Autoencoder

With the rapid development of Internet and cloud storage, data security sharing and copyright protection are becoming more and more important. In this paper, we introduce a robust image watermarking algorithm for copyright protection based on variational autoencoder networks. The proposed image watermarking embedding and extracting network consists of three parts: encoder subnetwork, decoder subnetwork, and detector subnetwork. In the training process, the encoder and decoder subnetworks learn a robust image representation model and further implement the embedding of 1-bit watermark image to the cover image. Meanwhile, the detector subnetwork learns to extract the 1-bit watermark image from the embedding image. Experimental results demonstrate that the watermarked images generated by the proposed algorithm have better visual effects and are more robust against geometric and noise attacks than traditional approaches in the transform domain.

Detection and Analysis of Heartbeats in Seismocardiogram Signals.

This paper presents an unsupervised methodology to analyze SeismoCardioGram (SCG) signals. Starting from raw accelerometric data, heartbeat complexes are extracted and annotated, using a two-step procedure. An unsupervised calibration procedure is added to better adapt to different user patterns. Results show that the performance scores achieved by the proposed methodology improve over related literature: on average, 98.5% sensitivity and 98.6% precision are achieved in beat detection, whereas RMS (Root Mean Square) error in heartbeat interval estimation is as low as 4.6 ms. This allows SCG heartbeat complexes to be reliably extracted. Then, the morphological information of such waveforms is further processed by means of a modular Convolutional Variational AutoEncoder network, aiming at extracting compressed, meaningful representation. After unsupervised training, the VAE network is able to recognize different signal morphologies, associating each user to its specific patterns with high accuracy, as indicated by specific performance metrics (including adjusted random and mutual information score, completeness, and homogeneity). Finally, a Linear Model is used to interpret the results of clustering in the learned latent space, highlighting the impact of different VAE architectural parameters (i.e., number of stacked convolutional units and dimension of latent space).

Partial Discharge Data Matching Method for GIS Case-Based Reasoning

With the accumulation of partial discharge (PD) detection data from substation, case-based reasoning (CBR), which computes the match degree between detected PD data and historical case data provides new ideas for the interpretation and evaluation of partial discharge data. Aiming at the problem of partial discharge data matching, this paper proposes a data matching method based on a variational autoencoder (VAE). A VAE network model for partial discharge data is constructed to extract the deep eigenvalues. Cosine distance is then used to calculate the match degree between different partial discharge data. To verify the advantages of the proposed method, a partial discharge dataset was established through a partial discharge experiment and live detections on substation site. The proposed method was compared with other feature extraction methods and matching methods including statistical features, deep belief networks (DBN), deep convolutional neural networks (CNN), Euclidean distances, and correlation coefficients. The experimental results show that the cosine distance match degree based on the VAE feature vector can effectively detect similar partial discharge data compared with other data matching methods.

Degradation feature extraction using multi-source monitoring data via logarithmic normal distribution based variational auto-encoder

Degeneration features extraction from multi-source monitoring data is significant for data-driven based state assessment and RUL prediction of complex equipment. However, the massive data gathered continuously from condition monitoring systems has created challenges to extract degradation features effectively, and meanwhile the current existing feature extraction methods such as PCA pays more attention to the integrity of the extracted information, but ignores the priori information. In this paper, we developed logarithmic normal distribution based variational auto-encoder algorithms which can ensure that the final feature extraction results follow the lognormal prior hypothesis to address above problems. The main procedure of the developed method is as follows. First, create the multi-source information dataset consist of different monitoring indicators, and normalize the dataset. Then, the KL divergence regularization is deduced based on the principle of variational Bayes inference. Next, the three-layer variational auto-encoder network model is constructed, and normalized datasets is used to train and test the model. The experiment results demonstrate that the degradation trajectory obtained using the proposed method offers stronger predictive capabilities than do those obtained using original methods, thereby improving the accuracy of predictions of the equipment remaining useful life.

Deep learning–based inverse method for layout design

Layout design with complex constraints is a challenging problem to solve due to the non-uniqueness of the solution and the difficulties in incorporating the constraints into the conventional optimization-based methods. In this paper, we propose a design method based on the recently developed machine learning technique, Variational Autoencoder (VAE). We utilize the learning capability of the VAE to learn the constraints and the generative capability of the VAE to generate design candidates that automatically satisfy all the constraints. As such, no constraints need to be imposed during the design stage. In addition, we show that the VAE network is also capable of learning the underlying physics of the design problem, leading to an efficient design tool that does not need any physical simulation once the network is constructed. We demonstrated the performance of the method on two cases: inverse design of surface diffusion induced morphology change and mask design for optical microlithography.

The Optimally Designed Variational Autoencoder Networks for Clustering and Recovery of Incomplete Multimedia Data.

Clustering analysis of massive data in wireless multimedia sensor networks (WMSN) has become a hot topic. However, most data clustering algorithms have difficulty in obtaining latent nonlinear correlations of data features, resulting in a low clustering accuracy. In addition, it is difficult to extract features from missing or corrupted data, so incomplete data are widely used in practical work. In this paper, the optimally designed variational autoencoder networks is proposed for extracting features of incomplete data and using high-order fuzzy c-means algorithm (HOFCM) to improve cluster performance of incomplete data. Specifically, the feature extraction model is improved by using variational autoencoder to learn the feature of incomplete data. To capture nonlinear correlations in different heterogeneous data patterns, tensor based fuzzy c-means algorithm is used to cluster low-dimensional features. The tensor distance is used as the distance measure to capture the unknown correlations of data as much as possible. Finally, in the case that the clustering results are obtained, the missing data can be restored by using the low-dimensional features. Experiments on real datasets show that the proposed algorithm not only can improve the clustering performance of incomplete data effectively, but also can fill in missing features and get better data reconstruction results.

Learning a Probabilistic Model for Diffeomorphic Registration.

We propose to learn a low-dimensional probabilistic deformation model from data which can be used for the registration and the analysis of deformations. The latent variable model maps similar deformations close to each other in an encoding space. It enables to compare deformations, to generate normal or pathological deformations for any new image, or to transport deformations from one image pair to any other image. Our unsupervised method is based on the variational inference. In particular, we use a conditional variational autoencoder network and constrain transformations to be symmetric and diffeomorphic by applying a differentiable exponentiation layer with a symmetric loss function. We also present a formulation that includes spatial regularization such as the diffusion-based filters. In addition, our framework provides multi-scale velocity field estimations. We evaluated our method on 3-D intra-subject registration using 334 cardiac cine-MRIs. On this dataset, our method showed the state-of-the-art performance with a mean DICE score of 81.2% and a mean Hausdorff distance of 7.3 mm using 32 latent dimensions compared to three state-of-the-art methods while also demonstrating more regular deformation fields. The average time per registration was 0.32 s. Besides, we visualized the learned latent space and showed that the encoded deformations can be used to transport deformations and to cluster diseases with a classification accuracy of 83% after applying a linear projection.