Cycle-consistency Constraints Research Articles

Predicting single-cell cellular responses to perturbations using cycle consistency learning.

Phenotype-based drug screening emerges as a powerful approach for identifying compounds that actively interact with cells. Transcriptional and proteomic profiling of cell lines and individual cells provide insights into the cellular state alterations that occur at the molecular level in response to external perturbations, such as drugs or genetic manipulations. In this paper, we propose cycleCDR, a novel deep learning framework to predict cellular response to external perturbations. We leverage the autoencoder to map the unperturbed cellular states to a latent space, in which we postulate the effects of drug perturbations on cellular states follow a linear additive model. Next, we introduce the cycle consistency constraints to ensure that unperturbed cellular state subjected to drug perturbation in the latent space would produces the perturbed cellular state through the decoder. Conversely, removal of perturbations from the perturbed cellular states can restore the unperturbed cellular state. The cycle consistency constraints and linear modeling in the latent space enable to learn transferable representations of external perturbations, so that our model can generalize well to unseen drugs during training stage. We validate our model on four different types of datasets, including bulk transcriptional responses, bulk proteomic responses, and single-cell transcriptional responses to drug/gene perturbations. The experimental results demonstrate that our model consistently outperforms existing state-of-the-art methods, indicating our method is highly versatile and applicable to a wide range of scenarios. The source code is available at: https://github.com/hliulab/cycleCDR.

Uncertainty Estimation in Unsupervised MR-CT Synthesis of Scoliotic Spines

<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Goal:</i> Uncertainty estimations through approximate Bayesian inference provide interesting insights to deep neural networks' behavior. In unsupervised learning tasks, where expert labels are unavailable, it becomes ever more important to critique the model through uncertainties. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Methods:</i> This paper presents a proof-of-concept for generalizing the aleatoric and epistemic uncertainties in unsupervised MR-CT synthesis of scoliotic spines. A novel adaptation of the cycle-consistency constraint in CycleGAN is proposed such that the model predicts the aleatoric uncertainty maps in addition to the standard volume-to-volume translation between Magnetic Resonance (MR) and Computed Tomography (CT) data. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Results:</i> Ablation experiments were performed to understand uncertainty estimation as an implicit regularizer and a measure of the model's confidence. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Conclusion:</i> The aleatoric uncertainty helps in distinguishing between the bone and soft-tissue regions in CT and MR data during translation, while the epistemic uncertainty provides interpretable information to the user for downstream tasks.

Learning zero-shot dense light field reconstruction from heterogeneous imaging

Due to the limited sensor size, light field (LF) imaging usually suffers from an essential trade-off between its spatial and angular resolutions. Recently, some approaches have been proposed to use hybrid lens imaging to enhance spatial resolution, but LF angular reconstruction has not been involved. In this paper, we propose a zero-shot learning-based method for reconstructing a dense LF image from a heterogeneous imaging system that integrates a 2D digital camera and a commercial LF camera. Our main insight is that the high-resolution information of the 2D digital image can assist in enhancing the spatial resolution of the LF image, and further, the spatially enhanced LF image can guide the angular reconstruction of the 2D digital image. To avoid ambiguity, it is pointed out that this paper focuses on the latter, i.e., enhancing the angular resolution of the 2D image. Specifically, we first construct two unsupervised modules to learn the geometric property of the acquired LF image and the correlation between the LF image and 2D image, so as to perform the initial reconstruction. Afterwards, an LF detail refiner is constructed to improve the reconstruction quality, and a cycle consistency constraint is designed to realize unsupervised training. Consequently, we can learn the dense LF reconstruction from the heterogeneous imaging data itself without extra training samples. Experimental results show that the proposed method can faithfully reconstruct high-quality densely-sampled LF images, and surpasses some state-of-the-art methods in both quantitative and qualitative evaluations. Furthermore, the proposed method can be combined with existing LF view stitching methods to further enhance the field of view of the reconstructed LF image. Our code will be released at https://github.com/YeyaoChen/ZLFSR.

Self-supervised medical slice interpolation network using controllable feature flow

Deep learning-based image interpolation methods are confronted with various challenges in its application to anisotropic medical volumetric data (i.e., CT and MR images) out of the complex nonlinear deformation and the scarcity of high-quality images. This paper proposes a self-supervised multiple medical slice interpolation network using controllable feature flow. Firstly, a Controllable Feature Flow Network (CFFNet) is proposed to estimate the complex nonlinear deformation of medical images. In CFFNet, we first use a deformation-aware network and a spatial channel modulation module to predict the bi-directional feature flows from the source slices to the target slices by considering an additional position parameter, and then the learned feature flows are used to synthesize the target intermediate features by using the deformable convolution. Secondly, an improved two-stage self-supervised framework is proposed to alleviate the dependency on high-quality medical images. In the first stage, the synthesized training pairs along the dense sagittal and coronal directions are adopted to pre-train the CFFNet. In the second one, the sparse axial slices are used to fine-tune the CFFNet with the cycle-consistency constraint and a feature domain smooth loss. Experimental results illustrate that the proposed CFFNet assumes superior performance on medical slice interpolation tasks with fewer parameters and the proposed self-supervised CFFNet obtains competitive results compared with other fully supervised algorithms. We also evaluate our model on HepaticVessel segmentation task, the proposed method can effectively enhance the continuity and smoothness of the 3D vessel structures, showing promising results on clinical application. The code is available at: https://codeocean.com/capsule/8945297/tree/v1.

Two-Stage Self-Supervised Cycle-Consistency Transformer Network for Reducing Slice Gap in MR Images.

Magnetic resonance (MR) images are usually acquired with large slice gap in clinical practice, i.e., low resolution (LR) along the through-plane direction. It is feasible to reduce the slice gap and reconstruct high-resolution (HR) images with the deep learning (DL) methods. To this end, the paired LR and HR images are generally required to train a DL model in a popular fully supervised manner. However, since the HR images are hardly acquired in clinical routine, it is difficult to get sufficient paired samples to train a robust model. Moreover, the widely used convolutional Neural Network (CNN) still cannot capture long-range image dependencies to combine useful information of similar contents, which are often spatially far away from each other across neighboring slices. To this end, a Two-stage Self-supervised Cycle-consistency Transformer Network (TSCTNet) is proposed to reduce the slice gap for MR images in this work. A novel self-supervised learning (SSL) strategy is designed with two stages respectively for robust network pre-training and specialized network refinement based on a cycle-consistency constraint. A hybrid Transformer and CNN structure is utilized to build an interpolation model, which explores both local and global slice representations. The experimental results on two public MR image datasets indicate that TSCTNet achieves superior performance over other compared SSL-based algorithms.

Automatic Update for Wi-Fi Fingerprinting Indoor Localization via Multi-Target Domain Adaptation

Wi-Fi fingerprinting system in the long term suffers from gradually deteriorative localization accuracy, leading to poor user experiences. To keep high accuracy yet at a low cost, we first study long-term variation of access points (APs) and characteristics of their Wi-Fi signals through over-one-year experiments. Motivated by the experimental findings, we then design MTLoc, a Multi-Target domain adaptation network-based Wi-Fi fingerprinting Localization system. As the core, MTDAN (Multi-Target Domain Adaptation Network) model adopts the framework of generative adversarial network to learn time-invariant, time-specific, and location-aware features from the source and target domains. To enhance the alignment among the source and targets, two-level cycle consistency constraints are proposed. Hence, MTDAN is able to transfer location knowledge from the source domain to multiple targets. In addition, domain selection and outlier detection are designed to avoid explosive growth of storage for targets and to limit the impact of random variations of Wi-Fi signals. Extensive experiments are carried out on five datasets collected over two years in various real-world indoor environments with a total area of 8, 350 m2. Experimental results demonstrate that MTLoc retains high localization accuracy with limited storage and training cost in the long term, which significantly outperforms its counterparts. We share our dataset to the community for other researchers to validate our results and conduct further research.

ActFloor-GAN: Activity-Guided Adversarial Networks for Human-Centric Floorplan Design.

We present a novel two-stage approach for automated floorplan design in residential buildings with a given exterior wall boundary. Our approach has the unique advantage of being human-centric, that is, the generated floorplans can be geometrically plausible, as well as topologically reasonable to enhance resident interaction with the environment. From the input boundary, we first synthesize a human-activity map that reflects both the spatial configuration and human-environment interaction in an architectural space. We propose to produce the human-activity map either automatically by a pre-trained generative adversarial network (GAN) model, or semi-automatically by synthesizing it with user manipulation of the furniture. Second, we feed the human-activity map into our deep framework ActFloor-GAN to guide a pixel-wise prediction of room types. We adopt a re-formulated cycle-consistency constraint in ActFloor-GAN to maximize the overall prediction performance, so that we can produce high-quality room layouts that are readily convertible to vectorized floorplans. Experimental results show several benefits of our approach. First, a quantitative comparison with prior methods shows superior performance of leveraging the human-activity map in predicting piecewise room types. Second, a subjective evaluation by architects shows that our results have compelling quality as professionally-designed floorplans and much better than those generated by existing methods in terms of the room layout topology. Last, our approach allows manipulating the furniture placement, considers the human activities in the environment, and enables the incorporation of user-design preferences.

Portrait Map Art Generation By Asymmetric Image-to-Image Translation

AbstractThe authors propose a deep neural network–based algorithm to automatically generate portrait map art (PMA), a modern art form created by British portrait artist Ed Fairburn. The authors formulate the generation of PMA as an adaptive dual-to-single image translation problem. The authors’ proposed model analyzes the appearance of one portrait and one map image using two encoder networks and utilizes their hidden encodings as representations of the portrait and map image to generate new PMA using a decoder network. An adaptive style harmonization module is proposed to fuse the two hidden encodings. Optimized by cycle-consistency constraint, the model can produce new PMA images without baselines.

Asynchronous Generative Adversarial Network for Asymmetric Unpaired Image-to-Image Translation

The unpaired image-to-image translation aims to translate input images from one source domain to some desired outputs in a target domain by learning from unpaired training data. Cycle-consistency constraint provides a general principle to estimate and measure forward and backward mapping functions between two domains. In many cases, the information entropy of images from the two domains is not equal, resulting in an information-rich domain and an information-poor domain. However, existing solutions based on cycle-consistency either completely discard the information asymmetry between the two domains (a common choice), which leads to inferior translation performance for the asymmetric unpaired image-to-image translation, or have to rely on special task-specific designs and introduce extra loss components. These elaborative designs especially for the relatively harder translation direction from the information-poor domain to the information-rich domain (poor-to-rich translation) require extra labor and are limited to some specific tasks. In this paper, we propose a novel asynchronous generative adversarial network named Async-GAN, which provides a model-agnostic framework for easily turning symmetrical models into powerful asymmetric counterparts that can handle asymmetric unpaired image-to-image translation much better. The key innovation is to iteratively build gradually improving intermediate domains for generating pseudo paired training samples, which provide stronger full supervision for assisting the poor-to-rich translation. Extensive experiments on various asymmetric unpaired translation tasks demonstrate the superiority of the proposal. Furthermore, the proposed training framework could be extended to various Cycle-GAN solutions and achieve a performance gain.

Cycle-Consistent Weakly Supervised Visual Grounding With Individual and Contextual Representations.

Visual grounding, aiming to align image regions with textual queries, is a fundamental task for cross-modal learning. We study the weakly supervised visual grounding, where only image-text pairs at a coarse-grained level are available. Due to the lack of fine-grained correspondence information, existing approaches often encounter matching ambiguity. To overcome this challenge, we introduce the cycle consistency constraint into region-phrase pairs, which strengthens correlated pairs and weakens unrelated pairs. This cycle pairing makes use of the bidirectional association between image regions and text phrases to alleviate matching ambiguity. Furthermore, we propose a parallel grounding framework, where backbone networks and subsequent relation modules extract individual and contextual representations to calculate context-free and context-aware similarities between regions and phrases separately. Those two representations characterize visual/linguistic individual concepts and inter-relationships, respectively, and then complement each other to achieve cross-modal alignment. The whole framework is trained by minimizing an image-text contrastive loss and a cycle consistency loss. During inference, the above two similarities are fused to give the final region-phrase matching score. Experiments on five popular datasets about visual grounding demonstrate a noticeable improvement in our method. The source code is available at https://github.com/Evergrow/WSVG.

Self-supervised cycle-consistent learning for scale-arbitrary real-world single image super-resolution

Whether conventional machine learning-based or current deep neural networks-based single image super-resolution (SISR) methods, they are generally trained and validated on synthetic datasets, in which low-resolution (LR) inputs are artificially produced by degrading high-resolution (HR) images based on a hand-crafted degradation model (e.g., bicubic downsampling). One of the main reasons for this is that it is challenging to build a realistic dataset composed of real-world LR–HR image pairs. However, a domain gap exists between synthetic and real-world data because the degradations in real scenarios are more complicated, limiting the performance in practical applications of SISR models trained with synthetic data. To address these problems, we propose a Self-supervised Cycle-consistent Learning-based Scale-Arbitrary Super-Resolution framework (SCL-SASR) for real-world images. Inspired by the Maximum a Posteriori estimation, our SCL-SASR consists of a Scale-Arbitrary Super-Resolution Network (SASRN) and an inverse Scale-Arbitrary Resolution-Degradation Network (SARDN). SARDN and SASRN restrain each other with the bidirectional cycle consistency constraints as well as image priors, making SASRN adapt to the image-specific degradation well. Meanwhile, considering the lack of targeted training images and the complexity of realistic degradations, SCL-SASR is designed to be online optimized solely with the LR input prior to the SR reconstruction. Benefitting from the flexible architecture and the self-supervised learning manner, SCL-SASR can easily super-resolve new images with arbitrary integer or non-integer scaling factors. Experiments on real-world images demonstrate the high flexibility and good applicability of SCL-SASR, which achieves better reconstruction performance than state-of-the-art self-supervised learning-based SISR methods as well as several external dataset-trained SISR models.

An End-to-end Supervised Domain Adaptation Framework for Cross-Domain Change Detection

Change detection is a crucial but extremely challenging task in remote sensing image analysis, and much progress has been made with the rapid development of deep learning. However, most existing deep learning-based change detection methods try to elaborately design complicated neural networks with powerful feature representations. However, they ignore the universal domain shift induced by time-varying land cover changes, including luminance fluctuations and seasonal changes between pre-event and post-event images, thereby producing suboptimal results. In this paper, we propose an end-to-end supervised domain adaptation framework for cross-domain change detection named SDACD, to effectively alleviate the domain shift between bi-temporal images for better change predictions. Specifically, our SDACD presents collaborative adaptations from both image and feature perspectives with supervised learning. Image adaptation exploits generative adversarial learning with cycle-consistency constraints to perform cross-domain style transformation, which effectively narrows the domain gap in a two-side generation fashion. As for feature adaptation, we extract domain-invariant features to align different feature distributions in the feature space, which could further reduce the domain gap of cross-domain images. To further improve the performance, we combine three types of bi-temporal images for the final change prediction, including the initial input bi-temporal images and two generated bi-temporal images from the pre-event and post-event domains. Extensive experiments and analyses conducted on two benchmarks demonstrate the effectiveness and generalizability of our proposed framework. Notably, our framework pushes several representative baseline models up to new State-Of-The-Art records, achieving 97.34% and 92.36% on the CDD and WHU building datasets, respectively. The source code and models are publicly available at https://github.com/Perfect-You/SDACD.

V-ShadowGAN: generative adversarial networks for removing and generating shadows associated with vehicles based on unpaired data

We propose a method to change the shadows cast by vehicles based on unpaired data, which is used to solve the problem of insufficient datasets for target detection in autonomous driving scenarios. This method can increase the training samples of road vehicles in different time periods and reduce the time needed to make complex annotations of datasets. Direct use of generative adversarial networks with cycle-consistency constraints can achieve mapping transformation of unpaired domains, but it is difficult to completely learn the mapping between different shadow domains. To solve this problem, V-ShadowGAN is proposed. First, to train the model, we construct an unpaired dataset of vehicle shadows in different time periods. Second, the shadow mask extractor and the shadow mask discriminator are constructed to extract shadow masks and guide the generation of shadows in the network. Finally, the spatial attention mechanism is added to the generator, which improves the overall quality of the generated image by changing the channel weight of the network and deepening the network. Various experiments show that V-ShadowGAN can effectively change the vehicle shadow.

A Domain Gap Aware Generative Adversarial Network for Multi-Domain Image Translation.

Recent image-to-image translation models have shown great success in mapping local textures between two domains. Existing approaches rely on a cycle-consistency constraint that supervises the generators to learn an inverse mapping. However, learning the inverse mapping introduces extra trainable parameters and it is unable to learn the inverse mapping for some domains. As a result, they are ineffective in the scenarios where (i) multiple visual image domains are involved; (ii) both structure and texture transformations are required; and (iii) semantic consistency is preserved. To solve these challenges, the paper proposes a unified model to translate images across multiple domains with significant domain gaps. Unlike previous models that constrain the generators with the ubiquitous cycle-consistency constraint to achieve the content similarity, the proposed model employs a perceptual self-regularization constraint. With a single unified generator, the model can maintain consistency over the global shapes as well as the local texture information across multiple domains. Extensive qualitative and quantitative evaluations demonstrate the effectiveness and superior performance over state-of-the-art models. It is more effective in representing shape deformation in challenging mappings with significant dataset variation across multiple domains.

Thin Cloud Removal for Remote Sensing Images Using a Physical-Model-Based CycleGAN With Unpaired Data

Thin cloud removal from remote sensing (RS) images is challenging. Recently, deep-learning-based methods have achieved excellent results using supervised training on paired image data. However, in practice, real paired image data are unavailable. Therefore, in this letter, we propose a novel thin cloud removal method, a physical-model-based CycleGAN (PM-CycleGAN), which can be trained using only unpaired data. The PM-CycleGAN training process comprises forward and backward loops. The forward loop first decomposes a cloudy image into a cloud-free image, thin cloud thickness map, and thickness coefficient using three generators. Then, it combines these three components using a physical model to reconstruct the original cloudy image to obtain the cycle consistency constraint. The backward loop first uses the physical model to synthesize a cloud-free image, thin cloud thickness map, and thickness coefficient into a cloudy image, which are then decomposed into the original three components using the three generators. Visual and quantitative comparisons against several state-of-the-art (SOTA) methods on a cloudy image dataset demonstrated the superiority of PM-CycleGAN.

Joint image and feature adaptative attention-aware networks for cross-modality semantic segmentation

Deep learning-based methods are widely used for the task of semantic segmentation in recent years. However, due to the difficulty and labor cost of collecting pixel-level annotations, it is hard to acquire sufficient training images for a certain imaging modality, which greatly hinders the performance of these methods. The intuitive solution to this issue is to train a pre-trained model on label-rich imaging modality (source domain) and then apply the pre-trained model to the label-poor imaging modality (target domain). Unsurprisingly, since the severe domain shift between different modalities, the pre-trained model would perform poorly on the target imaging modality. To this end, we propose a novel unsupervised domain adaptation framework, called Joint Image and Feature Adaptive Attention-aware Networks (JIFAAN), to alleviate the domain shift for cross-modality semantic segmentation. The proposed framework mainly consists of two procedures. The first procedure is image adaptation, which transforms the source domain images into target-like images using the adversarial learning with cycle-consistency constraint. For further bridging the gap between transformed images and target domain images, the second procedure employs feature adaptation to extract the domain-invariant features and thus aligns the distribution in feature space. In particular, we introduce an attention module in the feature adaptation to focus on noteworthy regions and generate attention-aware results. Lastly, we combine two procedures in an end-to-end manner. Experiments on two cross-modality semantic segmentation datasets demonstrate the effectiveness of our proposed framework. Specifically, JIFAAN surpasses the cutting-edge domain adaptation methods and achieves the state-of-the-art performance.

Atlas-based Segmentation of Intracochlear Anatomy in Metal Artifact Affected CT Images of the Ear with Co-trained Deep Neural Networks.

We propose an atlas-based method to segment the intracochlear anatomy (ICA) in the post-implantation CT (Post-CT) images of cochlear implant (CI) recipients that preserves the point-to-point correspondence between the meshes in the atlas and the segmented volumes. To solve this problem, which is challenging because of the strong artifacts produced by the implant, we use a pair of co-trained deep networks that generate dense deformation fields (DDFs) in opposite directions. One network is tasked with registering an atlas image to the Post-CT images and the other network is tasked with registering the Post-CT images to the atlas image. The networks are trained using loss functions based on voxel-wise labels, image content, fiducial registration error, and cycle-consistency constraint. The segmentation of the ICA in the Post-CT images is subsequently obtained by transferring the predefined segmentation meshes of the ICA in the atlas image to the Post-CT images using the corresponding DDFs generated by the trained registration networks. Our model can learn the underlying geometric features of the ICA even though they are obscured by the metal artifacts. We show that our end-to-end network produces results that are comparable to the current state of the art (SOTA) that relies on a two-steps approach that first uses conditional generative adversarial networks to synthesize artifact-free images from the Post-CT images and then uses an active shape model-based method to segment the ICA in the synthetic images. Our method requires a fraction of the time needed by the SOTA, which is important for end-user acceptance.

Joint Feature Synthesis and Embedding: Adversarial Cross-Modal Retrieval Revisited.

Recently, generative adversarial network (GAN) has shown its strong ability on modeling data distribution via adversarial learning. Cross-modal GAN, which attempts to utilize the power of GAN to model the cross-modal joint distribution and to learn compatible cross-modal features, is becoming the research hotspot. However, the existing cross-modal GAN approaches typically 1) require labeled multimodal data of massive labor cost to establish cross-modal correlation; 2) utilize the vanilla GAN model that results in unstable training procedure and meaningless synthetic features; and 3) lack of extensibility for retrieving cross-modal data of new classes. In this article, we revisit the adversarial learning in existing cross-modal GAN methods and propose Joint Feature Synthesis and Embedding (JFSE), a novel method that jointly performs multimodal feature synthesis and common embedding space learning to overcome the above three shortcomings. Specifically, JFSE deploys two coupled conditional Wassertein GAN modules for the input data of two modalities, to synthesize meaningful and correlated multimodal features under the guidance of the word embeddings of class labels. Moreover, three advanced distribution alignment schemes with advanced cycle-consistency constraints are proposed to preserve the semantic compatibility and enable the knowledge transfer in the common embedding space for both the true and synthetic cross-modal features. All these add-ons in JFSE not only help to learn more effective common embedding space that effectively captures the cross-modal correlation but also facilitate to transfer knowledge to multimodal data of new classes. Extensive experiments are conducted on four widely used cross-modal datasets, and the comparisons with more than ten state-of-the-art approaches show that our JFSE method achieves remarkably accuracy improvement on both standard retrieval and the newly explored zero-shot and generalized zero-shot retrieval tasks.

Semi-Supervised Face Frontalization in the Wild

Synthesizing a frontal view face from a single nonfrontal image, i.e. face frontalization, is a task of practical importance in a wide range of facial image analysis applications. However, to train the frontalization model in a supervised manner, most existing face frontalization methods rely on the availability of nonfrontal-frontal face pairs (typically from the Multi-PIE dataset) captured in a constrained environment. Such approaches, in return, limit the generalizability of their application to unconstrained scenarios. Unfortunately, although a large amount of in-the-wild face datasets are available, they cannot easily be utilized for face frontalization training since the nonfrontal and frontal facial images are not paired. To train a frontalization network which generalizes well to both constrained and unconstrained environments, we propose a semi-supervised learning framework which effectively uses both (labeled) indoor and (unlabeled) outdoor faces. Specifically, to achieve this goal, this article presents a Cycle-Consistent Face Frontalization Generative Adversarial Network (CCFF-GAN) which consists of both (1) the supervised and (2) the unsupervised components. For (1), we use the indoor paired (labeled) data to learn a roughly accurate frontalization network which may not generalize well to outdoor (in-the-wild) scenarios. For (2), to cope with the generalization issue, the unsupervised part uses the unpaired (unlabeled) images under the perceptual cycle consistency constraint in the semantic feature space to generalize the network from controlled (indoor) to uncontrolled (outdoor) environment. Extensive experiments demonstrate the effectiveness of the proposed method in comparison with the state-of-the-art face frontalization methods, especially under the in-the-wild scenarios.

Separate in Latent Space: Unsupervised Single Image Layer Separation

Many real world vision tasks, such as reflection removal from a transparent surface and intrinsic image decomposition, can be modeled as single image layer separation. However, this problem is highly ill-posed, requiring accurately aligned and hard to collect triplet data to train the CNN models. To address this problem, this paper proposes an unsupervised method that requires no ground truth data triplet in training. At the core of the method are two assumptions about data distributions in the latent spaces of different layers, based on which a novel unsupervised layer separation pipeline can be derived. Then the method can be constructed based on the GANs framework with self-supervision and cycle consistency constraints, etc. Experimental results demonstrate its successfulness in outperforming existing unsupervised methods in both synthetic and real world tasks. The method also shows its ability to solve a more challenging multi-layer separation task.