Unsupervised Translation Research Articles

Translation of MFL and UT data by using generative adversarial networks: A comparative study

Magnetic flux leakage (MFL) and ultrasonic testing (UT) are widely used in-line inspection technologies to detect corrosion defects along pipelines. The integration of MFL and UT data has the potential to provide complementary insights that facilitate a comprehensive assessment of pipeline integrity. However, due to the inherent dissimilarity with their underlying physical principles, these techniques yield notable disparities in signal characteristics, posing challenges in integrating these multimodal data. This study aims to establish a translation mapping between MFL and UT signals to achieve consistent physical interpretations across the two modalities. Thus, this study explored the feasibility of generative adversarial network (GAN) based models encompassing both supervised and unsupervised translation approaches contingent on the availability of aligned data. Furthermore, two translation modes, MFL-UT and UT-MFL, were analyzed, respectively, to understand the effectiveness of the translation direction. The experimental results demonstrate satisfactory performance for both aligned and unaligned data translation, with the UT-MFL translation direction yielding superior results. Overall, the translation approaches pave the way for future applications, especially in subsequent data analysis tasks such as registration, comparison, and fusion of multimodal data.

Mismatching-aware unsupervised translation quality estimation for low-resource languages

Mismatching-aware unsupervised translation quality estimation for low-resource languages

SeRL: Style-embedding representation learning for unsupervised CT images synthesis from unpaired MR images

Hepatocellular carcinoma (HCC) is the third leading cause of cancer-related deaths worldwide, and the fastest-growing cause of cancer deaths in the United States. Computed tomography (CT) and magnetic resonance (MR) imaging are the key image-based examination methods for HCC patients. Patients may alternatively undergo CT and MR exams given a radiation dose of the former and the cost of the latter. Due to organ representation differences between CT and MR, it’s a great challenge to detect the tumors from a time series of data consisting of both CT and MR by a single model. The annotations for these time series data will consume much time and labor for the physicians. Thus, we propose our style-embedding representation learning (SeRL) for unsupervised and unpaired abdomen CT and MR translation. Different from current medical image translation models, the style-representation information from real CT and real MR images has been embedded in the translation process to bypass some local minima during the convergence process and improve the synthesis results. Our patch-corrosion augmentation method enhances the style-embedding representation learning by bringing more diversity to the training data. Combined with the self-attention module, SeRL eliminates the noise caused by low grayscale pixel values during translation. Results on the unpaired HCC patient's CT and MR images show that our proposed SeRL is able to generate high quality CT images from MR ones. Evaluations such as Frechet Inception Distance (FID), Sliced Wasserstein Distance (SWD), and liver segmentation dice score are utilized to demonstrate our advantages over other state-of-the-art unsupervised methods.

Unsupervised image-to-image translation in multi-parametric MRI of bladder cancer

Detection of muscular invasive bladder cancer (MIBC) is critical for surgical selection of bladder cancer (BCa) patients. Currently, multi-parameter magnetic resonance imaging (mp-MRI) is the predominant approach for identifying MIBC. However, mp-MRI is still insufficient due to the presence of noise and artifacts. Our research aims to synthesize images from the existing sequences of mp-MRI to substitute missing or low signal-to-noise ratio sequences through image-to-image (I2I) translation. Using mp-MRI images of 255 BCa patients collected in our department, we here propose a one-to-many unsupervised I2I translation network with region-wise semantic enhancement to synthesize virtual samples. We introduce an improved adaptive instance normalization module to support the generator for synthesizing multi-domain images. In addition, a branch for region-wise semantic segmentation helps the generator to enhance the quality of image translation for a specific region. A semantically consistent loss is applied to maintaining the consistency between the synthesized and the input images via region-wise semantic segmentation. Experiments on the BraTS and BCa datasets indicate that our I2I translation approach outperforms several state of the art methods. Additionally, we perform clinical feasibility tests using the synthesis images. The clinicians reach a consensus between the Vesical Imaging Reporting and Data System (VI-RADS) scoring results from the synthesized and the real mp-MRI images. In addition, after the BCa training set has been expanded using the proposed generation model, the accuracy of the BCa muscular invasion classification is improved from 77.78% to 85.19%.

Unsupervised cross-domain translation via deep learning and adversarial attention neural networks and application to music-inspired protein designs

Taking inspiration from nature about how to design materials has been a fruitful approach, used by humans for millennia. In this paper we report a method that allows us to discover how patterns in disparate domains can be reversibly related using a computationally rigorous approach, the AttentionCrossTranslation model. The algorithm discovers cycle- and self-consistent relationships and offers a bidirectional translation of information across disparate knowledge domains. The approach is validated with a set of known translation problems, and then used to discover a mapping between musical data-based on the corpus of note sequences in J.S. Bach's Goldberg Variations created in 1741-and protein sequence data-information sampled more recently. Using protein folding algorithms, 3D structures of the predicted protein sequences are generated, and their stability is validated using explicit solvent molecular dynamics. Musical scores generated from protein sequences are sonified and rendered into audible sound.

Unsupervised many-to-many stain translation for histological image augmentation to improve classification accuracy

BackgroundDeep learning tasks, which require large numbers of images, are widely applied in digital pathology. This poses challenges especially for supervised tasks since manual image annotation is an expensive and laborious process. This situation deteriorates even more in the case of a large variability of images. Coping with this problem requires methods such as image augmentation and synthetic image generation. In this regard, unsupervised stain translation via GANs has gained much attention recently, but a separate network must be trained for each pair of source and target domains. This work enables unsupervised many-to-many translation of histopathological stains with a single network while seeking to maintain the shape and structure of the tissues. MethodsStarGAN-v2 is adapted for unsupervised many-to-many stain translation of histopathology images of breast tissues. An edge detector is incorporated to motivate the network to maintain the shape and structure of the tissues and to have an edge-preserving translation. Additionally, a subjective test is conducted on medical and technical experts in the field of digital pathology to evaluate the quality of generated images and to verify that they are indistinguishable from real images. As a proof of concept, breast cancer classifiers are trained with and without the generated images to quantify the effect of image augmentation using the synthetized images on classification accuracy. ResultsThe results show that adding an edge detector helps to improve the quality of translated images and to preserve the general structure of tissues. Quality control and subjective tests on our medical and technical experts show that the real and artificial images cannot be distinguished, thereby confirming that the synthetic images are technically plausible. Moreover, this research shows that, by augmenting the training dataset with the outputs of the proposed stain translation method, the accuracy of breast cancer classifier with ResNet-50 and VGG-16 improves by 8.0% and 9.3%, respectively. ConclusionsThis research indicates that a translation from an arbitrary source stain to other stains can be performed effectively within the proposed framework. The generated images are realistic and could be employed to train deep neural networks to improve their performance and cope with the problem of insufficient numbers of annotated images.

Unsupervised English Intelligent Machine Translation in Wireless Network Environment

Researchers suggest unsupervised English machine translation to address the absence of parallel corpus in English translation. Unsupervised pretraining techniques, denoising autoencoders, back translation, and shared latent representation mechanisms are used to simulate the translation task using just monolingual corpora. This paper uses pseudo-parallel data to construct unsupervised neural machine translation (NMT) and dissimilar language pair analysis. This paper firstly analyzes the low performance of unsupervised translation on dissimilar language pairs from three aspects: bilingual word embedding quality, shared words, and word order. And artificial shared word replacement and preordering strategies are proposed to increase the shared words between dissimilar language pairs and reduce the difference in their syntactic structure, thereby improving the translation performance on dissimilar language pairs. The denoising autoencoder and shared latent representation mechanism in unsupervised English machine translation are only required in the early stage of training, and learning the shared latent representation limits the further improvement of performance in different directions. While training the denoising autoencoder by repeatedly altering the training data slows down the convergence of the model, this is especially true for divergent languages. This paper presents an unsupervised NMT model based on pseudo-parallel data to address this issue. It trains two standard supervised neural machine translation models using the pseudo-parallel corpus generated by the unsupervised neural machine translation system, which enhances translation performance and speeds convergence. Finally, the English intelligent translation model is deployed in the wireless network server, and users can access it through the wireless network.

Re‐EnGAN: Unsupervised image‐to‐image translation based on reused feature encoder in CycleGAN

As an advanced task for image synthesis without labelled data, unsupervised image-to-image translation refers to the overall conversion of a certain characteristic image domain X to another domain Y. The key point is to learn a mapping relationship that can be transformed between different image domains. Existing methods mainly adopt GANs to generate authentic images. While the discriminators will be abandoned after the training process is completed. In order to avoid this waste of training resources, a feature encoder reusing method is proposed, which could reduce the number of parameters and accelerate the training speed. In addition, we add the adaptive perceptual loss for the purpose of paying more attention to the quality of generated images. This loss directly uses the encoder during training to perform feature-level constraints, and applies the L1-norm on the intermediate feature layer of the generated samples. The experiments illustrate that our framework can generate more natural images and provide an effective solution for unsupervised translation.

IPGAN: Identity-Preservation Generative Adversarial Network for unsupervised photo-to-caricature translation

Photo-to-caricature translation is an extremely challenging task because there are not only texture differences between caricatures and photos, but also various spatial deformations in caricatures. Most of existing methods tend to introduce difficult obtained additional information such as precise facial landmarks to guide caricature generation. In addition, identity preservation is a crucial characteristic of caricatures, but unfortunately there seems to be few methods to consider it. Motivated by the aforementioned observations, we propose an Identity-Preservation Generative Adversarial Network (IPGAN) for unsupervised photo-to-caricature translation. In particular, considering the importance of identity retention, we propose a novel identity preservation loss to hold the identity information of original photos and improve the quality of generated caricatures. To capture realistic caricature styles, we design a style differentiation loss to help our model produce caricatures with styles that remarkably differ from photos. Moreover, to learn satisfactory deformations without supervision, our model uses a warp controller to acquire exaggerations automatically that enable to customize diverse exaggerations. As an unsupervised translation method, our IPGAN can also be applied to caricature-to-photo translation. Experiments on the WebCaricature dataset suggest that our IPGAN achieves state-of-the-art performance and can generate realistic as well as identity preservation caricatures.

Unsupervised word translation for English-Hindi with different retrieval techniques

Word translation or incorporation of bilingual dictionaries is an important capability that impacts many multilingual language processing tasks. For translation from one language to another language, we either relied on parallel data or bilingual dictionaries. In this paper, we address this problem and generate best cross-lingual word embedding for English-Hindi language pair. Here, we neither use an aligned document or sentence aligned corpus, nor any bilingual dictionary. We are following the assumption of intra lingual similarity distribution that for the most frequent word the distribution graph is similar between Hindi and English corpus and embeddings are isometric. These cross-lingual words embedding can be used for unsupervised neural machine translation and cross-lingual transfer learning. Different retrieval techniques nearest neighbour, inverted nearest neighbours retrieval, inverted Softmax, and cross-lingual word scaling are performed and compared for the bi-lingual embedding of English-Hindi, which is trained for unsupervised and semi-supervised ways by passing seed dictionary. Bi-lingual word embedding is tested on generated English-Hindi dictionary.

Unsupervised Real-World Image Super-Resolution via Dual Synthetic-to-Realistic and Realistic-to-Synthetic Translations

Due to the challenges of collecting paired low-resolution (LR) and high-resolution (HR) images in real-world scenarios, most existing deep convolutional neural network (CNN)-based single image super-resolution (SR) models are trained with artificially synthesized LR-HR image pairs. However, the domain gap between the synthetic data for model training and the realistic data for testing degrades SR performance significantly, which discourages the application of SR models in practice. One possible solution is to learn from unpaired real-world LR and HR images for their accessibility. Predominant strategies are mainly based on unsupervised domain translation. Despite great advances, there are still noticeable domain gaps between the realistic-like/synthetic-like images generated by unpaired translation and the true realistic/synthetic ones. To address this problem, this letter proposes an effective unsupervised SR framework based on dual synthetic-to-realistic and realistic-to-synthetic translations, namely DTSR. Specifically, to bridge the domain gap between testing and training data, the SR model is optimized using HR images and their realistic-like LR counterparts produced by the synthetic-to-realistic translation. In turn, we propose to narrow the domain gap further via applying the realistic-to-synthetic translation to realistic LR images prior to super-resolving, which also makes the SR model super-resolve simpler examples in testing relative to model training. Moreover, focal frequency and bilateral filtering losses are particularly introduced into DTSR for better details restoration and artifacts suppression. Extensive experiments show that our DTSR outperforms several state-of-the-art models in terms of both quantitative and qualitative comparisons.

Fully unsupervised word translation from cross-lingual word embeddings especially for healthcare professionals

Fully unsupervised word translation from cross-lingual word embeddings especially for healthcare professionals

Learning Cycle-Consistent Cooperative Networks via Alternating MCMC Teaching for Unsupervised Cross-Domain Translation

This paper studies the unsupervised cross-domain translation problem by proposing a generative framework, in which the probability distribution of each domain is represented by a generative cooperative network that consists of an energy-based model and a latent variable model. The use of generative cooperative network enables maximum likelihood learning of the domain model by MCMC teaching, where the energy-based model seeks to fit the data distribution of domain and distills its knowledge to the latent variable model via MCMC. Specifically, in the MCMC teaching process, the latent variable model parameterized by an encoder-decoder maps examples from the source domain to the target domain, while the energy-based model further refines the mapped results by Langevin revision such that the revised results match to the examples in the target domain in terms of the statistical properties, which are defined by the learned energy function. For the purpose of building up a correspondence between two unpaired domains, the proposed framework simultaneously learns a pair of cooperative networks with cycle consistency, accounting for a two-way translation between two domains, by alternating MCMC teaching. Experiments show that the proposed framework is useful for unsupervised image-to-image translation and unpaired image sequence translation.

Unsupervised colonoscopic depth estimation by domain translations with a Lambertian-reflection keeping auxiliary task.

A three-dimensional (3D) structure extraction technique viewed from a two-dimensional image is essential for the development of a computer-aided diagnosis (CAD) system for colonoscopy. However, a straightforward application of existing depth-estimation methods to colonoscopic images is impossible or inappropriate due to several limitations of colonoscopes. In particular, the absence of ground-truth depth for colonoscopic images hinders the application of supervised machine learning methods. To circumvent these difficulties, we developed an unsupervised and accurate depth-estimation method. We propose a novel unsupervised depth-estimation method by introducing a Lambertian-reflection model as an auxiliary task to domain translation between real and virtual colonoscopic images. This auxiliary task contributes to accurate depth estimation by maintaining the Lambertian-reflection assumption. In our experiments, we qualitatively evaluate the proposed method by comparing it with state-of-the-art unsupervised methods. Furthermore, we present two quantitative evaluations of the proposed method using a measuring device, as well as a new 3D reconstruction technique and measured polyp sizes. Our proposed method achieved accurate depth estimation with an average estimation error of less than 1 mm for regions close to the colonoscope in both of two types of quantitative evaluations. Qualitative evaluation showed that the introduced auxiliary task reduces the effects of specular reflections and colon wall textures on depth estimation and our proposed method achieved smooth depth estimation without noise, thus validating the proposed method. We developed an accurate depth-estimation method with a new type of unsupervised domain translation with the auxiliary task. This method is useful for analysis of colonoscopic images and for the development of a CAD system since it can extract accurate 3D information.

Multimodal Unsupervised Speech Translation for Recognizing and Evaluating Second Language Speech

This paper addresses an automatic proficiency evaluation and speech recognition for second language (L2) speech. The proposed method recognizes the speech uttered by the L2 speaker, measures a variety of fluency scores, and evaluates the proficiency of the speaker’s spoken English. Stress and rhythm scores are one of the important factors used to evaluate fluency in spoken English and are computed by comparing the stress patterns and the rhythm distributions to those of native speakers. In order to compute the stress and rhythm scores even when the phonemic sequence of the L2 speaker’s English sentence is different from the native speaker’s one, we align the phonemic sequences based on a dynamic time-warping approach. We also improve the performance of the speech recognition system for non-native speakers and compute fluency features more accurately by augmenting the non-native training dataset and training an acoustic model with the augmented dataset. In this work, we augment the non-native speech by converting some speech signal characteristics (style) while preserving its linguistic information. The proposed variational autoencoder (VAE)-based speech conversion network trains the conversion model by decomposing the spectral features of the speech into a speaker-invariant content factor and a speaker-specific style factor to estimate diverse and robust speech styles. Experimental results show that the proposed method effectively measures the fluency scores and generates diverse output signals. Also, in the proficiency evaluation and speech recognition tests, the proposed method improves the proficiency score performance and speech recognition accuracy for all proficiency areas compared to a method employing conventional acoustic models.

Using Dependency-Based Contextualization for transferring Passive Constructions from English to Spanish

We hypothesize that parallel corpora as well as machine translation outputs contain many literal translations that are the result of transferring the constructions of the source language to the target language. When translating passive expressions from English to Spanish, there are several constructions available, however, both automatic and human (if of low quality) translations tend to select the periphrastic structure, which is the literal construction. The objective of this article is to make use of strategies trained on monolingual corpora to translate English passive expressions into Spanish so as to verify whether unsupervised translation with monolingual corpora benefits syntactic diversity. Special attention will be given to the monolingual-based strategy relying on dependency-based contextualization. The results of the experiments carried out show that the methods relying on monolingual corpora tend to offer more non-literal translations (middle-voice) than those trained on parallel corpora.

Unsupervised word translation for English-Hindi with different retrieval techniques

Unsupervised word translation for English-Hindi with different retrieval techniques

Unsupervised Bitext Mining and Translation via Self-Trained Contextual Embeddings

We describe an unsupervised method to create pseudo-parallel corpora for machine translation (MT) from unaligned text. We use multilingual BERT to create source and target sentence embeddings for nearest-neighbor search and adapt the model via self-training. We validate our technique by extracting parallel sentence pairs on the BUCC 2017 bitext mining task and observe up to a 24.5 point increase (absolute) in F1 scores over previous unsupervised methods. We then improve an XLM-based unsupervised neural MT system pre-trained on Wikipedia by supplementing it with pseudo-parallel text mined from the same corpus, boosting unsupervised translation performance by up to 3.5 BLEU on the WMT’14 French-English and WMT’16 German-English tasks and outperforming the previous state-of-the-art. Finally, we enrich the IWSLT’15 English-Vietnamese corpus with pseudo-parallel Wikipedia sentence pairs, yielding a 1.2 BLEU improvement on the low-resource MT task. We demonstrate that unsupervised bitext mining is an effective way of augmenting MT datasets and complements existing techniques like initializing with pre-trained contextual embeddings.

Domain-Independent Natural Language Processing of text using Unsupervised Translation

NLP is one of the very important domains of artificial intelligence. Nowadays, advancements are being made and NLP is one of the most developing fields. In this paper, we offer a mutual use of unsupervised translation with n-grams and Natural Language Processing techniques to challenge the difficulty of unsupervised translation extraction from textual data. To build a Text Meaning Extraction System, we have to deliver one important element which is input text. This studypresented a different algorithm to work out resemblances between natural languages, by using sequence package analysis and changing them into n-grams. Whenever the sentences that are grammatically difficult and quite lengthy are applied to see the results of the presented algorithm, there are quite efficient results in a semantic reaction. To enhance the experience in the field of AI and search engines, this research paper shows how to improve the handling capability of fuzzy concepts within computers. For example, when search jobs are executed in search engines small textual concepts or sentences might be semantically formed to switch the keyword-based queries. This ability may be functional to intelligent agents to even the procedure of communication between humans and machinery.

Unsupervised Word Translation with Adversarial Autoencoder

Crosslingual word embeddings learned from monolingual embeddings have a crucial role in many downstream tasks, ranging from machine translation to transfer learning. Adversarial training has shown impressive success in learning crosslingual embeddings and the associated word translation task without any parallel data by mapping monolingual embeddings to a shared space. However, recent work has shown superior performance for non-adversarial methods in more challenging language pairs. In this article, we investigate adversarial autoencoder for unsupervised word translation and propose two novel extensions to it that yield more stable training and improved results. Our method includes regularization terms to enforce cycle consistency and input reconstruction, and puts the target encoders as an adversary against the corresponding discriminator. We use two types of refinement procedures sequentially after obtaining the trained encoders and mappings from the adversarial training, namely, refinement with Procrustes solution and refinement with symmetric re-weighting. Extensive experimentations with high- and low-resource languages from two different data sets show that our method achieves better performance than existing adversarial and non-adversarial approaches and is also competitive with the supervised system. Along with performing comprehensive ablation studies to understand the contribution of different components of our adversarial model, we also conduct a thorough analysis of the refinement procedures to understand their effects.