Image Style Transfer Research Articles

LaneTD: Lane Feature Aggregator Based on Transformer and Dilated Convolution

Deep-learning-based algorithms successfully improve the detection accuracy of traffic lanes. Most of them employed stacked convolutional neural networks (CNNs) to extract semantics, but they have trouble gathering global visual information. What is more, the stacked CNNs perform multiple downsampling, which reduces the resolution of feature map, causing the tiny features easy to be ignored. To address these problems, LaneTD is proposed: a novel lane feature aggregator that extracts and fuses local and global features. This method adopts dilated convolution to extract local features of various scales and enhances global representation based on the transformer encoder. The local information and global information are then weighted fused using a bidirectional feature pyramid. In addition, we apply unsupervised image style transfer to improve the performance in nighttime and dark scenarios. The method is quantitatively evaluated on the public challenging CULane dataset, and the result shows that the proposed method significantly improves the accuracy and computational efficiency of lane detection in different scenarios. Furthermore, an ablation study is carried out, and a discussion of efficiency tradeoff choices can be applied in practice.

Garment image style transfer based on deep learning

Before neural networks, image style transfer procedures had a common idea: analyze images with a certain style, build a mathematical or statistical model for the style, and then change the image to be transferred, so that it better fits the established model. In this paper, k-means and semantic closed natural matting algorithm is combined for image segmentation, the style and content in the image are extracted based on neural network, and the resulting image is synthesized by image reconstruction technology to realize the migration of national costume styles. Due to the serious artifacts of the output image, an improved image style transfer algorithm is adopted to constrain the transformation from the input image to the output image in the local affine transformation of the color space, and this constraint is expressed as a completely differentiable parameter term, image distortion is suppressed effectively. In the process of real photo style transfer, there is also space inconsistency. Smoothing is done to ensure that the space style is consistent after style processing, it greatly speeds up the operation speed. On NVIDIA GTX1080TI graphics card, algorithm is tested with 256×256 resolution images. It includes three indicators of average running time, memory usage and the number of styles generated by a single model, which are 0.06 s, 136.06 MB and 1 respectively. These indicators can reflect the efficiency and flexibility of the algorithm.

A Compositional Transformer Based Autoencoder for Image Style Transfer

Image style transfer has become a key technique in modern photo-editing applications. Although significant progress has been made to blend content from one image with style from another image, the synthesized image may have a hallucinatory effect when the texture from the style image is rich when processing high-resolution image style transfer tasks. In this paper, we propose a novel attention mechanism, named compositional attention, to design a compositional transformer-based autoencoder (CTA) to solve this above-mentioned issue. With the support from this module, our model is capable of generating high-quality images when transferring from texture-riched style images to content images with semantics. Additionally, we embed region-based consistency terms in our loss function for ensuring internal structure semantic preservation in our synthesized image. Moreover, information theory-based CTA is discussed and Kullback–Leibler divergence loss is introduced to preserve more brightness information for photo-realistic style transfer. Extensive experimental results based on three benchmark datasets, namely Churches, Flickr Landscapes, and Flickr Faces HQ, confirmed excellent performance when compared to several state-of-the-art methods. Based on a user study assessment, the majority number of users, ranging from 61% to 66%, gave high scores on the transfer effects of our method compared to 9% users who supported the second best method. Further, for the questions of realism and style transfer quality, we achieved the best score, i.e., an average of 4.5 out of 5 compared to other style transfer methods.

Structure-Aware Feature Disentanglement With Knowledge Transfer for Appearance-Changing Place Recognition.

Long-term visual place recognition (VPR) is challenging as the environment is subject to drastic appearance changes across different temporal resolutions, such as time of the day, month, and season. A wide variety of existing methods address the problem by means of feature disentangling or image style transfer but ignore the structural information that often remains stable even under environmental condition changes. To overcome this limitation, this article presents a novel structure-aware feature disentanglement network (SFDNet) based on knowledge transfer and adversarial learning. Explicitly, probabilistic knowledge transfer (PKT) is employed to transfer knowledge obtained from the Canny edge detector to the structure encoder. An appearance teacher module is then designed to ensure that the learning of appearance encoder does not only rely on metric learning. The generated content features with structural information are used to measure the similarity of images. We finally evaluate the proposed approach and compare it to state-of-the-art place recognition methods using six datasets with extreme environmental changes. Experimental results demonstrate the effectiveness and improvements achieved using the proposed framework. Source code and some trained models will be available at http://www.tianshu.org.cn.

Multi-Modal Multi-Stage Underwater Side-Scan Sonar Target Recognition Based on Synthetic Images

Due to the small sample size of underwater acoustic data and the strong noise interference caused by seabed reverberation, recognizing underwater targets in Side-Scan Sonar (SSS) images is challenging. Using a transfer-learning-based recognition method to train the backbone network on a large optical dataset (ImageNet) and fine-tuning the head network with a small SSS image dataset can improve the classification of sonar images. However, optical and sonar images have different statistical characteristics, directly affecting transfer-learning-based target recognition. In order to improve the accuracy of underwater sonar image classification, a style transformation method between optical and SSS images is proposed in this study. In the proposed method, objects with the SSS style were synthesized through content image feature extraction and image style transfer to reduce the variability of different data sources. A staged optimization strategy using multi-modal data effectively captures the anti-noise features of sonar images, providing a new learning method for transfer learning. The results of the classification experiment showed that the approach is more stable when using synthetic data and other multi-modal datasets, with an overall accuracy of 100%.

Self-training adversarial learning for cross-domain retinal OCT fluid segmentation

Accurate measurements of the size, shape and volume of macular edema can provide important biomarkers to jointly assess disease progression and treatment outcome. Although many deep learning-based segmentation algorithms have achieved remarkable success in semantic segmentation, these methods have difficulty obtaining satisfactory segmentation results in retinal optical coherence tomography (OCT) fluid segmentation tasks due to low contrast, blurred boundaries, and varied distribution. Moreover, directly applying a well-trained model on one device to test the images from other devices may cause the performance degradation in the joint analysis of multi-domain OCT images. In this paper, we propose a self-training adversarial learning framework for unsupervised domain adaptation in retinal OCT fluid segmentation tasks. Specifically, we develop an image style transfer module and a fine-grained feature transfer module to reduce discrepancies in the appearance and high-level features of images from different devices. Importantly, we transfer the target images to the appearance of source images to ensure that no image information of the source domain for supervised training is lost. To capture specific features of the target domain, we design a self-training module based on a discrepancy and similarity strategy to select the images with better segmentation results from the target domain and then introduce them into the source domain for the iterative training segmentation model. Extensive experiments on two challenging datasets demonstrate the effectiveness of our proposed method. In Particular, our proposed method achieves comparable results on cross-domain retinal OCT fluid segmentation compared with the state-of-the-art methods.

TsrNet: A two-stage unsupervised approach for clothing region-specific textures style transfer

Style transfer has been widely applied in the fashion industry for design assistance. This paper focuses on object-level image style transfer for specific texture regions on fashion images. The current methods mostly utilize extra conditional inputs to obtain regions of interest in fashion images, which take extensive time and cost to prepare these data. To address this issue, we propose a two-stage unsupervised approach named TsrNet for clothing texture style transfer between images. Firstly, we construct a network named Mask-GAN to unsupervisedly split out the clothing texture region. Secondly, we improve CycleGAN to interchange the texture on specific target regions of the two images. Specifically, considering the diversity of fashion texture, we construct a multiscale module (Drf-module) with dynamic perceptual fields and design an image gradient-based texture structure loss (GTS) to perform texture transfer. Qualitative and quantitative experimental results show the effectiveness of our algorithm for clothing-specific region style transfer.

Towards harmonized regional style transfer and manipulation for facial images

Regional facial image synthesis conditioned on a semantic mask has achieved great attention in the field of computational visual media. However, the appearances of different regions may be inconsistent with each other after performing regional editing. In this paper, we focus on harmonized regional style transfer for facial images. A multi-scale encoder is proposed for accurate style code extraction. The key part of our work is a multi-region style attention module. It adapts multiple regional style embeddings from a reference image to a target image, to generate a harmonious result. We also propose style mapping networks for multi-modal style synthesis. We further employ an invertible flow model which can serve as mapping network to fine-tune the style code by inverting the code to latent space. Experiments on three widely used face datasets were used to evaluate our model by transferring regional facial appearance between datasets. The results show that our model can reliably perform style transfer and multi-modal manipulation, generating output comparable to the state of the art.

Image and Video Style Transfer Based on Transformer

The essence of image style transfer is to generate images that both maintain in the original content image and present the effect with artistic features under the guidance of style images. Deep learning’s quick rise has resulted in even more achievements in image style transfer, an otherwise popular study area. Nevertheless, due to the limitations of Convolutional Neural Networks (CNN), extracting and retaining the input images is problematic. Therefore, image style transfer based on traditional CNN is biased in the representation of content images. To address the above problems, this paper proposes STLTSF (Style Transfer based on Transformer), a transformer-based method that may achieve image style transfer based on the long-range dependencies of the input images. Compared with traditional visual transformers, STLTSF has two different transformer encoders, one for generating domain-specific content and the other for generating styles. We may convert the encoder to a decoding method that can be styled based on content sequences by using numerous layers of transformers. The suggested STLTSF approach outperforms traditional CNN-based methods in both qualitative and quantitative experiments.

Improved Image Style Transfer Based on VGG-16 Convolutional Neural Network Model

In the field of computer vision, image style transfer is an important research direction. With the promotion of artificial intelligence, this technology is becoming more and more popular. Compared with the current image style transfer technology based on artificial intelligence, the traditional technology appears to be complex, time-consuming and inefficient. Migration technology is mainly based on images of the current mainstream style VGG - 16 convolution neural network model, adopts TensorFlow framework, in this article to a picture before the smart migration style photo style, texture features such as the methods of improvement, by modifying the style loss function, makes the image content images can transfer a variety of style, Finally, it is verified by relevant experiments.

Image style transfer via curved stroke rendering

Image style transfer via curved stroke rendering

Optimal Transport-Based Patch Matching for Image Style Transfer

State-of-the-art image style transfer methods have achieved impressive results by using neural networks. However, neural style transfer (NST) methods either ignore the local details of the style image by using the global statistics for style modeling or cannot fully use shallow features of neural networks, leading to the synthesized image having fewer details. In this study, we proposed a new patch-based style transfer method that directly operates in the image pixel domain without using any neural networks, achieving fascinating style transfer results with rich image details. The proposed method was derived from classic texture synthesis methods. Most previous methods rely on nearest neighbor search (NNS) for patch matching. However, this greedy strategy cannot guarantee the similarity of patch distributions between the synthesized image and the style image, which limits the expressiveness of textures. We solved this problem by proposing an optimal patch matching algorithm formed on the Optimal Transport (OT) theory, which theoretically guarantees the similarity of the patch distributions and gives a flexible style modeling method. Various qualitative and quantitative experiments demonstrated that the proposed method achieves better synthesized results than state-of-the-art style transfer methods, including NST and classic methods based on texture synthesis.

Application of Image Style Transfer Based on Normalized Residual Network in Art Design

Application of Image Style Transfer Based on Normalized Residual Network in Art Design

Memory-Contrastive Unsupervised Domain Adaptation for Building Extraction of High-Resolution Remote Sensing Imagery

Deep learning-based semantic segmentation has been widely applied for building extraction. However, due to the domain gap, the extraction of building in high-resolution remote sensing imagery is difficult when the model trained on a source dataset is directly used to test on a target data. Considering that humans can retrieve memory to deal with correlative tasks in different domains, memory mechanisms have been developed effectively to assist cross-domain feature extraction. However, whether the memory mechanisms can achieve satisfactory result or not highly depends on the premise that the memory is relevant to the task. Therefore, the domain-invariant memory is crucial in cross-domain building extraction task. To this end, a memory-contrastive unsupervised domain adaptation method is proposed on the basis of a novel memory mechanism. Specifically, to facilitate the model to memorize domain-invariant features, we first conduct a normalization-based image style transfer strategy and a discriminator-based adversarial method at the image level and feature level, respectively. Subsequently, we carry out a memory-contrastive module to obtain domain-invariant features. Especially, a teacher–student network is exploited to help knowledge transferring by knowledge distillation to enhance the performance of the memory-contracted module. To narrow the distance between the two domains, a memory bank is designed to store and update category features obtained from the source domain, and then the similarity between category features in the target domain and memory bank is calculated. Results of the cross-domain experiments show that the proposed method can achieve optimal building extraction. (GitHub: https://github.com/RS-CSU/MDANet).

Retracted: Neural Network-Based Mapping Mining of Image Style Transfer in Big Data Systems.

[This retracts the article DOI: 10.1155/2021/8387382.].

Research on Training and Optimization of Image Style Transfer Model Based on CoreML

Research on Training and Optimization of Image Style Transfer Model Based on CoreML

Arbitrary Style Transfer With Fused Convolutional Block Attention Modules

The advancement of deep learning has rendered image style transfer a progressively intricate subject matter. The proposed solution aims to tackle the limitations of current methods in retaining the content image object contours and avoiding blurred image boundaries and mismatched color matching after stylization. To achieve this, an arbitrary-style transfer network is introduced, which leverages the attention mechanism. The network comprises an encoder-decoder module, a convolutional block attention module (CBAM), and an adaptive attention normalization network (AdaAttN) module. The CBAM attention mechanism is presented as an extension of the AdaAttN network, with the aim of producing stylized images that exhibit both global and local style coordination. This is achieved by leveraging long-range dependencies in the image. Additionally, a novel loss function, referred to as the structural similarity loss, is proposed to enhance the consistency of the generated images with respect to the underlying content structure. Finally, a new local feature loss is introduced to further enhance the visual quality of the stylized images at a local level. The study involved conducting style transfer training on a dataset comprising 82,783 real images and 81,446 artistic images. Furthermore, an additional set of 1,000 resultant images, generated from 100 real photos and 10 artistic portraits, was utilized for testing purposes. The study compares the experimental outcomes with four contemporary-style transfer techniques. Additionally, the efficacy of the CBAM module and SSIM loss function is demonstrated through ablation experiments. The findings of the experiment demonstrate that the network proposed has the ability to effectively adapt to the local style and can adeptly correspond the semantically proximate style features to the content features, thereby preserving superior spatial consistency.

Artistic Image Style Transfer Based on Laplacian Pyramid Network

Artistic Image Style Transfer Based on Laplacian Pyramid Network

TET: Text emotion transfer

Text style transfer aims at transforming the style of a piece of text while keeping its primary content. The style of the text is usually defined as a particular writing tone in different categories, such as formality, politeness, sentiment, and political slant. Recently, most of the work in the area has been devoted to the problem of sentiment transfer, which tries to transfer an opinionated text into a positive or negative perspective. It has applications in marketing, political news, chatbots, writing tools, and many others. On the other hand, emotions as the basic forms of sentiments have brought many attentions to different tasks, including image style transfer but they are not well expressed in text style transfer yet. This article presents a text emotion transfer model that transforms the style of a text to each of the predefined ‘anger’, ‘fear’, ‘joy’, and ‘sadness’ emotions. Relying on masked language modeling and transfer learning, the proposed model can perform efficiently on limited amounts of emotion-annotated data. Moreover, the model shows promising experimental results against other existing models considering style transfer accuracy, content preservation, and fluency in the ISEAR and TEC emotion corpora.

Neural Image Style Transfer Using Modified Histogram Matching

Neural Image Style Transfer is an algorithm which was first introduced in 2016 for transferring the style of an image using a CNN. Over the years, a large portion of the research has been devoted in balancing the trade-off between time taken by the algorithm to process the images and the quality of the results generated. In addition, the application of traditional image processing algorithms has been limited. In this paper, we aim to introduce a modification to the traditional method by utilizing a localized histogram matching algorithm combined with Contrast Limited Adaptive Histogram Equalization (CLAHE). The results show that the quality of style transferred and the colour of images obtained by this method is much better than the traditional method for the same number of iterations.