Style Transfer Research Articles

Research on the Application of Intelligent Algorithms in Color Matching in Painting

This paper aims to explore the transformation of art creation methods by technology. Color, as the core of visual art, traditionally relies on the artist's personal experience, intuition, and color theory. With the development of artificial intelligence technology, intelligent algorithms have achieved functions such as color intelligent matching, style transfer, and real-time rendering through deep learning and big data analysis, providing new possibilities for artistic creation. Comparative experiments show that intelligent algorithms outperform traditional methods in terms of color harmony, style consistency, and creative efficiency and significantly improve user satisfaction and color matching accuracy.

Community Transferrable Representation Learning for Image Style Classification

Style is prone to experience but hard to formulate, even for design professionals. The current style classification (SC) methods can be categorized into mono-tasking (style) and multi-tasking (style, artists, genre and other additional clues). However, the style strength spread-out phenomenon challenges them by the uncertain noises’ distribution and unstable distinguishable feature representation. The multiple sub-styles make the principal style recognition hard for current SC approaches with lower accuracy. This paper proposes the community transferrable representation learning (CTRL) framework inspired by community detection and style transfer studies. The community feature is the transferrable feature embeddings in latent space through all samples within the same style concept. Therefore, instead of learning the unique style features, we introduce the learnable transferrable embeddings, whose distance will be smaller for the intraclass and larger for the interclass, to represent the principal style concept. There are two generative processes, p-AE and q-AE, as the cooperative operations to learn the community styles by the proposed diamond cells (D-cells) in a stacked way. The proposed community loss includes two terms: the invariance term is used to make the principal style embedding consistent, and the redundancy reduction term can decorate the different vector embeddings of sub-styles. The style classification and style feature learning model are optimized simultaneously in a semi-supervised way to learn the class-level style representation. We conduct experiments on six style datasets (three oil paintings, one architecture, one fashion and one general image). Results show that the proposed framework brings a performance gain of 2%-7% in terms of accuracy compared with the SOTA approaches.

Neural Style Transfer: A Review and Analysis

Neural style transfer (NST) is a fascinating deep learning application that merges the style of one image with the content of another, resulting in visually compelling artistic images. This article reviews multiple research advances in the field of NST, including new style loss based on Sliced Wasserstein distance, VTNet model for video style conversion, Graph Neural Network based image style transfer method, multi style geometric deformation method, 3D neural style conversion, style transfer for 3D grids, traditional Chinese painting style transfer, and artistic conversion of QR codes. Through these studies, the author aims to improve the effectiveness and efficiency of image and video style conversion while maintaining the integrity of content information. The newly proposed style loss function utilizes SWD to improve computational efficiency and theoretically ensures the accuracy of style similarity. VTNet introduces prediction and generation branches to achieve high-quality, real-time video style conversion. In addition, the article also explores semi parametric methods based on GNN, multi style geometric transformations, and style transfer techniques for specific cultural artworks, demonstrating the potential application of neural style transfer in different fields and its contribution to artistic creation and design.

Multi-view Panoramic Image Style Transfer with Multi-scale Attention and Global Sharing

Style transfer for panoramic images is a challenging task, due to the problems associated with its unique structure, including edge discontinuities, pole distortion, fuzzy details, and memory limitation. In this paper, we propose a novel Multi-view Transformation network for Panorama Style Transfer (MuTPST). Firstly, this architecture has a multi-view panoramic transformation mechanism, which includes a multi-view cubic projection and a multi-view equirectangular re-projection of panoramic images. This can address pole distortion and edge discontinuity by skillfully applying multiple types of projections and transformations. To capture different levels of context and structure in the stylization stage, we carefully design a multi-scale attention content encoder, which can coordinate the distribution of visual attention across space and channels. Besides, by the sharing of global style features in thumbnails and patches, MuTPST can process ultra-high resolution panoramic images ( \(e.g.\) , 10,000 \(\times\) 5,000 pixels) with limited GPU memory. Extensive experiments illustrate that the proposed method outperforms the state-of-the-art with a discernible improvement in panoramic image style transfer. More results and interactive features can be found on https://weiyang001.github.io/MuTPST/ .

MLA-net: Multi-layer Attention Network for Arbitrary Style Transfer

MLA-net: Multi-layer Attention Network for Arbitrary Style Transfer

Fish Body Pattern Style Transfer Based on Wavelet Transformation and Gated Attention

To address the temporal jitter with low segmentation accuracy and the lack of high-precision transformations for specific object classes in video generation, we propose the fish body pattern sync-style network for ornamental fish videos. This network innovatively integrates dynamic texture transfer with instance segmentation, adopting a two-stage processing architecture. First, high-precision video frame segmentation is performed using Mask2Former to eliminate background elements that do not participate in the style transfer process. Then, we introduce the wavelet-gated styling network, which reconstructs a multi-scale feature space via discrete wavelet transform, enhancing the granularity of multi-scale style features during the image generation phase. Additionally, we embed a convolutional block attention module within the residual modules, not only improving the realism of the generated images but also effectively reducing boundary artifacts in foreground objects. Furthermore, to mitigate the frame-to-frame jitter commonly observed in generated videos, we incorporate a contrastive coherence preserving loss into the training process of the style transfer network. This enhances the perceptual loss function, thereby preventing video flickering and ensuring improved temporal consistency. In real-world aquarium scenes, compared to state-of-the-art methods, FSSNet effectively preserves localized texture details in generated videos and achieves competitive SSIM and PSNR scores. Moreover, temporal consistency is significantly improved. The flow warping error index decreases to 1.412. We chose FNST (fast neural style transfer) as our baseline model and demonstrate improvements in both model parameter count and runtime efficiency. According to user preferences, 43.75% of participants preferred the dynamic effects generated by this method.

The use of deep learning and artificial intelligence-based digital technologies in art education

In order to explore the application of deep learning (DL) and artificial intelligence (AI) technologies in art education, this work proposes and optimizes an innovative art creation system—Creative Intelligence Cloud (CIC). The system combines a deep generative adversarial network and convolutional neural network, aiming to enhance the automation level, consistency of artistic styles, and creation efficiency in art creation. This work first analyzes existing art creation methods. It points out the shortcomings of traditional systems in terms of image quality, style transfer, and computational performance, especially the application limitations in real teaching scenarios. Therefore, this work designs an art creation model optimized by DL and validates and evaluates it through extensive experiments. The experimental results show that CIC outperforms existing mainstream models in multiple dimensions, including image quality, computational performance, user experience, and style creation. For example, in image quality evaluation, CIC achieves high scores in clarity (0.89), detail performance (0.85), style consistency (0.87), and color accuracy (0.91). In terms of computational performance and resource consumption, CIC shows its superiority, with a training time of only 1500 s, memory consumption of 4.9GB, and a Graphics Processing Unit resource usage rate of 70%. Compared to models such as the Visual Perception Generative Adversarial Network and Artistic Recognition and Transfer Style Convolutional Neural Network, CIC is more efficient and consumes fewer resources. Furthermore, CIC’s scores in user experience and style transfer capability are significantly higher than those of other models, providing smoother and more creatively rich art creation tools for art education. Therefore, this work offers new ideas and methods for the application of DL and AI technologies in art creation and art education, and promotes the practical use of AI in art education. The work has certain academic contributions and practical value.

Deep learning-based style transfer research for traditional woodcut paper horse art images

Deep learning (DL)-based style transfer techniques have been employed to modify traditional woodcut paper horse art images. This art form struggles to maintain its cultural character while adapting to modern digital media. Style transfer, as a DL technique, allows for the combination of the innovative style of one image with the content of another, enabling the digital recreation of conventional aesthetics. Data collection involved the compilation of a dataset consisting of high-resolution images of traditional woodcut paper horse art. The dataset was preprocessed through normalization of image sizes, application of data augmentation techniques to enhance diversity and mitigate overfitting, and filtering of images to isolate specific features of paper horse art, including line work, shading, and texture. This research proposed the Intelligent Pied Kingfisher - Refined Convolutional Neural Network (IPK-RCNN) model, which is trained on both traditional woodcut images and modern digital art, aiming to create a hybrid style that respects traditional aesthetics while integrating contemporary stylistic elements. The style transfer process employs a Region-based Convolutional Neural Network (RCNN) that incorporates IPK optimization for the enhancement of convolutional layer weight parameters, thereby improving both efficiency and accuracy. The IPK-RCNN method effectively preserves the intricate textures and details of woodcut art while incorporating modern stylistic variations. The model achieves high-quality outputs, attaining a Peak Signal-to-Noise Ratio (PSNR) of 35.12 dB, a Structural Similarity Index (SSIM) of 0.96, an Area Under Curve (AUC) of 0.90, and an efficient execution time of 31.17 ms. This approach underscores the potential of DL in the preservation and reimagining of traditional art forms for digital platforms.

Neural Style Transfer and Clothes Segmentation for Creating New Batik Patterns on Clothing Design

Purpose: Applying the original batik image style to other object images and generating new batik patterns that applied to clothing. Methods: This research uses the Neural Style Transfer method to apply object images to batik to produce new batik patterns, and Clothes Segmentation is used to select areas of clothing in the image so the new batik patterns can be applied to clothing images. And Testing using SSIM, LPIPS and PSNR metrics. This research uses Google Colab, batik image data, and clothing mockup images taken from the internet. Result: This study shows high average results on SSIM, LPIPS and fair results on PSNR. The results show that the similarity is relatively high with high detected noise. Novelty: This research develops a new approach in the field of batik pattern innovation and its application to clothing design images. The novelty of this research lies in the implementation of Neural Style Transfer and Clothes Segmentation, which results in a method of exploring new batik patterns and applying them to clothing design images.

Causal Inference-Based Self-Supervised Cross-Domain Fundus Image Segmentation

Accurate glaucoma diagnosis relies on precise segmentation of the optic disc (OD) and optic cup (OC) in retinal images. However, despite the development of numerous automatic segmentation models, the lack of annotations in the target domain and domain shift among datasets continue to limit their segmentation performance. To address these issues, we propose a Causal Self-Supervised Network (CSSN) that leverages self-supervised learning to enhance model performance. First, we construct a Structural Causal Model (SCM) and employ backdoor adjustment to convert the conventional conditional distribution into an interventional distribution, effectively severing the influence of style information on feature extraction and pseudo-label generation. Subsequently, the low-frequency components of source and target domain images are exchanged via Fourier transform to simulate cross-domain style transfer. The original target images and their style-transferred counterparts are then processed by a dual-path segmentation network to extract their respective features, and a confidence-based pseudo-label fusion strategy is employed to generate more reliable pseudo-labels for self-supervised learning. In addition, we employ adversarial training and cross-domain contrastive learning to further reduce style discrepancies between domains. The former aligns feature distributions across domains using a feature discriminator, effectively mitigating the adverse effects of style inconsistency, while the latter minimizes the feature distance between original and style-transferred images, thereby ensuring structural consistency. Experimental results demonstrate that our method achieves more accurate OD and OC segmentation in the target domain during testing, thereby confirming its efficacy in cross-domain adaptation tasks.

White Light Imaging to Narrow-band Imaging via Style Transfer : A Feasibility Test for Enhancing Colonoscopic Diagnostics

White Light Imaging to Narrow-band Imaging via Style Transfer : A Feasibility Test for Enhancing Colonoscopic Diagnostics

ObjMST: Object-focused multimodal style transfer

ObjMST: Object-focused multimodal style transfer

SPAST: Arbitrary style transfer with style priors via pre-trained large-scale model

SPAST: Arbitrary style transfer with style priors via pre-trained large-scale model

Integrating Wavelet Transforms into Image Reconstruction Networks for Effective Style Transfer

Integrating Wavelet Transforms into Image Reconstruction Networks for Effective Style Transfer

Flexible style transfer from remote sensing images to maps

Flexible style transfer from remote sensing images to maps

An Adaptive Convolution Neural Network-based Multimedia Generative Adversarial Learning Framework for Style Transfer of Movie Videos

An Adaptive Convolution Neural Network-based Multimedia Generative Adversarial Learning Framework for Style Transfer of Movie Videos

A Feasibility Study on the Two-way Empowerment of Art Education Between Museums and Universities in the Era of AI

The iterative upgrading of artificial intelligence (AI) technology has provided an innovative path for the mutual empowerment of museums and art education in universities. This study focuses on the interaction mechanism between cultural resources and the education system in the AI era, revealing the feasibility and practical value of mutual empowerment. On the one hand, museums, by leveraging generative AI, neural radiance fields and other technologies, transform cultural relics into decomposable dynamic educational resource libraries. These systems facilitate the transformation of cultural and artistic works from physical forms to programmable educational primitives. Through parametric decomposition, style transfer, and spatio-temporal correlation analysis, a cognitive network that supports cross-disciplinary knowledge production is formed. On the other hand, through creative data feedback and algorithmic innovation in higher education practice, cultural connotations are cultivated, thereby transforming the role of museums from cultural custodians to educational enablers, and universities from knowledge disseminators to engines of cultural reproduction, promoting the modern translation of traditional cultural elements. This two-way empowerment mechanism provides technical solutions to historical problems such as the closed nature of cultural resources and the suspension of educational practice, and lays a theoretical foundation for the transformation of humanities education in the digital civilization era. From the perspective of the popularization of artificial intelligence technology, this article deeply explores the possibility of two-way empowerment between museums and university art education, and further explores the paths of two-way empowerment between museums and universities in the era of artificial intelligence.

Hierarchical style transfer for enhancing feature diversity in long-tailed visual recognition

Hierarchical style transfer for enhancing feature diversity in long-tailed visual recognition

3D reconstruction of digital rocks based on StyleGAN and transformer

The 3D reconstruction of digital rocks assumes a paramount role within various engineering applications, necessitating careful consideration of the methods employed. The available approaches are divided into two groups, i.e., the physical experimental method and the numerical simulation method. Although the former is reliable, it incurs high costs and is limited by sample size constraints. The latter is cost-effective but suffers from prolonged processing times and sometimes suboptimal performance. However, the advent of deep learning has paved the way for integrating these techniques into 3D digital rock reconstruction. A standout amongst the method of deep learning techniques is the generative adversarial network (GAN). Nonetheless, existing models of GAN lack complete integration of multi-scale information. To address this issue, this study proposed style-transformer GAN (STGAN), a novel GAN model founded upon style-based GAN (StyleGAN) and Transformer. By utilizing the attention mechanisms of Transformer, STGAN ameliorates its ability to extract features from multi-scale training images. Additionally, the incorporation of style transfer further enhances the quality of the generated images. By comparing it with other deep learning algorithms, the efficiency and applicability of the proposed method are demonstrated.

VariGAN: Enhancing Image Style Transfer via UNet Generator, Depthwise Discriminator, and LPIPS Loss in Adversarial Learning Framework.

Image style transfer is a challenging task that has gained significant attention in recent years due to its growing complexity. Training is typically performed using paradigms offered by GAN-based image style transfer networks. Cycle-based training methods provide an approach for handling unpaired data. Nevertheless, achieving high transfer quality remains a challenge with these methods due to the simplicity of the employed network. The purpose of this research is to present VariGAN, a novel approach that incorporates three additional strategies to optimize GAN-based image style transfer: (1) Improving the quality of transferred images by utilizing an effective UNet generator network in conjunction with a context-related feature extraction module. (2) Optimizing the training process while reducing dependency on the generator through the use of a depthwise discriminator. (3) Introducing LPIPS loss to further refine the loss function and enhance the overall generation quality of the framework. Through a series of experiments, we demonstrate that the VariGAN backbone exhibits superior performance across diverse content and style domains. VariGAN improved class IoU by 236% and participant identification by 195% compared to CycleGAN.