Virtual Try-on Research Articles

OccluMix: Towards De-Occlusion Virtual Try-on by Semantically-Guided Mixup

Image Virtual try-on aims at replacing the cloth on a personal image with a garment image (in-shop clothes), which has attracted increasing attention from the multimedia and computer vision communities. Prior methods successfully preserve the character of clothing images, however, occlusion remains a pernicious effect for realistic virtual try-on. In this work, we first present a comprehensive analysis of the occlusions and categorize them into two aspects: i) Inherent-Occlusion: the ghost of the former cloth still exists in the try-on image; ii) Acquired-Occlusion: the target cloth warps to the unreasonable body part. Based on the in-depth analysis, we find that the occlusions can be simulated by a novel semantically-guided mixup module, which can generate semantic-specific occluded images that work together with the try-on images to facilitate training a de-occlusion try-on (DOC-VTON) framework. Specifically, DOC-VTON first conducts a sharpened semantic parsing on the try-on person. Aided by semantics guidance and pose prior, various complexities of texture are selectively blending with human parts in a copy-and-paste manner. Then, the Generative Module (GM) is utilized to take charge of synthesizing the final try-on image and learning to de-occlusion jointly. In comparison to the state-of-the-art methods, DOC-VTON achieves better perceptual quality by reducing occlusion effects.

VTON-SCFA: A Virtual Try-On Network Based on the Semantic Constraints and Flow Alignment

An image-based virtual try-on system transfers an in-shop garment to the corresponding garment region of a reference person, which has huge application potential and commercial value in online clothing shopping. Existing methods have difficulty preserving garment texture and body details because of rough garment alignment and imperfect detail-retention strategies. To address this problem, we propose a virtual try-on network based on semantic constraints and flow alignment. The key idea of the framework is as follows: 1) a global-local semantic predictor (GLSP) is proposed to generate a reasonable target semantic map, which clearly guides the correct alignment of the in-shop garment with the body and the generation of try-on result; and 2) a novel appearance flow-based garment alignment network (AFGAN) is proposed to align the in-shop garment with the body, which is important to preserve maximum garment detail and ensure natural and realistic warping; and 3) we propose a synthesis strategy to integrate the aligned garment and the human body to preserve maximum body detail for generating a realistic result and preventing cross-occlusion and pixel confusion between different body parts. Experiments on the existing benchmark dataset demonstrate that the proposed method achieves the best performance on qualitative and quantitative experiments among the state-of-the-art virtual try-on techniques.

Virtual Try-On With Garment Self-Occlusion Conditions

Image-based virtual try-on focuses on changing the model's garment item to the target ones and preserving other visual features. To preserve the texture detail of the given in-shop garment, former methods use geometry-based methods (e.g., Thin-plate-spline interpolation) to realize garment warping. However, due to limited degree of freedom, geometry-based methods perform poorly when garment self-occlusion occurs, which is common in daily life. To address this challenge, we propose a novel occlusion-focused virtual try-on system. Compared to previous ones, our system contains three critical submodules, namely, Garment Part Modeling (GPM), a group of Garment Part Generators (GPGs), and Overlap Relation Estimator (ORE). GPM takes the pose landmarks as input, and progressively models the mask of body parts and garments. Based on these masks, GPGs are introduced to generate each garment part. Finally, ORE is proposed to model the overlap relationships between each garment part, and we bind the generated garments under the guidance of overlap relationships predicted by ORE. To make the most of extracted overlap relationships, we proposed an IoU-based hard example mining method for loss terms to handle the sparsity of the self-occlusion samples in the dataset. Furthermore, we introduce part affinity field as pose representation instead of landmark used widely by previous methods and achieve accuracy improvement on try-on layout estimation stage. We evaluate our model on the VITON dataset and found it can outperform previous approaches, especially on samples with garment self-occlusion.

Multi-sensory Perception and Deep Learning-based Virtual Try-on Technologies for Body Shape and Attractiveness, Unrealistic Societal Beauty Standards, and Thinness Ideal Internalization

Multi-sensory Perception and Deep Learning-based Virtual Try-on Technologies for Body Shape and Attractiveness, Unrealistic Societal Beauty Standards, and Thinness Ideal Internalization

Eying Sales Gains with Virtual Try-On Technology: A Natural Experiment in Webrooming

Eying Sales Gains with Virtual Try-On Technology: A Natural Experiment in Webrooming

POVNet: Image-Based Virtual Try-On Through Accurate Warping and Residual.

Virtual dressing room applications help online shoppers visualize outfits. Such a system, to be commercially viable, must satisfy a set of performance criteria. The system must produce high quality images that faithfully preserve garment properties, allow users to mix and match garments of various types and support human models varying in skin tone, hair color, body shape, and so on. This paper describes POVNet, a framework that meets all these requirements (except body shapes variations). Our system uses warping methods together with residual data to preserve garment texture at fine scales and high resolution. Our warping procedure adapts to a wide range of garments and allows swapping in and out of individual garments. A learned rendering procedure using an adversarial loss ensures that fine shading, etc. is accurately reflected. A distance transform representation ensures that hems, cuffs, stripes, and so on are correctly placed. We demonstrate improvements in garment rendering over state of the art resulting from these procedures. We demonstrate that the framework is scalable, responds in real-time, and works robustly with a variety of garment categories. Finally, we demonstrate that using this system as a virtual dressing room interface for fashion e-commerce websites has significantly boosted user-engagement rates.

3D Virtual Garment Simulation and Facial Expression Recognition Technologies, Beauty Filter-based Appealing Self-presentation and Generative Artificial Intelligence Virtual Try-on Tools, and Deep Learning-based Gesture Recognition and Augmented Reality Face Makeup Systems for Body-Image Affective Judgments and Dysfunctional Appearance Beliefs

3D Virtual Garment Simulation and Facial Expression Recognition Technologies, Beauty Filter-based Appealing Self-presentation and Generative Artificial Intelligence Virtual Try-on Tools, and Deep Learning-based Gesture Recognition and Augmented Reality Face Makeup Systems for Body-Image Affective Judgments and Dysfunctional Appearance Beliefs

Strategic responses to the pandemic: a case study of the US department store industry

PurposeThe study focuses on (1) the success of three strategies employed during the pandemic – two “persevering” strategies, curbside pickup and return window extension and one innovative strategy, virtual try-on technology and (2) whether the strategies are likely to be successful in the post-pandemic world.Design/methodology/approachThe authors utilize a panel dataset containing 17 department store chains in the US The panel includes weekly sales by the retailers at the city level from 2018 to 2021, encompassing both a pre-COVID-19 period and a period during the pandemic. A two-way fixed effects model, including retailer-city fixed effects and year-week fixed effects, is used to estimate department store sales.FindingsThe authors find that the two persevering strategies offset the negative impact of government-imposed containment and health measures on sales performance. On the other hand, the innovative strategy is more effective with a low level of containment and health measures, leading to our observation that virtual try-on may be more sustainable than the other two strategies in a post-pandemic environment.Originality/valueThis paper makes the following contributions: First, the authors contribute to the literature on strategies that may be used to respond to crises. Second, the authors contribute to the retail management literature, assessing the impact of the three retail strategies on department store sales. Finally, the authors compare the impact on sales of the two persevering strategies to the innovative strategy and conclude that a mix of these types of strategies may be most effective at generating short-term sales during a crisis and longer-term sales post crisis.

Dress-up: deep neural framework for image-based human appearance transfer.

The fashion industry is at the brink of radical transformation. The emergence of Artificial Intelligence (AI) in fashion applications creates many opportunities for this industry and make fashion a better space for everyone. Interesting to this matter, we proposed a virtual try-on interface to stimulate consumers purchase intentions and facilitate their online buying decision process. Thus, we present, in this paper, our flexible person generation system for virtual try-on that aiming to treat the task of human appearance transfer across images while preserving texture details and structural coherence of the generated outfit. This challenging task has drawn increasing attention and made huge development of intelligent fashion applications. However, it requires different challenges, especially in the case of a wide divergences between the source and target images. To solve this problem, we proposed a flexible person generation framework called Dress-up to treat the 2D virtual try-on task. Dress-up is an end-to-end generation pipeline with three modules based on the task of image-to-image translation aiming to sequentially interchange garments between images, and produce dressing effects not achievable by existing works. The core idea of our solution is to explicitly encode the body pose and the target clothes by a pre-processing module based on the semantic segmentation process. Then, a conditional adversarial network is implemented to generate target segmentation feeding respectively, to the alignment and translation networks to generate the final output results. The novelty of this work lies in realizing the appearance transfer across images with high quality by reconstructing garments on a person in different orders and looks from simlpy semantic maps and 2D images without using 3D modeling. Our system can produce dressing effects and provide significant results over the state-of-the-art methods on the widely used DeepFashion dataset. Extensive evaluations show that Dress-up outperforms other recent methods in terms of output quality, and handles a wide range of editing functions for which there is no direct supervision. Different types of results were computed to verify the performance of our proposed framework and show that the robustness and effectiveness are high by utilizing our method.

ST-VTON: Self-supervised vision transformer for image-based virtual try-on

Virtual try-on techniques are in the spotlight due to the promise of enhancing the user experience of online shopping in the apparel industry. Among these technologies, image-based virtual try-on methods have received more attention due to their efficiency and low cost. These methods commonly warp the specific clothing item to the desired shape and then fuse it with the target person image. However, existing approaches demand the laborious paired dataset of clothes and persons. While recent works alleviate the dependence on paired datasets by transferring clothing from one person to another, the pose information is still a rigid demand to guide the synthesis. To solve the above-mentioned problems, we propose ST-VTON, a two-stage self-supervised try-on approach that can synthesize photo-realistic try-on results with only person images and a standard ViT structure. Specifically, we first utilize massive person and clothing images to pre-train a powerful ViT encoder following the MAE approach to enhance the generalization ability and accelerate model training. Secondly, we propose the Clothing Recovery Module (CRM) to reconstruct in-shop clothes from person images and use the paired generated clothing and corresponding person images to train a new ViT decoder while optimizing the pre-trained encoder. Noted that, the CRM, as an independent module, can be used to effectively expand existing paired try-on datasets and enhance other try-on approaches in terms of training data. Extensive experiments conducted on the VITON dataset and UnPaired virtual Try-on (UPT) dataset with both paired and unpaired approaches demonstrate that try-on results rendered by our model can match or even outperform supervised methods. • We propose a novel ViT-backed try-on method with lightweight preprocessing and a standard ViT without elaborate modification. • .We introduce a two-stage self-supervised training method which can expand paired try-on datasets from separate person images. • Our model can achieve competitive results for both paired try-on and unpaired clothing transfer.

Avatar personalisation vs. privacy in a virtual try-on app for apparel shopping

ABSTRACT Virtual try-on (VTO) apps are now used by many fashion consumers, but VTOs for the apparel category have met with resistance. This study examines privacy concern, body image and social value as antecedents to adoption intention towards an apparel VTO with two types of photorealistic avatars. Twenty users first tried out the app in lab sessions, then 301 completed an online survey with a video of the VTO. A majority of participants were concerned about potential misuse of their uploaded picture and preferred to use a pre-loaded avatar of a model with a similar body. This option explains why privacy concern had a weak negative impact on adoption intention in our model, albeit at the expense of self-presentation benefits. The trait of privacy disposition best predicted consumer responses overall, yet other motives were also revealed. Discussed are the implications of this study’s results and limitations to privacy calculus research.

A Multi-Level Consistency Network for High-Fidelity Virtual Try-On

The 2D virtual try-on task aims to transfer a target clothing image to the corresponding region of a person image. Although an extensive amount of research has been conducted due to its immense applications, this task still remains a great challenge to handle some complicated issues (e.g., non-rigid shapes, large occlusions and arbitrary poses). To this end, we propose a novel network with structural and textural consistency-preserving mechanism for producing high-fidelity try-on images. Specifically, we first generate the semantic layout of a clothing-agnostic person to obtain the segmentation map, which is used as the transforming conditions of the target clothes. Based on a recurrent network structure, the transform lookup is performed to iteratively update a dense flow. Then, we adopt a thin-plate-spline-based warping method to estimate the coarse offset flow for all key-point positions. Guided by this sparse flow, a multi-scale deformable convolution module is designed to further iteratively predict the fine offsets for densely sampled positions, by which the clothing item and person shape can be accurately aligned. Finally, we develop a refinement module to effectively fuse the global and local features, which can render accurate geometric structures of the body parts and maintain texture sharpness of the clothes. Extensive experiments on benchmark datasets demonstrate that our method outperforms other state-of-the-art methods in terms of quantitative and qualitative try-on results. The code is available on: https://github.com/TJU-WEIHAO/MLCN .

Two‐dimensional virtual try‐on algorithm and application research for personalized dressing

To reduce the cost of virtual try-on, a method of image deformation by body part size is proposed for the traditional two-dimensional virtual try-on method, which is challenging to represent the personalized characteristics of the body size of the fitting subject. On the basis of the input information of the user's body size, the method can generate a fitting effect that shows the user's characteristics with the corresponding clothes. The image segmentation algorithm is used to snap out the garment from the background garment image, and then the size and position of the garment are adjusted according to the dressing position of the standard mannequin image and fit the mannequin image. The final mesh with dense vertices is generated using surface subdivision. The experimental results show that this method can show good results in user personalized try-on.

The rapid construction method of human body model for virtual try-on on mobile terminal based on MDD-Net

Traditional anthropometric evaluation needs professional measuring tools and operations, and it is time-consuming, expensive, and not suitable for virtual try-on. As the mobile internet develops, the issue of human body reconstruction toward virtual try-on needs to be tackled. This paper proposes a rapid human body reconstruction method for virtual try-on based on multidimensional dense net (MDD-Net) on mobile terminal. MDD-Net takes fusion features as input and predicts 3D human body model. The acquisition of fusion features and the display of 3D human body are implemented on mobile toward virtual try-on. In the learning fuzzy anthropometric feature module, the example-guided fuzzy anthropometric feature matrix is acquired and default coding elements are interpolated. In the learning multi-perspective silhouette feature module, the fine human body shape features are learned based on DenseNet201. A related fusion feature data set is generated for the training and testing of MDD-Net. Compared to shape pose estimation models, the shape representation spaces of HMR and SMPLify are only 20.34% and 7.59% of our method, and their prediction accuracies are approximately 50% of our method. Compared to accurate shape estimation models, our method is more robust against the pose and perspective noise. The prediction accuracies of our method are improved by 13.34%, 55.77%, 34.6%, and 43.4%, 37.2% 9.0% on four test sets. Extensive experiments have demonstrated the superiority of our method for human body shape estimation toward virtual try-on.

NeuralTailor

The fields of SocialVR, performance capture, and virtual try-on are often faced with a need to faithfully reproduce real garments in the virtual world. One critical task is the disentanglement of the intrinsic garment shape from deformations due to fabric properties, physical forces, and contact with the body. We propose to use a garment sewing pattern, a realistic and compact garment descriptor, to facilitate the intrinsic garment shape estimation. Another major challenge is a high diversity of shapes and designs in the domain. The most common approach for Deep Learning on 3D garments is to build specialized models for individual garments or garment types. We argue that building a unified model for various garment designs has the benefit of generalization to novel garment types, hence covering a larger design domain than individual models would. We introduce NeuralTailor, a novel architecture based on point-level attention for set regression with variable cardinality, and apply it to the task of reconstructing 2D garment sewing patterns from the 3D point cloud garment models. Our experiments show that NeuralTailor successfully reconstructs sewing patterns and generalizes to garment types with pattern topologies unseen during training.

A design in system architecture based on mobile cloud computing for a virtual try-on solution

Cloud computing is an emerging technology in this digital century. It provides an excellent but low-cost source for database storage, computing power, applications, and services through an internet-delivered cloud platform. Thanks to the cost savings in investing and maintaining physical data centers, as well as the stability of Quality of Service (QoS), it has no restrictions on company size or sector for enterprises to shift their operations to a cloud platform. The fashion industry, particularly the fashion e-commerce sector, is a case study in leveraging the cloud platform via a technology called “Virtual try-on” (VTO). VTO solution allows fashion brands to increase the shopping experience, however, requires installing and maintaining a bulky system for implementation. There are different methods and approaches to design architectures using cloud computing, however, there have not been many studies addressing tasks related to machine learning that uses the high Graphics Processing Unit (GPU) encountered in VTO solutions. To design a scheduler that could optimize the system performance while lowering operational expenses in VTO solutions, this research proposes a system to (1) handle synchronous model and asynchronous model separately and clearly, (2) perform multi-layered task processing architecture by hashing task ID and implementing a queue management system. This method would satisfy three major requirements: (1) Avoid complex hardware requirements for users, (2) Ensure the system stability and the ease of horizontal and vertical extension, and (3) Protect user information privacy.

Glass Virtual Try-On

Abstract: Virtual try-on is a technology that allows people to virtually check the appearance of accessories, makeup, hairstyle, hair color, clothes and potentially more on themselves. The virtual try-on presents many advantages over real try-on, it speeds up the process providing the possibility to test hundreds of products without the need to reach a physical store. The try-on process is performed on a 3D face reconstructed from the input image allowing the user to see the virtual face and glasses from different viewpoints. The try-on process is fully automated and does not require the user to provide anything else but the picture and name of the glasses frames to test. Keywords: Virtual try-on, Virtual glasses, Augmented reality

Designing an AI-Based Virtual Try-On Web Application.

In the last few years, Augmented Reality, Virtual Reality, and Artificial Intelligence (AI) have been increasingly employed in different application domains. Among them, the retail market presents the opportunity to allow people to check the appearance of accessories, makeup, hairstyle, hair color, and clothes on themselves, exploiting virtual try-on applications. In this paper, we propose an eyewear virtual try-on experience based on a framework that leverages advanced deep learning-based computer vision techniques. The virtual try-on is performed on a 3D face reconstructed from a single input image. In designing our system, we started by studying the underlying architecture, components, and their interactions. Then, we assessed and compared existing face reconstruction approaches. To this end, we performed an extensive analysis and experiments for evaluating their design, complexity, geometry reconstruction errors, and reconstructed texture quality. The experiments allowed us to select the most suitable approach for our proposed try-on framework. Our system considers actual glasses and face sizes to provide a realistic fit estimation using a markerless approach. The user interacts with the system by using a web application optimized for desktop and mobile devices. Finally, we performed a usability study that showed an above-average score of our eyewear virtual try-on application.

Exploring the role of augmented reality in online impulse behaviour

PurposeThis study aims to examine the role of augmented reality (AR) in online impulse behaviour for highbody-involvement products. This study further explores whether flow and spatial presence mediate the link between AR and online impulse behaviour.Design/methodology/approachThe authors collected 255 responses from shopping mall visitors and used SPSS (21.0) (PROCESS macro) and AMOS 21.0 to test the hypothesised model.FindingsThe findings reveal that AR virtual try-on significantly influences online impulse behaviour by providing hedonic value and reducing product risk prior to purchase. Second, flow and spatial presence partially and complementarily mediate the relationship between AR characteristics, hedonic value, and product risk.Originality/valueTheoretically, this study extends the literature on AR and online impulse behaviour from a psychological perspective, and it broadens managers' understanding of how they can use AR as a tool to increase sales.

A Reviewed Analysis of the Technological Trends in E-Commerce.

Abstract: E-commerce stores are growing increasingly popular among all types of enterprises. Ever since the Covid-19 pandemic strike, both Small and Large Enterprises have started to shift their businesses digitally with the help of e-commerce platforms. As a result of this increased popularity, many new technological trends are being introduced in the E-commerce sector. Businesses are attempting to digitalize their operations by creating an effective digitally enhanced E-commerce platform in order to attract new customers and maximize business prospects. Many new technological trends are being introduced so that consumers can get the best digitalized virtual experience. This review paper looks at the technological trends in the e-commerce industry. Knowledge of these new technological trends will undoubtedly assist businesses in strategic planning and aligning their E-commerce sites with the most appropriate technology, thereby increasing client acquisition and business growth rate. Keywords: E-commerce, Technological Trends, AR - VR, Customers, ROPO, Virtual Try-On, Security, Drones, Voice Search, Blockchain, Omni-Channel, Smart Product Recommendation.