Visual Artifacts Research Articles

Learning to predict perceptual visibility of rendering deterioration in computer games.

Contemporary computer gaming affords players the agency to manually tailor rendering settings, a capability crucial for optimizing computational demands following their hardware performance. Specifically, adjustments to texture resolution, shadow map intricacies, and anti-aliasing complexities facilitate seamless animation generation, particularly on systems equipped with budget-friendly graphical units. Nonetheless, the intricacy and extensive interdependencies among these rendering parameters render selecting an optimal configuration a multifaceted challenge. Our approach involves training a proprietary convolutional neural network (CNN) to streamline this intricate process. This CNN quantitatively compares reference images rendered at peak quality to their counterparts subjected to quality reduction. Its function involves identifying and classifying artifacts within the altered images, and evaluating their perceptibility to human observers. Our neural network undergoes rigorous training using an expansive dataset derived from prevalent game engine scenes. A meticulous process to establish accurate classification data involves manually annotating image regions afflicted by diminished quality. As a testament to our methodology, we implement a prototype forward renderer based on OpenGL. Employing the trained neural network within this application enables the evaluation of image quality across diverse anti-aliasing settings, culminating in identifying settings minimizing artifact visibility. Empirical validation via user studies substantiates our network's efficacy, demonstrating its superior discernment of artifact visibility compared to established image quality metrics.

Prospective 3D Fat Navigator (FatNav) motion correction for 7T Terra MRI.

Ultra-high field (7T) MRI allows scans at sub-millimetre resolution with exquisite signal-to-noise ratio (SNR). As 7T MRI becomes more widely used clinically, the challenge of patient motion must be overcome. Retrospective motion correction is used successfully for some protocols, but for acquisitions such as slice-by-slice scans only prospective motion correction can deliver the full potential of 7T MRI. We report the first implementation of prospective 3D Fat Navigator ("FatNav") motion correction for the Siemens 7T Terra MRI. We implemented a modular Sequence Building Block for FatNav and embedded it into the vendor's gradient-recalled echo (GRE) sequence. We modified the reconstruction pipeline to reconstruct FatNav images online, coregistering them and sending motion updates to the host sequence online. We tested five registration algorithms for performance and accuracy on synthetic FatNav data. We implemented the best three of these in our sequence and tested them online. We acquired T1 and T2* weighted brain images of healthy volunteers correcting every other image for motion to visualise the effectiveness of online motion correction. Data were acquired with and without head immobilisation. We also tested performance while correcting every measurement for motion. Our implementation uses a 1.23 s 3D FatNav acquisition module and delivers motion updates in less than 3s, which is sufficient for motion updates every few k-space lines in typical scans. Corrected images are crisper with fewer visible motion artefacts. This improved sharpness is reflected quantitatively by an increase in the variance of the image Laplacian which is 1.59 x better for corrected vs uncorrected images. Profiles across the cerebral falx are 33% steeper for corrected vs uncorrected images. Prospective FatNav improves GRE image quality in the brain. Our modular Sequence Building Block provides a simple method to integrate motion correction in 7T MRI pulse sequences.

Spatially and Temporally Optimized Audio‐Driven Talking Face Generation

AbstractAudio‐driven talking face generation is essentially a cross‐modal mapping from audio to video frames. The main challenge lies in the intricate one‐to‐many mapping, which affects lip sync accuracy. And the loss of facial details during image reconstruction often results in visual artifacts in the generated video. To overcome these challenges, this paper proposes to enhance the quality of generated talking faces with a new spatio‐temporal consistency. Specifically, the temporal consistency is achieved through consecutive frames of the each phoneme, which form temporal modules that exhibit similar lip appearance changes. This allows for adaptive adjustment in the lip movement for accurate sync. The spatial consistency pertains to the uniform distribution of textures within local regions, which form spatial modules and regulate the texture distribution in the generator. This yields fine details in the reconstructed facial images. Extensive experiments show that our method can generate more natural talking faces than previous state‐of‐the‐art methods in both accurate lip sync and realistic facial details.

Seamless and Aligned Texture Optimization for 3D Reconstruction

AbstractRestoring the appearance of the model is a crucial step for achieving realistic 3D reconstruction. High‐fidelity textures can also conceal some geometric defects. Since the estimated camera parameters and reconstructed geometry usually contain errors, subsequent texture mapping often suffers from undesirable visual artifacts such as blurring, ghosting, and visual seams. In particular, significant misalignment between the reconstructed model and the registered images will lead to texturing the mesh with inconsistent image regions. However, eliminating various artifacts to generate high‐quality textures remains a challenge. In this paper, we address this issue by designing a texture optimization method to generate seamless and aligned textures for 3D reconstruction. The main idea is to detect misalignment regions between images and geometry and exclude them from texture mapping. To handle the texture holes caused by these excluded regions, a cross‐patch texture hole‐filling method is proposed, which can also synthesize plausible textures for invisible faces. Moreover, for better stitching of the textures from different views, an improved camera pose optimization is present by introducing color adjustment and boundary point sampling. Experimental results show that the proposed method can eliminate the artifacts caused by inaccurate input data robustly and produce high‐quality texture results compared with state‐of‐the‐art methods.

“Firsties”—Where It All Began: A Qualitative Analysis of Photographs in Couple Relationship Initiation

Photography plays a crucial role in couple relationships, particularly during relationship initiation. This study explores “firsties,” that is, pictures depicting relationship initiation. Based on qualitative interviews and visual elicitation with 21 couples in Switzerland, four image types in the firsties category were identified, “first trips,” “first picture/early pictures of the couple,” “the ‘paparazzi’-picture,” and “the ‘hidden’ couple,” and their characteristics were carved out. The findings highlight the value of firsties as significant visual artifacts serving as indicators and proof of commitment. Firsties remain important and gain significance as the relationship progresses. This research enhances understanding of relationship initiation as a pivotal moment in couples’ relationship trajectories and the integral role of photography therein.

Autoencoder-based image fusion network with enhanced channels and feature saliency

The existing deep learning based infrared and visible image fusion technologies have made significant progress, but there are still many problems need to be solved, such as information loss (targets and texture, etc.) of both infrared and visible images, noise and artifacts existing in fused image. To address these issues in fusion, an infrared and visible image fusion method based on autoencoder network is proposed in this paper. Firstly, novel enhanced channels are designed and input parallelly with source images into the network to enhance the specific features and reduce information loss in feature fusion. Then, the feature maps are obtained by the encoder. Next, a feature fusion method based on feature saliency is proposed, using a pre-trained classifier to measure the saliency of features, and the fused image is obtained by the decoder finally. Experimental results demonstrate that the targets are obvious and the textures are plentiful in the fused images generated by the proposed method. Also, the objective metrics of the proposed method are higher than the state of the art methods, which demonstrate that the proposed method is effective to fuse the infrared and visible images.

Dynamic Spatiotemporal Scheduling for Construction Building Projects

For building projects, the manager is responsible for coordinating the work of subcontractors at the construction site. This includes operations, material flows, and storage. In summary, one of their main roles is to ensure smooth team rotation, maintain fluid circulation, and avoid congestion or relaxation on the site. However, traditional tools lack the ability to consider the planning and management of worksite spaces when calculating the execution schedule and critical path. Consequently, three-week planning is usually carried out separately on independent plans, often using spreadsheets. In addition, a construction site is highly dynamic and mobile in nature, and the positioning of resources and workers can change daily. This makes the management of available space even more complex, and effective space management becomes an imperative. To address this challenge, this paper develops visual dynamic artifacts that present different operation types. The methodology and the conceptual framework facilitate the calculation of the Occupancy Rate (OR) that enables construction project managers to create simple yet dynamic spatiotemporal models of the construction schedule. By incorporating factors such as crew turnover and occupancy evolution, managers can simplify the calculation process and effectively optimize construction work by utilizing site occupancy rates. In summary, this paper presents the Dynamic Model of the Occupancy Rate Schedule (DMORS), a methodology developed through design science. This model utilizes created artifacts representing various operation types to ensure accurate calculations of dynamic occupancy by floor and sector in a site. Consequently, it enables the construction of a more realistic schedule based on critical space ideologies. The DMORS enables managers to use the OR for different floors and sectors of a site, allowing for better space management. A proof of concept demonstrates that this tool can enhance the efficiency and productivity of construction projects by optimizing crew schedules and resource allocation based on site OR.

Thin Cloud Removal Generative Adversarial Network Based on Sparse Transformer in Remote Sensing Images

Thin clouds in Remote Sensing (RS) imagery can negatively impact subsequent applications. Current Deep Learning (DL) approaches often prioritize information recovery in cloud-covered areas but may not adequately preserve information in cloud-free regions, leading to color distortion, detail loss, and visual artifacts. This study proposes a Sparse Transformer-based Generative Adversarial Network (SpT-GAN) to solve these problems. First, a global enhancement feature extraction module is added to the generator’s top layer to enhance the model’s ability to preserve ground feature information in cloud-free areas. Then, the processed feature map is reconstructed using the sparse transformer-based encoder and decoder with an adaptive threshold filtering mechanism to ensure sparsity. This mechanism enables that the model preserves robust long-range modeling capabilities while disregarding irrelevant details. In addition, inverted residual Fourier transformation blocks are added at each level of the structure to filter redundant information and enhance the quality of the generated cloud-free images. Finally, a composite loss function is created to minimize error in the generated images, resulting in improved resolution and color fidelity. SpT-GAN achieves outstanding results in removing clouds both quantitatively and visually, with Structural Similarity Index (SSIM) values of 98.06% and 92.19% and Peak Signal-to-Noise Ratio (PSNR) values of 36.19 dB and 30.53 dB on the RICE1 and T-Cloud datasets, respectively. On the T-Cloud dataset, especially with more complex cloud components, the superior ability of SpT-GAN to restore ground details is more evident.

Iterative Removal of G-PCC Attribute Compression Artifacts Based on a Graph Neural Network

As a compression standard, Geometry-based Point Cloud Compression (G-PCC) can effectively reduce data by compressing both geometric and attribute information. Even so, due to coding errors and data loss, point clouds (PCs) still face distortion challenges, such as the encoding of attribute information may lead to spatial detail loss and visible artifacts, which negatively impact visual quality. To address these challenges, this paper proposes an iterative removal method for attribute compression artifacts based on a graph neural network. First, the geometric coordinates of the PCs are used to construct a graph that accurately reflects the spatial structure, with the PC attributes treated as signals on the graph’s vertices. Adaptive graph convolution is then employed to dynamically focus on the areas most affected by compression, while a bi-branch attention block is used to restore high-frequency details. To maintain overall visual quality, a spatial consistency mechanism is applied to the recovered PCs. Additionally, an iterative strategy is introduced to correct systematic distortions, such as additive bias, introduced during compression. The experimental results demonstrate that the proposed method produces finer and more realistic visual details, compared to state-of-the-art techniques for PC attribute compression artifact removal. Furthermore, the proposed method significantly reduces the network runtime, enhancing processing efficiency.

GeGra: Approaching a generic model for quantitative grain size analysis from materials microscopy data using deep learning

Grain size has a significant impact on the properties of materials, and is crucial for predicting material properties. Traditional grain size measurement relies heavily on human operators, leading to subjective results, and existing machine learning methods are typically material-specific, requiring significant labeling and training efforts for each new material. This paper provides insight into developing a deep learning-based generic grain boundary detection model (GeGra) from different material micrographs. The model is trained on 1006 images from various microscopy techniques such as light optical, Kerr, and scanning electron microscopy, acquired at different magnifications for different materials such as copper, austenite, brass, sintered hard magnet, hard metal, bronze, nickel silver, and aluminum. The developed GeGra model effectively handles visual artifacts and substructures such as twin grains, which often pose challenges for material-specific, state-of-the-art grain boundary segmentation models. The developed model achieved an IoU score of 69 % on a diverse test set and enables accurate grain size analysis using external image analysis software in less than one minute, according to ASTM standards, which is more than 5 times faster than the manual method. The developed model prioritizes generality with objective that it can have broader applicability for various materials instead of high-precision grain boundary detection. Additionally, the model has the potential to be a foundational tool for generalized grain size analysis in material microscopy, reducing the effort required for such analysis and assisting both material science experts and machine learning users.

Unveiling dynamics of language visibility and vitality in café menus near Labrang Monastery.

This study explores the dynamics of Linguistic Landscapes (LL) in the commercial tourist district near Labrang Monastery by analyzing the use of Tibetan, Mandarin, and English on café menus. Traditionally, LL research has focused on language visibility as an indicator of vitality, overlooking contextual complexities. We challenge this approach by considering geocultural contexts, instigators' language proficiency, and intended audience. This study employed a triangulation approach, combining multiple methods of data collection and analysis. First, we conducted a preliminary observation of shop names in the region. Second, with the consent of shop owners, we comprehensively documented visual and textual artifacts both inside and outside the cafés, including signboards, menus, wall billboards, and displayed books. Third, we conducted extensive semi-structured interviews with shop owners and customers. These interviews, transcribed for analysis, provided insights into language usage preferences and decision-making processes. By advocating for a holistic LL research approach, integrating qualitative insights with quantitative data, this study contributes to a deeper understanding of LL dynamics. Our findings show that geocultural contexts, instigators' language proficiency, and intended audience all play a role in shaping language representation and vitality. The absence of a language on signage does not necessarily signify diminished vitality but can reflect strategic decisions influenced by religious, individual and commercial factors. Beyond mere visibility, languages on café menus serve as symbolic markers of ethnic identity and reflect the functions they assume within the speech community, offering insights into language vitality across different usage contexts. This research enriches scholarly discourse on LL, particularly within the Chinese context, by emphasizing the multifaceted nature of language presence and its vitality.

High-quality FLORET UTE imaging for clinical translation.

To develop a robust 3D ultrashort-TE (UTE) protocol that can reproducibly provide high-quality images, assessed by the ability to yield clinically diagnostic images, and is suitable for clinical translation. Building on previous work, a UTE sampled with Fermat looped orthogonally encoded trajectories (FLORET) was chosen as a starting point due to its shorter, clinically reasonable scan times. Modifications to previous FLORET implementations included gradient waveform frequency limitations, a new trajectory ordering scheme, a balanced SSFP implementation, fast gradient spoiling, and full inline reconstruction. FLORET images were collected in phantoms and humans on multiple scanners and sites to demonstrate these improvements. The updates to FLORET provided high-quality images in phantom, musculoskeletal, and pulmonary applications. The gradient waveform modifications and new trajectory ordering scheme significantly reduced visible artifacts. Fast spoiling reduced acquisition time by 20%-28%. Across the various scanners and sites, the inline image quality was consistent and of diagnostic quality. Total image acquisition plus reconstruction time was less than 4 min for musculoskeletal and pulmonary applications with reconstructions taking less than 1 min. Recently developed improvements for the FLORET sequence have enabled robust, high-quality UTE acquisitions with short acquisition and reconstruction times. This enables clinical UTE imaging as demonstrated by the implementation of the sequence and acquisition on five MRI scanners, at three different sites, without the need for any additional system characterization or measurements.

SynFlowMap: A synchronized optical flow remapping for video motion magnification

Motion magnification refers to the process of spatially amplifying small movements in a video to reveal important information about a scene. Several motion magnification methods have been proposed, but most of them introduce perceptible and annoying visual artifacts. In this paper, we propose a method that first analyzes the optical flow between the original frame and the corresponding frames, which are motion-magnified with other methods. Then, the method uses the generated optical flow map and the original video to synthesize a combined motion-magnified video. The method is able to amplify the motion by larger values, invert the direction of the motion, and combine filtered motion from multiple frequencies and Eulerian methods. Amongst other advantages, the proposed approach eliminates artifacts caused by Eulerian motion-magnification methods. We present an extensive qualitative and quantitative analysis of the results compared to the main approaches for Eulerian methods. A final contribution of this work is a new video database for motion magnification that allows the evaluation of quantitative motion magnification.

Detection of real-time deep fakes and face forgery in video conferencing employing generative adversarial networks

As facial modification technology advances rapidly, it poses a challenge to methods used to detect fake faces. The advent of deep learning and AI-based technologies has led to the creation of counterfeit photographs that are more difficult to discern apart from real ones. Existing Deep fake detection systems excel at spotting fake content with low visual quality and are easily recognized by visual artifacts. The study employed a unique active forensic strategy Compact Ensemble-based discriminators architecture using Deep Conditional Generative Adversarial Networks (CED-DCGAN), for identifying real-time deep fakes in video conferencing. DCGAN focuses on video-deep fake detection on features since technologies for creating convincing fakes are improving rapidly. As a first step towards recognizing DCGAN-generated images, split real-time video images into frames containing essential elements and then use that bandwidth to train an ensemble-based discriminator as a classifier. Spectra anomalies are produced by up-sampling processes, standard procedures in GAN systems for making large amounts of fake data films. The Compact Ensemble discriminator (CED) concentrates on the most distinguishing feature between the natural and synthetic images, giving the generators a robust training signal. As empirical results on publicly available datasets show, the suggested algorithms outperform state-of-the-art methods and the proposed CED-DCGAN technique successfully detects high-fidelity deep fakes in video conferencing and generalizes well when comparing with other techniques. Python tool is used for implementing this proposed study and the accuracy obtained for proposed work is 98.23 %.

Mixed methods with social network analysis for networked learning

In our research we study small group interaction and meaning making in the context of a larger community of people and artifacts. Our research methodology combines social network analysis and content analysis in different ways. The primary purpose of this paper is to explore approaches and demonstrate the feasibility of mixed methods research combining network-level and content-level methods. We report our experiences from three case studies (Get Satisfaction, Canvas, r/place), which include individual variation (innovative approaches toward integration) and a common approach of “zooming in,” or shifting perspective between bird’s eye and detailed levels of interaction data during analysis (message content, dialogic structure, or visual artifact vs. patterns of users and their interactions). We show that the two sets of methods in combination can eliminate shortcomings of the separate methods used independently.

A Novel Technique for High-Efficiency Characterization of Complex Cracks with Visual Artifacts

A Novel Technique for High-Efficiency Characterization of Complex Cracks with Visual Artifacts

Ethnic culture in urban space: a theoretical review

Relevance. Cities are centers of attraction for people of different nationalities and cultures. In the context of urbanization and globalization, various ethnic cultures interact, which leads to their transformation. The study of ethnic culture in urban space makes it possible to understand how culture changes under the influence of new living conditions, which is especially important for the formation of intercultural relations and tolerant attitude towards representatives of other cultures and nationalities.The purpose is to analyze theoretical studies of ethnic culture in urban space, including issues of tolerance, identity, and urban mentality.Objectives: to consider the elements of ethnic culture; to analyze the connection between ethnic culture and urban space; to determine how identity manifests itself in urban space; to characterize cultural ethnic practices in urban space; to characterize the visual artifacts of ethnicity in urban space.Methodology. The socio-cultural paradigm of studying urban space was used as the methodological basis of the study. The authors also applied general scientific methods: analysis, comparison, generalization.Results. A theoretical analysis of the developments of domestic authors on the topic of manifestations of ethnic culture in urban space, including aspects of national symbols, architecture, mentality, identity, tolerance, is carried out.Conclusions. Ethnic culture in urban space is a complex phenomenon encompassing socio-cultural and ethnodemographic aspects of urban community life. It finds expression in architecture, cultural practices of events, ornaments, everyday life, and the media. Today, despite the unification of everyday life and lifestyle, there are trends towards strengthening national identity, preserving national identity and cultural traditions, in particular through elements of urban public spaces.

“Flat Surface” as Material Metaphor: “Bad” Cover Design, “Good” Storytelling, and Post-Fordist Sensibility in Chinese Web Novels

The article deploys the material metaphor of “flat surface” to read the unexpected cross-modal affinities between amateurish cover design and professional storytelling in Chinese web novels. Canvassing the ways visual and verbal artifacts put into play metaphoric associations with “flat surface,” this essay interprets the popular fascination with flatness as a symptom of China's alignment with ubiquitous computing in its transition to a post-Fordist cultural economy when the world is increasingly conceived as a flat horizon. The author first exhibits a selection of spoof covers and argues that their willful indulgence in two-dimensionality is an antidote to the uncanny similarity of illusionist covers born out of what Mario Carpo calls “mass customization” in the digital era. Then the author turns to examine the web novel Zhuixu 赘婿 (Matrilocal Son-in-Law) by Fennu de Xiangjiao 愤怒的香蕉 (Angry Banana), a highly acclaimed alternate history fiction with a curiously lousy cover, focusing on how Angry Banana's cover designs and narrative storytelling convey the flattening effect. The author attributes the fascination with flatness to the rise of post-Fordism in China's cultural economy by demonstrating how the protagonist in Zhuixu succeeds in flattening “all under heaven” (tianxia 天下) by way of nonhumanizing his existence and weaponizing an internet of things (IoT) while embracing a post-Fordist network of work, life, and governance. Finally, the author furnishes a few more examples to suggest the broader trends toward flatness in Chinese web novels and concludes by discussing the critical edge of the metaphor of flatness in its applicability to post-Fordist cultural analysis.

Hypothesis Testing for Progressive Kernel Estimation and VCM Framework.

Identifying an appropriate radius for unbiased kernel estimation is crucial for the efficiency of radiance estimation. However, determining both the radius and unbiasedness still faces big challenges. In this paper, we first propose a statistical model of photon samples and associated contributions for progressive kernel estimation, under which the kernel estimation is unbiased if the null hypothesis of this statistical model stands. Then, we present a method to decide whether to reject the null hypothesis about the statistical population (i.e., photon samples) by the F-test in the Analysis of Variance. Hereby, we implement a progressive photon mapping (PPM) algorithm, wherein the kernel radius is determined by this hypothesis test for unbiased radiance estimation. Second, we propose VCM+, a reinforcement of Vertex Connection and Merging (VCM), and derive its theoretically unbiased formulation. VCM+ combines hypothesis testing-based PPM with bidirectional path tracing (BDPT) via multiple importance sampling (MIS), wherein our kernel radius can leverage the contributions from PPM and BDPT. We test our new algorithms, improved PPM and VCM+, on diverse scenarios with different lighting settings. The experimental results demonstrate that our method can alleviate light leaks and visual blur artifacts of prior radiance estimate algorithms. We also evaluate the asymptotic performance of our approach and observe an overall improvement over the baseline in all testing scenarios.

AI-IMAGE REPRESENTATION AND LINEAR REPRENDER RENDERING

Image representation and rendering have become critical in numerous applications such as virtual reality, medical imaging, and computer graphics. Traditional rendering techniques often face challenges in efficiently handling complex scenes and achieving photorealistic results while maintaining low computational costs. The problem lies in the high-dimensional nature of image data, leading to slow processing times and reduced scalability. This research presents an AI-enhanced technique called Linear RepRender, which leverages deep learning to transform high-dimensional image representations into simplified linear forms for faster rendering. The proposed method employs a combination of convolutional neural networks (CNNs) and linear regression models to reduce image complexity. Specifically, the CNN extracts low-level and high-level features from the image, while the linear regression step approximates the scene’s core visual elements. This hybrid approach significantly improves rendering speed without sacrificing image quality. Furthermore, the method incorporates a loss function optimized for minimizing discrepancies between the rendered and ground truth images. Experimental results demonstrate that Linear RepRender outperforms traditional rendering algorithms, such as ray tracing and rasterization, in terms of computational efficiency and visual accuracy. On a dataset of complex 3D scenes, the proposed method achieved a 35% reduction in rendering time and a 22% improvement in peak signal-to-noise ratio (PSNR) compared to state-of-the-art methods. Additionally, Linear RepRender was able to handle up to 1.5 million polygons per scene with minimal visual artifacts, making it suitable for real-time applications.