High-fidelity Head Research Articles

HAvatar: High-fidelity Head Avatar via Facial Model Conditioned Neural Radiance Field

The problem of modeling an animatable 3D human head avatar under lightweight setups is of significant importance but has not been well solved. Existing 3D representations either perform well in the realism of portrait images synthesis or the accuracy of expression control, but not both. To address the problem, we introduce a novel hybrid explicit-implicit 3D representation, Facial Model Conditioned Neural Radiance Field, which integrates the expressiveness of NeRF and the prior information from the parametric template. At the core of our representation, a synthetic-renderings-based condition method is proposed to fuse the prior information from the parametric model into the implicit field without constraining its topological flexibility. Besides, based on the hybrid representation, we properly overcome the inconsistent shape issue presented in existing methods and improve the animation stability. Moreover, by adopting an overall GAN-based architecture using an image-to-image translation network, we achieve high-resolution, realistic and view-consistent synthesis of dynamic head appearance. Experiments demonstrate that our method can achieve state-of-the-art performance for 3D head avatar animation compared with previous methods.

Background Noise Filtering and Distribution Dividing for Crowd Counting.

Crowd counting is a challenging problem due to the diverse crowd distribution and background interference. In this paper, we propose a new approach for head size estimation to reduce the impact of different crowd scale and background noise. Different from just using local information of distance between human heads, the global information of the people distribution in the whole image is also under consideration. We obey the order of far- to near-region (small to large) to spread head size, and ensure that the propagation is uninterrupted by inserting dummy head points. The estimated head size is further exploited, such as dividing the crowd into parts of different densities and generating a high-fidelity head mask. On the other hand, we design three different head mask usage mechanisms and the corresponding head masks to analyze where and which mask could lead to better background filtering1. Based on the learned masks, two competitive models are proposed which can perform robust crowd estimation against background noise and diverse crowd scale. We evaluate the proposed method on three public crowd counting datasets of ShanghaiTech [2], UCFQNRF [3] and UCFCC_50 [4]. Experimental results demonstrate that the proposed algorithm performs favorably against the state-of-the-art crowd counting approaches.

Transcranial Direct Current Stimulation Optimization - From Physics-Based Computer Simulations to High-Fidelity Head Phantom Fabrication and Measurements.

BackgroundTranscranial direct current stimulation (tDCS) modulates neural networks. Computer simulations, while used to identify how currents behave within tissues of different conductivity properties, still need to be complemented by physical models.Objective/HypothesisTo better understand tDCS effects on biology-mimicking tissues by developing and testing the feasibility of a high-fidelity 3D head phantom model that has sensing capabilities at different compartmental levels.MethodsModels obtained from MRI images generated 3D printed molds. Agar phantoms were fabricated, and 18 monitoring electrodes were placed on specific phantom brain areas.ResultsWhen using rectangular electrodes, the measured and simulated voltages at the monitoring electrodes agreed reasonably well, except at excitation locations. The electric field distribution in different phantom layers appeared better confined with circular electrodes compared to rectangular electrodes.ConclusionThe high-fidelity 3D head model was found to be feasible and comparable with computer-based electrical simulations, with high correlation between simulated and measured brain voltages. This feasibility study supports testing to further assess the reliability of this model.

Multi-View Stereo on Consistent Face Topology

We present a multi-view stereo reconstruction technique that directly produces a complete high-fidelity head model with consistent facial mesh topology. While existing techniques decouple shape estimation and facial tracking, our framework jointly optimizes for stereo constraints and consistent mesh parameterization. Our method is therefore free from drift and fully parallelizable for dynamic facial performance capture. We produce highly detailed facial geometries with artist-quality UV parameterization, including secondary elements such as eyeballs, mouth pockets, nostrils, and the back of the head. Our approach consists of deforming a common template model to match multi-view input images of the subject, while satisfying cross-view, cross-subject, and cross-pose consistencies using a combination of 2D landmark detection, optical flow, and surface and volumetric Laplacian regularization. Since the flow is never computed between frames, our method is trivially parallelized by processing each frame independently. Accurate rigid head pose is extracted using a PCA-based dimension reduction and denoising scheme. We demonstrate high-fidelity performance capture results with challenging head motion and complex facial expressions around eye and mouth regions. While the quality of our results is on par with the current state-of-the-art, our approach can be fully parallelized, does not suffer from drift, and produces face models with production-quality mesh topologies.

Continuous wave simulations on the propagation of electromagnetic fields through the human head.

Characterizing the human head as a propagation medium is vital for the design of both on-body and implanted antennas and radio-frequency sensors. The following problem has been addressed: find the best radio-frequency path through the brain for a given receiver position-on the top of the sinus cavity. Two parameters, transmitter position and radiating frequency, should be optimized simultaneously such that 1) the propagation path through the brain is the longest; and 2) the received power is maximized. To solve this problem, we have performed a systematic and comprehensive study of the electromagnetic fields excited in the head by small on-body magnetic dipoles (small coil antennas). An anatomically accurate high-fidelity head mesh has been generated from the Visible Human Project data. The base radiator was constructed of two orthogonal magnetic dipoles in quadrature, which enables us to create a directive beam into the head. We have found at least one optimum solution. This solution implies that a distinct RF channel may be established in the brain at a certain frequency and transmitter location.

Live Speech Driven Head-and-Eye Motion Generators

This paper describes a fully automated framework to generate realistic head motion, eye gaze, and eyelid motion simultaneously based on live (or recorded) speech input. Its central idea is to learn separate yet interrelated statistical models for each component (head motion, gaze, or eyelid motion) from a prerecorded facial motion data set: 1) Gaussian Mixture Models and gradient descent optimization algorithm are employed to generate head motion from speech features; 2) Nonlinear Dynamic Canonical Correlation Analysis model is used to synthesize eye gaze from head motion and speech features, and 3) nonnegative linear regression is used to model voluntary eye lid motion and log-normal distribution is used to describe involuntary eye blinks. Several user studies are conducted to evaluate the effectiveness of the proposed speech-driven head and eye motion generator using the well-established paired comparison methodology. Our evaluation results clearly show that this approach can significantly outperform the state-of-the-art head and eye motion generation algorithms. In addition, a novel mocap+video hybrid data acquisition technique is introduced to record high-fidelity head movement, eye gaze, and eyelid motion simultaneously.