Talking Head Video Research Articles

Video Creation Using Facial Animation and Speech Synthesis

We present an end-to-end system for creating talking head video from text and a single image automatically. A two-dimensional clone of an individual made entirely of a still image of a person, that conveys the given text message to the target audience through replicated facial expressions. It can send any message that is delivered in a text to it with a voice that is cloned from a few seconds of their actual voice input. Our technique yields videos of natural facial movements including blinks and brow movements, as well as mouth movements that are in synchronization with the audio. The proposed method only uses a single base model to create a personalized video, avoiding the hassle of training it every time.

MakeltTalk

We present a method that generates expressive talking-head videos from a single facial image with audio as the only input. In contrast to previous attempts to learn direct mappings from audio to raw pixels for creating talking faces, our method first disentangles the content and speaker information in the input audio signal. The audio content robustly controls the motion of lips and nearby facial regions, while the speaker information determines the specifics of facial expressions and the rest of the talking-head dynamics. Another key component of our method is the prediction of facial landmarks reflecting the speaker-aware dynamics. Based on this intermediate representation, our method works with many portrait images in a single unified framework, including artistic paintings, sketches, 2D cartoon characters, Japanese mangas, and stylized caricatures. In addition, our method generalizes well for faces and characters that were not observed during training. We present extensive quantitative and qualitative evaluation of our method, in addition to user studies, demonstrating generated talking-heads of significantly higher quality compared to prior state-of-the-art methods.

Oliceridine is Associated with Reduced Risk of Vomiting and Need for Rescue Antiemetics Compared to Morphine: Exploratory Analysis from Two Phase 3 Randomized Placebo and Active Controlled Trials.

IntroductionUse of parenteral opioids is a major risk factor for postoperative nausea and vomiting. Conventional opioids bind to µ-opioid receptors (MOR), stimulate both the G-protein signaling (achieving analgesia); and the β-arrestin pathway (associated with opioid-related adverse effects). Oliceridine, a next-generation IV opioid, is a G-protein selective MOR agonist, with limited recruitment of β-arrestin. In two randomized, placebo- and morphine-controlled phase 3 studies of patients with moderate-to-severe acute pain following bunionectomy or abdominoplasty, oliceridine at demand doses of 0.1, 0.35, and 0.5 mg provided rapid and sustained analgesia vs. placebo with favorable gastrointestinal (GI) tolerability. In this exploratory analysis, we utilized a clinical endpoint assessing gastrointestinal tolerability, “complete GI response” defined as the proportion of patients with no vomiting and no use of rescue antiemetic to characterize the GI tolerability profile of oliceridine vs. morphine.MethodsA logistic regression model was utilized to compare oliceridine (pooled regimens) vs. morphine, after controlling for analgesia (using the sum of pain intensity difference [SPID]-48/24 [bunionectomy/abdominoplasty] with pre-rescue scores carried forward for 6 h). This analysis excluded patients receiving placebo and was performed for each study separately and for pooled data from both studies.ResultsIn the unadjusted analysis, a significantly greater proportion of patients in the placebo (76.4%), oliceridine 0.1 mg (68.0%), and 0.35 mg (46.2%) demand dose achieved complete GI response vs. morphine 1 mg (30.8%), p ≤ 0.005. In the adjusted analysis, after controlling for analgesia, the odds ratio of experiencing a complete GI response with oliceridine (pooled regimens) vs. morphine was 3.14 (95% CI: 1.78, 5.56; p < 0.0001) in bunionectomy study and 1.92 (95% CI: 1.09, 3.36; p = 0.024) in abdominoplasty study.ConclusionsWhen controlled for the analgesic effects (constant SPID-48/24), the odds ratio for complete GI response was higher with oliceridine than morphine, suggesting better GI tolerability with oliceridine. Oliceridine is Associated with Reduced Risk of Vomiting and Need for Rescue Antiemetics Compared to Morphine: Exploratory Analysis from Two Phase 3 Randomized Placebo and Active Controlled Trials - Talking-head video (1,06,013 kb) Electronic Supplementary MaterialThe online version of this article (10.1007/s40122-020-00216-x) contains supplementary material, which is available to authorized users.

Text-based editing of talking-head video

Editing talking-head video to change the speech content or to remove filler words is challenging. We propose a novel method to edit talking-head video based on its transcript to produce a realistic output video in which the dialogue of the speaker has been modified, while maintaining a seamless audio-visual flow (i.e. no jump cuts). Our method automatically annotates an input talking-head video with phonemes, visemes, 3D face pose and geometry, reflectance, expression and scene illumination per frame. To edit a video, the user has to only edit the transcript, and an optimization strategy then chooses segments of the input corpus as base material. The annotated parameters corresponding to the selected segments are seamlessly stitched together and used to produce an intermediate video representation in which the lower half of the face is rendered with a parametric face model. Finally, a recurrent video generation network transforms this representation to a photorealistic video that matches the edited transcript. We demonstrate a large variety of edits, such as the addition, removal, and alteration of words, as well as convincing language translation and full sentence synthesis.

Comparing video styles and study strategies during video-recorded lectures: effects on secondary school mathematics students’ preference and learning

ABSTRACT Despite the increasing popularity of using video-recorded lectures, we still understand little about how different video styles and study strategies during video lectures may affect student learning. In this paper, we present a systematic investigation of video-recorded lectures in the context of secondary school mathematics education. Study 1 investigated students’ (n = 24) preferences and teacher’s comments regarding six different video styles in a flipped learning environment. Study 2 consisted of two stages and examined the learning effects of different video styles and study strategies. In Study 2A, we randomly assigned 129 students to one of the three most preferred video styles from Study 1. No significant difference was found among the three video styles in terms of learning performance. Nevertheless, Khan-style + teacher’s talking head videos required the least teacher effort and registered the highest mean scores in both the recall and application questions. Study 2B thus focused on this video style and randomly assigned 87 students to two different study strategies: interpolated short-answer quizzes and interpolated copying of worked answers. The results suggest that the interpolation of video lectures with short-answer quizzes significantly improves students’ ability to solve novel problems. Overall, we recommend that teachers use Khan-style + teacher’s talking head videos with interpolated short-answer quizzes in the future practice of video-recorded lecturing for secondary school mathematics.

Erratum: "Assisting E-Government Users with Animated Talking Faces" by P. Foglia, F. Giuntoli, C. A. Prete, and M. Zanda

The references were inadvertently removed from an article entitled "Assisting E-Government Users with Animated Talking Faces" in the January-February 2007 issue of &lt;interactions&gt; . We apologize for this oversight. The information is available below. 1. Cullen, R., O'Connor D., Veritt A. An Evaluation of Local Government Web sites in New Zealand. In The World of EGovernment (ed. Curtin G.C., Sommer M.H., Sommer V.), Haworth, 2003. 2. Foglia P., Prete C.A., Zanda M. Modelling Public Administration Portals. In Encyclopaedia of Portal Technology and Application (ed. Tatnall A.), Idea Group (in press). 3. Haxby J.V., Gobbini M.I., Furey M.L., Ishai A., Schouten, J.L., Pietrini P. Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science, Sept. 28; 293(5539):2405-7, 2001. 4. Nielsen J. Talking-Head video is boring online, AlertBox, Dec. 5, 2005. 5. Qiu L., Benbasat I. An Investigation into the effects of text-to-speech voice and 3D avatars on the perception of presence and flow of live help in electronic commerce. ACM Transactions on Computer Human Interaction (TOCHI), 12(4); 2005. 6. Reis F. E-Government: Internet based interaction with the European businesses and citizens; Statistics in Focus; Eurostat, 2005. 7. Stewart O., Huerta J.M. Transition relevance place: a proposal for adaptive user interface in natural language dialog management systems. ACM SIGCHI Conference Work in progress, 2006.

A revolution in education

A classroom is equipped with strategically placed cameras to show the instructor, a zoomed view of material chosen by the instructor, the board, and all students in the class. Enough cameras provide different views to closely observe all activities of interest in the classroom. The board itself may be electronic-a computer display as well as a tablet used for input to a computer. The combined system builds a model of the classroom at every instant and permits interactive viewing of it. In live mode, the system lets any person anywhere be telepresent in the class and participate in it. You can observe and explore as you wish, rather than relying on what the cameraman wants you to see. In a real classroom, you have a fixed location and cannot move around without disturbing the class. Not so in remote mode. A remote student can electronically copy the contents of the classroom material, including the board display. A remote student can ask questions in class at any time, and their video may be displayed to the class at any time. In fact, it is possible to display all remote students as a video gallery to provide feedback to the instructor. This classroom removes the requirement of physically being in a class to attend. In archive mode, the system can reconstruct the class proceedings in telepresence mode to let a student visit, or revisit, the class from any viewpoint. All class activities are archived for this purpose. Postprocessing can further enhance features as required. Many other information sources can be attached to provide all information at one place. The archival system lets students tele-attend a class at their own convenience and pace. This does not mean just viewing a talking-head video, but immersion in the classroom.