On-screen Events Research Articles

For 19-Month-Olds, What Happens On-Screen Stays On-Screen.

Humans rely extensively on external representations such as drawings, maps, and animations. While animations are widely used in infancy research, little is known about how infants interpret them. In this study, we asked whether 19-month-olds take what they see on a screen to be happening here and now, or whether they think that on-screen events are decoupled from the immediate environment. In Experiments 1–3, we found that infants did not expect a falling animated ball to end up in boxes below the screen, even though they could track the ball (i) when the ball was real or (ii) when the boxes were also part of the animation. In Experiment 4, we tested whether infants think of screens as spatially bounded physical containers that do not allow objects to pass through. When two location cues were pitted against each other, infants individuated the protagonist of an animation by its virtual location (the animation to which it belonged), not by its physical location (the screen on which the animation was presented). Thus, 19-month-olds reject animation-reality crossovers but accept the depiction of the same animated environment on multiple screens. These results are consistent with the possibility that 19-month-olds interpret animations as external representations.

Finding Untitled Goose Game’s Dynamic Music in the World of Silent Cinema

There are three unusual things about Untitled Goose Game’s music. First, for an independent video game produced by a small studio, the music is dynamic and reactive to a high degree. The game uses pre-recorded, non-generative musical performances and yet will respond to onscreen events within a buffer of only a few seconds at maximum. Second, the music takes inspiration not from other dynamic music systems in video games but from the varying practices of musical accompaniment for silent cinema and early comedy, aiming to replicate affect rather than process. Finally, the music for Untitled Goose Game takes the unusual step of adapting pre-existing classical music from the public domain—in this case, six of Claude Debussy’s Préludes for solo piano—rather than creating an original score intended from its conception to be dynamic. Accordingly, this article outlines the dynamic music system at work in Untitled Goose Game and the influence drawn on for this system from non–video game approaches to musical accompaniment. The article discusses the varying practices for music for the silent era of cinema, the theoretical frameworks used to conceptualize these many divergent approaches, and how closely we might recognize their legacy at work in Untitled Goose Game’s soundtrack. Ultimately, this article argues that by looking to approaches beyond more familiar debates about dynamic music for video games, Untitled Goose Game helped shortcut familiar problems that confront developers and composers when working with dynamic and reactive music.

Trauma, empathy and complicity in The Last of Us Part II: playing as enemy characters

The Last of Us Part II continues Ellie’s story and introduces the new playable character Abby as they inhabit a post-apocalyptic world. This article aims to analyze how the game portrays trauma, which is a central component of the narrative. Considering the existence of two different diegetic agents, the article also discusses the concepts of empathy and complicity. The results indicate that through the experience of playing as enemy characters, the game successfully builds empathy for people with distinct motivations while also making the player feel complicit with the violence committed by the characters. The article also concludes that video games and their capacity for making the player feel empathic and complicit with the on-screen events present a very interesting possibility for a readerly engagement with trauma.

EXTENDING GAMING DEMAND: SUBJECTIVE EXPERIENCES OF COGNITIVE, EMOTIONAL, PHYSICAL, AND SOCIAL VIDEOGAMING REQUIREMENTS

Videogames directly involve players as co-creators of on-screen events, and this interactivity is assumed to be a core source of their attraction as a successful entertainment medium. Although interactivity is an inherent property of the videogame, it is variably perceived by the end user—for some users, perceived as a more demanding process, taxing their already-limited attentional resources. At least four such demands have been explicated in extant literature: cognitive (making sense of game logics/tasks), emotional (affective responses to game events/outcomes), physical (managing controller inputs and interfaces), and social (responding to human/nonhuman in-game others). Past work has reported empirical support of these concepts through validation of closed-ended survey metrics (e.g., Video Game Demand Scale). The current study challenges and extends the demand concept through an analysis of players’ own language when describing videogame demands in short essays about gaming experiences—critical given that people may experience a phenomenon in ways not accounted for in deductive data approaches. A secondary analysis of qualitative data made freely available by VGDS authors revealed both convergence with and divergence from prior work. Comporting with VGDS, cognitive demands are mostly experienced by players as ludic concerns and physical demands are mostly experienced in relation to handheld controller perceptions. Diverging from VGDS, players’ emotional demands represented both basic and complex emotional states, and social demands manifest different depending on whether or not the social “other” is human or non-human: humans are considered demanding on interpersonal terms, whereas non-humans are considered demanding as personified evocative objects.

The piano mécanique in 1930s French cinema

ABSTRACT Several French poetic realist films in the 1930s featured player pianos or other kinds of pianos mécaniques. Their visual and narrative presence often carried with it the potential for aural disruption: mechanical instruments like the player piano were ideal anempathetic musical sources, creating music that was both uncanny in its inhumanness and threatening in its indifference to on-screen events. The piano mécanique held additional extra-diegetic significance in France: its importance as a tool for modernist composition. This article traces the history of the piano mécanique in 1930s French cinema. It places the player piano’s presence in several iconic films of the decade within a broader history of early twentieth-century avant-garde composition. The article additionally considers its role as a reflexive commentary on cinema’s transition to synchronised sound in the early 1930s. It concludes with an analysis of scenes from two iconic films: Pépé le Moko (Julien Duvivier, 1937) and La Règle du jeu (Jean Renoir, 1939), which demonstrates the evolving reception of the player piano in 1930s France. The analysis reveals a strong connection between cinematic and modernist musical practices and demonstrates the effect of rapid technological and cultural change on the evolution of 1930s French film music.

Beyond Behavioral Inhibition: A Computer Avatar Task Designed to Assess Behavioral Inhibition Extends to Harm Avoidance.

Personality factors such as behavioral inhibition (BI), a temperamental tendency for avoidance in the face of unfamiliar situations, have been identified as risk factors for anxiety disorders. Personality factors are generally identified through self-report inventories. However, this tendency to avoid may affect the accuracy of these self-report inventories. Previously, a computer based task was developed in which the participant guides an on-screen “avatar” through a series of onscreen events; performance on the task could accurately predict participants’ BI, measured by a standard paper and pencil questionnaire (Adult Measure of Behavioral Inhibition, or AMBI). Here, we sought to replicate this finding as well as compare performance on the avatar task to another measure related to BI, the harm avoidance (HA) scale of the Tridimensional Personality Questionnaire (TPQ). The TPQ includes HA scales as well as scales assessing reward dependence (RD), novelty seeking (NS) and persistence. One hundred and one undergraduates voluntarily completed the avatar task and the paper and pencil inventories in a counter-balanced order. Scores on the avatar task were strongly correlated with BI assessed via the AMBI questionnaire, which replicates prior findings. Females exhibited higher HA scores than males, but did not differ on scores on the avatar task. There was a strong positive relationship between scores on the avatar task and HA scores. One aspect of HA, fear of uncertainty was found to moderately mediate the relationship between AMBI scores and avatar scores. NS had a strong negative relationship with scores on the avatar task, but there was no significant relationship between RD and scores on the avatar task. These findings indicate the effectiveness of the avatar task as a behavioral alternative to self-report measures to assess avoidance. In addition, the use of computer based behavioral tasks are a viable alternative to paper and pencil self-report inventories, particularly when assessing anxiety and avoidance.

Personalized Object-Based Audio for Hearing Impaired TV Viewers

Age demographics have led to an increase in the proportion of the population suffering from some form of hearing loss. The introduction of object-based audio to television broadcast has the potential to improve the viewing experience for millions of hearing impaired people. Personalization of object-based audio can assist in overcoming difficulties in understanding speech and understanding the narrative of broadcast media. The research presented here documents a Multi-Dimensional Audio (MDA) implementation of object-based clean audio to present independent object streams based on object category elicitation. Evaluations were carried out with hearing impaired people and participants were able to personalize audio levels independently for four object-categories using an on-screen menu: speech, music, background effects and foreground effects related to on-screen events. Results show considerable preference variation across subjects but indicate that expanding object-category personalization beyond a binary speech/non-speech categorization can substantially improve the viewing experience for some hearing impaired people.

Watch what I do, not what I say I do: Computer-based avatars to assess behavioral inhibition, a vulnerability factor for anxiety disorders

Behavioral inhibition (BI), a tendency to withdraw from or avoid novel social and non-social situations, is a personality trait which can confer risk for anxiety disorders. Like many personality traits, BI is often assessed via self-report questionnaires where respondents rate themselves for frequency of certain behaviors or feelings. However, questionnaires have inherent limitations, particularly in psychiatric populations where there may be unawareness of deficit. A viable alternative may be virtual environments, in which the participant guides an on-screen “avatar” through a series of onscreen events meant to simulate real-world situations. Here, we report on initial development of such an assessment tool, involving several onscreen scenarios with choice points where the participant can select from response options corresponding to inhibited or uninhibited behaviors. In two experiments involving over 300 college students, scores on the computer-based task were strongly correlated with BI scores attained through self-report questionnaire (r > .780, p < .001); this relationship held regardless of participant gender and experience with computer games. The results suggest that virtual environments may hold promise as alternative formats for assessment of personality traits in populations unsuited to traditional paper-and-pencil questionnaire formats due to psychopathology, limited attention span, or poor vocabulary and/or literacy skills.

‘Half of it is just the fun of finding the right music’: Music and the films of Adam Curtis

ABSTRACTThis article introduces the use of music in Adam Curtis' documentary films, focusing on The Power Of Nightmares (2005) and All Watched Over By Machines of Loving Grace (2011). Curtis' work is notable for its use of found footage and archive material, as well as an eclectic range of pre-existing musics from cinematic, popular, industrial and classical sources. Music is not used as a passive accompaniment to on-screen events; it takes an active role in directing the documentary narrative. This article argues that deciphering the uses of music in these works is key to understanding Curtis' authorial intent.

Inhibition of eye blinking reveals subjective perceptions of stimulus salience

Spontaneous eye blinking serves a critical physiological function, but it also interrupts incoming visual information. This tradeoff suggests that the inhibition of eye blinks might constitute an adaptive reaction to minimize the loss of visual information, particularly information that a viewer perceives to be important. To test this hypothesis, we examined whether the timing of blink inhibition, during natural viewing, is modulated between as well as within tasks, and also whether the timing of blink inhibition varies as a function of viewer engagement and stimulus event type. While viewing video scenes, we measured the timing of blinks and blink inhibition, as well as visual scanning, in a group of typical two-year-olds, and in a group of two-year-olds known for attenuated reactivity to affective stimuli: toddlers with Autism Spectrum Disorders (ASD). Although both groups dynamically adjusted the timing of their blink inhibition at levels greater than expected by chance, they inhibited their blinking and shifted visual fixation differentially with respect to salient onscreen events. Moreover, typical toddlers inhibited their blinking earlier than toddlers with ASD, indicating active anticipation of the unfolding of those events. These findings indicate that measures of blink inhibition can serve as temporally precise markers of perceived stimulus salience and are useful quantifiers of atypical processing of social affective signals in toddlers with ASD.

Eyetracking for two-person tasks with manipulation of a virtual world

Eyetracking facilities are typically restricted to monitoring a single person viewing static images or pre-recorded video. In the present article, we describe a system that makes it possible to study visual attention in coordination with other activity during joint action. The software links two eyetracking systems in parallel and provides an on-screen task. By locating eye movements against dynamic screen regions, it permits automatic tracking of moving on-screen objects. Using existing SR technology, the system can also cross-project each participant's eyetrack and mouse location onto the other's on-screen work space. Keeping a complete record of eyetrack and on-screen events in the same format as subsequent human coding, the system permits the analysis of multiple modalities. The software offers new approaches to spontaneous multimodal communication: joint action and joint attention. These capacities are demonstrated using an experimental paradigm for cooperative on-screen assembly of a two-dimensional model. The software is available under an open source license.

COVIEWING PRESCHOOL TELEVISION IN THE US

Past studies have shown that naturalistic parent–child coviewing of television provides a context for interactions that promote comprehension and more general language development. To determine whether onscreen text can facilitate such interactions, 58 parent–child dyads (with children aged 3 to 5 years) were observed while viewing a preschool video in which the narrative was accompanied by either: (1) an onscreen “Mommy bar” of subtitles including jokes and general parenting tips targeted at parents; (2) a parent‐directed “educationally enhanced bar” that modeled the sorts of educationally valuable comments observed in past research (e.g. discussing characters' emotions, tying onscreen events to the child's own experience); or (3) no onscreen subtitles for parents. Results indicated that, during viewing, parents who watched the “educationally enhanced” version were significantly more likely to make comments involving inferences about characters' emotions, tying onscreen events to children's own experiences, and encouraging children to engage in viewer participation. Thus, onscreen prompts can successfully elicit the types of parent–child interactions that can contribute to story comprehension, language development, and literacy.

The Speed Death of the Eye: The Ideology of Hollywood Film Special Effects

In the late 20th and early 21st centuries, increased computing power has made possible extraordinary leaps in film special effects. This article argues that special effects developed since the beginning of digital animation, when coupled with standard editing room techniques (jump cuts, cutaways), have brought us to an era where the eye cannot keep pace with on-screen events. It is arguable that video gamers are best equipped to handle the visual overload produced by action films' effects. The article enumerates a series of techniques used in current action films to bring about visual excess that has an upsetting, exciting, overwhelming somatic effect on the viewer; these same effects are indispensable for the success of contemporary blockbuster action movies. Following theorist Paul Virilio's arguments, this article suggests that a machine ideology drives the perceptual system to dizzying limits, resulting in the “speed death of the eye.”

Game

Game

Odyssey Renewed

The first home video-gaming console, the Magnavox Odyssey, was released in 1972. Its limited graphical capacities led Magnavox to ship it with a number of plastic overlays for the user's television that would admit a little variety into the then relatively crude gaming experience, limited to a built-in, Pong-like game. Computer and video games have come a long way since then, but it often seems as if critical approaches to gaming have continued shuffling through these plastic films, taking transformations of the screen, or on-screen events, for the whole of the gaming experience. It seems to me that reflection has been paralysed, becoming a discourse of regulation as it revolves around anxieties about gender, violence and narrative. I'd like to explore these anxieties as they've emerged in a few places, and then see if I can articulate the beginnings of an approach that might afford us a more complex, less pessimistic aesthetics of gaming.