Virtual Reality Narratives Research Articles

Immersive machine learning for social attitude detection in virtual reality narrative games

People can understand how human interaction unfolds and can pinpoint social attitudes such as showing interest or social engagement with a conversational partner. However, summarising this with a set of rules is difficult, as our judgement is sometimes subtle and subconscious. Hence, it is challenging to program Non-Player Characters (NPCs) to react towards social signals appropriately, which is important for immersive narrative games in Virtual Reality (VR). We collaborated with two game studios to develop an immersive machine learning (ML) pipeline for detecting social engagement. We collected data from participants-NPC interaction in VR, which was then annotated in the same immersive environment. Game design is a creative process and it is vital to respect designer’s creative vision and judgement. We therefore view annotation as a key part of the creative process. We trained a reinforcement learning algorithm (PPO) with imitation learning rewards using raw data (e.g. head position) and socially meaningful derived data (e.g. proxemics); we compared different ML configurations including pre-training and a temporal memory (LSTM). The pre-training and LSTM configuration using derived data performed the best (84% F1-score, 83% accuracy). The models using raw data did not generalise. Overall, this work introduces an immersive ML pipeline for detecting social engagement and demonstrates how creatives could use ML and VR to expand their ability to design more engaging experiences. Given the pipeline’s results for social engagement detection, we generalise it for detecting human-defined social attitudes.

The possibilities of illness narratives in virtual reality for bodies at the margins

ABSTRACT Through decades of scholarly analysis and application, the practice of illness narratives has been established as an effective therapeutic intervention for dealing with illness-related emotional well-being (Couser; Frank; Irvine and Charon). Scholars of illness narratives argue that the medium works to bring agency back to the body following the neoliberal relinquishing of one’s life story in the patient-physician encounter. Contemporary scholarly work is mapping the growth of illness narrative forms from the traditional book to emerging digital-born narratives; however, there is limited research on the medium’s intersection with virtual reality (VR) technologies. Working with Marie-Laure Ryan’s theoretical framework of possible worlds theory, this paper explores the transformative potential of VR illness narratives for pathologized identities found when VR resists the call to fall into one of two categories: pure transhumanism where VR reality is emancipated from actual reality or an artificial experience that has no lasting effect on the self.

VR Storytelling: Potentials and Limitations of Virtual Reality Narratives

In this introduction we will aim to outline some of the principal transformations that VR is imparting to audiovisual language and experience, with a particular focus on the impact of these innovations on storytelling. The essays of the section span a variety of theoretical approaches and backgrounds, argumentative styles and case studies (e.g. documentaries, games, art), but all of them share a similar rhetorical attitude: underscoring the potential and novelty of VR but, at the same time, identifying a series of remediations and limitations. In particular, they will show that most of the contemporary discourses on VR need to be reframed critically in order to reflect more objectively on complex notions and dynamics such as screen, interactivity, immersion, presence, embodiment, illusion and empathy.

PHOTOGRAMMETRIC UNDERWATER AND UAS SURVEYS OF ARCHAEOLOGICAL SITES: THE CASE STUDY OF THE ROMAN SHIPWRECK OF TORRE SANTA SABINA

Abstract. The 2020 underwater archaeological research in the inlet of Torre Santa Sabina – Baia dei Camerini (Municipality of Carovigno, Brindisi, Italy) represented the first phase of the pilot intervention of the Interreg Italia-Croatia UnderwaterMuse project. The project aims to enhance and make accessible the vast underwater heritage of the areas involved; this will be done by creating submerged archaeological parks and using virtual reality's narrative and communicative tools and platforms. During the 2020 campaign, different survey activities were carried out using several techniques and methodology to produce the area's multi-scale documentation. First of all, the entire stretch of coast was mapped with UASs (Uncrewed Aerial Systems) flights to reconstruct the coastal landscape in various phases. Furthermore, an underwater photogrammetric survey carried out by expert scuba divers has been achieved in the wreck's aft area (the site analyzed during the 2020 excavation activities). The fruitful synergy between the various actors involved and the support of the territory and the community has allowed the achievement of this campaign's objectives, preliminary to the broader and more articulated intervention foreseen for the following year.

Samuel Beckett in Virtual Reality: Exploring Narrative Using Free Viewpoint Video

Abstract This article describes an investigation of interactive narrative in virtual reality (VR) through Samuel Beckett's theatrical text Play. Actors are captured in a green screen environment using free-viewpoint video (FVV). Built in a game engine, the scene is complete with binaural spatial audio and six degrees of freedom of movement. The project explores how ludic qualities in the original text elicit the conversational and interactive specificities of the digital medium. The work affirms potential for interactive narrative in VR, opens new experiences of the text and highlights the reorganization of the author-audience dynamic.

Talking Realities: Audio Guides in Virtual Reality Visualizations.

Building upon the ideas of storytelling and explorable explanations, we introduce Talking Realities, a concept for producing data-driven interactive narratives in virtual reality. It combines an audio narrative with an immersive visualization to communicate analysis results. The narrative is automatically produced using template-based natural language generation and adapts to data and user interactions. The synchronized animation of visual elements in accordance with the audio connects the two representations. In addition, we discuss various modes of explanation ranging from fully guided tours to free exploration of the data. We demonstrate the applicability of our concept by developing a virtual reality visualization for air traffic data. Furthermore, generalizability is exhibited by sketching mock-ups for two more application scenarios in the context of information and scientific visualization.

Culture and vision in virtual reality narratives

AbstractHansen states that “FL studies must learn to conceive of culture as an open, multi‐voiced and dialogical interaction full of contradictions.” One advocated approach to teach transcultural understanding is through the analysis of cultural narratives. Kearney defines cultural narratives as “the multiple (sometimes competing), conventional storylines that cultural groups produce and use to make sense of and attribute meaning to their shared experiences.” This article will showcase a project in a beginning French course in which four different Parisians from the same quarter were asked to document and share the stories of their lives with a virtual reality (VR) camera. Findings reveal that the VR narratives allowed students to envision, experience, and understand diverse facets of Parisian culture and more vividly imagine their future role as participants in Parisian communities.

Hacia un modelo de narrativa en periodismo inmersivo

Introducción: Ante el incremento de uso por parte de periodistas y medios de comunicación, de la realidad virtual y los contenidos inmersivos como formato para informar, nos hemos propuesto aportar un posible modelo de narrativa de realidad virtual para la producción de estos contenidos. Metodología: Se ha optado por un estudio mixto, donde lo cuantitativo está compuesto por encuestas realizadas a 60 individuos Generación Z españoles, al tiempo que dos grupos de discusión con 5-6 individuos de similar grupo etario. Asimismo, se consideró un análisis comparativo entre elementos de la narrativa inmersiva y la narrativa audiovisual tradicional, para alcanzar el fin deseado. Resultados y conclusiones: se evidencia la complejidad de la segmentación de audiencias en el ámbito digital, aunque los individuos pertenezcan al mismo entorno social. Los instrumentos de medición permitieron disponer de algunos elementos básicos para la creación de contenidos noticiosos en formatos multimedia con capacidad inmersiva, los cuales deben ir acompañados de un compromiso y constancia por parte de los medios de comunicación y periodistas implicados en procesos de innovación.

Narrative in Virtual Reality? Anatomy of a Dream Reborn

Narrative in Virtual Reality? Anatomy of a Dream Reborn

From BIM to VR: defining a level of detail to guide virtual reality narratives

In 2012, the Carleton Immersive Media Studio (CIMS) started a research relationship with Public Services and Procurement Canada to develop a building information model (BIM) of the Parliament Hill National Historic Site of Canada. The model was created to facilitate a multi-year rehabilitation of the site and was developed using both historical records and highly detailed geo-referenced point cloud data. In the process of planning the model, CIMS developed a unique Level of Detail (LOD) specification for heritage buildings that, in addition to standard specifications, considered cultural heritage value as part of the LOD. As the rehabilitation project unfolded, the possibility of using the BIM for public engagement through the creation of virtual reality (VR) experiences was proposed. In this paper, we discuss the transferal of CIMS’ LOD from a BIM to a VR environment, arguing that the BIM LOD’s focus on cultural heritage value is consistent with virtual reality LOD in that it can be used to guide participants through a virtual reality narrative by inferring that areas of higher fidelity have greater value.

Lithium hindsight 360: Increasing access and ensuring preservation of virtual reality narratives

ABSTRACTThis alternative event combines a virtual reality (VR) demonstration and interactive discussion to brainstorm access and preservation strategies for individual VR creators. A specific challenge to address will be how to build these considerations into the creative process rather than after the work has been completed. The Lithium Hindsight 360 medical humanities narrative will be used as the demo to raise additional questions about ethics that may not normally be present in a new media artwork.

Surveying VR storytelling: Investigating key terminology and the role of the procedural author

Current discussions around virtual reality (VR) storytelling have surged in the last few years, along with the growth of virtual reality technology and content. There is a focus on differentiating and defining this medium by discussing the immersive, interactive qualities that give it new ways to tell stories. Questions circle around what it means when audiences are ‘present’ in the narrative and how this creates a desire to have agency. I argue these terms – immersion, presence, interactivity, and agency – can be afforded more nuance as they are often used to discuss two different yet interconnected aspects of telling stories with technology: the technology itself and narrative techniques. In an attempt to understand how to tell stories with VR, I have discussed the terms immersion, presence, interactivity and agency from a technology perspective and a narrative perspective. I discuss the translation of literary techniques to VR narratives to demonstrate how storytellers can alter the way audiences are immersed, have agency and feel present and can interact in digital narratives.

Exploring Enjoyment of Cinematic Narratives in Virtual Reality: A Comparison Study

This study compares watching a film in a traditional theater setting to watching the same film in a virtual theater using a virtual reality (VR) head-mounted display (HMD). The study seeks to determine whether watching a film in a virtual theater is a comfortable experience that viewers can tolerate; and whether watching in VR produces similar feelings of identification, transportation, and enjoyment for the films as watching in a traditional theater. Using the Oculus Rift DK2 HMD and the Riftmax virtual theater software, participants watched either Signs or Ferris Bueller's Day Off in a virtual theater. Other participants watched the same films in a traditional theater setting, and independent comparisons were made. Results indicate that over 90% of the VR participants were able to watch the entire movie in the virtual theater, and feelings of identification, transportation, and enjoyment were quite similar between conditions. Differences found were attributable to the films' content, with an interaction between VR and theater conditions. Implications and future research are discussed.

Hearts and Minds: The Interrogations Project

Hearts and Minds: The Interrogations Project is an interactive virtual reality narrative performance made for the EVL’s CAVE2™ large-scale 320-degree panoramic virtual reality environment that visualizes stories of violence and the post-traumatic stress experienced by ordinary American soldiers who became torturers in the course of serving their country. During the American-led counterinsurgency and counter-terrorism campaigns in Iraq in the years after 11 September 2001, the torture and abuse of detainees was a commonplace tactic.

An ‘Amuse-Bouche at Best'

Much has been written about the function of narrative in virtual reality (VR) productions (Aylett & Louchart, 2003; Aylett et al, 2005; Ryan, 2001; 2005; 2008; 2009), but the role of the audience, and the relative degree of control that they have over the content, has led some scholars to believe that there is an ontological problem with describing VR in narrative terms. This article investigates some of these assumptions, via an analysis of an undertheorised aspect of VR that has emerged in recent years – the 360° film. It argues that 360° film represents a much more important aspect of VR than has been previously recognised. In so doing, the article establishes this medium as an important field of study, and argues that ultimately, it will be the commercial infrastructure for this content which will define the parameters of immersive storytelling.