Perceptual Immersion Research Articles

Particle-Velocity-Based Mixed-Source Sound Field Translation for Binaural Reproduction

Following the rise of virtual reality is a demand for sound field reproduction techniques that allow the user to interact and move within acoustic reproductions with six-degrees-of-freedom. To this end, a mixed-source model of near-field and far-field virtual sources has been introduced to improve the performance of sound field translation in binaural reproductions of spatial audio recordings. The previous works, however, expand the sound field in terms of the mixed sources based on sound pressure. In this paper, we develop a new mixed-source expansion based on particle velocity, which contributes to more precise reconstruction of the interaural phase difference and, therefore, contributes to improved human perception of sound localization. We represent particle velocity over space using velocity coefficients in the spherical harmonic domain, and the driving signals of the virtual mixed-sources are estimated by constructing cost functions to optimize the velocity coefficients. Compared to the state-of-the-art method, sound-pressure-based mixed-source expansion, we show through numerical simulations that the proposed particle-velocity-based mixed-source expansion has better reconstruction performance in sparse solutions, allowing for sound field translation with better perceptual immersion over a larger space. Finally, we perceptually validate the proposed method through a Multiple Stimulus with Hidden Reference and Anchor (MUSHRA) experiment for a single source scenario. The experimental results support the better perceptual immersion of the proposed method.

3D hand reconstruction with both shape and appearance from an RGB image

Modeling 3D hands with geometry details and appearance can increase perceptual immersion and realism in many applications. However, traditional 3D representations such as meshes and voxel grids can not represent high-quality hand shapes with their property of the discrete surface and fixed topology. To address these problems, we introduce implicit function for more detailed shape reconstruction, and design a Structure-aware Signed Distance Function (S-SDF) to reconstruct hand shape in arbitrary resolution. In this way, our method not only focuses on independent fingers but also keeps the relationship between fingers. Such implicit function of 3D representation associated with part semantics and structure of the hand is central to generating realistic hand shapes of diverse variations. Meanwhile, in order to avoid time-consuming and laborious 3D annotating of texture, we present a self-supervised appearance synthesis approach. Extensive experiments on widely used datasets demonstrate that the proposed method reconstructs more realistic hands compared with previous methods.

IMPLICATED GAMING: CHOICE AND COMPLICITY IN LUDIC HOLOCAUST MEMORY

ABSTRACTHolocaust memorial sites and institutions have begun to embrace new media and digital technologies as methods of communication, public engagement, and memorialization. Despite increasing numbers of interactive digital media projects focused on Holocaust education, there is a significant gulf between the topics addressed by digital Holocaust works and those conceptualized and studied at a higher level in scholarly Holocaust literature. Most notably, challenging questions regarding the categories of bystandership, complicity, and perpetration are largely ignored in favor of traditional victim‐focused narratives. I suggest that, in their eagerness to adopt virtual reality (VR) and mixed reality (MR) technologies, digital Holocaust memory projects have neglected the significant potential of nonimmersive video games to address questions regarding bystandership and complicity. By moving away from the perceptual immersion of VR and toward the ludic and simulative arguments of video games, digitally interactive Holocaust projects may be able to lessen the risks of over‐immersion and retraumatization that are antithetical to critical historical thinking and understanding. This article examines the 2013 game Papers, Please as an example of how video games can present sophisticated arguments about human agency, bystandership, and complicity. By placing players in a historical problem space laden with impossible moral choices, Papers, Please demonstrates the systemic forces that structure human behavior under extreme, violent, and authoritarian conditions. I argue that Holocaust‐based ludic digital media modeled after Papers, Please could explore bystandership and complicity in similarly nuanced and powerful ways, potentially touching on academic Holocaust concepts such as the “choiceless choices” of “the gray zone.” Addressing these topics in ludic historical media could help to bridge the gap between popular and scholarly understandings of the Holocaust in the twenty‐first century.

Perceptual Immersion, or What Computer Games Can Be Used for in Architectural Education

Perceptual Immersion, or What Computer Games Can Be Used for in Architectural Education

Movie viewing elicits rich and reliable brain state dynamics

Adaptive brain function requires that sensory impressions of the social and natural milieu are dynamically incorporated into intrinsic brain activity. While dynamic switches between brain states have been well characterised in resting state acquisitions, the remodelling of these state transitions by engagement in naturalistic stimuli remains poorly understood. Here, we show that the temporal dynamics of brain states, as measured in fMRI, are reshaped from predominantly bistable transitions between two relatively indistinct states at rest, toward a sequence of well-defined functional states during movie viewing whose transitions are temporally aligned to specific features of the movie. The expression of these brain states covaries with different physiological states and reflects subjectively rated engagement in the movie. In sum, a data-driven decoding of brain states reveals the distinct reshaping of functional network expression and reliable state transitions that accompany the switch from resting state to perceptual immersion in an ecologically valid sensory experience.

Categorizing the Educational Affordances of 3 Dimensional Immersive Digital Environments

Aim/Purpose: This paper provides a general-purpose categorization scheme for assessing the utility of new and emerging three-dimensional interactive digital environments (3D-IDEs), along with specific pedagogic approaches that are known to work. It argues for the use of 3D-IDEs on the basis of their ludic appeal and ability to provide intrinsic motivation to the learner, and their openness that allows the learner to gain a more holistic understanding of a topic. Background: Researchers have investigated the affordances, benefits, and drawbacks of individual 3D-IDEs, such as virtual worlds, but teachers lack a general-purpose approach to assessing new 3D-IDEs as they appear and applying them to teaching practice. Methodology: The categorization scheme is based on the analysis, reflection, and comprehension of the research on limitations, challenges, and opportunities for teaching in virtual environments by Angel Rueda, Valdes Godines and Guzmán Flores; the scheme is discussed in terms of an experiment to trial virtual genetics labs in Second Life. Contribution: The paper describes a general-purpose approach to applying existing and new 3D virtual spaces to education, shows a worked example of the use of the categories, and describes six approaches to consider in applying these technologies. Findings: 3D-IDEs are categorized in terms of the way in which they interface with the user’s senses and their ability to provide ‘immersion’; two forms of immersion are examined: digital perceptual immersion – the generated sense of reality – and ludic narrative immersion – a less cognitive and more emotional engagement with the learning environment. Recommendations for Practitioners: Six specific forms of pedagogy appropriate for 3D-IDEs are examined and discussed, in terms of the affordances and technology required, as assessed by the categorization scheme. More broadly, the paper argues for a change in the assessment of new digital technologies from the technology’s features to its affordances and the pedagogies it can support. Recommendation for Researchers: The paper offers a practical approach to choosing and using 3D-IDEs for education, based upon previous work. The next step is to trial the scheme with teachers to ascertain its ease of use and effectiveness. Impact on Society: The paper argues strongly for a new approach to teaching, where the learner is encouraged to use 3D-IDEs in a ludic manner in order to generate internal motivation to learn, and to explore the topic according to their individual learning needs in addition to the teacher’s planned route through the learning material. Future Research: The categorization scheme is intended to be applied to new technologies as they are introduced. Future research is needed to assess its effectiveness and if necessary update the scheme.

Perceptual Conflicts in a Multi-Stereoscopic Immersive Virtual Environment: Case Study on Face-to-Face Interaction through an Avatar

With multi-stereoscopy technology, novel projection-based immersive systems now can support multiple users by providing each one with an independent stereoscopic view of the virtual scene. When users work face-to-face, they may have an incorrect view if objects are located between them. In this case, avatars can be introduced to enable face-to-face interaction in the virtual world, whereas they are side-by-side in the real device. As a consequence, such multi-user systems provide the users with a new kind of perceptual immersion and related cognitive experiences, because users must handle both information from the real world (i.e., other users' bodies) and those from the virtual scene (i.e., other users' avatars) at the same time. In this study, we experimentally created special interaction situations to examine the perceptual conflicts generated by the dual-presence of the real and virtual visual and audio stimuli. In a two-user scenario, participants performed an object-picking task according to three types of instructions (verbal, gestural, or multimodal instructions) given by an experimenter. This co-located experimenter was also virtually present by an avatar in the virtual world to enable face-to-face interactions with the participants. Our goal was to observe to what extent the perceptual conflicts induced by the dual-presence of the experimenter can be integrated without significantly altering the performance of the participants. For that, we studied the influence of such perceptual conflicts on participants' choice of collaborator (whether they interacted with the avatar or the real experimenter) and on their task efficiency. As the results showed, first users had an a priori choice of collaborator (avatar or real person) and this choice did not change under different experimental conditions. Second, perceptual conflicts had an impact on users' performance in terms of task completion time. We discuss the implications of these results for designing a better immersive system for co-located collaboration between multiple users.

3D in Depth: <em>Coraline, Hugo</em>, and a Sustainable Aesthetic

Digital 3D presents a rare opportunity to study the aesthetic impact of a new technology. In the face of its ever-diminishing novelty, filmmakers are seeking a sustainable formal response to 3D. Some are moving away from protrusion effects, which are associated with disruptive gimmickry, and exploring depth as a means of extending the technology's narrative reach. This style, exemplified by Henry Selick's Coraline (2009) and Martin Scorsese's Hugo (2011), courts narrative integration and critical acceptance by downplaying protrusion in favor of precise manipulations of perspective and volume. The essay argues against the concept of perceptual immersion as an explanatory framework for 3D, and focuses instead on the prospects for diegetic absorption.

신뢰와 인지적 몰입 매개변수가 스마트폰의 사용과 만족도에 미치는 영향 분석

The purpose of this study is to explore determinants which affect the significant increase in the user acceptance of smart phone. This study also analyzes the effect of each variable on the actual acceptance by empirical methods. In this study, first, the system quality and the service quality are defined as independent variables based on developed IS success model of DeLone & McLean(2003). Second, we proposed the research model by providing trust and perceptual immersion as intermediate variables, and user satisfaction and actual use as dependent variables by the proceeding research for accepting information technology and new service. Third, the statistical analysis is conducted by surveying to 200 smart phone users for verifying a validity of research models and hypotheses. As a result, almost hypotheses are accepted in confidence interval except for the hypothesis between security and trust variable.

Cognitive and Meta-cognitive Dimensions of Psychoses

This paper outlines cognitive approaches to understanding and treating positive psychotic symptoms, such as hallucinations, delusions, and dissociations. Recent cognitive accounts of psychosis are reviewed along with the claim that it is not the symptoms themselves but cognitive and meta-cognitive appraisals (attributions and beliefs) about the significance of the symptoms that cause distress and dysfunction. Psychotic symptoms do lie on a continuum with normal experience. Cognitive appraisal dimensions may interact with reasoning styles such as inferential confusion, cognitive slippage, fantasy proneness, and perceptual immersion (styles also normally distributed in the population) and together persuade the person with psychosis to live in fictional narratives as if they were real. Recent clinical studies suggest that addressing beliefs about symptoms modifying inferential styles and normalizing experiences may help symptom management.

Social presence in distributed group environments: The role of social identity

This paper argues that to achieve social presence in a distributed environment, it is not necessary to emulate face-to-face conditions of increased cues to the interpersonal. Rather, it is argued, that a sense of belongingness to the group, or perceptual immersion in the group, can be realised through the creation of a shared social identity between group members. From this perspective, social presence is a function of the cognitive representation of the group by group members and not the interpersonal bonds between group members. Furthermore, specific design features and characteristics of the distributed learning environment can be utilised to achieve and maintain this shared group identity. This approach, encapsulated by the SIDE model, is discussed and supported by two case studies of distributed students, each consisting of 10 groups, collaborating for a period of 5 weeks on group projects.