Immersive View Research Articles

Immersive reality for robotic surgical training: a pilot study using 3D visors for immersive view of the operating field.

Immersive intracorporeal vision is a key feature of robotic surgery, limited today to only one trainee per operation when the dual console is available. We developed a tool that provides a virtually unlimited number of surgeons with the operator's view, with the possibility to also watch the surgeon's hand movements and the operating table. In this study, we aim to assess trainees' reaction to this innovative training method. Medical students and surgery residents were offered an immersive experience with head-mounted devices, showing a didactic video in a 360° virtual space with 3D intracorporeal robotic vision, the surgeon's hand movements and the surrounding operating room during a robotic rectal resection with total mesorectal excision. Subsequently, participants were asked to fill a questionnaire evaluating the user's reaction to the new training tool including the validated System Usability Scale (SUS) and Simulator Sickness Questionnaire (SSQ), and non-validated questions. 102 participants took part in the training and the assessment questionnaires, 94 (92%) medical students and 8 (8%) surgery residents. Users' feedback was overall positive. In the engagement and intention to use items, almost 90% of the respondents voted for a complete or near complete agreement. The median SUS score was 80 [IQR 70-90]. The median SSQ score was 44.88 [IQR 22.44-82.28]. Exposing trainees to immersive robotic vision of the surgical field had a positive reaction from our audience. Our initial results encourage further implementing this technology in surgical training of medical students and residents to prove its efficacy.

Reclaiming Energy Flows: Energy GeoHumanities and the Socio-Ecologies of Rivers in Latin American Hydro-Modernities

This article explores fluvial forms of art and activism to reclaim new meanings of energy at a time when questions about the energy transition are central in geography studies and global environmental discourses. Departing from Serpent River Book by Carolina Caycedo, we propose affective, embodied, and creative entanglements to understand energy flows in river ecologies in Latin America for an exploration of energy geohumanities. We then discuss the methodology of this energy geohumanities project in our engagements with Caycedo’s art, reflecting on our sedimentary writing and thinking process enacted throughout the article. By centering this article on what energies art conjures and releases in affective and political terms, we explore the meanings of doing energy geohumanities in the deployment of a poetics of embodied tensions between flow and containment and between aerial and immersive views that challenge the dichotomies of subject/object, nature/culture and geo/human. Through unfolding a “sedimentary blog writing” method based on geo-mimicry, we hope to make a case for how entangled thinking and writing offer a novel approach to examining the multiple meanings of energy beyond electricity in the extractive zones of hydro-modernity.

AIDEN, a visualization tool in virtual reality for RDF graph

This paper presents AIDEN (Artificial Intelligence for Data Exploration and Navigation), a system based on virtual reality (VR) to provide an immersive and intuitive application for exploring, analyzing, and modifying RDF knowledge graphs in an interactive three-dimensional environment. The system aims to enhance knowledge capitalization and decision-support processes by helping user to navigate through complex RDF knowledge graphs in an intuitive manner. By facilitating the physical navigation of virtual space with its multi-modal interactions such as gestures, movements, and voice commands; AIDEN allows technical data readily accessible to non-experts. This immersive approach takes advantage of humans’ innate ability to interact with three-dimensional environments, thereby enhancing the user experience by reducing cognitive load while exploiting the potential of a third dimension for data visualization purposes. Two distinct knowledge graph representations have been designed to optimize navigation and user experience. The first type, the "overall view," is designed for pattern recognition and group identification. The second, called "immersive view," submerges the user into the graph for a more local perspective, enhancing the readability of individual node relationships. This paper discusses the design and implementation of AIDEN and explores its potential to improve the way to interact with complex data structures.

Photorealistic 3-Dimensional Models of the Anatomy and Neurosurgical Approaches to the V2, V3, and V4 Segments of the Vertebral Artery.

The vertebral artery (VA) has a tortuous course subdivided into 4 segments (V1-V4). For neurosurgeons, a thorough knowledge of the 3-dimensional (3D) anatomy at different segments is a prerequisite for safe surgery. New technologies allowing creation of photorealistic 3D models may enhance the anatomic understanding of this complex region. To create photorealistic 3D models illustrating the anatomy and surgical steps needed for safe neurosurgical exposure of the VA. We dissected 2 latex injected cadaver heads. Anatomic layered dissections were performed on the first specimen. On the second specimen, the two classical approaches to the VA (far lateral and anterolateral) were realized. Every step of dissection was scanned using photogrammetry technology that allowed processing of 3D data from 2-dimensional photographs by a simplified algorithm mainly based on a dedicated mobile phone application and open-source 3D modeling software. For selected microscopic 3D anatomy, we used an operating microscope to generate 3D models. Classic anatomic (n=17) and microsurgical (n=12) 3D photorealistic models based on cadaver dissections were created. The models allow observation of the spatial relations of each anatomic structure of interest and have an immersive view of the approaches to the V2-V4 segments of the VA. Once generated, these models may easily be shared on any digital device or web-based platforms for 3D visualization. Photorealistic 3D scanning technology is a promising tool to present complex anatomy in a more comprehensive way. These 3D models can be used for education, training, and potentially preoperative planning.

THE WHOLE IS OTHER THAN THE SUM OF ITS PARTS: SENSIBILITY ANALYSIS OF 360° URBAN IMAGE SPLITTING

Abstract. 360° imagery has been increasingly used to estimate the subjective qualities of the urban space, such as the feeling of safety or the liveliness of a place. These spherical panoramas offer an immersive view of the urban scene, close to the experience of a pedestrian. In recent years, Deep Learning approaches have been developed for this estimation task, only using flat images because these images are easier to annotate and process with standard CNNs. Thus to qualify the whole urban space, the panoramic images are divided into four flat sub-images that can be processed by the trained neural networks. The sub-images cover the 360° field of view, e.g. front, back, left, and right views. The four scores obtained are averaged to represent the level of the quality at the location of the panorama. However, this split introduces a bias since some elements of the urban space are halved over two images and the global context is lost. Based on the Place Pulse 2.0 dataset, this paper investigates the impact of splitting 360° panoramas on the perceptual scores predicted by neural networks. For each panorama, we predict the score for thirty-six overlapping sub-images. The scores were shown to have high variability and be highly dependent on the direction of the camera for the perspective images. This indicates that four images are not sufficient to capture the complexity of the perceptual qualities of the urban space.

Relaxation and Fascination through Outside Views of Mexican Dwellings

Exposure to outside views creates opportunities to distract and experience feelings of relaxation. To explore the relationship between the environmental qualities of the views with such psychological states, 89 participants from seven Mexican states evaluated the views they contemplated during the confinement due to the COVID-19 pandemic. Items on fascination, cognitive well-being, and how relaxing and helpful the views were to withstand the confinement were answered. Participants took photographs of the views, which were evaluated according to 41 environmental dimensions, considering the built elements, vegetation, and visibility. Based on these dimensions, a classification of the views into categories was realized with multidimensional scaling. The five categories obtained were (a) immersive views of extensive landscapes with vegetation, (b) non-immersive views of landscapes with vegetation, (c) views of courtyards with vegetation, (d) views of commonplace scenes, and (e) views of mostly built elements. The categories generating the highest and lowest relaxation, fascination, and cognitive well-being were identified. The views of extensive landscapes with vegetation and the views of courtyards were the categories presenting the most favorable psychological effects. Furthermore, a partial correlation network found direct relations between the environmental and psychological dimensions. Fascination relates to the observation of distant elements, mountains, and trees. Meanwhile, relaxation correlates with the presence of plants and anticorrelates with car visibility, the quantity of the windows of the visible buildings, and the variety of built elements. Relaxation was the psychological state with the highest direct relation with the environmental dimensions. Meanwhile, the perceived immersion (the feeling of being outdoors), the quantity of plants, and the attractiveness of the built elements were the environmental aspects most directly related to the psychological dimensions. The multiplicity of environmental and architectural qualities considered allowed specific implications for architecture to be obtained. An integrated configuration of the natural and the built elements, and a limited quantity and variation of the built elements were qualities that generated positive outcomes in the observers of the views.

Space On and Off Screen: The Détournement of Documentary into Video Installation

This essay proposes the détournement of the documentary film Displacement (Nogueira 2021) into a video installation, as a process to subvert the sequential documentary account of reality, as well as to interrogate space and movement on and off screen. Instead of editing and presenting the images and sounds in a continuous f low, the setup of the installation fragments the narrative and replaces the sequential format by a projected, sculptural, four-channel experience, composed of a prologue and three acts: (1) the family’s daily activities; (2) disruption of the quotidian routines; (3) the family’s displacement. While the installation projects the prologue onto clear white translucent plastic (like the ones used to cover construction sites) at the entrance to the darkroom, the three acts are projected on the three walls surrounding the audience in the darkroom. The result is an intersection of images and sounds: a juxtaposition and intertextuality of the content to offer an immersive view into family life. It also raises the question of embodiment in video installations, especially the notions of spectatorship, authorship, reality, performance, and, most importantly, what the boundaries of the screen are in the expanded practice of documentary film.

Multi-View Merging for Robot Teleoperation With Virtual Reality

In robotic teleoperation, an operator usually needs to be trained for long hours. One of the factors leading to this steep learning curve is the quality of telepresence. This letter proposes a novel and user-friendly telepresence interface for manipulation. Visual information from different views is merged and presented to the operator in an intuitive way to facilitate the task based on state-of-the-art virtual reality technology. Besides rendering the scene, a virtual robot is also rendered in the immersive view so that the robot is visible even when it is occluded. We performed a series of user studies to evaluate this interface. The results show that the proposed interface can achieve a better task performance compared to a standard approach in both the manipulation efficiency and the users' preferences.

Excision of a Retrochiasmatic Craniopharyngioma by Transcallosal, Interforniceal Approach With Exoscope Assistance: 2-Dimensional Operative Video.

Retrochiasmatic craniopharyngiomas are difficult to treat due to their close proximity to critical neurovascular structures. Several surgical approaches with distinct advantages and limitations have been described to access these tumors, including extended transnasal endoscopic approach (ETEA), subtemporal, translamina terminalis, and transpetrosal approach.1-3 We present a 51-yr-old male with a large retrochiasmatic craniopharyngioma extending into the third ventricle, causing obstructive hydrocephalus. Preoperative magnetic resonance imaging (MRI) showed a tumor cyst abutting the fornices expanding the space between two internal cerebral veins (ICV). After surgical consent, we decided to take advantage of this corridor to approach the tumor in its long axis. Surgical goal was to achieve cyst decompression with "safe maximal" resection of the solid component at last to preserve the pituitary function. Though the long axis of the tumor could be approached using ETEA, we preferred this approach in view of cyst decompression early in the surgery while completely avoiding risks such as cerebrospinal fluid (CSF) rhinorrhea, internal carotid artery (ICA) injury, and sinonasal complications. We utilized a 3-dimensional 4 K exoscope, which provides an excellent ergonomic position, and a high-resolution immersive view compared to a microscope or endoscope. Cyst decompression and near-total resection of the solid component was achieved. Postoperatively, his headaches improved and he was neurologically intact with intact neuroendocrine function. Approach-related risks may include but not limited to hemorrhage due to the rupture of venous sinuses or ICV, stalk or hypothalamus injury, and memory disturbances due to forniceal injury. To conclude, the transcallosal, interforniceal approach to retrochiasmatic craniopharyngiomas may provide a safe surgical corridor in select cases. Patient consented to the proposed procedure. All radiological images have been anonymized. IRB/ethics committee approval was not required.

Improving Access to Safe Anesthetic Care in Rural and Remote Communities in Affluent Countries.

Inadequate access to anesthesia and surgical services is often considered to be a problem of low- and middle-income countries. However, affluent nations, including Canada, Australia, and the United States, also face shortages of anesthesia and surgical care in rural and remote communities. Inadequate services often disproportionately affect indigenous populations. A lack of anesthesia care providers has been identified as a major contributing factor to the shortfall of surgical and obstetrical care in rural and remote areas of these countries. This report summarizes the challenges facing the provision of anesthesia services in rural and remote regions. The current landscape of anesthesia providers and their training is described. We also explore innovative strategies and emerging technologies that could better support physician-led anesthesia care teams working in rural and remote areas. Ultimately, we believe that it is the responsibility of specialist anesthesiologists and academic health sciences centers to facilitate access to high-quality care through partnership with other stakeholders. Professional medical organizations also play an important role in ensuring the quality of care and continuing professional development. Enhanced collaboration between academic anesthesiologists and other stakeholders is required to meet the challenge issued by the World Health Organization to ensure access to essential anesthesia and surgical services for all.

Pixels and Panoramas: An Enhanced Cubic Mapping Scheme for Video\/Image-Based Virtual-Reality Scenes

Virtual reality (VR) is rapidly appearing in various fields, such as navigation, robotics, and documentation. Spherical panoramic video, compared to 3D modeling, provides immersive and omnidirectional views in a much more convenient way. However, state-of-the-art video or image-encoding techniques, such as high-efficiency video coding or JPEG, require rectangular input sequences. Spherical videos are traditionally projected onto a plane or a cube for convenient encoding, but mapping quality and encoding efficiency are not considered. In this article, we propose GVScube projection, a method using a cube-Snyder (Scube) projection along with a gradually varied (GV) sampling method to generate panoramic video. This method achieves better pixel uniformity and less area deviation than other methods.

Earthquake Damage Data Collection Using Omnidirectional Imagery

The unique perspectives and viewpoints offered by omnidirectional camera technology has the potential to help improve the outcomes of technical post-earthquake reconnaissance missions. Omnidirectional imagery can be used to virtually ‘walk through’ damaged streets post hoc with a 360-degree, immersive view. A common reconnaissance mission aim is to accurately collect damage data; however, there are time challenges for surveyors in the field. The manuscript explores the potential for using omnidirectional imagery to improve damage surveying, firstly by comparing results from damage surveys completed in the field with results obtained using omnidirectional images collected during a mission and surveyed by an experienced engineer virtually and secondly by comparing damage assessment obtained through omnidirectional imagery collected on the ground with the EU Copernicus damage assessment maps. The omnidirectional imagery data was collected during two separate Earthquake Engineering Field Investigation Team post-earthquake reconnaissance missions, namely the area affected by the 2016, 7.8 Muisne Earthquake in Ecuador and the area struck by the 2016, 6.2 Amatrice earthquake in Italy. Notwithstanding the diverse geographic scale, terrain and urban context of the two reconnaissance missions, the results consistently show significant capabilities for this technology in the identification of construction typologies, number of storeys, aggregated ‘low’ and ‘high’ damage grades, and failure modes. The work highlights potential issues with correct identification of disaggregated lower damage grades (e.g. European Macroseismic Scale (EMS-98) damage grades 0 to 3). Challenges identified in the virtual survey process included poor image quality, insufficient photo sphere captures, and obstructions such as trees, walls or vehicles. The omnidirectional imagery represents a substantial improvement in damage assessment accuracy in respect to satellite imagery, especially for lower damage grades, while it is an essential tool for comprehensive surveys in reduced access zones with high levels of damage.

A Child's-Eye View: An Examination of Point-of-View Camera Use in Four Informal Education Settings

ABSTRACTA point-of-view (POV) camera is a video recording device attached to the person of a research participant. The cameras collect observational data from the individual wearing them and those in close proximity. The cameras can be used to collect data that provide researchers with an immersive view of visitor experience by documenting visits from the perspective of the individual. The purpose of this article is to outline the benefits and weaknesses of using POV cameras for answering broad visitor studies research questions, compare their use to other common methods for visitor studies research, and examine their use within four informal educational settings—a zoo, a museum exhibit, a bus tour, and an afterschool program. In particular, the POV cameras are useful for capturing data beyond a single exhibit, recording the dialogue and gestures of individual visitors, and documenting interactions among group members. POV cameras may be especially useful for answering questions about hands-on activities such as building or tinkering. Limitations that should be considered when attempting to use POV cameras include participants tampering with the recording, the inability to see facial expressions, and the limited angle of captured video. Care should also be taken to minimize the discomfort of visitors in studies utilizing POV cameras.

Rotation Removed Stabilization of Omnidirectional Videos Using Optical Flow

Omnidirectional cameras are often used with robots to provide an immersive view of the surroundings. However, robots often have unstable motion and undergo rotation. In this work, we formulate a method to stabilize the viewpoint of Omnidirectional videos by removing the rotation using dense optical flow fields. The method works by first projecting each Omnidirectional video frame on a unit sphere, measuring the optical flow at every point on the sphere, and finding the direction that minimizes its rotational component in a frame by frame manner. The Omnidirectional video is de-rotated and a ‘rotation-less, translation-only’ viewpoint is generated. Such a technique is well suited to work in any environment, even with sparse texture or repeating patterns where feature correspondence based methods may fail.

An image-space algorithm for immersive views in 3-manifolds and orbifolds

We describe the first image-space ray-tracing algorithm to generate interior views of flat or hyperbolic 3-manifolds, orbifolds, and similar polyhedral complexes. The complexity of our algorithm is linear in the number of echoes for a given number of polygons, compared to exponential complexity for the object-space algorithm chosen as standard.

An immersive view approach by secure interactive multimedia proof-of-concept implementation

Live media streaming is a field that recently has had a great impact in the scientific community, especially in the case of interactive media streaming. In this paper we propose a reference architecture conceptualizing an immersive view effect by considering heterogeneous tracking devices and enriched movement control over heterogeneous stream image sources. A proof-of-concept prototype implementation of the reference architecture is presented, called Live Interactive FramE (LIFE), illustrating the potential and value of the immersive view concept. Our work is part of the DESEOS research project that aims at applying an Ambient Assisted Living paradigm to a targeted scenario of hospitalised children. The main goal of LIFE is reducing stress and isolation during hospitalisation by enabling an immersive view of school activities via live media streaming. Functional and security aspects of LIFE are presented along with details of implementation and performance evaluation. Conclusions of experiments show that LIFE enables practical secure media streaming solution with optimal video quality settings.

Modulations of Amplitude in Mobile Acoustical Heritage

Nowadays, cultural heritage has been preserved digitally more than physically. However, most of the research works have been focusing more on preservation of video elements even though preservation of audio is equally important. In this paper, the possibilities of generating different sounds in an acoustical heritage through the modulations of sound characteristics are explored on mobile devices. The modulations of the amplitude are based on tiles. In the proposed method, the user uses the one-tap gesture to initiate the start of sound changes on the physical interface of an immersive panoramic view on mobile platform by altering the amplitude of recorded audio piece and consequently producing different audible results. User interactions happen along the x and y-axes following the standard iOS system coordinate. This paper also demonstrates the programming method based on tiles and direct modulations of acoustical heritage in panoramic view with the interaction of the users in order to provide enhanced sensorial audio experience and acoustical engagement in action space on mobile platform. The result of the evaluation has shown that the volume playback has different amplitudes from the different coordinates that the users have tapped on.

Shivers Down Your Spine: Cinema, Museums and the Immersive View

Among the books at the intersection of cultural studies, media studies and arts theory, some can also be usefully applied in the field of film studies. These works generally address the interrelation between the spectator gaze, the device regulating this gaze, and the various modes of representation resulting from this particular interaction. Foremost among these books, both European and American, published in the last thirty years are Wolfgang Schivelbusch's Geschichte der Eisenbahnreise, Max Milner's La Fantasmagorie and Jonathan Crary's Techniques of the Observer.1 Alison Griffiths's Shivers Down Your Spine can now be added to this list. In this review of Griffiths's book, commentary on the nature of cinema – a medium that mummifies and embalms time – within an exhaustive study of the history of museums and museology (specifically the museology of scientific exhibitions) may clarify such previously hazy concepts as immersion and interactivity. In her book, Griffiths recounts how in 1799 a spectator saw a panorama exhibition which he described as having sent ‘shivers’ down his spine. Thus, despite the current hype surrounding the concepts of ‘immersion’ and ‘interactivity’ informed by their presumed novelty, it would seem that immersive modes of spectatorship are far from new. Choosing the panorama spectator's experience as the title for her book, Griffiths argues that the concepts of immersion and interactivity are older than some might think, being associated, among other things, with museums, panoramas and mediaeval cathedrals, ‘antecedents for today's so-called immersive and interactive new media’ (p. 4). However, another important factor common to these environments is their size. The size of the universe which is created through a particular visual representation plays a key role in the emotional response of the spectator: in other words, size can inform the quality of our shivers. As Jean-Luc Godard said, ‘When we go to the movies, we raise our head. When we watch television, we lower it.’ When we are looking up at the sky, we may be more likely to be swept away by what we observe.

Shivers Down Your Spine: Cinema, Museums, and the Immersive View (review)

Reviewed by: Shivers Down Your Spine: Cinema, Museums, and the Immersive View Malgorzata Rymsza-Pawlowska (bio) Shivers Down Your Spine: Cinema, Museums, and the Immersive View. By Alison Griffiths. New York: Columbia University Press, 2008. Pp. xv+372. $45. IMAX, touch-screen computers, audio tours, and other interactive media are standard fare in most museums, so much so that their presence gets folded into larger interpretations and is undertheorized on its own merit. In this compelling study, Alison Griffiths works to remedy that problem by constructing the cultural history of immersive spectatorship, beginning from the premise that this mode of viewing, because of the way in which it locates its viewer inside the spectacle rather than assuming a physical distance, is different from other, less-involved types of spectatorship. The beautifully illustrated and aptly titled Shivers Down Your Spine (named for the sensation produced in the viewer) traces a trajectory of interactive and immersive spectatorship from the “revered gaze” of the cathedral through the nineteenth-century panorama and planetarium, and ends with the beginning of IMAX. The book then zeroes in on the modern museum as the locale of this mode of spectatorship and locates interactive displays as a major facet of the ongoing debate between education and entertainment in museums. Griffiths deftly makes connections among her diverse examples and contextualizes them within their larger cultural connotations. In her discussion of the panorama in Europe and the United States, for example, she notes that this medium evolved at the same time that the public became interested in more distant parts of the world on one hand, and the reconstruction of historical events on the other. Moving beyond a simple discussion of the composition of the panorama, she examines how it was built, how it was advertised, and how audiences were moved through its spaces. This bolsters the author’s point that interactive vision did not start with developments in the realm of new technology. Indeed, she argues that aspects of these older modes of spectatorship still inform immersive viewing today. If the first portion of the book sets the parameters and introduces the themes of interactive spectatorship, the second applies them to museum exhibition from both past and present, enabling Griffiths to cement her history and touch on an important aspect of interactivity: the tension between its educational and commercial values. She examines a wide variety of exhibits in various museums, large and small, tracing both the ideas underpinning their creation and the ways in which their interactive aspects have been used and understood by curators and audiences. Among the exhibits to which she turns her critical eye are African Voices and the Insect Zoo at the National Museum of Natural History, IMAX at the National Air and [End Page 951] Space Museum, and the National Waterfront Museum in Swansea, Wales. In her examination of the latter, a newly redesigned space that advertises itself as an “interactive experience,” Griffiths notes how the museum has coopted some aspects of modern branding and seeks to bill itself as much a space of entertainment as of education. A major strength of the book is its interdisciplinarity, standing as it does at the intersection of history, art history, new media studies, and museum studies. It is this theoretical flexibility that helps make Shivers Down Your Spine unique—capable of presenting multiple perspectives on technological innovations in interactive viewing, carefully tracing their development, and examining their ongoing cultural impact. Although this history is convincing, Griffiths makes the point that it is by no means complete. She acknowledges the porousness of her model and suggests plenty of examples for further study: waxworks, world’s fairs, My Space, the Discovery channel. This fascinating book both introduces a critical examination of some of the newest immersive technologies and suggests ideas for further study. In a world where iPhone tours, real-time image tagging, and individualized online exhibition are all quickly becoming a reality, investigations into diverse modes of spectatorship are badly needed. Shivers Down Your Spine is an engaging and important beginning. Malgorzata Rymsza-Pawlowska Malgorzata Rymsza-Pawlowska is a Ph.D. candidate in the Department of American Civilization at Brown University in Providence, Rhode Island. Copyright © 2009...

Shivers down your spine: cinema, museums, and the immersive view

Acknowledgments List of Illustrations Introduction Part I. From Cathedral to IMAX Screen: Case Studies in Immersive Spectatorship 1. Immersive Viewing and the Revered Gaze 2. Spectacle and Immersion in the Nineteenth-Century Panorama 3. Expanded Vision IMAX Style: Traveling as Far as the Eye Can See 4. A Moving Picture of the Heavens: Immersion in the Planetarium Space Show Part II. Museums and Screen Culture: Immersion and Interactivity Over Centuries 5. Back to the (Interactive) Future: The Legacy of the Nineteenth-Century Science Museum 6. From Daguerreotype to IMAX Screen: Multimedia and IMAX at the Smithsonian Institution 7. Film and Interactive Media in the Museum Gallery: From Roto-Radio to Immersive Video 8. Conclusion Notes Filmography Bibliography Index