Multisensory Technology Research Articles

Message on a bottle: the use of augmented reality as a form of disruptive rhetoric in wine marketing

ABSTRACT Wine, a product steeped in tradition and history, is often challenging for consumers to decipher due to its intricate characteristics. Appealing to non-wine connoisseurs to break into untapped market segments requires both innovation and creative thinking. To accomplish this, three wine brands utilized augmented reality (AR) technology to craft unique brand storytelling experiences that centered around the label, without focusing on physical product attributes, ultimately resonating with new market segments. This article explores the use cases of three wine brands; 19 Crimes, Barefoot, and Enosophia Wines, that employed AR technology as a form of disruptive rhetoric, to supplement their wine label storytelling. The study sheds light on how multisensory technology such as AR can be used to carve out new wine target markets and proposes an AR experience model with accompanying propositions outlining how the experiential realms can be leveraged to achieve key brand objectives.

Virtual Reality During Chemotherapy Infusion: An Innovative Intervention in Holistic Nursing Practice.

Patients with cancer receiving infusional chemotherapy show negative symptoms such as worry about their survival, anxiety, anguish, depression, fear, magnified perception of the passage of time, and difficulty managing boredom. Patients also suffer various side effects produced by chemotherapy such as nausea, vomiting, pain, and fatigue, which, together with psychological distress, drastically reduce their quality of life and adherence to therapy with a corresponding reduction in the probability of the individual's survival. Virtual Reality is one of the most innovative and promising digital health interventions, capable of quickly and effectively producing a positive influence on the psychosomatic axis, improving patients' quality of life during chemotherapy. Virtual Reality, through its 3-dimensional multisensory technology, isolates sensory channels from the negative external environment and enables an experience of being physically and psychologically present within virtual scenarios, in which patients can perceive sensations, emotions, cognitions, and interactions as if they really were in different surroundings. This article systematically expounds the scientific conditions necessary for effective, appropriate, and safe implementation of Virtual Reality interventions in holistic nursing practice, describing the underpinning conceptual framework, the types, technological characteristics, methods of use, duration, type of virtual content, and implementation procedure of Virtual Reality.

Multisensory 360 Videos Under Varying Resolution Levels Enhance Presence.

Omnidirectional videos have become a leading multimedia format for Virtual Reality applications. While live 360 videos offer a unique immersive experience, streaming of omnidirectional content at high resolutions is not always feasible in bandwidth-limited networks. While in the case of flat videos, scaling to lower resolutions works well, 360 video quality is seriously degraded because of the viewing distances involved in head-mounted displays. Hence, in this paper, we investigate first how quality degradation impacts the sense of presence in immersive Virtual Reality applications. Then, we are pushing the boundaries of 360 technology through the enhancement with multisensory stimuli. 48 participants experimented both 360 scenarios (with and without multisensory content), while they were divided randomly between four conditions characterised by different encoding qualities (HD, FullHD, 2.5K, 4K). The results showed that presence is not mediated by streaming at a higher bitrate. The trend we identified revealed however that presence is positively and significantly impacted by the enhancement with multisensory content. This shows that multisensory technology is crucial in creating more immersive experiences.

Biomedical gastronomy in the interventions of smell and taste disorders in ‘altered eating’

Biomedical gastronomy in the interventions of smell and taste disorders in ‘altered eating’

Efficacy of Multisensory Technology in Post-Stroke Cognitive Rehabilitation: A Systematic Review.

Post-stroke, in addition to sensorimotor signs and symptoms, could lead to cognitive deficits. Theories of embodiment stress the role of sensorimotor system and multisensory integration in sustaining high-order cognitive domains. Despite conventional post-stroke cognitive rehabilitation being effective, innovative technologies could overcome some limitations of standard interventions and exploit bodily information during cognitive rehabilitation. This systematic review aims to investigate whether 'multisensory technologies' compared to usual care treatment can be a viable alternative for cognitive rehabilitation. By applying PRISMA guidelines, we extracted data and assessed the bias of 10 studies that met the required criteria. We found that multisensory technologies were at least comparable to standard treatment but particularly effective for attention, spatial cognition, global cognition, and memory. Multisensory technologies consisted principally of virtual reality alone or combined with a motion tracking system. Multisensory technologies without motion tracking were more effective than standard procedures, whereas those with motion tracking showed balanced results for the two treatments. Limitations of the included studies regarded the population (e.g., no study on acute stroke), assessment (e.g., lack of multimodal/multisensory pre-post evaluation), and methodology (e.g., sample size, blinding bias). Recent advancements in technological development and metaverse open new opportunities to design embodied rehabilitative programs.

Multisensory Perception and Learning: Linking Pedagogy, Psychophysics, and Human-Computer Interaction.

In this review, we discuss how specific sensory channels can mediate the learning of properties of the environment. In recent years, schools have increasingly been using multisensory technology for teaching. However, it still needs to be sufficiently grounded in neuroscientific and pedagogical evidence. Researchers have recently renewed understanding around the role of communication between sensory modalities during development. In the current review, we outline four principles that will aid technological development based on theoretical models of multisensory development and embodiment to foster in-depth, perceptual, and conceptual learning of mathematics. We also discuss how a multidisciplinary approach offers a unique contribution to development of new practical solutions for learning in school. Scientists, engineers, and pedagogical experts offer their interdisciplinary points of view on this topic. At the end of the review, we present our results, showing that one can use multiple sensory inputs and sensorimotor associations in multisensory technology to improve the discrimination of angles, but also possibly for educational purposes. Finally, we present an application, the 'RobotAngle' developed for primary (i.e., elementary) school children, which uses sounds and body movements to learn about angles.

The translational neuroscience of sleep: A contextual framework.

Sleep is entwined across many physiologic processes in the brain and periphery, thereby exerting tremendous influence on our well-being. Yet sleep exists in a social-environmental context. Contextualizing sleep health with respect to its determinants—from individual- to societal-level factors—would enable neuroscientists to more effectively translate sleep health into clinical practice. Key challenges and opportunities pertain to (i) recognizing and exploring sleep’s functional roles, (ii) clarifying causal mechanisms in relation to key outcomes, (iii) developing richer model systems, (iv) linking models to known contextual factors, and (v) leveraging advances in multisensory technology. Meeting these challenges and opportunities would help transcend disciplinary boundaries such that social-environmental considerations related to sleep would become an ever-greater presence in the clinic.

Improving reading and comprehension in K-12: Evidence from a large-scale AI technology intervention in India

Abstract This paper presents evidence on the impact of technology-aided instruction on literacy using an AI-based multi-sensory technology platform across a large cross-section of government schools in India. The study focused on reading and comprehension in the English language. The intervention enhances the instructional effectiveness of the teachers and the learning ability of the children within the existing instructional environment without any new instructional design or pedagogy or content. Besides, the intervention is implemented by existing teachers and not outside volunteers. A total of 1 million children and 15,000 teachers across 5000 government schools in the states of Maharashtra, West Bengal, Punjab, Tamil Nadu, Telangana, Gujarat and Karnataka used the technology for the 2016-17 academic year. Using a randomized control-treatment assessment, the study finds a 20–40% overall gain in learning outcomes in the treatment sample. Gains within individual states and grades vary. Learning outcomes rose across the entire range of proficiency levels in a grade. Ongoing self-administered assessments report even higher impact in the 50–60% range. In addition, teachers also reported improving their skills as a result of using the technology, suggesting that the intervention can also alleviate teacher shortage and inadequate teacher training. The paper also reports briefly on the growing adoption of the program in several countries outside India. The results hold significant promise for disrupting the low and stagnating literacy levels across government schools in India and other similar environments.

Mechanism to capture learner\u2019s interaction in VR-based learning environment: design and application

Virtual Reality (VR) is a multi-sensory technology that stimulates learning and has the potential for pedagogical applications. While researchers in VR have demonstrated several applications to support understanding and learning in STEM education, the research regarding which features of VR leverage learning is in its infancy. The existing studies exploring how learners interact with VR are based on human observations or learners’ perceptions. This paper describes a novel mechanism to capture learner’s interaction behavior, in the context of a mobile-based static VR to learn the human circulatory system. The data capturing mechanism is based on screen recordings of VR interaction, which is further annotated manually to form a time-sequenced action series. In a preliminary test conducted with three learners, the interaction data was analyzed based on the time spent in each action in the VR environment, frequently co-occurring actions, and sequence of actions. The test results are described and the implications of using such a mechanism to capture learners’ interaction behavior is discussed. We conclude that capturing data in this manner gives a rich and detailed profile of learners and enables use of various analytics methods to provide personalized and adaptive support to learners.

Multisensory Technology for Flavor Augmentation: A Mini Review.

There is growing interest in the development of new technologies that capitalize on our emerging understanding of the multisensory influences on flavor perception in order to enhance human–food interaction design. This review focuses on the role of (extrinsic) visual, auditory, and haptic/tactile elements in modulating flavor perception and more generally, our food and drink experiences. We review some of the most exciting examples of recent multisensory technologies for augmenting such experiences. Here, we discuss applications for these technologies, for example, in the field of food experience design, in the support of healthy eating, and in the rapidly growing world of sensory marketing. However, as the review makes clear, while there are many opportunities for novel human–food interaction design, there are also a number of challenges that will need to be tackled before new technologies can be meaningfully integrated into our everyday food and drink experiences.

A Multimodal Approach for the Safeguarding and Transmission of Intangible Cultural Heritage: The Case of i-Treasures

Intangible Cultural Heritage (ICH) creations include, amongst other, music, dance, singing, theatre, human skills, and craftsmanship. These cultural expressions are usually transmitted orally and/or using gestures and are modified over a period of time, through a process of collective recreation. As the world becomes more interconnected and many different cultures come into contact, local communities run the risk of losing important elements of their ICH, while young people find it difficult to maintain the connection with the cultural heritage treasured by their elders. In this paper, we present a novel holistic approach for the safeguarding and transmission of ICH that goes beyond the mere digitization of ICH content. Based on multisensory technology for the capturing of ICH, the proposed approach enables the generation of completely novel cultural content. High-level semantics are extracted from the acquired data, enabling researchers to identify possible implicit or hidden correlations between different ICH expressions or interpretation styles and study the evolution of a specific ICH. These data, coupled with other cultural resources, are accessible through the i-Treasures Web-platform, which provides the means for supporting knowledge exchange between researchers as well as know-how transmission from ICH bearers to apprentices.

Creating e-shopping multisensory flow experience through augmented-reality interactive technology

PurposeDrawing on virtual liminoid theory, the purpose of this paper is to examine factors that induce a multisensory flow experience in an e-shopping context through the use of augmented-reality interactive technology (ARIT).Design/methodology/approachTo validate the research framework, a task-based laboratory study was performed. Participants were recruited through a snowball e-mail method and requested to freely and independently use ARIT for clothes fitting in the laboratory, after which they completed a questionnaire; 336 valid responses were received.FindingsEmpirical results revealed that three decorating psychological states (sense of body ownership, sense of ownership control, and self-explorative engagement) directly induced a multisensory flow experience. Furthermore, two multisensory factors (sense of self-location and haptic imagery) mediated the multisensory flow experience through these three decorating psychological states.Practical implicationsConsumers not only rely on generating self-display for optimal fitting in virtual avatar decoration in an e-shopping context but also concentrate more on expression and control of self-body. As consumers’ decorating psychological states require an ideal form of self-expression, a high degree of autonomy in exploring self-decoration options will create more value for consumers. Considering that expressing and controlling the self-body in addition to self-explorative engagement in virtual avatar decoration will trigger the flow experience in an e-shopping context, increasing the use of multisensory ARIT to trigger decorating psychological states in e-shopping contexts is highly recommended.Originality/valueIn this study, a relationship was constructed among virtual liminoid theory, flow theory, and multisensory technology, and an integrated conceptual framework was developed for the relationship between decorating psychological states and multisensory flow experience.

A review of multi-sensory technologies in a Science, Technology, Engineering, Arts and Mathematics (STEAM) classroom

<p class="JLDAbstract">This article reviews the literature on multi-sensory technology and, in particular, looks at answering the question: ‘What multi-sensory technologies are available to use in a science, technology, engineering, arts and mathematics (STEAM) classroom, and do they affect student engagement and learning outcomes?’ Here engagement is defined as motivation, interest, curiosity and attitude. This review identifies tools and software from a burgeoning, yet limited literature. To answer this question, this review will include the following: defining learning styles; background on multi-sensory instruction; multi-sensory education today; STEM and the rise of STEAM; a look at some multi-sensory tools; and the benefits of using multi-sensory technologies in education. </p>

가상 해저 시뮬레이션의 현실감 향상을 위한 다감각 효과 재현 시스템 개발 및 평가

With recent development of virtual reality technology, coupled with the growth of the marine industry, virtual underwater simulation systems are under development in various studies, for educational purposes and to simulate virtual reality experiences. Current literature indicates many underwater simulation systems to date have focused on the quality of visual stimulus delivered through three-dimensional graphics user interface, limiting the reality of the experience. In order to improve the quality of the reality delivered by such virtual simulations, it is crucial to develop multi-sensory technology rather than focus on the conventional audio-visual interaction, which limits experiencer from the sense of underwater immersion and existence within the simulation. This work proposes the immersive multi-sensory effector system, delivering the users with a more realistic underwater experience. The sense of reality perceived was evaluated, as the main factor of the virtual reality system.

A Novel Conceptual Fire Hazard Ranking Distribution System based on Multisensory Technology

Multisensory fire detection is widely accepted as the next generation of fire detection technology. A novel conceptual fire hazard ranking distribution system is proposed based on multisensory technology with the aim to assist building managers in fire emergency management, and fire brigades in fire rescue or fire fighting. The system is consisted of a fire node network representing fire hazard ranking of the divided control units in a building plan. Concept of fire information cloud is proposed inside the system to conduct most of the calculation and storage related to fire information. The system has the potential to provide space dimension in addition to traditional time and sensor type dimensions of a fire detection system. Fire location, fire situation assessment, and fire development intensity of a building in fire can be provided by the system for emergency management reference. Some long-term significances of the proposed conceptual system are also presented.

Using Multisensory Technology to Create a Therapeutic Environment for People with Dementia in an Adult Day Center: A Pilot Study

This quantitative pilot study examined the feasibility of using a multisensory environment (MSE) as a nonpharmacological intervention for people with behavioral and psychological symptoms of dementia (BPSD) within an adult day care center. Outcome measures were participant agitation, participant reaction to MSE, and caregiver satisfaction with MSE as a management strategy for older adults with BPSD. A within-subjects, repeated measures research design was used to obtain results from a total of 8 older adults with BPSD and 4 formal caregivers. Results indicated that the majority of participants enjoyed the MSE and improvements in some BPSD were observed after using the MSE. Reports from formal caregivers indicated that they believed the MSE was a useful nonpharmacological therapy for reducing anxiety and agitation among the day care participants who exhibited BPSD. Caregivers also suggested that this intervention should be incorporated into the design of the building and that a dedicated staff be assigned to facilitate the intervention for older adults each day.

The use of virtual reality for pain control: A review

Virtual reality (VR) is a relatively new technology that enables individuals to immerse themselves in a virtual world. This multisensory technology has been used in a variety of fields, and most recently has been applied clinically as a method of distraction for pain management during medical procedures. Investigators have posited that VR creates a nonpharmacologic form of analgesia by changing the activity of the body's intricate pain modulation system. However, the efficacy of VR has not been proven and the exact mechanisms behind VR's action remain unknown. This article presents a comprehensive review of the literature to date exploring the clinical and experimental applications of VR for pain control. The review details specific research methodologies and popular virtual environments. Limitations of the research, recommendations for improvement of future studies, and clinical experiences with VR are also discussed.

Utilty of a Tactile Display for Cueing Faults

Tactile displays have been proposed as a multisensory interface technology that can relieve the typically overburdened visual channel of operators. This study compared the ability of operators, while simultaneously completing a tracking task, to detect and identify system faults in a monitoring task with three types of alert cues: tactile, visual, and redundant tactile and visual. For the tactile display, the location and vibration pulse speed of two tactors were mapped to four system faults. Response time was significantly faster with the tactile cue. Also, the tactile cue resulted in less interference with the concurrent tracking task, while not degrading vigilance to an additional concurrent visual monitoring task. These results suggest that further tactile cue research is warranted to examine potential applications in complex systems, such as control stations for unmanned aerial vehicles.