Virtual Stimuli Research Articles

Rehabilitation exoskeleton system with bidirectional virtual reality feedback training strategy

AbstractVirtual reality (VR) technology revitalises rehabilitation training by creating rich, interactive virtual rehabilitation scenes and tasks that deeply engage patients. Robotics with immersive VR environments have the potential to significantly enhance the sense of immersion for patients during training. This paper proposes a rehabilitation robot system. The system integrates a VR environment, the exoskeleton entity, and research on rehabilitation assessment metrics derived from surface electromyographic signal (sEMG). Employing more realistic and engaging virtual stimuli, this method guides patients to actively participate, thereby enhancing the effectiveness of neural connection reconstruction—an essential aspect of rehabilitation. Furthermore, this study introduces a muscle activation model that merges linear and non‐linear states of muscle, avoiding the impact of non‐linear shape factors on model accuracy present in traditional models. A muscle strength assessment model based on optimised generalised regression (WOA‐GRNN) is also proposed, with a root mean square error of 0.017,347 and a mean absolute percentage error of 1.2461%, serving as critical assessment indicators for the effectiveness of rehabilitation. Finally, the system is preliminarily applied in human movement experiments, validating the practicality and potential effectiveness of VR‐centred rehabilitation strategies in medical recovery.

Consumer behavior in immersive virtual reality retail environments: A systematic literature review using the stimuli‐organisms‐responses (S‐O‐r) model

AbstractWith the rising popularity of immersive virtual reality (iVR) technologies, retailers are increasingly seeking innovative ways to create unique digital shopping experiences for their consumers. However, existing literature lacks a unified and comprehensive review that examines the interplay between virtual stimuli and consumer behavior in iVR shopping environments. To fill this gap, we conducted a systematic literature review, employing the Stimulus‐Organisms‐Responses (S‐O‐R) model as the underlying theoretical framework. This review analyzed empirical research on consumer behavior in iVR retail environments by focusing on experimental studies. Following the thematic analysis, we categorized the outcomes into descriptive themes to better comprehend consumer behavior within each theme. Our findings provide valuable insights for retailers and marketers aiming to enhance the consumer shopping experience using iVR technologies and suggest directions for future research.

Lightness constancy in reality, in virtual reality, and on flat-panel displays.

Virtual reality (VR) displays are being used in an increasingly wide range of applications. However, previous work shows that viewers often perceive scene properties very differently in real and virtual environments and so realistic perception of virtual stimuli should always be a carefully tested conclusion, not an assumption. One important property for realistic scene perception is surface color. To evaluate how well virtual platforms support realistic perception of achromatic surface color, we assessed lightness constancy in a physical apparatus with real lights and surfaces, in a commercial VR headset, and on a traditional flat-panel display. We found that lightness constancy was good in all three environments, though significantly better in the real environment than on the flat-panel display. We also found that variability across observers was significantly greater in VR and on the flat-panel display than in the physical environment. We conclude that these discrepancies should be taken into account in applications where realistic perception is critical but also that in many cases VR can be used as a flexible alternative to flat-panel displays and a reasonable proxy for real environments.

The Suite for the Assessment of Low-Level cues on Orientation (SALLO): The psychophysics of spatial orientation in virtual reality

Spatial orientation is a complex ability that emerges from the interaction of several systems in a way that is still unclear. One of the reasons limiting the research on the topic is the lack of methodologies aimed at studying multimodal psychophysics in an ecological manner and with affordable settings. Virtual reality can provide a workaround to this impasse by using virtual stimuli rather than real ones. However, the available virtual reality development platforms are not meant for psychophysical testing; therefore, using them as such can be very difficult for newcomers, especially the ones new to coding. For this reason, we developed SALLO, the Suite for the Assessment of Low-Level cues on Orientation, which is a suite of utilities that simplifies assessing the psychophysics of multimodal spatial orientation in virtual reality. The tools in it cover all the fundamental steps to design a psychophysical experiment. Plus, dedicated tracks guide the users in extending the suite components to simplify developing new experiments. An experimental use-case used SALLO and virtual reality to show that the head posture affects both the egocentric and the allocentric mental representations of spatial orientation. Such a use-case demonstrated how SALLO and virtual reality can be used to accelerate hypothesis testing concerning the psychophysics of spatial orientation and, more broadly, how the community of researchers in the field may benefit from such a tool to carry out their investigations.

Towards a yardstick of spatial impression: Psychophysical experiments using virtual piano stimuli

A rich “spatial impression” is an important factor in the product value of musical instruments. However, it also represents a development challenge since there is no established method to measure and control it. We introduce a novel yardstick for spatial impression, changing tacit knowledge into a measurable property. We focused on the difference in spatial impression between digital pianos and grand pianos, and conducted psychophysical experiments using piano sounds to identify the features that contribute to grand pianos’ spatial impression. We used multiple virtual stimuli which are created by varying physical parameters and conducted the experiment in a virtual environment including head-tracking from a VR headset. This allowed us to find a trend in the spatial impression and identify a physical parameter that correlates to the perceptual results of the experiment. This parameter, “phase entropy,” denotes the time complexity of the interaural phase difference. The correlation coefficient between the psychophysical experiment results based on virtual stimuli and the phase entropy was 0.8752. In addition to the virtual stimuli, we confirmed the correlation between perceptual score and phase entropy in an experiment using actual piano recordings; the correlation coefficient in this case was 0.6500.

Generating 3-D auditory stimuli for the evaluation of spatial impression in musical instruments

Spatial hearing plays an important, but not well understood, role in the perception of musical instruments and audio equipment. Psychophysical experiments are necessary to elucidate the processes which convey the spatial impression of sound sources. To this end, we propose a way to generate virtual stimuli for research on the spatial impression of musical instruments, specifically pianos. Our proposal, based on acoustical holography, takes a series of measurements over a dense grid and designs a virtual vibrating plate with the same directivity as the piano, as observed from the measuring grid. The number of active modes in the left-right and front-back directions of this virtual source can be controlled to reduce the radiation pattern complexity and generate a wide range of virtual stimuli. The resulting sound fields are rendered using binaural Ambisonics, using the position of the piano player as the spherical harmonic expansion center. The proposal was used in actual psychophysical experiments looking to shed light into the spatial impression of digital and acoustic pianos. The results show that the proposal does generate spatial sound signals which are consistently associated with different degrees of richness in the spatial impression.

Examining the Impact of Virtual Animal Stimuli on College Students' Affect and Perception of their Academic Advising Experience.

The benefits of animal-assisted interventions (AAI) involving animals in therapy are widely accepted. The presence of animals in therapy can decrease a patient's reservation about therapy and promote a sense of comfort and rapport during the therapy process. Using survey data from college students (n = 152) attending a large public four-year institution, this study is the first to investigate the benefits of virtual animal stimuli during academic advising appointments. It posits that exposure to virtual animal stimuli can influence positive mental health and well-being in academic advising settings. Specifically, the research questions explored how different types of video content influence students' affect and how virtual animal stimuli impact students' perception of their advisor and university. College students were randomly assigned to watch one of four types of virtual stimuli (wild animals, companion animals, nature, and a control) prior to their advising session. Subjective measures were collected at baseline and after the advising session. Results indicated animal stimuli increase positive affect, and companion animal stimuli influence the student's perception of the advisor. This study supports the notion that companion animal videos positively impact students' well-being and interactions with their advisors and may have broader implications beyond the academic setting.

Efficient visual learning by bumble bees in virtual-reality conditions: Size does not matter.

Recent developments allowed establishing virtual-reality (VR) setups to study multiple aspects of visual learning in honey bees under controlled experimental conditions. Here, we adopted a VR environment to investigate the visual learning in the buff-tailed bumble bee Bombus terrestris. Based on responses to appetitive and aversive reinforcements used for conditioning, we show that bumble bees had the proper appetitive motivation to engage in the VR experiments and that they learned efficiently elemental color discriminations. In doing so, they reduced the latency to make a choice, increased the proportion of direct paths toward the virtual stimuli and walked faster toward them. Performance in a short-term retention test showed that bumble bees chose and fixated longer on the correct stimulus in the absence of reinforcement. Body size and weight, although variable across individuals, did not affect cognitive performances and had a mild impact on motor performances. Overall, we show that bumble bees are suitable experimental subjects for experiments on visual learning under VR conditions, which opens important perspectives for invasive studies on the neural and molecular bases of such learning given the robustness of these insects and the accessibility of their brain.

New is not always better: Virtual reality does not necessarily enhance mnemonic processing.

Integrating new technologies such as Virtual Reality (VR) can contribute to increasing efficiency in several areas relevant to society. VR can be applied in various contexts and has the potential to improve mnemonic processes and memory performance. However, the specific conditions under which VR is more beneficial than conventional learning methods remain unclear. To further investigate the value of VR for mnemonic processing, participants performed a memory task under three different conditions. For that task, they were presented with rules regarding the spatial arrangement of building blocks with a written text or a video in 2D on a screen or in 3D/360° with a head-mounted display. Following the learning session, memory performance was measured by a recognition test involving a multiple-choice questionnaire, in which participants had to mark the correct arrangement of building blocks, and a construction test, in which they had to arrange five different building blocks according to the rules learned. Additionally, participants had to arrange 38 building blocks according to the rules in a free recall test the following day. Surprisingly, results revealed no superiority effect for learning in VR. Instead, learning the rules with the text yielded the best memory performance results, indicating that prior experience with conventional learning methods facilitates declarative knowledge acquisition. Considering previous findings regarding cognitive processing in VR, our results suggest that in passive learning, processing the more salient and personally relevant virtual stimuli in the surrounding VR environment requires more attentional resources. Therefore, VR impairs focusing on the relevant declarative information and impedes the transfer of the learned knowledge to different contexts. When considering to implement VR, the value to the particular domain and specific learning task should be taken into consideration: For learning basic declarative information without actively involving the students, conventional learning methods seem sufficient and more efficient for mnemonic processing compared to new technologies.

Spatial rehabilitation using virtual auditory space training paradigm in individuals with sensorineural hearing impairment.

The present study aimed to quantify the effects of spatial training using virtual sources on a battery of spatial acuity measures in listeners with sensorineural hearing impairment (SNHI). An intervention-based time-series comparison design involving 82 participants divided into three groups was adopted. Group I (n = 27, SNHI-spatially trained) and group II (n = 25, SNHI-untrained) consisted of SNHI listeners, while group III (n = 30) had listeners with normal hearing (NH). The study was conducted in three phases. In the pre-training phase, all the participants underwent a comprehensive assessment of their spatial processing abilities using a battery of tests including spatial acuity in free-field and closed-field scenarios, tests for binaural processing abilities (interaural time threshold [ITD] and level difference threshold [ILD]), and subjective ratings. While spatial acuity in the free field was assessed using a loudspeaker-based localization test, the closed-field source identification test was performed using virtual stimuli delivered through headphones. The ITD and ILD thresholds were obtained using a MATLAB psychoacoustic toolbox, while the participant ratings on the spatial subsection of speech, spatial, and qualities questionnaire in Kannada were used for the subjective ratings. Group I listeners underwent virtual auditory spatial training (VAST), following pre-evaluation assessments. All tests were re-administered on the group I listeners halfway through training (mid-training evaluation phase) and after training completion (post-training evaluation phase), whereas group II underwent these tests without any training at the same time intervals. Statistical analysis showed the main effect of groups in all tests at the pre-training evaluation phase, with post hoc comparisons that revealed group equivalency in spatial performance of both SNHI groups (groups I and II). The effect of VAST in group I was evident on all the tests, with the localization test showing the highest predictive power for capturing VAST-related changes on Fischer discriminant analysis (FDA). In contrast, group II demonstrated no changes in spatial acuity across timelines of measurements. FDA revealed increased errors in the categorization of NH as SNHI-trained at post-training evaluation compared to pre-training evaluation, as the spatial performance of the latter improved with VAST in the post-training phase. The study demonstrated positive outcomes of spatial training using VAST in listeners with SNHI. The utility of this training program can be extended to other clinical population with spatial auditory processing deficits such as auditory neuropathy spectrum disorder, cochlear implants, central auditory processing disorders etc.

Improving Naturalness in Transparent Augmented Reality with Image Gamma and Black Level

In this study, we used a custom-built Optical See-Through Augmented Reality (OST AR) system to conduct a psychophysical experiment to determine the preferred gamma and black level for high naturalness perception in OST-AR. We utilized 6 different fruit stimuli and 11 different backgrounds to do this experiment. We used two-way ANOVA to analyze the data and concluded that only the effect of different fruits on virtual objects’ gamma preference for high naturalness is considered statistically significant. Surprisingly, all ANOVA analyses indicate background’s color does not contribute to observers’ gamma or black level preference for naturalness. We found that gamma preference has a strong correlation with the average lightness of the virtual stimuli. There is no clear correlation between chroma, hue, and gamma preference in terms of naturalness perception. This finding suggests that the background can be ignored in future imaging pipelines emphasizing high naturalness appearance in Augmented Reality.

An adaptation algorithm for personalised virtual reality exposure therapy

BackgroundAnxiety disorders are highly prevalent in mental health problems. The lives of people suffering from an anxiety disorder can be severely impaired. Virtual Reality Exposure Therapy (VRET) is an effective treatment, which immerses patients in a controlled Virtual Environment (VE). This creates the opportunity to confront feared stimuli and learn how to deal with them, which may result in the reduction of anxiety. The configuration of these VEs requires extensive effort to maximise the potential of Virtual Reality (VR) and the effectiveness of the therapy. Manual configuration becomes infeasible when the number of possible virtual stimuli combinations is infinite. Due to the growing complexity, acquiring the skills to truly master a VR system is difficult and it increases the threshold for psychotherapists to use such useful systems. We therefore developed a prototype of a supportive algorithm to facilitate the use of VRET in a clinical setting. This automatised system assists psychotherapists to use the wide range of functionalities without burdening them with technical challenges. Thus, psychotherapists can focus their attention on the patient. MethodsIn this paper both the prototype of the algorithm and a first proof of concept are described. The algorithm suggests environment configurations for VRET, tailored to the individual therapeutic needs of each patient. The system aims to maximise learning during exposure therapy for different combinations of stimuli by using the Rescorla–Wagner model as a predictor for learning. In a first proof of concept, the VE configurations suggested by the algorithm for three anonymised clinical vignettes were compared with prior manual configurations by two psychotherapists. ResultsThe prototype of the algorithm and a first proof of concept are described. The first proof of concept demonstrated the relevance and potential of the proposed system, as it managed to propose similar configurations for the clinical vignettes compared to those made by therapists. Nonetheless, because of the exploratory nature of the study, no claims can yet be made about its efficacy. ConclusionsWith the increasing ubiquity of immersive technologies, this technology for assisted configuration of VEs could make VRET a valuable tool for psychotherapists.

Inter- and Transcultural Learning in Social Virtual Reality: A Proposal for an Inter- and Transcultural Virtual Object Database to be Used in the Implementation, Reflection, and Evaluation of Virtual Encounters

Visual stimuli are frequently used to improve memory, language learning or perception, and understanding of metacognitive processes. However, in virtual reality (VR), there are few systematically and empirically derived databases. This paper proposes the first collection of virtual objects based on empirical evaluation for inter-and transcultural encounters between English- and German-speaking learners. We used explicit and implicit measurement methods to identify cultural associations and the degree of stereotypical perception for each virtual stimuli (n = 293) through two online studies, including native German and English-speaking participants. The analysis resulted in a final well-describable database of 128 objects (called InteractionSuitcase). In future applications, the objects can be used as a great interaction or conversation asset and behavioral measurement tool in social VR applications, especially in the field of foreign language education. For example, encounters can use the objects to describe their culture, or teachers can intuitively assess stereotyped attitudes of the encounters.

Virtual Reality-Based Stimuli for Immersive Car Clinics: A Performance Evaluation Model

This study proposes a model to evaluate the performance of virtual reality-based stimuli for immersive car clinics. The model considered Attribute Importance, Stimuli Efficacy and Stimuli Cost factors and the method was divided into three stages: we defined the importance of fourteen attributes relevant to a car clinic based on the perceptions of Marketing and Design experts; then we defined the efficacy of five virtual stimuli based on the perceptions of Product Development and Virtual Reality experts; and we used a cost factor to calculate the efficiency of the five virtual stimuli in relation to the physical. The Marketing and Design experts identified a new attribute, Scope; eleven of the fifteen attributes were rated as Important or Very Important, while four were removed from the model due to being considered irrelevant. According to our performance evaluation model, virtual stimuli have the same efficacy as physical stimuli. However, when cost is considered, virtual stimuli outperform physical stimuli, particularly virtual stimuli with glasses. We conclude that virtual stimuli have the potential to reduce the cost and time required to develop new stimuli in car clinics, but with concerns related to hardware, software, and other definitions.

Comparing Vestibular Responses to Linear and Angular Whole-Body Accelerations in Real and Immersive Environments.

The vestibular end organs differ in terms of anatomical and physiological characteristics. Sensory modalities' stimuli including visual stimuli and vestibular sensation can influence these organs differently. This paper explores differences between vestibular responses to axial tilts in physical and virtual environments. Four passive whole-body movements (linear: up-down, and angular: yaw, pitch, and roll) were applied to twenty-seven healthy participants once using a hydraulic chair (physical) and once visually using a head-mounted display (virtual). Electrovestibulography (EVestG) was used as the outcome measure to investigate the magnitude of vestibular-response-change in both ears for physical and virtual stimuli. Three features including average action potential (AP) area, AP amplitude, and mean detected firing rate change were used as indices of response. The results show that for both physical and virtual stimuli (1) generally the pitch and roll tilts produce the largest EVestG changes compared to other tilts (2) roll and pitch tilt responses are not significantly different from each other and (3) right side and left side roll tilts' responses are not significantly different. The findings indicate although visually- and physically-induced vestibular responses are different in terms of afferent activity, visual stimuli can still result in distinct responses when exposed to different axial tilts.

Stereoscopic depth constancy for physical objects and their virtual counterparts.

Stereopsis plays an important role in depth perception; if so, disparity-defined depth should not vary with distance. However, studies of stereoscopic depth constancy often report systematic distortions in depth judgments over distance, particularly for virtual stimuli. Our aim was to understand how depth estimation is impacted by viewing distance and display-based cue conflicts by replicating physical objects in virtual counterparts. To this end, we measured perceived depth using virtual textured half-cylinders and identical three-dimensional (3D) printed versions at two viewing distances under monocular and binocular conditions. Virtual stimuli were viewed using a mirror stereoscope and an Oculus Rift head-mounted display (HMD), while physical stimuli were viewed in a controlled test environment. Depth judgments were similar in both virtual apparatuses, which suggests that variations in the viewing geometry and optics of the HMD have little impact on perceived depth. When viewing physical stimuli binocularly, judgments were accurate and exhibited stereoscopic depth constancy. However, in all cases, depth was underestimated for virtual stimuli and failed to achieve depth constancy. It is clear that depth constancy is only complete for cue-rich physical stimuli and that the failure of constancy in virtual stimuli is due to the presence of the vergence-accommodation conflict. Further, our post hoc analysis revealed that prior experience with virtual and physical environments had a strong effect on depth judgments. That is, performance in virtual environments was enhanced by limited exposure to a related task using physical objects.

Visual learning in a virtual reality environment upregulates immediate early gene expression in the mushroom bodies of honey bees

Free-flying bees learn efficiently to solve numerous visual tasks. Yet, the neural underpinnings of this capacity remain unexplored. We used a 3D virtual reality (VR) environment to study visual learning and determine if it leads to changes in immediate early gene (IEG) expression in specific areas of the bee brain. We focused on kakusei, Hr38 and Egr1, three IEGs that have been related to bee foraging and orientation, and compared their relative expression in the calyces of the mushroom bodies, the optic lobes and the rest of the brain after color discrimination learning. Bees learned to discriminate virtual stimuli displaying different colors and retained the information learned. Successful learners exhibited Egr1 upregulation only in the calyces of the mushroom bodies, thus uncovering a privileged involvement of these brain regions in associative color learning and the usefulness of Egr1 as a marker of neural activity induced by this phenomenon.

Embodiment in Virtual Reality Intensifies Emotional Responses to Virtual Stimuli

Modulating emotional responses to virtual stimuli is a fundamental goal of many immersive interactive applications. In this study, we leverage the illusion of illusory embodiment and show that owning a virtual body provides means to modulate emotional responses. In a single-factor repeated-measures experiment, we manipulated the degree of illusory embodiment and assessed the emotional responses to virtual stimuli. We presented emotional stimuli in the same environment as the virtual body. Participants experienced higher arousal, dominance, and more intense valence in the high embodiment condition compared to the low embodiment condition. The illusion of embodiment thus intensifies the emotional processing of the virtual environment. This result suggests that artificial bodies can increase the effectiveness of immersive applications psychotherapy, entertainment, computer-mediated social interactions, or health applications.

Are parrots naive realists? Kea behave as if the real and virtual worlds are continuous.

Human psychology and animal cognition have increasingly used virtual stimuli to test cognitive abilities, with the expectation that participants are 'naive realists', that is, that they perceive virtual environments as both equivalent and continuous with real-life equivalents. However, there have been no attempts to investigate whether nonhuman subjects in fact behave as if physical processes in the virtual and real worlds are continuous. As kea parrots have previously shown the ability to transfer knowledge between real stimuli and both images on paper and images on touchscreens, here we test whether kea behave as naive realists and so expect physical processes to be continuous between the physical and virtual worlds. We find that, unlike infants, kea do not discriminate between these two contexts, and that they do not exhibit a preference for either. Our findings therefore validate the use of virtual stimuli as a powerful tool for testing the cognition of nonhuman animal species.

Cue vetoing in depth estimation: Physical and virtual stimuli

Motion parallax and binocular disparity contribute to the perceived depth of three-dimensional (3D) objects. However, depth is often misperceived, even when both cues are available. This may be due in part to conflicts with unmodelled cues endemic to computerized displays. Here we evaluated the impact of display-based cue conflicts on depth cue integration by comparing perceived depth for physical and virtual objects. Truncated square pyramids were rendered using Blender and 3D printed. We assessed perceived depth using a discrimination task with motion parallax, binocular disparity, and their combination. Physical stimuli were presented with precise control over position and lighting. Virtual stimuli were viewed using a head-mounted display. To generate motion parallax, observers made lateral head movements using a chin rest on a motion platform. Observers indicated if the width of the front face appeared greater or less than the distance between this surface and the base. We found that accuracy was similar for virtual and physical pyramids. All estimates were more precise when depth was defined by binocular disparity than motion parallax. Our probabilistic model shows that a linear combination model does not adequately describe performance in either physical or virtual conditions. While there was inter-observer variability in weights, performance in all conditions was best predicted by a veto model that excludes the less reliable depth cue, in this case motion parallax.