Auditory Instructions Research Articles

Helping Blind People Grasp: Evaluating a Tactile Bracelet for Remotely Guiding Grasping Movements.

The problem of supporting visually impaired and blind people in meaningful interactions with objects is often neglected. To address this issue, we adapted a tactile belt for enhanced spatial navigation into a bracelet worn on the wrist that allows visually impaired people to grasp target objects. Participants' performance in locating and grasping target items when guided using the bracelet, which provides direction commands via vibrotactile signals, was compared to their performance when receiving auditory instructions. While participants were faster with the auditory commands, they also performed well with the bracelet, encouraging future development of this system and similar systems.

Effects of Visual and Auditory Instructions on Space Station Procedural Tasks

In order to compare the effects of visual and auditory instructions on the crew when guiding astronauts to complete the procedural tasks in the space station, in this study, subjects were recruited to complete the programmed task of starting from the node module, locating the scientific cabinet and spectrometer, and finally operating the orbital replaceable unit on the spectrometer. Meanwhile, the task performance, eye movement parameters, and cognitive load induced by 2 kinds of instructions in the task were statistically analyzed. The results showed that there were highly significant differences in terms of task completion time, the NASA-TLX (Task Load Index) total score, and eye movement index between the 2 instructions ( P < 0.01). There were also significant differences in error rate and effort ( P < 0.05). This study proves that visual instruction interaction is better than auditory instruction. Our work provides important reference for the selection of human–computer interaction mode for procedural tasks on space stations. It also provides the experience and theoretical evidence missing so far and proves the benefits of augmented reality assistance in terms of task performance and human factors.

It takes two (seconds): decreasing encoding time for two-choice functional near-infrared spectroscopy brain-computer interface communication.

Brain-computer interfaces (BCIs) can provide severely motor-impaired patients with a motor-independent communication channel. Functional near-infrared spectroscopy (fNIRS) constitutes a promising BCI-input modality given its high mobility, safety, user comfort, cost-efficiency, and relatively low motion sensitivity. The present study aimed at developing an efficient and convenient two-choice fNIRS communication BCI by implementing a relatively short encoding time (2s), considerably increasing communication speed, and decreasing the cognitive load of BCI users. To encode binary answers to 10 biographical questions, 10 healthy adults repeatedly performed a combined motor-speech imagery task within 2 different time windows guided by auditory instructions. Each answer-encoding run consisted of 10 trials. Answers were decoded during the ongoing experiment from the time course of the individually identified most-informative fNIRS channel-by-chromophore combination. The answers of participants were decoded online with an accuracy of 85.8% (run-based group mean). Post-hoc analysis yielded an average single-trial accuracy of 68.1%. Analysis of the effect of number of trial repetitions showed that the best information-transfer rate could be obtained by combining four encoding trials. The study demonstrates that an encoding time as short as 2s can enable immediate, efficient, and convenient fNIRS-BCI communication.

Finding Keywords in a YouTube Video: A Fun Tool to Learn Atomic Structure in Middle School Students

Knowing about the evolution of atomic models throughout history is a fundamental topic to learn about atomic structure. This paper describes how identifying keywords from a musical YouTube video provides a stimulating activity for learning this topic in a chemistry class in middle school. The main goal was to show how a fun and interesting activity can be planned using a free access video and how great results can be achieved during the educational process of “teaching-learning” when generating a high level in interest in the classroom. For this purpose, a group of students was previously characterized reporting that many students showed preferences for visual and auditory instructions in their learning processes. Three (3) experimental groups and one (1) control group were visited to evaluate learning goals. Within the control group, only 22% achieved the test as it was carried out, while in other groups, 98% passed the test using methods from this research. This paper explains that the activity was planned and evaluated in three (3) ways: video relevance, development, and methodology of final evaluation, with successful results and quite positive comments about the experience from students.

Effect of Encoding on Prospective Memory.

Event-based prospective memory (ProM) refers to remembering to execute planned actions in response to a target ProM cues. Encoding modality influences ProM performance; visual encoding has been studied more than auditory encoding. Further, it has not yet been examined whether different encoding may influence ProM performance in different encoding modalities. This study examines the effects of encoding modality (visual vs. auditory), cue-encoding specificity (specific cue vs. non-specific cue), and encoding modes (standard vs. implementation intention) on event-based ProM tasks. In Experiment 1, cue specificity and encoding modality were manipulated as a within-groups encoding of visual cues is more commonly and between-groups variable. Results revealed the facilitative effect of cue specificity on ProM performance. Also, with respect to encoding modality, participants showed better performance when receiving auditory instructions compared with the visual encoding condition. In Experiment 2, as in Experiment 1, cue specificity and encoding modality were manipulated. Encoding mode was added as a new between-group variable. Result revealed that there was a significant interaction between encoding modality and encoding modes. Visual implementation intention encoding was a more effective method for improving ProM performance compared with visual standard encoding. Furthermore, there was a significant interaction between cue-encoding specificity and encoding modes. Implementation intention encoding enhances ProM performance in non-specific cue-encoding conditions. Overall, the present study found that (1) auditory encoding modality showed superior ProM performance compared with visual encoding, although implementation intention had facilitative on ProM performance regardless of the encoding modalities, and (2) there was better ProM performance under specific encoding compared with non-specific encoding, and implementation intention had a facilitative effect on ProM performance in the non-specific condition.

See, Hear, or Feel - to Speak: A Versatile Multiple-Choice Functional Near-Infrared Spectroscopy-Brain-Computer Interface Feasible With Visual, Auditory, or Tactile Instructions.

Severely motor-disabled patients, such as those suffering from the so-called “locked-in” syndrome, cannot communicate naturally. They may benefit from brain-computer interfaces (BCIs) exploiting brain signals for communication and therewith circumventing the muscular system. One BCI technique that has gained attention recently is functional near-infrared spectroscopy (fNIRS). Typically, fNIRS-based BCIs allow for brain-based communication via voluntarily modulation of brain activity through mental task performance guided by visual or auditory instructions. While the development of fNIRS-BCIs has made great progress, the reliability of fNIRS-BCIs across time and environments has rarely been assessed. In the present fNIRS-BCI study, we tested six healthy participants across three consecutive days using a straightforward four-choice fNIRS-BCI communication paradigm that allows answer encoding based on instructions using various sensory modalities. To encode an answer, participants performed a motor imagery task (mental drawing) in one out of four time periods. Answer encoding was guided by either the visual, auditory, or tactile sensory modality. Two participants were tested outside the laboratory in a cafeteria. Answers were decoded from the time course of the most-informative fNIRS channel-by-chromophore combination. Across the three testing days, we obtained mean single- and multi-trial (joint analysis of four consecutive trials) accuracies of 62.5 and 85.19%, respectively. Obtained multi-trial accuracies were 86.11% for visual, 80.56% for auditory, and 88.89% for tactile sensory encoding. The two participants that used the fNIRS-BCI in a cafeteria obtained the best single- (72.22 and 77.78%) and multi-trial accuracies (100 and 94.44%). Communication was reliable over the three recording sessions with multi-trial accuracies of 86.11% on day 1, 86.11% on day 2, and 83.33% on day 3. To gauge the trade-off between number of optodes and decoding accuracy, averaging across two and three promising fNIRS channels was compared to the one-channel approach. Multi-trial accuracy increased from 85.19% (one-channel approach) to 91.67% (two-/three-channel approach). In sum, the presented fNIRS-BCI yielded robust decoding results using three alternative sensory encoding modalities. Further, fNIRS-BCI communication was stable over the course of three consecutive days, even in a natural (social) environment. Therewith, the developed fNIRS-BCI demonstrated high flexibility, reliability and robustness, crucial requirements for future clinical applicability.

Portable Device for the Measurement and Assessment of the Human Equilibrium.

This work presents the design and development of a new alternative tool to measure the Center of Pressure (CoP) displacements, intended to evaluate the human balance. The device is based on a modified commercial balance board used for video games, resulting in a low-cost, portable device capable of computing the CoP, providing 24 of the most used indexes to test the human balance. The proposed standalone device runs on rechargeable batteries, weighs only 3.5 kg, and has a data storage capacity for over 1000 tests. Visual and auditory instructions assist its user interface. Thus, contrary to the commercial systems designed for laboratory use, this device enables the measurement of quantitative balance parameters in non-laboratory places, allowing the study of the balance of vulnerable populations directly on their typical environments. To evaluate the device, 20 older adults (68.60 ± 1.23years) were tested, and the resulting values were compared with a similar study using a force platform; 19 indexes showed a similarity with those reported using force platform and 12 of these were statistically equivalent. The proposed device represents an open-source alternative tool for researchers and healthcare personnel to acquire reliable data to evaluate human balance.

Imaging Real-Time Tactile Interaction With Two-Person Dual-Coil fMRI.

Studies of brain mechanisms supporting social interaction are demanding because real interaction only occurs when persons are in contact. Instead, most brain imaging studies scan subjects individually. Here we present a proof-of-concept demonstration of two-person blood oxygenation dependent (BOLD) imaging of brain activity from two individuals interacting inside the bore of a single MRI scanner. We developed a custom 16-channel (8 + 8 channels) two-helmet coil with two separate receiver-coil pairs providing whole-brain coverage, while bringing participants into a shared physical space and realistic face-to-face contact. Ten subject pairs were scanned with the setup. During the experiment, subjects took turns in tapping each other’s lip versus observing and feeling the taps timed by auditory instructions. Networks of sensorimotor brain areas were engaged alternatingly in the subjects during executing motor actions as well as observing and feeling them; these responses were clearly distinguishable from the auditory responses occurring similarly in both participants. Even though the signal-to-noise ratio of our coil system was compromised compared with standard 32-channel head coils, our results show that the two-person fMRI scanning is feasible for studying the brain basis of social interaction.

Comparing Tactile to Auditory Guidance for Blind Individuals.

The ability to travel independently is crucial to an individual’s quality of life but compromised by visual impairment. Several navigational aids have been developed for blind people to address this limitation. These devices typically employ auditory instructions to guide users to desired waypoints. Unfortunately, auditory instructions may interfere with users’ awareness of environmental sounds that signal dangers or provide cues for spatial orientation. Accordingly, there is a need to explore the use of non-auditory modalities to convey information for safe and independent travel. Here, we explored the efficacy of a tactile navigational aid that provides turn signals via vibrations on a hip-worn belt. We compared the performance of 12 blind participants as they navigated a series of paths under the direction of the tactile belt or conventional auditory turn commands; furthermore, we assessed the effect of repeated testing, both in the presence and absence of simulated street sounds. A computer-controlled system triggered each turn command, measured participants’ time-to-path-completion, and detected major navigational errors. When participants navigated in a silent environment, they performed somewhat worse with the tactile belt than the auditory device, taking longer to complete each trial and committing more errors. When participants navigated in the presence of simulated street noises, the difference in completion time between auditory and tactile navigation diminished. These results suggest that tactile navigation holds promise as an effective method in everyday environments characterized by ambient noise such as street sounds.

Brain dynamics and connectivity networks under natural auditory stimulation

The analysis of functional magnetic resonance imaging (fMRI) data is challenging when subjects are under exposure to natural sensory stimulation. In this study, a two-stage approach was developed to enable the identification of connectivity networks involved in the processing of information in the brain under natural sensory stimulation. In the first stage, the degree of concordance between the results of inter-subject and intra-subject correlation analyses is assessed statistically. The microstructurally (i.e., cytoarchitectonically) defined brain areas are designated either as concordant in which the results of both correlation analyses are in agreement, or as discordant in which one analysis method shows a higher proportion of supra-threshold voxels than does the other. In the second stage, connectivity networks are identified using the time courses of supra-threshold voxels in brain areas contingent upon the classifications derived in the first stage. In an empirical study, fMRI data were collected from 40 young adults (19 males, average age 22.76 ± 3.25), who underwent auditory stimulation involving sound clips of human voices and animal vocalizations under two operational conditions (i.e., eyes-closed and eyes-open). The operational conditions were designed to assess confounding effects due to auditory instructions or visual perception. The proposed two-stage analysis demonstrated that stress modulation (affective) and language networks in the limbic and cortical structures were respectively engaged during sound stimulation, and presented considerable variability among subjects. The network involved in regulating visuomotor control was sensitive to the eyes-open instruction, and presented only small variations among subjects. A high degree of concordance was observed between the two analyses in the primary auditory cortex which was highly sensitive to the pitch of sound clips. Our results have indicated that brain areas can be identified as concordant or discordant based on the two correlation analyses. This may further facilitate the search for connectivity networks involved in the processing of information under natural sensory stimulation.

Measuring cooking experience implicitly and explicitly: Physiology, facial expression and subjective ratings

Understanding consumers’ emotional experience during the process of cooking is important to enable the development of food products. In addition to verbal (‘explicit’) reports, physiological variables and facial expression may be helpful measures since they do not interfere with the experience itself and are of a continuous nature. This study investigated the potential of a range of implicit and explicit measures (1) to differentiate between subtle differences in pleasantness of ingredients, and (2) to identify emotionally salient phases during the process of cooking. 74 participants cooked and tasted a curry dish following standardized timed auditory instructions, either with ‘basic’ or ‘premium’ versions of ingredients. Heart rate, skin conductance, EEG and facial expression were recorded continuously during cooking and tasting. Subjective ratings of valence and arousal were taken directly after. Before and after cooking, participants performed ‘dry cooking’ sessions without ingredients to acquire changes in the physiological variables caused by physical activity only. We found no differences between the ‘basic’ and ‘premium’ groups, neither in implicit, nor in explicit measures. However, there were several robust physiological effects reflecting different cooking phases. Most notably, heart rate was relatively high for two specific phases: adding curry paste from a sachet during cooking, and tasting the prepared dish. The verbal reports of valence and arousal showed similar patterns over phases. Thus, our method suggests that physiological variables can be used as continuous, implicit measures to identify phases of affective relevance during cooking and may be a valuable addition to explicit measures of emotion.

The effects of auditory satellite navigation instructions and visual blur on road hazard perception

The distracting effects of mobile telephone use while driving are well known, however the effects of other sources of distraction, such as auditory navigation devices, are less well understood. Whether the effects of auditory distraction might interact with other sensory impairments, such as vision impairment, is of interest given that visual impairment is relatively common within the population, particularly as a result of uncorrected refractive error. In this experiment, 20 current drivers (mean age of 29.4 ± 3.2 years), binocularly viewed video recordings of traffic scenes presented as part of the Hazard Perception Test and responded to potential hazards within the traffic scenes. Half of the presented scenes included auditory navigation instructions as an auditory distractor. Additionally, some of the scenes were viewed through optical lenses to induce different levels of refractive blur (+0.50 DS, +1.00 DS and +2.00 DS). Hazard perception response times increased significantly (p < 0.05) with increasing blur. Participants were significantly slower in reacting to hazards for the +1.00 DS and +2.00 DS blur conditions compared to the control condition (with no blur). There was also a significant increase in response times to hazards in the presence of the auditory navigation instructions. The combined effect of blur and auditory instructions was additive, with the worst performance being in the presence of both blur and auditory instructions. These results suggest that the delivery of auditory navigation guidance for those with visual impairments, such as blur, which are relatively common in the population, should be further investigated.

The Paradox of Control: Visual Clue vs. Auditory Instruction in Game Designing

Visual clue and auditory instruction are information that used in games. the genuine information is developed on game design conceptualization. this paper explains the study of the paradox of control, which was applied in the process of game designing. Basically, the research investigates the differences of powerful game information between visual clue and auditory instruction to gain complexity of game levelling. the research method used game design experiments, playing observations, and the interview to clarify the respondents’ decision-making. the visual clues and auditory instructions were occurred randomly based on the path of the respondents’ chosen. However, in one level of observation, the information of visual clue &amp; auditory instruction were happening together in the same time and those were contradictive each other. this condition was called the paradox of control on information handling in the game. interestingly, in this circumstance, most of the respondents decided to follow the visual clues, although it was not the right direction. that condition potentially led by visual perception. this paper will describe the respondents’ motivation on their decision-making. Finally, the result of this study could give recommendation for level designers in game designing, especially on complexity of the challenge in the game’s dramatic element.

Helping older pedestrians navigate in the city: comparisons of visual, auditory and haptic guidance instructions in a virtual environment

ABSTRACTPreserving older pedestrians’ navigation skills in urban environments is a challenge for maintaining their quality of life. However, existing aids do not take into account older people’s perceptual and cognitive declines nor their user experience, and they call upon sensory modalities that are already used during walking. The present study was aimed at comparing different guidance instructions using visual, auditory, and haptic feedback in order to identify the most efficient and best accepted one(s). Sixteen middle-age (non-retired) adults, 21 younger-old (young-retired) adults, and 21 older-old (old-retired) adults performed a navigation task in a virtual environment. The task was performed with visual feedback (directional arrows superimposed on the visual scenes), auditory feedback (sounds in the left/right ear), haptic feedback (vibrotactile information delivered by a wristband), combinations of different types of sensory feedback, or a paper map. The results showed that older people benefited from the sensory guidance instructions, as compared to the paper map. Visual and auditory feedbacks were associated with better performance and user experience than haptic feedback or the paper map, and the benefits were the greatest among the older-old participants, even though the paper-map familiarity was appreciated. Several recommendations for designing pedestrian navigation aids are proposed.

One way to guide them all: Wayfinding strategies and the examination of gender-specific navigational instructions in a real-driving context

Previous research suggests that men outperform women when they are required to use Euclidean information such as distances for orientation tasks, whereas women are superior in the use of landmarks. Our study examines whether this finding stands up to a test if it is put into an application context. Besides comparing self-reported wayfinding strategies between gender groups, we investigated if auditory instructions in navigation systems can be adjusted to potential gender differences in order to improve performance and to decrease distraction from the driving-task.Participants (N = 76) respond to a questionnaire for wayfinding strategy use and take part in a real-world 20 min, standardized driving task. Driving behavior as well as glance and blink data are recorded via data-logging and eye-tracking. Overall, male and female orientation behavior is much more similar than indicated by previous research. Both men and women predominantly report to use landmark rather than Euclidean information. Correspondingly, even though they selectively respond to distance-versus landmark-based auditory navigational instructions when navigating while driving, the utilization of landmarks as navigation queries results in a lower visual distraction and mental workload for both, men and women. Driving performance is not selectively affected. Our results indicate that the use of landmarks as auditory navigational cues can make driving both safer and more comfortable for men and women. Consequently, in-car navigation systems can benefit from the implementation of landmark navigation prompts.

Voice Navigation Effects on Real-World Lane Change Driving Analysis Using an Electroencephalogram

Improving the degree of assistance given by in-car navigation systems is an important issue for the safety of both drivers and passengers. There is a vast body of research that assesses the usability and interfaces of the existing navigation systems but very few investigations study the impact on the brain activity based on navigation-based driving. In this paper, a real-world experiment is designed to acquire the electroencephalography (EEG) and in-car information to analyze the dynamic brain activity while the driver is performing the lane-changing task based on the auditory instructions from an in-car navigation system. The results show that auditory cues can influence the speed and increase the frontal EEG delta and beta power, which is related to motor preparation and decision making during a lane change. However, there were no significant results on the alpha power. A better lane-change assessment can be obtained using specific vehicle information (lateral acceleration and heading angle) with EEG features for future naturalized driving study.

Cortical Activation during Landmark-Centered vs. Gaze-Centered Memory of Saccade Targets in the Human: An FMRI Study.

A remembered saccade target could be encoded in egocentric coordinates such as gaze-centered, or relative to some external allocentric landmark that is independent of the target or gaze (landmark-centered). In comparison to egocentric mechanisms, very little is known about such a landmark-centered representation. Here, we used an event-related fMRI design to identify brain areas supporting these two types of spatial coding (i.e., landmark-centered vs. gaze-centered) for target memory during the Delay phase where only target location, not saccade direction, was specified. The paradigm included three tasks with identical display of visual stimuli but different auditory instructions: Landmark Saccade (remember target location relative to a visual landmark, independent of gaze), Control Saccade (remember original target location relative to gaze fixation, independent of the landmark), and a non-spatial control, Color Report (report target color). During the Delay phase, the Control and Landmark Saccade tasks activated overlapping areas in posterior parietal cortex (PPC) and frontal cortex as compared to the color control, but with higher activation in PPC for target coding in the Control Saccade task and higher activation in temporal and occipital cortex for target coding in Landmark Saccade task. Gaze-centered directional selectivity was observed in superior occipital gyrus and inferior occipital gyrus, whereas landmark-centered directional selectivity was observed in precuneus and midposterior intraparietal sulcus. During the Response phase after saccade direction was specified, the parietofrontal network in the left hemisphere showed higher activation for rightward than leftward saccades. Our results suggest that cortical activation for coding saccade target direction relative to a visual landmark differs from gaze-centered directional selectivity for target memory, from the mechanisms for other types of allocentric tasks, and from the directionally selective mechanisms for saccade planning and execution.

THE EFFECTS OF GUIDANCE METHOD ON DETECTION AND SCANNING AT INTERSECTIONS - A PILOT STUDY.

Older drivers are frequently involved in collisions at intersections. One reason may be inadequate head and eye scanning when approaching the intersection. Prior driving simulator research on scanning at intersections has employed two main methods to guide subjects through the simulated world: auditory instructions similar to GPS navigation and following a lead vehicle. However, these two methods may have differing effects on head and eye scanning behaviors. We therefore conducted a pilot study to assess the effects of guidance method on participants' head and eye movements as well as their detection of motorcycle hazards at intersections. Detection rates were significantly higher when following a lead vehicle than when following GPS instructions, but participants were closer to the intersection when they responded. Preliminary examination of the head and eye movement data suggests participants scanned less frequently when following the lead vehicle.

Listeners use speaker identity to access representations of spatial perspective during online language comprehension

Little is known about how listeners represent another person’s spatial perspective during language processing (e.g., two people looking at a map from different angles). Can listeners use contextual cues such as speaker identity to access a representation of the interlocutor’s spatial perspective? In two eye-tracking experiments, participants received auditory instructions to move objects around a screen from two randomly alternating spatial perspectives (45° vs. 315° or 135° vs. 225° rotations from the participant’s viewpoint). Instructions were spoken either by one voice, where the speaker’s perspective switched at random, or by two voices, where each speaker maintained one perspective. Analysis of participant eye-gaze showed that interpretation of the instructions improved when each viewpoint was associated with a different voice. These findings demonstrate that listeners can learn mappings between individual talkers and viewpoints, and use these mappings to guide online language processing.

Neck Training @ home with Intelligent Headset

Context: Non-specific neck and shoulder pain (NSS) is an increasingly common disorder in Denmark and has statistically been experienced by half of the adult population within the last six months [1, 2]. The estimated societal and personal costs related to NSS amount to at least 13 billion DKR yearly. The reason for the NSS disorder reaching epidemic levels in Denmark and worldwide is primarily the introduction of computers, tablets and smartphones in peoples’ private and professional lives and the resulting work habits and physical postures imposed by such devices [3]. Curing NSS often requires several visits to physiotherapists or chiropractic clinics and in most cases the patients receive treatment as well as instructions in rehabilitative exercises that can be performed at home. Purpose: With this M.Sc. project we propose a tele-medical training assistant that will instruct and monitor the patient in doing these home exercises in the correct manner. The training assistant consists of the motion-sensor-equipped 3D audio headset, Intelligent Headset, developed by GN [4]. This headset is connected with a smartphone and controlled by the training assistant app developed. Using this headset, we can monitor the head movements of the patient very closely as well as give immediate auditory instructions and feedback. Methods: We first validate the precision of the headset motion sensors and the resulting ability to detect certain patterns of movement using the high-precision 3D motion capture system, Vicon, as a reference. We then develop and implement an application which is able to recognize and give feedback on three common and scientifically approved types of neck rehabilitation exercises. We do this through a user-centred process involving both healthy users and users with neck problems, while receiving continuous feedback from domain experts, such as chiropractors and physiotherapists.