Auditory Cues Research Articles

Handwriting for efficient text entry in industrial VR applications: influence of board orientation and sensory feedback on performance.

Text entry in Virtual Reality (VR) is becoming an increasingly important task as the availability of hardware increases and the range of VR applications widens. This is especially true for VR industrial applications where users need to input data frequently. Large-scale industrial adoption of VR is still hampered by the productivity gap between entering data via a physical keyboard and VR data entry methods. Data entry needs to be efficient, easy-to-use and to learn and not frustrating. In this paper, we present a new data entry method based on handwriting recognition (HWR). Users can input text by simply writing on a virtual surface. We conduct a user study to determine the best writing conditions when it comes to surface orientation and sensory feedback. This feedback consists of visual, haptic, and auditory cues. We find that using a slanted board with sensory feedback is best to maximize writing speeds and minimize physical demand. We also evaluate the performance of our method in terms of text entry speed, error rate, usability and workload. The results show that handwriting in VR has high entry speed, usability with little training compared to other controller-based virtual text entry techniques. The system could be further improved by reducing high error rates through the use of more efficient handwriting recognition tools. In fact, the total error rate is 9.28% in the best condition. After 40 phrases of training, participants reach an average of 14.5 WPM, while a group with high VR familiarity reach 16.16 WPM after the same training. The highest observed textual data entry speed is 21.11 WPM.

Breathy Voice as a Compensatory Strategy for Essential Vocal Tremor: A Single Case Experiment Across Participants

Essential vocal tremor (EVT) manifests as nearly rhythmic modulations of fundamental frequency (fo) and intensity and causes a shaky voice. Although research on behavioral therapy for EVT is limited, previous studies have shown that a breathy voice reduces the perception of simulated EVT and might be an effective compensatory strategy. The current study aimed to measure the acoustical and perceptual effects of a breathy voice strategy in speakers with EVT. A single-case experimental design (SCED) was used to investigate acoustical effects, and a pair comparison listening task was used to investigate perceptual effects. Three participants with EVT matched five different levels of breathiness produced by a computational model. Acoustical analyses of the rate and extent of fo and intensity modulation were performed to estimate EVT severity for baseline and treatment trials. Statistical analyses were completed using Tau-U. Twenty novice listeners participated in the perceptual study to determine if treatment trials sounded less "shaky" than baseline trials. Statistical analyses were completed using one sample ttests. The acoustical effects were variable across different levels of breathiness for all three participants. All participants were able to adjust their levels of breathiness based on auditory models and verbal cues but did not achieve the target levels of breathiness. Listeners did not perceive the breathy trials to be less "shaky" than baseline trials for any level of breathiness. The breathy voice strategy had variable effects on the acoustical measures of EVT, supporting the use of a single-case experimental design to investigate the effectiveness of behavioral treatment for EVT. Listeners perceived breathy trials to be "shakier" than baseline trials, which was inconsistent with our hypothesis and previous studies of EVT. Future research should evaluate the use of a breathy voice in speakers with confirmed laryngeal tremor, optimize cueing for breathiness, and provide more opportunities to practice the breathy strategy.

Driver Safety System

Abstract: Blind spots in vehicles and driver drowsiness are significant safety concerns that contribute to road accidents. To address these issues, we propose a comprehensive driver drowsiness and blind spot detection system using advanced technologies and image processing algorithms. The blind spot detection system employs ultrasonic sensors and an Arduino microcontroller board to gather real-time information about potential collision objects in the blind spot area. When a risk is detected, an LED alarm alerts the driver, enhancing their awareness and reducing the likelihood of accidents. Additionally, we explore the implementation of a camera-based blind spot detection system using deep learning techniques, offering a viable option for autonomous vehicles. For driver drowsiness detection, we present a non-intrusive method using a camera to continuously monitor the driver's facial features, such as eye and head movements. By analyzing these features, the system can accurately identify three driver states: awake, drowsy, and sleeping. When drowsiness is detected, the system activates alerts, such as visual and auditory cues, to awaken the driver and ensure their safety. The literature review greatly impacted our system's design, facilitating informed decisions and integration of cutting-edge technologies for an effective driver drowsiness and blind spot detection solution. The research paper also discusses the integration of these systems into various platforms, including mobile apps, Python programs, and IP cameras, providing flexible and cost-effective solutions for different vehicle types. Results from prototype implementations demonstrate the effectiveness and reliability of the proposed methods in detecting blind spots and drowsiness. Overall, this research enhances road safety via efficient blind spot & drowsiness detection, saving lives by addressing driver fatigue. Utilizing tech & machine learning, proposed systems advance active safety & driver assistance

FNIRS-based evaluation of the impact of SARS-CoV-2 infection central auditory processing.

Coronavirus disease-2019 due to SARS-CoV-2 infection has been associated with neurological and neuropsychiatric illnesses as well as auditory system problems. In this study, we aimed to evaluate the impact of SARS-CoV-2 infection on the central auditory system by assessing the hemodynamic activation changes using functional near-infrared spectroscopy (fNIRS). Three participants who had SARS-CoV-2 infection (study group) and four participants who had no SARS-CoV-2 infection (control group) were included in the study. During the auditory oddball task in which two different frequencies of tonal stimulation were presented at 80dB HL, the participants were asked to pay attention to the rare tonal stimulation and mentally count these target stimuli throughout the task. During this task, oxygenated hemodynamic response functions were evaluated with fNIRS. Significantly increased oxygenated hemodynamic responses were observed in both groups during the task (p<.05), which was significantly higher in the study group (p<.05). Significantly more HbO activation was observed in the vmPFC, superior temporal gyrus, and medial temporal gyrus in the study group compared to controls (p<.05). Significantly higher hemodynamic activation was observed in the right hemisphere in both groups, which was significantly higher in the study group (p<.05). SARS-CoV-2 infections may impact on central auditory processing or auditory attention due to changes in oxyhemoglobin levels in the frontal and temporal brain regions. It seems that SARS-CoV-2 infection is associated with an additional load on neural activity, and difficulties in focusing in auditory attention, following speech and hearing in noise as well as increased effort to perceive auditory cues.

Joint Contributions of Auditory, Proprioceptive and Visual Cues on Human Balance

Abstract One’s ability to maintain their center of mass within their base of support (i.e., balance) is believed to be the result of multisensory integration. Much of the research in this literature has focused on integration of visual, vestibular, and proprioceptive cues. However, several recent studies have found evidence that auditory cues can impact balance control metrics. In the present study, we sought to better characterize the impact of auditory cues on narrow stance balance task performance with different combinations of visual stimuli (virtual and real world) and support surfaces (firm and compliant). In line with past results, we found that reducing the reliability of proprioceptive cues and visual cues yielded consistent increases in center-of-pressure (CoP) sway metrics, indicating more imbalance. Masking ambient auditory cues with broadband noise led to less consistent findings; however, when effects were observed they were substantially smaller for auditory cues than for proprioceptive and visual cues — and in the opposite direction (i.e., masking ambient auditory cues with broadband noise reduced sway in some situations). Additionally, trials that used virtual and real-world visual stimuli did not differ unless participants were standing on a surface that disrupted proprioceptive cues; disruption of proprioception led to increased CoP sway metrics in the virtual visual condition. This is the first manuscript to report the effect size of different perturbations in this context, and the first to study the impact of acoustically complex environments on balance in comparison to visual and proprioceptive contributions. Future research is needed to better characterize the impact of different acoustic environments on balance.

Beyond the Eye: Multisensory Contributions to the Sensation of Illusory Self-Motion (Vection)

Abstract Vection is typically defined as the embodied illusion of self-motion in the absence of real physical movement through space. Vection can occur in real-life situations (e.g., ‘train illusion’) and in virtual environments and simulators. The vast majority of vection research focuses on vection caused by visual stimulation. Even though visually induced vection is arguably the most compelling type of vection, the role of nonvisual sensory inputs, such as auditory, biomechanical, tactile, and vestibular cues, have recently gained more attention. Non-visual cues can play an important role in inducing vection in two ways. First, nonvisual cues can affect the occurrence and strength of vection when added to corresponding visual information. Second, nonvisual cues can also elicit vection in the absence of visual information, for instance when observers are blindfolded or tested in darkness. The present paper provides a narrative review of the literature on multimodal contributions to vection. We will discuss both the theoretical and applied relevance of multisensory processing as related to the experience of vection and provide design considerations on how to enhance vection in various contexts.

Effect of musical cues on gait in individuals with Parkinson disease with comorbid dementia

BackgroundIndividuals with Parkinson disease and comorbid dementia (PDD) demonstrate gait impairments, but little is known about how these individuals respond to interventions for gait dysfunction. Rhythmic auditory stimulation (RAS), which utilizes music or other auditory cues to alter gait, has been shown to be effective for improving gait in individuals with PD without dementia, but has not been explored in individuals with PDD. Research questionCan individuals with PDD modulate their gait in response to music and mental singing cues? MethodsThis single center, cross-sectional, interventional study included 17 individuals with PDD. Participants received Music and Mental singing cues at tempos of 90 %, 100 %, 110 %, and 120 % of their uncued walking cadence. Participants were instructed to walk to the beat of the song. Gait variables were collected using APDM Opal sensors. Data were analyzed using mixed effect models to explore the impact of tempo and cue type (Music vs Mental) on selected gait parameters of velocity, cadence, and stride length. ResultsMixed effects models showed a significant effect of tempo but not for cue type for velocity (F=11.51, p < .001), cadence (F=11.13, p < .001), and stride length (F=5.68, p = .002). When looking at the marginal means, velocity at a cue rate of 90 % was significantly different from 100 %, indicating participants walked slower with a cue rate of 90 %. Participants did not significantly increase their velocity, cadence, or stride length with faster cue rates of 110 % and 120 % SignificanceIndividuals with PDD appear to be able to slow their velocity in response to slower cues, but do not appear to be able to increase their velocity, cadence, or stride length in response to faster cue tempos. This is different from what has been reported in individuals with PD without dementia. Further research is necessary to understand the underlying mechanism for these differences.

Efficacy of MCIMT with Auditory Cueing in order to Augment Functional Motor Recovery of Chronic Hemiparetic Arm

Background: Stroke is a leading cause of functional impairments globally, often resulting in long-term disability and a substantial impact on individuals, families, and caregivers. Learned nonuse, a phenomenon where motor deficits persist after a stroke due to central nervous system suppression, is a significant challenge in stroke rehabilitation. Constraint-Induced Movement Therapy (CIMT) is a well-established neurorehabilitation approach based on behavioural training, which includes repetitive task-specific training, behavioural techniques, and constraining the use of the more affected upper extremity. CIMT has been successful in improving functionality in chronic hemiparetic stroke patients, promoting cortical reorganization and neural plasticity.&#x0D; Method: This study involved 60 stroke survivors aged 20 or older, selected purposively and meeting specific criteria. Participants underwent an 8-week Modified CIMT (M-CIMT) program with auditory cueing for the hemiparetic upper limb. Data collection included pre-test and post-test assessments using the Wolf Motor Function Test (WMFT) and the Motor Activity Log (MAL) to evaluate functional activity. The M-CIMT program comprised warm-up sessions, M-CIMT protocols, and cool-down exercises, conducted five days a week.&#x0D; Results: Following participation in the M-CIMT program, significant improvements were observed in WMFT and MAL scores. The mean pre-test score of 46.60 seconds decreased to a mean post-test score of 16.85 seconds, with highly significant t-values of 14.292 and -51.356 and a p-value of 0.0, indicating a strong correlation between the two datasets. The analysis revealed that M-CIMT effectively enhanced upper extremity function in chronic stroke patients.&#x0D; Conclusion: This study highlights the effectiveness of the Modified CIMT protocol, emphasizing repeated use and a distributed practice schedule, in reducing upper-limb impairment and improving upper-limb use and function in chronic stroke patients. The significant improvement in post-test scores underscores the promise of M-CIMT as a valuable approach for upper extremity rehabilitation following chronic stroke, offering hope for enhanced recovery and improved quality of life for affected individuals.&#x0D; Keywords: Stroke; Rehabilitation; Upper Extremity; Constraint-Induced Movement Therapy; Motor Activity

The Role of Auditory and Visual Cues in the Perception of Mandarin Emotional Speech in Male Drug Addicts

Evidence from previous neurological studies has revealed that drugs can cause severe damage to the human brain structure, leading to significant cognitive disorders in emotion processing, such as psychotic-like symptoms (e.g., speech illusion: reporting positive/negative responses when hearing white noise) and negative reinforcement. Due to these emotion processing disorders, drug addicts may experience difficulties in emotion recognition and speech illusion, which are essential for interpersonal communication and a healthy life experience. However, previous research has yielded divergent results regarding whether drug addicts are more attracted to negative stimuli or positive stimuli. Additionally, little attention has been paid to the speech illusion experienced by drug addicts. Therefore, the current study aimed to investigate the effect of drugs on patterns of emotion recognition through two basic channels: auditory (speech) and visual (facial expression), as well as the speech illusions of drug addicts. The current study conducted a perceptual experiment in which 52 stimuli of four emotions (happy, angry, sad, and neutral) in three modalities (auditory, visual, auditory + visual [congruent & incongruent]) were presented to address Question 1 regarding the multi-modal emotional speech perception of drug addicts. Additionally, 26 stimuli of white noise and speech of three emotions in two noise conditions were presented to investigate Question 2 concerning the speech illusion of drug addicts. A total of thirty-five male drug addicts (25 heroin addicts and 10 ketamine addicts) and thirty-five male healthy controls were recruited for the perception experiment. The results, with heroin and ketamine addicts as examples, revealed that drug addicts exhibited lower accuracies in multi-modal emotional speech perception and relied more on visual cues for emotion recognition, especially when auditory and visual inputs were incongruent. Furthermore, both heroin and ketamine addicts showed a higher incidence of emotional responses when only exposed to white noise, suggesting the presence of psychotic-like symptoms (i.e., speech illusion) in drug addicts. Our results preliminarily indicate a disorder or deficit in multi-modal emotional speech processing among drug addicts, and the use of visual cues (e.g., facial expressions) may be recommended to improve their interpretation of emotional expressions. Moreover, the speech illusions experienced by drug addicts warrant greater attention and awareness. This paper not only fills the research gap in understanding multi-modal emotion processing and speech illusion in drug addicts but also contributes to a deeper understanding of the effects of drugs on human behavior and provides insights for the theoretical foundations of detoxification and speech rehabilitation for drug addicts.

Holographic Hintways: A systems feasibility and usability study of augmented reality cueing for gait adaptation

BackgroundThrough providing on-demand visual and auditory cues while walking,augmented reality (AR) can theoretically cue spatiotemporal gait adaptations in, populations such as those with Parkinson’s disease. However, given the novelty of the, technology, the type and extent of gait adaptations in response to such a cueing, system are unknown. Before such systems can be incorporated into rehabilitation, approaches, it is important to understand how people interact with the technology. Research questions: What are the effects of visual and auditory cues delivered, through AR on spatiotemporal walking patterns and variability in a healthy, young, population? Is there a relationship between system usability and gait variability?, MethodsTwenty healthy, young participants walked in four different cueing conditions using an AR headset: No Cues (NC) (i.e., natural gait), Auditory (A), Visual (V), and Auditory + Visual (AV). Inertial measurement units recorded spatiotemporal gait data at 200 Hz, a System Usability Survey was administered afterward, and linear regression models were developed to examine whether gait variability is predictive of system usability. ResultsAll cueing conditions exhibited a significantly slower cadence compared to, NC trials. Cadence variability was significantly higher for A trials compared to V and, NC. V trials exhibited significantly decreased stride lengths compared to NC. Increased, reported system usability was significantly correlated with decreased stance phase, time variability across A trials. SignificanceOur findings support that holographic spatial-visual and auditory cues, are promising to evoke spatiotemporal gait adaptations. Results also support the, notion that the type of system and cue delivery design may impact gait outcomes.,Before an AR cueing system can be applied to a specific population in future, interventions, a more holistic approach towards finding the relationship between this, technology and its users is needed.

Sex difference in the facilitation of fear learning by prior fear conditioning

There is now ample evidence that the strength and underlying mechanisms of memory formation can be drastically altered by prior experience. However, the prior work using rodent models on this topic has used only males as subjects, and as a result, we do know whether or not the effects of prior experience on subsequent learning are similar in both sexes. As a first step towards addressing this shortcoming, rats of both sexes were given auditory fear conditioning, or fear conditioning with unsignaled shocks, followed an hour or a day later by a single pairing of light and shock. Fear memory for each experience was assessed by measuring freezing behavior to the auditory cue and fear-potentiated startle to the light. Results showed that males trained with auditory fear conditioning showed facilitated learning to the subsequent visual fear conditioning session when the two training sessions were separated by one hour or one day. Females showed evidence of facilitation in rats given auditory conditioning when they were spaced by an hour but not when they were spaced by one day. Contextual fear conditioning did not support the facilitation of subsequent learning under any conditions. These results indicate that the mechanism by which prior fear conditioning facilitates subsequent learning differs between sexes, and they set the stage for mechanistic studies to understand the neurobiological basis of this sex difference.

Targeted memory reactivation during post-learning sleep does not enhance motor memory consolidation in older adults.

Targeted memory reactivation (TMR) during sleep enhances memory consolidation in young adults by modulating electrophysiological markers of neuroplasticity. Interestingly, older adults exhibit deficits in motor memory consolidation, an impairment that has been linked to age-related degradations in the same sleep features sensitive to TMR. We hypothesised that TMR would enhance consolidation in older adults via the modulation of these markers. A total of 17 older participants were trained on a motor task involving two auditory-cued sequences. During a post-learning nap, two auditory cues were played: one associated to a learned (i.e., reactivated) sequence and one control. Performance during two delayed re-tests did not differ between reactivated and non-reactivated sequences. Moreover, both associated and control sounds modulated brain responses, yet there were no consistent differences between the auditory cue types. Our results collectively demonstrate that older adults do not benefit from specific reactivation of a motor memory trace by an associated auditory cue during post-learning sleep. Based on previous research, it is possible that auditory stimulation during post-learning sleep could have boosted motor memory consolidation in a non-specific manner.

Enhancing accessibility of web-based SVG buttons: An optimization method and best practices

The Internet’s rapid development has made web page access an integral part of people’s lives, catering to entertainment, socializing, and information retrieval. However, people with visual impairments face challenges in comprehending information and rely on auditory cues. Thankfully, web pages have played a crucial role in bridging this gap by providing them with valuable knowledge about the world. Scalable Vector Graphics (SVG) has emerged as a versatile icon style in web development, surpassing image-based models due to its operational efficiency and non-deformable properties. Nonetheless, our empirical research reveals several accessibility issues with SVG buttons, particularly the absence of alternative text. Addressing this issue entails overcoming three key challenges, as reflected in it is hard to locate SVG buttons, inability to provide standardized alternative text, and excessive resource costs. To tackle these challenges, we introduce an automated tool called SAM (SVG Accessibility Optimization Method), which supplements missing alternative text in SVG buttons. Our approach involves extensive research, including the identification of 18 common categories of SVG buttons and the creation of 25 corresponding feature templates. SAM employs classification, semantic optimization, and role optimization to enhance the accessibility of SVG buttons. We evaluate SAM by applying it to 30 real-world web pages, successfully providing alternative text with an average F1-score of 92.6%. Compared to other models and image-based baseline methods, SAM demonstrates superior performance with an average F1-score increase of 14.5%. Through the efficiency analysis and a user study, we discuss the advantages of SAM, highlighting its lightweight nature (with First Contentful Paint (FCP) at 0.762 s, Time To Interactive (TTI) at 1.238 s, and memory usage at 18.32 MB). Furthermore, visually impaired users express positive feedback regarding web pages repaired by our tool, underscoring the usefulness of our method.

Music literacy training in dermatology: A mixed methods cohort study of postgraduate education

AbstractBackgroundWe have previously pioneered arts‐based training to enhance clinical skills in senior and resident dermatologists. However, the utility of music to support the development of dermatological skills such as listening, observation, description and communication is unexplored.ObjectivesWe hypothesised that music literacy training (MLT) would enhance the clinical dermatology skills of newly qualified doctors and could be effectively delivered in a single‐session virtual format.MethodsOur training comprised a 2.5‐h course held in October 2021 on the virtual meeting platform Zoom. The course was led by a music educator and included sessions which placed particular emphasis on listening to music, discriminating auditory cues to interpret imagery and meaning, and describe this to others. Participants explored concepts such as pitch, melody, pace, rhythm, dynamics and timbre by working collaboratively, independently and in small break‐out groups. All participants documented their learning journey through pre‐ and post‐course reflective essays and the training was evaluated by a pre‐ and post‐course dermatology clinical assessment.ResultsMusic served as a catalyst for reflection and discussion, supporting significant improvement key clinical skills such as listening, clinical observation and description in newly qualified doctors. Participants reported increased confidence communicating with others and found the course structure enjoyable and fun.ConclusionsCombined qualitative and quantitative evidence highlights the benefit of MLT in developing key clinical skills in newly qualified doctors which is effective when delivered in a single‐session, virtual format.

Decoding effects of psychoactive drugs in a high-dimensional space of eye movements in monkeys.

Oculomotor behavior has been shown to be correlated with mental disorders in clinics, making it promising for disease diagnosis. Here we developed a thorough oculomotor test toolkit, involving saccade, smooth pursuit, and fixation, allowing the examination of multiple oculomotor parameters in monkey models induced by psychoactive drugs. Eye movements were recorded after daily injections of phencyclidine (PCP) (3.0mg/kg), ketamine (0.8mg/kg) or controlled saline in two macaque monkeys. Both drugs led to robust reduction in accuracy and increment in reaction time during high cognitive-demanding tasks. Saccades, smooth pursuit, and fixation stability were also significantly impaired. During fixation, the involuntary microsaccades exhibited increased amplitudes and were biased toward the lower visual field. Pupillary response was reduced during cognitive tasks. Both drugs also increased sensitivity to auditory cues as reflected in auditory evoked potentials (AEPs). Thus, our animal model induced by psychoactive drugs produced largely similar abnormalities to that in patients with schizophrenia. Importantly, a classifier based on dimension reduction and machine learning could reliably identify altered states induced by different drugs (PCP, ketamine and saline, accuracy=93%). The high performance of the classifier was reserved even when data from one monkey were used for training and testing the other subject (averaged classification accuracy=90%). Thus, despite heterogeneity in baseline oculomotor behavior between the two monkeys, our model allows data transferability across individuals, which could be beneficial for future evaluation of pharmaceutical or physical therapy validity.

Comparing the influence of auditory streaming cues on binaural pitch fusion in normal-hearing and hearing-impaired listeners

Binaural pitch fusion occurs when dichotic signals eliciting a different pitch are perceived as one sound. Previously, we investigated the effects of auditory streaming cues on binaural fusion using a modification of the ABA streaming paradigm. A dichotic stimulus (fixed tone at frequency fref in the reference ear and variable frequency in the contralateral ear) was presented at alternating intervals with up to 3 “capture” tones at fref. Participants reported whether they perceived galloping or two streams. Binaural fusion frequency range decreased with the number of capture tones and longer inter-stimulus intervals (ISIs), suggesting competition of sequential grouping with capture tones versus simultaneous grouping (fusion) with the dichotic tone. In the present study, we compared the effects in NH and hearing-impaired (HI) listeners. Preliminary data from 13 NH and 5 HI listeners suggest greater reductions in fusion range with number of capture tones and with longer ISI in HI listeners, especially in listeners with broad binaural fusion. These findings indicate that auditory grouping cues can reduce binaural fusion in both NH and HI listeners. Funded by NIH/NIDCD R01DC013307.

Crossmodal cueing of masker location in a localization-in-noise task

Simpson et al. [JASA 129(4), 2489, (2011)] found that localization of a target presented in a simultaneous masker improved when, on a trial-by-trial basis, a preview of the masking stimulus was presented at the location of the upcoming masker. Subsequent research revealed that this improvement resulted solely from cueing the location of the masker (i.e., cueing spectro-temporal properties of the masker provided no additional benefit). In the present study, we examined the degree to which the benefit of cueing masker location depended on the modality in which this cue was provided. In separate blocks of trials, the location of the masker was cued auditorily, visually (LED activated at the masker location) and audio-visually, and compared to performance in a no-cue baseline. The results revealed a substantial (∼10 dB) benefit of cueing the masker auditorily over the no-cue condition, consistent with previous results. Importantly, there was also a substantial (albeit smaller) benefit of a visual cue (∼6 dB). Performance with a bimodal cue was no different than performance with an auditory cue. Thus, although hearing a sound from the upcoming masker location was most effective, the ability to effectively cue space crossmodally suggests that the spatial information itself is most critical.

Head movement during natural group conversation and inter-annotator agreement on manual annotation

During speech communication and conversational turn-taking, listeners direct their head and eyes to receive meaningful auditory and visual cues. Features of these behaviors may convey listener intent. This study designed a test environment, data collection protocol and procedures, and investigated head movement behaviors during self-driven conversations among multiple partners. Nine participants were tested in cohorts of three. Participants wore a headset with sensors tracked by an infrared camera system. Participants watched an audio-video clip, followed by a 5-min undirected discussion. The entire session was video recorded for annotation purposes. Two annotators independently coded the video files using the EUDICO Linguistic Annotator software application. Annotations were then co-registered with the head tracking data in post processing. Inter-annotator agreement demonstrated the desired reliability, thereby validating the procedures designed. Movement trajectories showed that there were individual differences on the head yaw distribution. The combination of objective measures of head movement and manual annotation of conversation behaviors provides a rich data set for characterizing natural conversations in ecologically valid settings. The measurement procedures and coding system developed here is a first step towards characterizing head movements during conversations needed to predict listening intent and to create actions based on those predictions.

Challenges in providing augmented hearing for individuals with hearing loss

Recently there has been an emphasis on developing augmented reality systems that can enhance the sensory signals listeners normally perceive in the real world. An early example of a practical augmented hearing system is the hearing aid, which amplifies the signals that would normally enter the ear in order to improve their audibility for hearing impaired listeners. Many applications are now envisioning the use of virtual audio technology to add synthetic, spatialized sounds to the “passthrough” audio signal that would normally be produced by a hearing aid. One challenge for the designers of both augmented and virtual audio systems is to ensure they preserve the ability of both normal and hearing-impaired listeners to determine the spatial locations of sounds in the environment. The auditory cues required for localizing sounds are well understood for normal hearing listeners, but less so for hearing impaired listeners. Here we discuss the results from two experiments where hearing-impaired listeners performed substantially worse than expected with virtual or augmented audio cues, and present preliminary results suggesting that the unexpectedly poor results from hearing-impaired listeners may be the result of a different weighting of localization cues across frequency regions than is typically observed for listeners with normal hearing.

Harmony behind bars: Unravelling the impacts of acoustic design in prisons

Design briefs for prisons often include criteria regulating noise from mechanical plant or break-in, partition performance and reverberation times. However, criteria are often taken (without modification) from non-carceral related standards used for design of residential or health-care projects. While carceral spaces are used for sleeping and health care, this paper will show the acoustic needs (and wants) of people inside prisons can be significantly different from those of people outside. This paper provides insights from the emergent field of sensory criminology into what makes the acoustic environment inside prisons different from other residential or health care contexts. It highlights the role that acoustics plays in the lived experience for the occupants, including incarcerated individuals and corrections staff. In prisons, acoustics facilitates communication and information gathering for incarcerated individuals, and corrections staff rely on auditory cues to gauge the prevailing tension, or “heat.” The intricate dynamics of prison cultures and insights from sensory criminology can readily evade the purview of acousticians. By leveraging complementary disciplines, acousticians can more effectively design acoustics which foster pro-social communication while mitigating risks associated with undesirable social dynamics. This approach ensures the acoustic design becomes well-informed and purposeful, yielding benefits for incarcerated individuals and corrections staff.