Audio-only Presentation Research Articles

Comparison of subjective evaluations in virtual and real environments for soundscape researcha).

Emerging technologies of virtual reality (VR) and augmented reality (AR) are enhancing soundscape research, potentially producing new insights by enabling controlled conditions while preserving the context of a virtual gestalt within the soundscape concept. This study explored the ecological validity of virtual environments for subjective evaluations in soundscape research, focusing on the authenticity of virtual audio-visual environments for reproducibility. Different technologies for creating and reproducing virtual environments were compared, including field recording, simulated VR, AR, and audio-only presentation, in two audio-visual reproduction settings, a head-mounted display with head-tracked headphones and a VR lab with head-locked headphones. Via a series of soundwalk- and lab-based experiments, the results indicate that field recording technologies provided the most authentic audio-visual environments, followed by AR, simulated VR, and audio-only approaches. The authenticity level influenced subjective evaluations of virtual environments, e.g., arousal/eventfulness and pleasantness. The field recording and AR-based technologies closely matched the on-site soundwalk ratings in arousal, while the other approaches scored lower. All the approaches had significantly lower pleasantness ratings compared to on-site evaluations. The choice of audio-visual reproduction technology did not significantly impact the evaluations. Overall, the results suggest virtual environments with high authenticity can be useful for future soundscape research and design.

Using network science to examine audio-visual speech perception with a multi-layer graph.

To examine visual speech perception (i.e., lip-reading), we created a multi-layer network (the AV-net) that contained: (1) an auditory layer with nodes representing phonological word-forms and edges connecting words that were phonologically related, and (2) a visual layer with nodes representing the viseme representations of words and edges connecting viseme representations that differed by a single viseme (and additional edges to connect related nodes in the two layers). The results of several computer simulations (in which activation diffused across the network to simulate word identification) are reported and compared to the performance of human participants who identified the same words in a condition in which audio and visual information were both presented (Simulation 1), in an audio-only presentation condition (Simulation 2), and a visual-only presentation condition (Simulation 3). Another simulation (Simulation 4) examined the influence of phonological information on visual speech perception by comparing performance in the multi-layer AV-net to a single-layer network that contained only a visual layer with nodes representing the viseme representations of words and edges connecting viseme representations that differed by a single viseme. We also report the results of several analyses of the errors made by human participants in the visual-only presentation condition. The results of our analyses have implications for future research and training of lip-reading, and for the development of automatic lip-reading devices and software for individuals with certain developmental or acquired disorders or for listeners with normal hearing in noisy conditions.

The audio-visual integration effect on music emotion: Behavioral and physiological evidence.

Previous research has indicated that, compared to audio-only presentation, audio-visual congruent presentation can lead to a more intense emotional response. In the present study, we investigated the audio-visual integration effect on emotions elicited by positive or negative music and the role of visual information presentation durations. The participants were presented with audio-only condition, audio-visual congruent condition, and audio-visual incongruent condition and then required to judge the intensity of emotional experience elicited by the music. Their emotional responses to the music were measured using self-ratings and physiological aspects, including heart rate, skin temperature, EMG root mean square and prefrontal EEG. Relative to the audio-only presentation, the audio-visual congruent presentation led to a more intense emotional response. More importantly, the audio-visual integration occurred both in the positive music and in the negative music. Furthermore, the audio-visual integration effect was larger for positive music than for negative music; meanwhile the audio-visual integration effect was strongest with the visual information presented within 80s for negative music, which indicated that this integration effect was more likely to occur in the negative music. These results suggest that when the music was positive, the effect of audio-visual integration was greater. When the music was negative, the modulation effect of the presentation durations of visual information on the music-induced emotion was more significant.

Audio Engagement and Learning

ABSTRACTSound effects and music can be used to complement instructional narrations in recorded media presentations. College students (n = 143) participated in a study examining the effects of music and sound effects on learning outcomes when compared to voice-only narration. Four groups of participants listened to a recorded short story and answered questions about the story. The control group’s story was a voice-only narration. The three treatment groups’ story was enhanced with one of the following: sound effects, music, or a combination of sound effects and music. Means for each group were not significantly different, and the difference of means between and within each of the groups was also found not to be significant. The results indicated that the presence of sound effects and music in an audio-only presentation did not significantly increase or decrease learning.

A virtual speaker in noisy classroom conditions: supporting or disrupting children’s listening comprehension?

Aim: Seeing a speaker’s face facilitates speech recognition, particularly under noisy conditions. Evidence for how it might affect comprehension of the content of the speech is more sparse. We investigated how children’s listening comprehension is affected by multi-talker babble noise, with or without presentation of a digitally animated virtual speaker, and whether successful comprehension is related to performance on a test of executive functioning.Materials and Methods: We performed a mixed-design experiment with 55 (34 female) participants (8- to 9-year-olds), recruited from Swedish elementary schools. The children were presented with four different narratives, each in one of four conditions: audio-only presentation in a quiet setting, audio-only presentation in noisy setting, audio-visual presentation in a quiet setting, and audio-visual presentation in a noisy setting. After each narrative, the children answered questions on the content and rated their perceived listening effort. Finally, they performed a test of executive functioning.Results: We found significantly fewer correct answers to explicit content questions after listening in noise. This negative effect was only mitigated to a marginally significant degree by audio-visual presentation. Strong executive function only predicted more correct answers in quiet settings.Conclusions: Altogether, our results are inconclusive regarding how seeing a virtual speaker affects listening comprehension. We discuss how methodological adjustments, including modifications to our virtual speaker, can be used to discriminate between possible explanations to our results and contribute to understanding the listening conditions children face in a typical classroom.

The Effect of Implied Performer Age and Group Membership on Evaluations of Music Performances

This study examined the effects of implied performer age and group membership on listeners’ evaluations of music performances. Undergraduate music majors ( n = 23), nonmusic majors ( n = 17), and members of a New Horizons ensemble ( n = 16) were presented with six 30-second excerpts of concert band performances. Excerpts were presented to all respondents in three implied performer age conditions: audio-only (no implied age), audiovisual presentations that featured digital images of middle school concert bands, and audiovisual presentations that featured digital images of older adult concert bands. Respondents ( N = 56) used 10-point scales to rate each performance on tone, rhythmic precision, and dynamic contrast. Results of a repeated-measures analysis of variance indicated that audio-only presentations were rated significantly lower than audiovisual presentations, and music majors rated performances significantly lower than other respondents.

Influence of Education, Performance Level, and Presentation Medium on Evaluations of String Orchestra Performances

Abstract The purpose of this study was to examine the effect of education level (high school orchestra students, undergraduate string music education majors, and in-service orchestra teachers), ensemble performance level (good, average, and weak), and presentation medium (audio-only and audiovisual) on the evaluations of string orchestra performances. Participants (N = 96) evaluated audio-only or audiovisual recordings of good, average, and weak string ensembles. A repeated-measures ANOVA test revealed that string education majors rated intonation more favorably than inservice teachers. All participants were able to discriminate between the good, average, and weak performances when evaluating tone, intonation, and rhythmic precision. Ratings for musical effect did not differ significantly between the good and average groups. Participants assigned significantly lower tone and intonation ratings in the audio-only presentation medium than the audiovisual. Finding evidence of when musicians learn to accurately assess music performances can be useful both for in-service music teachers and teacher educators. If this ability is apparent in high school students, they can have a larger role in the rehearsal process.

The effects of relationship context and modality on ratings of funniness

There is evidence to suggest that humour is an important part of mate choice and that humour may serve as an indicator of genetic quality. The current study investigated how rated funniness from a video clip was related to an individual’s attractiveness as a short-term or long-term partner. We additionally tested for the presence of an attractiveness halo effect on humour ratings by comparing ratings of funniness from video clips, audio-only presentations, and photographs. We found that funniness was most strongly correlated with attractiveness for short-term relationships, especially in videos of males. We also found that attractiveness was related to funniness ratings differently across video, audio-only clips, and photographs. Relative to their rated funniness in the audio-only condition, with no appearance cues, attractive individuals were rated as funnier in video clips than less attractive individuals. An additional study demonstrated that ratings of flirtatiousness and funniness were strongly correlated. Perceived similarity between producing humour and flirting may explain why humour is more preferable in a short-term partner as flirting may be seen to signal proceptivity. The effects of attractiveness on humour judgement may also be explained by an association with flirtation as flirting may be most enjoyable when directed by attractive individuals.

Voice Perceptions and Quality of Life of Transgender People

Despite the plethora of research documenting that the voice and quality of life (QoL) are related, the exact nature of this relationship is vague. Studies have not addressed people who consider their voice to influence their life and identity, but would not be considered to have a voice "disorder" (e.g., transgender individuals). Individuals seeking vocal feminization may or may not have vocal pathology and often have concerns not addressed on the standard psychosocial measures of voice impact. Recent development of a voice-related QoL measure specific to the needs of transgender care (Transgender Self-Evaluation Questionnaire [TSEQ]) affords opportunity to explore relationships between self-perceived QoL and perceptions of femininity and likability associated with transgender voice. Twenty male-to-female transgender individuals living as a female 100% of the time completed the TSEQ and contributed a speech sample describing Norman Rockwell's "The Waiting Room" picture. Twenty-five undergraduate listeners rated voice femininity and voice likability after audio-only presentation of each speech sample. Speakers also self-rated their voices on these parameters. For male-to-female transgender clients, QoL is moderately correlated with how others perceive their voice. QoL ratings correlate more strongly with speaker's self-rated perception of voice compared with others' perceptions, more so for likability than femininity. This study complements previous research reports that subjective measures from clients and listeners may be valuable for evaluating the effectiveness of treatment in terms of how treatment influences voice-related QoL issues for transgender people.

Listener perceptions of stuttering across two presentation modes: A quantitative and qualitative approach

The purpose of this study was to compare participants’ quantitative and qualitative judgments of various forms and frequencies of stuttering, during either audiovisual or audio-only presentation modes. A total of 64 participants voluntarily agreed to participate in the study. Each participant was randomly assigned and exposed to only one of the four conditions of varying stuttering severity levels. The assigned speech sample was presented to the participant in either the audiovisual or audio-only mode. After watching or listening to the sample, participants completed a six-item Likert scale and were asked four open-ended questions. Comments made by listeners were developed into positive and negative comments as well as sorted into five theme clusters. The data analyses revealed a number of non-significant differences across mean Likert scale ratings for the six statements and for the number of positive and negative comments between the two presentation modes. However, the results provided partial support for the notion that as the frequency of stuttering increases, listeners tend to make increasingly more negative comments about the speaker. Additionally, there were no clear differences in comments across the five theme clusters between the two presentation modes. The findings suggest that the type of presentation mode does not appear to affect listeners’ perceptions of stuttering to the same extent that stuttering severity does. Educational objectives: The reader will be able to: (1) describe how listeners perceive and react to various frequencies of stuttering; (2) explain how audiovisual versus audio-only samples impact listener perceptions of stuttering and (3) provide examples of the ways in which qualitative data can enhance the understanding of how listeners react to various levels of stuttering.

The effect of interactive media on children's story memory.

The authors examined the influence of interactive media on children’s story memory. First-grade children (6 –7-year-olds) experienced a computer-based story in 1 of 4 presentation modes: One group heard only the narration, analogous to radio. A 2nd group saw an audiovisual presentation, analogous to television. A 3rd group viewed the story and interacted with animated areas of the screen. A 4th group was yoked to the interaction group such that they observed but did not control the interaction. The audio-only group consistently recalled and comprehended poorly, but there were no differences among the other media groups. In the interaction groups, there was also no relation between the amount of interaction with the story and subsequent memory. Overall, the results for interactive media were similar to the findings for the television-like presentation. Children today have the opportunity to learn through a range of different media, including audiotapes and radio, television and videotapes, and computer-based activities. In particular, computer access and use by children is becoming increasingly important, both at home and in school settings (Calvert, 1999; Jennings, 2001). As computers become more integrated into the classroom, new questions arise about the potential impact of interactive media on children’s ability to comprehend and remember information. Specifically, most computer programs allow children to control the activity and to interact with the content in a way that is quite different from other forms of presentation, such as radio and television. It is important to investigate whether interactivity might influence children’s learning and memory. The question of whether children’s learning might be shaped by the type of media through which the information is presented is not new. For example, researchers compared the impact of radio and television presentations. One theory had it that the highly salient visual information available through a televised presentation would interfere with children’s processing of the auditory information and thus reduce learning or at least shift children’s attention away from the narration toward the visually depicted information. Support for this “visual superiority” hypothesis was found when children who viewed television recalled more actions than children who heard an audio-only presentation. The latter were more likely to remember dialogue and to make inferences about

The Relative Effectiveness of Audio, Video, and Static Visual Computer-Mediated Presentations

Research on non-computerized media such as television, slides, animation, and film has developed an information-processing framework that can be adapted to research on computerized media. We examined the relative effectiveness of three different forms of computer presentation: digital video presentations with text, audio-only presentations with text, and static visual presentations with text. ANOVA results showed that although the achievement scores of participants in different treatment groups did not differ significantly, there was a significant difference in the time participants took to complete the presentations. From an information-processing perspective, these differences may be attributed to bisensory interference or bisensory facilitation. We concluded that when computer-mediated presentations are being designed, the comparative efficiency of the learning medium might often be more pertinent than the effectiveness of media implementation. Les auteurs ont analyse l'efficacite relative de trois formes de presentation informatisee: la presentation video numerique avec texte, la presentation exclusivement audio et la prdsentation visuelle statique avec texte. Les resultats de l'analyse de la variance revelent une nette difference dans le temps pris par les participants pour effectuer les presentations. Du point de vue du traitement de l'information, ces diff6rences peuvent s'expliquer par l'interference bisensorielle ou la facilitation bisensorielle. Les auteurs concluent que, dans les presentations assistees par ordinateur, l'efficience relative du media d'apprentissage peut souvent etre consid6ree plus importante que l'efficacite reelle de l'utilisation du media. As integrated computer-mediated instruction enters the mainstream of instructional development at all levels of education and training, the implications of the use of multi-media techniques have become essential to program design and development. Exploration of the relative effectiveness of various multi-media techniques and their impact on design efficiency, as well as on learning efficiency, has the potential to provide programmers and instructional designers with a theoretical and practical basis from which to select the appropriate medium for individual learning objectives. The effectiveness of different multimedia formats in facilitating learning is a major consideration within this framework. An additional concern, however, must be the efficiency of the pedagogical program. In other words, although a specific

Dysarthric sentence intelligibility: contribution of iconic gestures and message predictiveness.

This study examined changes in the sentence intelligibility scores of speakers with dysarthria in association with different signal-independent factors (contextual influences). This investigation focused on the presence or absence of iconic gestures while speaking sentences with low or high semantic predictiveness. The speakers were 4 individuals with dysarthria, who varied from one another in terms of their level of speech intelligibility impairment, gestural abilities, and overall level of motor functioning. Ninety-six inexperienced listeners (24 assigned to each speaker) orthographically transcribed 16 test sentences presented in an audio + video or audio-only format. The sentences had either low or high semantic predictiveness and were spoken by each speaker with and without the corresponding gestures. The effects of signal-independent factors (presence or absence of iconic gestures, low or high semantic predictiveness, and audio + video or audio-only presentation formats) were analyzed for individual speakers. Not all signal-independent information benefited speakers similarly. Results indicated that use of gestures and high semantic predictiveness improved sentence intelligibility for 2 speakers. The other 2 speakers benefited from high predictive messages. The audio + video presentation mode enhanced listener understanding for all speakers, although there were interactions related to specific speaking situations. Overall, the contributions of relevant signal-independent information were greater for the speakers with more severely impaired intelligibility. The results are discussed in terms of understanding the contribution of signal-independent factors to the communicative process.

Current status of laryngectomee rehabilitation: III. Understanding of esophageal speech

To assess those aspects of esophageal speech that contribute to effective communication, a series of measures were applied to the esophageal speech of a prospectively studied group of recent laryngectomees (n = 20) six months following completion of their cancer therapy and to that of a second group (n = 27) studied retrospectively 6.6 years postoperatively. Ages of the two groups did not differ, but there were more whites and more males in the group studied retrospectively. The technical proficiency of the subjects studied prospectively was slightly but significantly less than that of those studied retrospectively. Measures of technical proficiency correlated well with other measures of quality of esophageal speech and were much easier to perform. The understandability of the esophageal speech of the group studied retrospectively was significantly superior. For both groups, understandability was enhanced significantly during audiovisual compared with audio-only presentations. Naive judges had great variability in understanding esophageal speech.