Audio-only Condition Research Articles

Effects of Non-Musical Attributes and Excerpt Duration on Ratings of High-Level Piano Performances

The purpose of this study was to determine if judgments of expert piano performances would be affected by excerpt duration, non-musical attributes of performer attractiveness, dress, and stage behavior, and by an interaction between duration and these attributes. Given that non-musical attributes would be evident from the beginning of a performance but that information about performance quality would accrue with time spent listening, it was hypothesized that non-musical attribute biases would weaken with increasing excerpt length. Thirty-three undergraduate and graduate music majors rated performers on non-musical attributes by observing performers with the sound turned off. One hundred fourteen participants rated 15 performances on 6 test items, either under an audio-only condition or under an audiovisual condition. Excerpts were 25, 55, and 115 seconds. Results showing that high-attractive women held an advantage over low-attractive women for 25-second excerpts but not for longer ones confirmed the hypothesis that the importance of attractiveness declines with increased musical exposure. This hypothesis was less strongly supported for ratings of male performers, for whom differences in dress appeared to be of greater import. It was also found that there was significantly greater agreement between test items within excerpt durations than across them. Ratings were higher at 55 seconds and 115 seconds than they were at 25 seconds, although reliability declined slightly as excerpts became longer. Finally, performances judged under the audiovisual condition were rated significantly higher than performances judged under the audio-only condition.

User Evaluation of a Communication System That Automatically Generates Captions to Improve Telephone Communication

This study examined the subjective benefit obtained from automatically generated captions during telephone-speech comprehension in the presence of babble noise. Short stories were presented by telephone either with or without captions that were generated offline by an automatic speech recognition (ASR) system. To simulate online ASR, the word accuracy (WA) level of the captions was 60% or 70% and the text was presented delayed to the speech. After each test, the hearing impaired participants (n = 20) completed the NASA-Task Load Index and several rating scales evaluating the support from the captions. Participants indicated that using the erroneous text in speech comprehension was difficult and the reported task load did not differ between the audio + text and audio-only conditions. In a follow-up experiment (n = 10), the perceived benefit of presenting captions increased with an increase of WA levels to 80% and 90%, and elimination of the text delay. However, in general, the task load did not decrease when captions were presented. These results suggest that the extra effort required to process the text could have been compensated for by less effort required to comprehend the speech. Future research should aim at reducing the complexity of the task to increase the willingness of hearing impaired persons to use an assistive communication system automatically providing captions. The current results underline the need for obtaining both objective and subjective measures of benefit when evaluating assistive communication systems.

The interplay between the auditory and visual modality for end-of-utterance detection

The existence of auditory cues such as intonation, rhythm, and pausing that facilitate end-of-utterance detection is by now well established. It has been argued repeatedly that speakers may also employ visual cues to indicate that they are at the end of their utterance. This raises at least two questions, which are addressed in the current paper. First, which modalities do speakers use for signalling finality and nonfinality, and second, how sensitive are observers to these signals. Our goal is to investigate the relative contribution of three different conditions to end-of-utterance detection: the two unimodal ones, vision only and audio only, and their bimodal combination. Speaker utterances were collected via a novel semicontrolled production experiment, in which participants provided lists of words in an interview setting. The data thus collected were used in two perception experiments, which systematically compared responses to unimodal (audio only and vision only) and bimodal (audio-visual) stimuli. Experiment I is a reaction time experiment, which revealed that humans are significantly quicker in end-of-utterance detection when confronted with bimodal or audio-only stimuli, than for vision-only stimuli. No significant differences in reaction times were found between the bimodal and audio-only condition, and therefore a second experiment was conducted. Experiment II is a classification experiment, and showed that participants perform significantly better in the bimodal condition than in the two unimodal ones. Both the first and the second experiment revealed interesting differences between speakers in the various conditions, which indicates that some speakers are more expressive in the visual and others in the auditory modality.

The effects of various physical characteristics of high-level performers on adjudicators’ performance ratings

The purpose of this study was to examine the relationship between physical attractiveness and performance ratings of high-level pianists. Eighteen of the 30 competitors who participated in the Eleventh Van Cliburn International Piano Competition were rated by undergraduate and graduate music students and faculty. Participants were divided into three groups who rated the performances under either audio, audiovisual, or visual-only conditions. Visual-only participants rated performers on physical attractiveness, dress, and stage behaviour. Results suggest that high-level pianists are not affected in the same way by the apparent attractiveness bias that has been found in studies of novice and college-level musicians. Performers rated low on behaviour received consistently higher performance scores than high behaviour performers regardless of treatment condition (audio or audiovisual). Performers who rated high on each of the visual components benefited from the audiovisual condition, as compared with the audio-only condition, on note accuracy, but not on five other measures of performance. Interactions between each visual component (attractiveness, dress, and behaviour) and gender of the rater raise questions about gender differences in the perception of attractiveness.

Informational masking of speech in children: auditory-visual integration.

The focus of this study was the release from informational masking that could be obtained in a speech task by viewing a video of the target talker. A closed-set speech recognition paradigm was used to measure informational masking in 23 children (ages 6-16 years) and 10 adults. An audio-only condition required attention to a monaural target speech message that was presented to the same ear with a time-synchronized distracter message. In an audiovisual condition, a synchronized video of the target talker was also presented to assess the release from informational masking that could be achieved by speechreading. Children required higher target/distracter ratios than adults to reach comparable performance levels in the audio-only condition, reflecting a greater extent of informational masking in these listeners. There was a monotonic age effect, such that even the children in the oldest age group (12-16.9 years) demonstrated performance somewhat poorer than adults. Older children and adults improved significantly in the audiovisual condition, producing a release from informational masking of 15 dB or more in some adult listeners. Audiovisual presentation produced no informational masking release for the youngest children. Across all ages, the benefit of a synchronized video was strongly associated with speechreading ability.

音声対話における反応潜時の同調傾向が目標達成の評価に及ぼす影響

It is assumed that the durations of response latencies of two speakers (i. e., latencies before responding to the conversational partner's last utterance) becoming similar over the time course (i. e., convergence) influences performance evaluation within dialogues. Twelve dyads in which the participants had opposing opinions regarding a particular topic tried to reach a compromise through discussion, under audio-only conditions. Six dyads participated in each of the two experimental conditions: a decreasing transmission delay condition in which the transmission durations decrease gradually, so that response latencies tend to converge, and an increasing transmission delay condition in which the transmission durations increase gradually, so that response latencies do not converge. At the end of the 15-minute dialogue, the participants evaluated their achievement of compromise and the impression about their partners. Further, the duration of response latencies were measured from the recorded vocalizations. The results suggest that the speakers whose response latency durations became similar to those of their conversational partners over the time course evaluated that they reached a compromise. The influence of convergence on performance evaluation was further discussed.

Effect of gazing at the camera during a video link on recall

The impact of looking into the camera during a presentation over a video link (resulting in the perception of mutual gaze) on information recall was investigated. In a face-to-face context mutual gaze has been shown to facilitate the encoding and subsequent recall of information [Fry, R., Smith, G.F., 1975. The effects of feedback and eye contact on performance of a digit-coding task. J. Soc. Psychol. 96, 145–146; Otteson, J.D., Otteson, C.R., 1980. Effect of teacher's gaze on children's story recall. Percept. Motor Skill. 50, 35–42; Sherwood, J.V., 1988. Facilitative effects of gaze upon learning. Percept. Motor Skill. 64 (3 Part 2), 1275–1278]. One explanation for these findings is that gaze acts as an arousal stimulus, which increases attentional focus and therefore enhances memory [Kelley, D.H., Gorham, J., 1988. Effects of immediacy on recall of information. Commun. Edu. 37(3), 198–207]. Two studies were conducted in order to test whether gazing at the camera during video-mediated presentations resulted in similar benefits as mutual gaze in a face-to-face context. In s tudy 1 a confederate presented information about two fictitious soap products. In one condition, the confederate gazed at the camera for 30% of the presentation, therefore giving the participants the impression that he was gazing in their direction. In the other condition the confederate did not gaze at the camera. Participants viewed the sales presentations from both conditions. In the condition where gaze was directed at the camera, participants recalled significantly more information about the sales presentation. Study 2 employed the same pre-recorded sales presentations used in study 1, however they were delivered to the participants under audio-only conditions (therefore, the image was switched off). Results from study 2 indicated no recall differences between the two conditions. Findings from these studies would seem to indicate that the perception of gaze aversion over a video link (a consequence of the salesman not looking into the camera) has a negative impact on information recall. This has practical implications for video-mediated presentations. In a distance learning environment lecturers could be advised to look into the camera in order to promote more efficient learning in students.

Seeing to hear better: evidence for early audio-visual interactions in speech identification

Lip reading is the ability to partially understand speech by looking at the speaker's lips. It improves the intelligibility of speech in noise when audio-visual perception is compared with audio-only perception. A recent set of experiments showed that seeing the speaker's lips also enhances sensitivity to acoustic information, decreasing the auditory detection threshold of speech embedded in noise [J. Acoust. Soc. Am. 109 (2001) 2272; J. Acoust. Soc. Am. 108 (2000) 1197]. However, detection is different from comprehension, and it remains to be seen whether improved sensitivity also results in an intelligibility gain in audio-visual speech perception. In this work, we use an original paradigm to show that seeing the speaker's lips enables the listener to hear better and hence to understand better. The audio-visual stimuli used here could not be differentiated by lip reading per se since they contained exactly the same lip gesture matched with different compatible speech sounds. Nevertheless, the noise-masked stimuli were more intelligible in the audio-visual condition than in the audio-only condition due to the contribution of visual information to the extraction of acoustic cues. Replacing the lip gesture by a non-speech visual input with exactly the same time course, providing the same temporal cues for extraction, removed the intelligibility benefit. This early contribution to audio-visual speech identification is discussed in relationships with recent neurophysiological data on audio-visual perception.

Comparing the roles of 3D representations in audio and audio-visual collaborations

This study investigates the effects of performance and communication within audio-visual (shared representations) and audio-only conditions. Two three-dimensional (3D) representations were presented in each communication condition. The goal of the study was to examine both explicit and implicit references made during verbal interactions, and to gather subjective usability evaluations of each representation. Sixty dyads performed a series of problem solving tasks in three experimental conditions: mixed, 3D cylinder and 3D helix representations. Assessment measures included overall performance time and accuracy, and user attitudes pertaining to the usability of the displays. Although no differences in task performance were observed, qualitative measures revealed differences between representation and communication groups. User preferences for 3D cylinder and 3D helix representations were observed, with disparate strategies being adopted between groups. In general, the analyses indicated that the presence of shared visual information enhances collaborative problem solving.

Seeing to hear better: evidence for early audio-visual interactions in speech identification

Lip reading is the ability to partially understand speech by looking at the speaker's lips. It improves the intelligibility of speech in noise when audio-visual perception is compared with audio-only perception. A recent set of experiments showed that seeing the speaker's lips also enhances sensitivity to acoustic information, decreasing the auditory detection threshold of speech embedded in noise [J. Acoust. Soc. Am. 109 (2001) 2272; J. Acoust. Soc. Am. 108 (2000) 1197]. However, detection is different from comprehension, and it remains to be seen whether improved sensitivity also results in an intelligibility gain in audio-visual speech perception. In this work, we use an original paradigm to show that seeing the speaker's lips enables the listener to hear better and hence to understand better. The audio-visual stimuli used here could not be differentiated by lip reading per se since they contained exactly the same lip gesture matched with different compatible speech sounds. Nevertheless, the noise-masked stimuli were more intelligible in the audio-visual condition than in the audio-only condition due to the contribution of visual information to the extraction of acoustic cues. Replacing the lip gesture by a non-speech visual input with exactly the same time course, providing the same temporal cues for extraction, removed the intelligibility benefit. This early contribution to audio-visual speech identification is discussed in relationships with recent neurophysiological data on audio-visual perception.

Distraction Effects of Background Soap Operas on Homework Performance: An experimental study enriched with observational data

An experiment was conducted to examine the impact of background soap operas on homework performance and time. Students in grade eight (aged 14) ( n =192) did paper-and-pencil and memorisation assignments with two types of soap opera episodes in the background, or the soundtrack of soap operas, or no medium. In each condition, half of the students were observed. Results indicated that students in the television conditions performed worse and used more time than students in the control condition. No significant differences were found between the audio-only and control conditions. Observational data showed that the extension of time in the television conditions was completely due to the fact that students used time to look at the screen. Although the television did not reduce time spent looking at the task, performance did decrease, probably because the alternation of resources between homework and television led to less thorough processing of the assignments.

Audibility-index predictions of normal-hearing and hearing-impaired listeners' performance on the connected speech test.

In a previous study (Sherbecoe & Studebaker, 2002), we derived a frequency-importance function and a transfer function for the audio compact disc version of the Connected Speech Test (CST). The current investigation evaluated the validity of these audibility-index (AI) functions based on how well they predicted data from four published studies that presented the CST to normal-hearing and hearing-impaired subjects. AI values were calculated for the test conditions received by 78 normal-hearing and 72 hearing-impaired subjects from the selected studies. The observed CST scores and AI values for these conditions/subjects were then plotted and the dispersion of the data compared to the expected range based on critical differences. The AI values for the conditions/subjects were also converted into expected CST scores and subtracted from their corresponding observed scores to determine the distribution of the resulting difference scores and the relationship between the difference scores and subject age. Good predictions were obtained for normal-hearing subjects who had been tested under audio-only conditions but not those who had received audiovisual tests. The expected scores for the latter subjects were too low when the AI accounted only for audibility and too high when it included the correction for visual cues from ANSI S3.5-1997. All of the hearing-impaired subjects had been tested under audio-only conditions. In their case, the mean difference between the observed and the expected scores was comparable with the audio-only mean for the normal-hearing subjects when the AI included corrections for speech level distortion and hearing loss desensitization. However, the hearing-impaired subject data had greater variability. The predictions for these subjects also decreased in accuracy when subject age increased beyond 70 yr despite the application of an AI correction for age. The results of this study suggest that the AI functions derived for the CST satisfactorily predict the scores of normal-hearing subjects when they listen in speech babble under audio-only conditions but not when they receive visual cues. To obtain accurate predictions for the audiovisual form of the CST, it will be necessary to develop new ANSI-style AI correction equations for visual cues or new AI functions based on audiovisual test scores. If the current AI functions are used to predict the scores of hearing-impaired listeners tested under audio-only conditions, the AI should include corrections for the effects of speech level and hearing loss. A correction for subject age also could be applied, if it seems appropriate to do so. In either case, however, the predictions are still likely to be less accurate than the predictions for normal-hearing subjects. This may be because speech recognition deficits in people with hearing loss are not due solely to diminished audibility. Hearing-impaired subjects, particularly if they are elderly, also may be more susceptible to masking effects or other factors not accounted for by the AI.

Voice dialling can reduce the interference between concurrent tasks of driving and phoning

Previous research has found that using a mobile phone while driving can be detrimental to driving performance. Voice dialling has the potential to reduce the interference between concurrent tasks of driving and phoning. A laboratory experiment was designed to test two variations of a speech interface against a standard manual interface. Forty-eight participants carried out driving-related tasks while simultaneously dialling familiar numbers. The speech interfaces significantly reduced both tracking error and the reaction time to peripheral targets, compared to the manual interface. Despite an increase in dialling transaction times, speech was preferred by users, and felt to be less demanding than manual control. The speech interface with visual plus auditory feedback of recognition results performed worse than the audio-only condition. The results are explained in terms of the physical and cognitive demands of phoning while driving, and have important implications for interface design.

Effects of Performer Attractiveness, Stage Behavior, and Dress on Evaluation of Children's Piano Performances

The purpose of this study was to determine whether selected nonmusical attributes of sixth-grade pianists would affect ratings of their performances. Twenty pianists (10 girl and 10 boys) were videotaped. They and their performances were rated by 123 musically trained evaluators. Members of the visual group viewed a videotape with the sound turned off. They rated pianists on appropriateness of dress, stage behavior, and physical attractiveness. These ratings were the basis for grouping students as being high or low on each of these three attributes. Audiovisual and audio group members rated musical performance on five test items. Results revealed support for the existence of a bias: although high pianists were rated higher than low pianists under the audio condition for all three attributes, the differences between them often were significantly greater under the audiovisual condition than under the audio-only condition. In addition, and unlike finding of earlier studies, videotaped performances were not rated higher than audiotaped performances. Also, female judges were more lenient than male judges. Finally, male and female pianists were affected differently by nonmusical attributes for about half of the test items.

Time-Interval Measurement of Stuttering

The study reported in this paper was designed to replicate and extend the results of an earlier study (Ingham, Cordes, & Gow, 1993) that investigated time-interval judgments of stuttering. Results confirmed earlier findings that interjudge agreement is higher for these interval-recording tasks than has been previously reported for event-based analyses of stuttering judgments or for time-interval analyses of event judgments. Results also confirmed an earlier finding that judges with intrajudge agreement levels of 90% or better show higher interjudge agreement than judges with lower intrajudge agreement scores. This study failed to find differences between audiovisual and audio-only judgment conditions; between relatively experienced and relatively inexperienced student judges; and, most importantly, between the judgments made, and the agreement levels achieved, by judges from two different clinical research settings. The implications of these findings for attempts to develop a reliable measurement method for stuttering are discussed.

MESSAGE COMPLEXITY AND ATTENTION TO TELEVISION

Three experiments investigated the processing costs of watching television messages. Processing costs were indexed with a secondary task reaction time measure in which subjects were asked to pay attention to commercial messages while responding with button presses to randomly occuring tones or flashes. Response time to the secondary tasks was used as a measure of attention to the primary task (watching the messages). Audio and video complexity of the messages were within-subject variables, and the channels presented to subjects (audio-only, video-only, or both) was a between-subjects variable. Results indicated that: (1) for a tone secondary task, multiple-channel presentations demanded more capacity than single-channel presentations (video or audio channel only); (2) more capacity was required to process simple video and auditory information than complex information; and (3) complexity of information in an absent channel (e.g., visual information in the audio-only condition) produced the same slowed reaction times as those occurring when the channel was present.

ISSUE VERSUS IMAGE ORIENTATION AND EFFECTS OF POLITICAL ADVERTISING

Melding uses and gratifications and information-processing perspectives, an experiment was constructed to investigate how media channel and audience motivation for attending to a political advertisement govern the processing and effect of the ad. A 2×2 factorial design (N=153) was used. Subjects were instructed to attend to a political ad “to learn where the candidate stands on issues” (issue motivational set) or “to form an impression of the candidate's personality” (image motivational set). In addition, subjects either viewed a two-channel political television ad (audio-video condition) or listened only to the audio portion (audio-only condition). Subjects attending to learn issue information were most likely to become informed and were more confident in their learning than were subjects attending to form a personality impression. Those attending to form a personality impression paid greater attention to the video portion of the ad and had greater confidence in their recall of video information. They were also more likely than were issue attenders to base their inferences of the candidate's traits on information contained in the ad rather than on assumptions about how traits are interrelated. No differences by channel condition were found.

The Affective Sensitivity Test as a Function of Stimulus Modality: Does Complete Information Hinder Performance?

The study investigated performance on the Affective Sensitivity Test as a function of the stimulus modality. Subjects were administered one of four variations of the test: (1) the standard sound film version, (2) the audio but not the visual portion of the film, (3) the visual but not the audio portion of the film, (4) the test booklet with neither the visual nor audio portion. Contrary to prediction, the standard administration (audio plus visual) resulted in significantly lower scores than did either the minimal information-booklet-only condition or the audio-only condition. Several explanations of these findings are discussed; data suggest the need for more research on the effect of different types of information on affective sensitivity.

MEDIUM OF COMMUNICATION, DIFFERENTIAL POWER, AND PHASING OF CONCESSIONS: NEGOTIATING SUCCESS AND ATTRIBUTIONS TO THE OPPONENT

The effect of three variables thought to be important to the negotiation process was investigated via an attributional analysis. Three different communication modes (audio, audio/video, face-to-face), three levels of power (high, equal, low), and three prior concession-phasing strategies (alternating, increasingly cooperative, decreasingly cooperative) were combined in a factorial design to determine their effect on negotiation outcomes and negotiator attributions. The data indicated that the communication and power variables were the most potent, with the face-to-face communication mode producing the best joint outcomes, followed by the audio/video and audio-only conditions. The various “prior” concession strategies had little effect on subsequent negotiation outcomes. Several significant internal/external attributions of locus of causality were found, as were significantly different impression ratings according to conditions.