Speech Sources Research Articles

The Influence of Speed on Error Rates in Simultaneous Interpreting: An Observational Study (English to Arabic)

In simultaneous interpreting, the rate of source speech (SR) plays an important role in the quality of the interpreter. As the rate of speech increases, interpreters have to process and translate words faster, which can lead to more errors. This phenomenon is because faster speech harms the interpreter's attention and work, causing more omissions, additions, and other linguistic and paralinguistic errors. The impact of these errors can affect the accuracy and quality of interpretation, which highlights the need to understand how changes in speech rate affect performance and the production of good strategies to solve problems. This study examines how the rate of speech (SR) affects the quality of simultaneous interpreting (SI), particularly to assess the impact of the rate of speech on the interpreter. For this purpose, a method was adopted that was divided into seven categories based on Barik's (1971) classification. These categories include segment omission, word and phrase-level omission, deviation of meaning, addition, incomplete sentence, full pause, and long pause. In this study, the performance of six United Nations interpreters with many years of experience was evaluated using six audios ranging from 124 to 164 words per minute (wpm). Findings indicated that higher speaking frequency led to more errors, especially in word and segment omissions. This emphasizes the importance of addressing specific strategies to overcome the challenges that rapid speech can pose to the accuracy and effectiveness of simultaneous interpreting.

Development of an experimental technique for measuring polar diagrams of adaptive directivity implemented in hearing aids

Nowadays, hearing aids have efficient signal processing options to improve speech intelligibility in noise. This work concerns the characterization of the microphone systems directivity available in hearing aids. In recent decades, "adaptive" directivities have been implemented in hearing aids, thus making it possible to focus on useful sound sources in the hearing aid patient environment. These options use the principles of sound environments detection, acoustic beamforming and noise reduction. Performances of these options can be estimated with subjective methods (as speech audiometry in noise), and objective methods (listening effort, intelligibility indicators as HASPI, SII, STI, etc.). But these methods provide an overall assessment of all the processing involved in speech enhancement and are not designed to characterize the effects of adaptive directivity. We developed an experimental technique based on sound sources separation with phase opposition diffusion (known as "Hagerman" method) to draw polar diagrams of adaptive directivity, for realistic sound environments. We used a sound multi-diffusion system to generate various sound environments with noise and speech sources around an artificial head equipped with hearing aids. The first results show the adaptive directivity effects associated with noise reduction provided by the latest hearing aids generation on everyday hearing scenes.

Objective validation of alternative speech test sources for person-to-person speech intelligibility assessments

A sound source that mimics the acoustics characteristics of a human talker is required in objective assessments of speech intelligibility in person-to-person speech communications (PP) scenarios. The few special speech test sources available that meet the strict specifications of the relevant standards are costly and can be inaccessible to practitioners. This can prevent investigations or can lead to non-special and more affordable loudspeakers to be used as alternative sources without knowledge of their suitability. This study evaluates the validity of non-special loudspeakers in PP assessments and supplements a previous preliminary examination by consolidating and expanding early findings. Speech Transmission Index Public Address (STIPA) assessments were conducted employing three representative non-special loudspeakers and one standardized reference speech test source under laboratory conditions. Absolute errors of STIPA measurements were analysed and evaluated against two just perceivable difference (JND) threshold levels. The mean absolute errors obtained for all non-special loudspeakers for almost all scenarios were within one JND of any of the two JND levels. The striking and consistent low level of error found validated the proposition that non-special and affordable loudspeakers may be used as suitable speech sources in pilot or survey-grade speech intelligibility assessments in place of a special standardized speech source when that source is not available, or in full investigations when the relevant standard to adhere to allows the use of those alternative speech sources.

Cognitive load in remote simultaneous interpreting: place name translation in two Mandarin variants

This study examined the accuracy of place name translation during remote simultaneous interpreting (RSI) when interpreters were required to use a different variant of their native language. Participants, professional English/ Mandarin from mainland China, were divided into two groups: the Taiwan group, using the Taiwanese Mandarin variant, and the control group, using the mainland variant. The source speech consisted of three lists of place names, some of which had the same Chinese translation in both the mainland China Mandarin variant and the Taiwanese Mandarin variant, while others did not. The two groups of participants performed RSI from English into Chinese. The analysis focused on the third item in each list, chosen due to the differences in Chinese translations between the two Mandarin variants. Results showed no significant difference in the number of incorrect translations for the first test item between the two groups. However, significant variations were observed for the second and third test items, potentially linked to progressive mental fatigue and increased cognitive demand faced by the Taiwan group participants. This suggests that using a non-habitual language variant can impact the accuracy of RSI.

Head-orienting behaviors during simultaneous speech detection and localization.

Head movement plays a vital role in auditory processing by contributing to spatial awareness and the ability to identify and locate sound sources. Here we investigate head-orienting behaviors using a dual-task experimental paradigm to measure: (a) localization of a speech source; and (b) detection of meaningful speech (numbers), within a complex acoustic background. Ten younger adults with normal hearing and 20 older adults with mild-to-severe sensorineural hearing loss were evaluated in the free field on two head-movement conditions: (1) head fixed to the front and (2) head moving to a source location; and two context conditions: (1) with audio only or (2) with audio plus visual cues. Head-tracking analyses quantified the target location relative to head location, as well as the peak velocity during head movements. Evaluation of head-orienting behaviors revealed that both groups tended to undershoot the auditory target for targets beyond 60° in azimuth. Listeners with hearing loss had higher head-turn errors than the normal-hearing listeners, even when a visual location cue was provided. Digit detection accuracy was better for the normal-hearing than hearing-loss groups, with a main effect of signal-to-noise ratio (SNR). When performing the dual-task paradigm in the most difficult listening environments, participants consistently demonstrated a wait-and-listen head-movement strategy, characterized by a short pause during which they maintained their head orientation and gathered information before orienting to the target location.

Transducer Variability in Speech-in-Noise Testing: Considerations Related to Stimulus Bandwidth.

Clinical audiologists typically assume that headphones and insert phones will produce comparable results when they are used to conduct speech-in-noise or other audiological tests; however, this may not always be the case. Here, we show that there are significant differences in the scores that previous studies have reported for headphone and insert-phone transducers on the Words-in-Noise (WIN) Test, and discuss the possibility that the variations in high-frequency output that are allowable under the speech source specifications of American National Standards Institute S3.6 might be contributing to transducer-dependent differences in performance for the WIN and other tests that are presented through the auxiliary input channels of clinical audiometers. A literature review was conducted to identify articles that reported WIN Test results for both listeners with normal hearing and with hearing impairment and specified the type of transducer (insert or TDH-50) used for the data collection. Among the 19 included studies, participants with normal hearing using inserts exhibited systematically worse WIN Test scores compared to those using TDH-50 headphones, while participants with hearing loss showed comparable average scores across transducer types. The results highlight the importance of considering transducer type when interpreting WIN Test outcomes, particularly when comparing to normative scores obtained from individuals with normal hearing. Although further research is needed to elucidate the underlying mechanisms driving differences in test performance across transducer types, these findings underscore the need for standardized test administration protocols and careful documentation of transducer type when administering speech-in-noise tests for clinical or research applications.

A developmental model of audio-visual attention (MAVA) for bimodal language learning in infants and robots

A social individual needs to effectively manage the amount of complex information in his or her environment relative to his or her own purpose to obtain relevant information. This paper presents a neural architecture aiming to reproduce attention mechanisms (alerting/orienting/selecting) that are efficient in humans during audiovisual tasks in robots. We evaluated the system based on its ability to identify relevant sources of information on faces of subjects emitting vowels. We propose a developmental model of audio-visual attention (MAVA) combining Hebbian learning and a competition between saliency maps based on visual movement and audio energy. MAVA effectively combines bottom-up and top-down information to orient the system toward pertinent areas. The system has several advantages, including online and autonomous learning abilities, low computation time and robustness to environmental noise. MAVA outperforms other artificial models for detecting speech sources under various noise conditions.

Investigating the impact of visual access on trainee interpreters’ simultaneous interpreting performance

ABSTRACT Visual access has long been considered crucial for interpreters, providing essential kinesic cues such as gaze and facial expressions that enhance understanding and emphasis. Despite its acknowledged importance, empirical studies examining the impact of visual access on interpreters’ performance remain limited. To date, conclusive evidence establishing a direct link between visual access and interpreters’ performance is lacking. This study addresses this gap by employing a data-driven approach to explore the influence of visual components in source speeches on interpreting performance. A total of 48 graduate interpreter trainees (first language/L1: Chinese, second language/L2: English) participated in English-to-Chinese simultaneous interpreting tasks under both video and audio-only conditions. Two raters (L1: Chinese, L2: English) assessed their performances using three criteria: fidelity, target language quality (language), and fluency of delivery (delivery). Data analysis revealed that, overall, participants exhibited superior performance under the video condition (M = 4.92, SD = 1.62) compared to the audio-only condition (M = 4.19, SD = 1.67). Specifically, their performance in both fidelity and delivery was notably higher under the video condition (fidelity: M = 5.12, SD = 1.59; delivery: M = 4.58, SD = 0.23) than under the audio condition (fidelity: M = 4.38, SD = 1.63; delivery: M = 4.00, SD = 0.20). However, no statistically significant difference was observed in language performance between the two conditions. The implications are discussed at the end.

Saudi Interpreters’ Adaptive Cognitive Strategic Behaviors in Consecutive Interpretation

This study investigates the adaptive cognitive strategies used by Saudi interpreters in consecutive interpretation (CI) guided by Gile’s (2009) Effort Models. Using a structured questionnaire, insights were gathered from 102 Saudi interpreters, divided into undergraduate students and professional interpreters with varying levels of experience. The research focuses on how interpreters manage the selection of interpretation strategies across four phases of CI: listening, note-taking, note-decoding, and reformulating. Notable findings include the contrast between experienced professionals, who seldom rely on common sense, omitting, or paraphrasing, and trainees, who employ these strategies more frequently in response to unfamiliar topics. Professionals, regardless of experience, effectively use paraphrasing when faced with numerical complexities. When dealing with fast-speaking speakers, experienced professionals avoid omitting content, while trainees rely on this strategy. In conclusion, to manage concentration during long speeches, trainees prioritize attention to the source speech and use selective omission, while experienced professionals gravitate toward generalizing and summarizing approaches. This research highlights the influence of experience on strategy choice, aligning with Jääskeläinen’s (1996) claim that experienced translators allocate more cognitive resources to production strategies than novices. A key recommendation of this study is the need for specialized training for novice interpreters, particularly in strategy development, to handle various CI process challenges. Training focused on enhancing note-taking skills is vital, especially for trainees, as clear and understandable notes are crucial for success in the note-decoding and reformulating stages of CI.

Visual processing during computer-assisted consecutive interpreting

Abstract This study investigates the visual processing patterns during computer-assisted consecutive interpreting (CACI). In phase I of the proposed CACI workflow, the interpreter listens to the source speech and respeaks it into speech recognition (SR) software. In phase II, the interpreter produces target speech supported by the SR text and its machine translation (MT) output. A group of students performed CACI with their eye movements tracked. In phase I, the participants devoted the majority of their attention to listening and respeaking, with very limited attention distributed to the SR text. However, a positive correlation was found between the percentage of dwell time on the SR text and the quality of respeaking, which suggests that active monitoring could be important. In phase II, the participants devoted more visual attention to the MT text than to the SR text and engaged in deeper and more effortful processing when reading the MT text. We identified a positive correlation between the percentage of dwell time on the MT text and interpreting quality in the L2–L1 direction but not in the L1–L2 direction. These results contribute to our understanding of computer-assisted interpreting and can provide insights for future research and training in this area.

Effects of note-taking on the accuracy and fluency of consecutive interpreters' immediate free recall of source texts: A three-stage developmental study

Note-taking for consecutive interpreting (CI) is a special type of note-taking in that selective notes are taken in a highly multi-tasking condition under time pressure during a CI speech segment but are used to enable a complete recall of the source speech immediately. Given the specialty of interpreters' note-taking, it is still unclear in what ways note-taking affects interpreters' memory performance. Moreover, the effect of note-taking is theoretically assumed to depend on interpreting expertise, which has not been substantiated by empirical research. This study addresses these questions by investigating the effect of note-taking on the accuracy and fluency of interpreters' memory of the source text at different training stages. An oral recall experiment was administered to student interpreters at three critical note-taking training stages: the initial stage (with one-month training in note-taking), after the critical development stage (with six-month training in note-taking), and the advanced stage (with two-year training in note-taking), with a sample size of 53, 53 and 35 at each stage respectively. Participants listened to and immediately recalled speech segments consecutively in two note-taking conditions: with and without note-taking. Recall accuracy was measured by completeness of information recalled, and recall fluency was measured by oral fluency during recall. Results showed that note-taking significantly enhanced both accuracy and fluency of interpreters' recall of the source text at all the three stages, revealing a facilitative effect of note-taking on source text memory. The size of the note-taking effect increased as interpreting training experience accumulated, indicating that interpreting training enhanced the facilitative role of note-taking. However, the developmental trajectories of the note-taking effect on recall accuracy and fluency differed. As interpreting training proceeded, the note-taking effect on recall fluency was first improved (from Stage 1 to Stage 2), and later was the note-taking effect on recall accuracy (from Stage 1 to Stage 3). The findings shed light on the cognitive mechanisms of how note-taking functions for consecutive interpreters. A potential practical implication for interpreter training is that a step-by-step note-taking training approach may be beneficial.

Coherence and comprehensibility in second language speakers’ academic speaking performance

AbstractThis study examined the role of discourse organization in second language (L2) comprehensibility ratings. Twelve English for Academic Purposes teachers listened to 60 L2 speech samples elicited through a TOEFL–type integrated speaking task, evaluating each sample for comprehensibility and coherence (perceived interconnectedness of ideas). The samples were analyzed for the occurrence of discourse features at micro and macro levels. Results revealed a strong association between coherence and comprehensibility (r = .70). Whereas L2 speakers’ use of additive connectives (e.g., and) uniquely predicted comprehensibility, ordering of ideas and source–speech similarity in speakers’ performances predicted coherence. Lexical overlaps predicted both constructs. Findings underscore the importance of coherence to comprehensible academic L2 speech demonstrating that the two constructs include partially overlapping yet distinct characteristics.

WaveVC: Speech and Fundamental Frequency Consistent Raw Audio Voice Conversion

Voice conversion (VC) is a task for changing the speech of a source speaker to the target voice while preserving linguistic information of the source speech. The existing VC methods typically use mel-spectrogram as both input and output, so a separate vocoder is required to transform mel-spectrogram into waveform. Therefore, the VC performance varies depending on the vocoder performance, and noisy speech can be generated due to problems such as train-test mismatch. In this paper, we propose a speech and fundamental frequency consistent raw audio voice conversion method called WaveVC. Unlike other methods, WaveVC does not require a separate vocoder and can perform VC directly on raw audio waveform using 1D convolution. This eliminates the issue of performance degradation caused by the train-test mismatch of the vocoder. In the training phase, WaveVC employs speech loss and F0 loss to preserve the content of the source speech and generate F0 consistent speech using the pre-trained networks. WaveVC is capable of converting voices while maintaining consistency in speech and fundamental frequency. In the test phase, the F0 feature of the source speech is concatenated with a content embedding vector to ensure the converted speech follows the fundamental frequency flow of the source speech. WaveVC achieves higher performances than baseline methods in both many-to-many VC and any-to-any VC. The converted samples are available online.

Effective Monoaural Speech Separation through Convolutional Top-Down Multi-View Network

Speech separation, sometimes known as the “cocktail party problem”, is the process of separating individual speech signals from an audio mixture that includes ambient noises and several speakers. The goal is to extract the target speech in this complicated sound scenario and either make it easier to understand or increase its quality so that it may be used in subsequent processing. Speech separation on overlapping audio data is important for many speech-processing tasks, including natural language processing, automatic speech recognition, and intelligent personal assistants. New speech separation algorithms are often built on a deep neural network (DNN) structure, which seeks to learn the complex relationship between the speech mixture and any specific speech source of interest. DNN-based speech separation algorithms outperform conventional statistics-based methods, although they typically need a lot of processing and/or a larger model size. This study presents a new end-to-end speech separation network called ESC-MASD-Net (effective speaker separation through convolutional multi-view attention and SuDoRM-RF network), which has relatively fewer model parameters compared with the state-of-the-art speech separation architectures. The network is partly inspired by the SuDoRM-RF++ network, which uses multiple time-resolution features with downsampling and resampling for effective speech separation. ESC-MASD-Net incorporates the multi-view attention and residual conformer modules into SuDoRM-RF++. Additionally, the U-Convolutional block in ESC-MASD-Net is refined with a conformer layer. Experiments conducted on the WHAM! dataset show that ESC-MASD-Net outperforms SuDoRM-RF++ significantly in the SI-SDRi metric. Furthermore, the use of the conformer layer has also improved the performance of ESC-MASD-Net.

Dentistry in Ancient Egypt

Dentistry, dated in Ancient Egypt as an independent profession since the early 3rd millennium BC, was the first specialty recorded in a document in ancient Egypt, but nothing proves teeth interventions in mummies. The mouth and dental preservation was very important in rituals and theology because, as a source of exterior speech, it allowed man to express his spiritual feelings. The emphasis put on the mouth, lips, tongue, palate and teeth, was connected with the soul, the expression, and articulation of words. In the ‘Opening of the mouth’ ceremony, the dead jaws were freed to rebirth, give a voice and utter the words and names of power. Thus the white and healthy child Horus’ teeth allow to make up the undamaged mouth (PT 79 a) and five onions, their symbolic representation, were given the dead as a sacred offering. The emphasis put on the mouth, lips, tongue, palate and teeth, was connected with the soul, the expression, and articulation of words. As source of exterior speech, they allow man to express his spiritual feelings. In the Memphite theology the intellectual God Creator Ptah have generated the world pronouncing their names. Teeth and lips represent the magical verbal action and the tongue means speech, the _logos_. Dental problems were well known and some indications of doctor’s different diagnosis and treatments were found in medical papyri. The most common dental diseases were: attrition, frequent until pulp exposure, receding gums, caries and the destruction of enamel, chronic suppurative periodontitis that causes mobility, dental abscess and sepsis, loose teeth, ulcerative stomatitis, alveolar diseases and heavy deposits of calculus. Egyptian paleopathology and molecular biology provide new medical information and had revolutionized all aspects of dentistry, changing previous beliefs. However Egyptians were creative or witty and had devised many treatments for the various affections, both chirurgical and medicinal in nature, using different drugs. By studying a critical scrutiny of the development of dentistry profession in ancient times, it has been possible an exploration of dental problems, diseases and treatments and establishing guidelines and difficulties.

Dialogue between the Lines: Deaf and Hearing Interpreters' Interaction during Intralingual Co-Interpretation

Abstract: This study focuses on the practice of simultaneous cointerpreting in the media, specifically examining the interaction between a hearing interpreter (HI) and a deaf interpreter (DI). The HI's role in this context is to interpret the spoken source speech in Belgian French into French Belgian Sign Language (LSFB) for the DI, who, in turn, conveys the discourse to a deaf audience. The main objective of this research is to explore the phenomenon of adjustment within the HI-DI interaction and the strategies used by the HIs to overcome the inherent challenges of this configuration. To do so, the study analyzes a collection of recorded press conferences related to the COVID-19 crisis, featuring four HI-DI teams. By analyzing a set of video data from press conferences, this study aims to identify the types of information exchanged during the interpretation process, such as specific concepts, meta-comments, the speaker's intentions, and so on. Additionally, the study aims to identify the strategies used by the HIs throughout the process. This analysis reveals the complex dynamics of the HI-DI interaction, shedding light on the HIs' efforts to convey content and meaning from the source speech, as well as their use of metalinguistic indications to make effective adjustments. This article focuses on three specific strategies employed by the HIs: compression strategies, whose goal is to provide the DI with time and space to reformulate the (pre)interpreted message; explicitation strategies, used to make explicit logical connections and prosodic information in the discourse; and interactional signaling strategies, which facilitate interaction between the interpreters.

Broad and fine acoustic categories in bod, bond, bald, and bard: A step toward acoustic phonology

Acoustics is central to the study of speech communication, but it is conspicuously under-represented in abstract representations of speech. Many flavors of phonological analysis tend toward articulatory descriptions, and transcriptions focus on strings of articulatory actions. All this is despite acoustics being easier to measure than articulation with current technology. The present study explores basic concepts for an acoustic phonology, with two types of postulated categories: broad and fine. Resonant, turbulent, transient, and occludent types of sounds comprise the broad categories, as general methods of filtering the speech source. Fine categories are conceptualized as specific types of acoustic actions within a broad category. These acoustic actions are goal-oriented, as for achieving a particular acoustic effect like the presence of antiformants or a lowered F2 or F3. However, these actions are not explicitly restricted to manipulating traditional phonetic features like formants. By default, fine categories are assumed to be produced in parallel when possible, yielding overlap effects like anticipatory nasalization, lateralization, and rhotacization. These concepts are explored in a microanalysis of bod, bond, bald, and bard from the speaker in the Massive Auditory Lexical Decision data set, with an eye to seeding the ground for a future acoustic phonology.

Acoustic feature analysis of directional sound from parametric array speakers with application to mitigate attacks on voice-activated systems

A number of studies have identified the vulnerabilities of voice-activated devices to attacks from amplitude-demodulated sound from directional speakers. In this research, a controlled set of speech data, both live and played back from directional speakers, is analyzed to identify the acoustic features integral to speaker recognition in smart devices. A category of features, including Mel-frequency cepstral coefficients derived from Mel filter banks, are examined for potential to differentiate and classify speech sources. Potential vulnerabilities in voice-activated systems are systematically identified by examining the acoustic characteristics of diverse voice samples. This analytical approach aims to unveil potential weaknesses that could be exploited, triggering responses from these systems. The findings of this study contribute to the comprehension of the security landscape surrounding voice-activated technologies, paving the way for enhanced cybersecurity measures within this domain.

Relationship between attention and spatial processing abilities

Reduction in an individual’s hearing ability and their working memory (WM) capacity are hypothesized to be two of the major contributors to the decline in listener performance seen in complex listening environments. Generally, the improvement in speech intelligibility that may occur when a target is spatially separated from competing talkers is quantified as spatial release from masking (SRM). The goal of this study was to estimate working memory capacity based on a divided attention version of the classic spatial release from masking task (the temporal overlap task) and the classic abbreviated reading span task (aRST). Temporal overlap threshold was estimated by adaptively varying the maximum amount of temporal overlap of the signals where a listener could still correctly identify the speech source presented directly ahead of the listener when the speech material was either colocated or spatially separated. Initial analyses of the data revealed a strong relationship between SRM and temporal overlap thresholds, suggesting that SRM is driven by the listeners’ ability to modulate their attentional mechanisms. The relationship between SRM, temporal overlap thresholds, and reading span measures will also be discussed.

Speech detection models for effective communicable disease risk assessment in air travel environments

In environments characterized by elevated noise levels, such as airports or aircraft cabins, travelers often find themselves involuntarily speaking loudly and drawing closer to one another in an effort to enhance communication and speech intelligibility. Unfortunately, this unintentional behaviour increases the risk of respiratory particles dispersion, potentially carrying infectious agents like bacteria which makes the contagion control more challenging. The accurate characterization of the risk associated to speaking, in such a challenging noise environment with multiple overlapping speech sources, is therefore of outmost importance. Among the most advanced signal processing strategies that can be used to accurately determine who spoke when and with whom and for how long but most importantly how loudly, at one location, artificial intelligence-based speaker diarization approaches were considered and adapted for this task. This article details the implementation of speaker diarization algorithms, customized to extract speaker and speech parameters discreetly. Validation and preliminary study results are also provided. The algorithms calculate speech duration and sound pressure level for each sentence and speaker, aiding in assessing viral contaminant spread. The paper focuses on applying these algorithms in noisy environments, particularly in confined spaces with multiple speakers. The findings contribute to proactive measures for containing and managing communicable diseases in air travel settings.