High-speed Videoendoscopy Data Research Articles

Deep Learning-Based Analysis of Glottal Attack and Offset Times in Adductor Laryngeal Dystonia.

Diagnosis of adductor laryngeal dystonia (AdLD) is challenging as it mimics voice features of other voice disorders. This could lead to misdiagnosis (or delayed diagnosis) and ineffective treatments of AdLD. This paper develops automated measurements of glottal attack time (GAT) and glottal offset time (GOT) from high-speed videoendoscopy (HSV) in connected speech as objective measures that can potentially facilitate the diagnosis of this disorder in the future. HSV data were recorded from vocally normal adults and patients with AdLD during the reading of the "Rainbow Passage" and six CAPE-V (Consensus Auditory-Perceptual Evaluation of Voice) sentences. A deep learning framework was designed and trained to segment the glottal area and detect the vocal fold edges in the HSV dataset. This automated framework allowed us to automatically measure and quantify the GATs and GOTs for the participants. Accordingly, a comparison was held between the obtained measurements among vocally normal speakers and those with AdLD. The automated framework was successfully developed and able to accurately segment the glottal area/edges. The precise automated measurements of GAT and GOT revealed minor, nonsignificant differences compared to the results of manual analysis-showing a strong correlation between the measures by the automated and manual methods. The results showed significant differences in the GAT values between the vocally normal subjects and AdLD patients, with larger variability in both the GAT and GOT measures in the AdLD group. The developed automated approach for GAT and GOT measurement can be valuable in clinical practice. These quantitative measurements can be used as meaningful biomarkers of the impaired vocal function in AdLD and help its differential diagnosis in the future.

Studying the glottal vibration onset and offset using laryngeal high-speed videoendoscopy in connected speech

Laryngeal high-speed videoendoscopy (HSV) is a powerful tool to study the detailed vibratory characteristics of the vocal folds. The use of HSV in connected speech enables us to study the transitory behaviors of the vocal folds during different vocalizations. The onset and offset of voicing contain critical information about the health and function of the vocal folds. In this work, HSV is used to study how the voicing begins/ends at the laryngeal level at the onset/offset of phonation. HSV data were obtained during production of connected speech from participants with non-pathological voices and patients with different neurogenic voice disorders. The data collection was performed using a Photron FASTCAM mini AX200 high-speed camera coupled with a flexible nasolaryngoscope. The glottal attack time and glottal offset time were measured at the onset and offset of different vocalizations, respectively, from the HSV data. The findings show differences in the onset and offset measures between the two groups of participants. The vocal fold vibratory behaviors will be discussed qualitatively and quantitatively using these measures. The use of HSV in connected speech could help us develop accurate voice assessment procedures to assess the vocal folds function during connected speech.

Laryngeal Imaging Study of Glottal Attack/Offset Time in Adductor Spasmodic Dysphonia during Connected Speech.

Adductor spasmodic dysphonia (AdSD) disrupts laryngeal muscle control during speech and, therefore, affects the onset and offset of phonation. In this study, the goal is to use laryngeal high-speed videoendoscopy (HSV) to measure the glottal attack time (GAT) and glottal offset time (GOT) during connected speech for normophonic (vocally normal) and AdSD voices. A monochrome HSV system was used to record readings of six CAPE-V sentences and part of the "Rainbow Passage" from the participants. Three raters visually analyzed the HSV data using a playback software to measure the GAT and GOT. The results show that the GAT was greater in the AdSD group than in the normophonic group; however, the clinical significance of the amount of this difference needs to be studied further. More variability was observed in both GATs and GOTs of the disorder group. Additionally, the GAT and GOT time series were found to be nonstationary for the AdSD group while they were stationary for the normophonic voices. This study shows that the GAT and GOT measures can be potentially used as objective markers to characterize AdSD. The findings will potentially help in the development of standardized measures for voice evaluation and the accurate diagnosis of AdSD.

Deep-Learning-Based Representation of Vocal Fold Dynamics in Adductor Spasmodic Dysphonia during Connected Speech in High-Speed Videoendoscopy

Adductor spasmodic dysphonia (AdSD) is a neurogenic dystonia, which causes spasms of the laryngeal muscles. This disorder mainly affects production of connected speech. To understand how AdSD affects vocal fold (VF) movements and hence, the speech signal, it is necessary to study VF kinematics during the running speech. This paper introduces an automated method for analysis of VF vibrations in AdSD using laryngeal high-speed videoendoscopy (HSV) in running speech. A monochrome HSV system was used to obtain video recordings from vocally normal individuals and AdSD patients during production of the six CAPE-V sentences and the "Rainbow Passage." A deep neural network was designed based on the UNet architecture. The network was developed for glottal area segmentation in HSV data providing a tool for quantitative analysis of VF vibrations in both norm and AdSD. The network was trained and validated using the manually labeled HSV frames. After training the network, the segmentation quality was quantitatively evaluated against visual analysis results of a test dataset including segregated HSV frames and a short sequence of VF vibrations in consecutive frames. The developed convolutional network was successfully trained and demonstrated an accurate segmentation on the testing dataset with a mean Intersection over Union (IoU) of 0.81 and a mean Boundary-F1 score of 0.93. Moreover, the visual assessment of the automated technique showed an accurate detection of the glottal edges/area in the HSV data even with challenging image quality and excessive laryngeal maneuvers of AdSD patients during the running speech. The introduced automated approach provides an accurate representation of the glottal edges/area during connected speech in HSV data for norm and AdSD patients. This method facilitates the development of HSV-based measures to quantify VF dynamics in AdSD. Using HSV to automatically analyze VF vibrations in AdSD can allow for understanding AdSD vocal mechanisms and characteristics.

A Deep Learning Approach for Quantifying Vocal Fold Dynamics During Connected Speech Using Laryngeal High-Speed Videoendoscopy.

Voice disorders are best assessed by examining vocal fold dynamics in connected speech. This can be achieved using flexible laryngeal high-speed videoendoscopy (HSV), which enables us to study vocal fold mechanics with high temporal details. Analysis of vocal fold vibration using HSV requires accurate segmentation of the vocal fold edges. This article presents an automated deep-learning scheme to segment the glottal area in HSV from which the glottal edges are derived during connected speech. Using a custom-built HSV system, data were obtained from a vocally healthy participant reciting the "Rainbow Passage." A deep neural network was designed for glottal area segmentation in the HSV data. A recently introduced hybrid approach by the authors was utilized as an automated labeling tool to train the network on a set of HSV frames, where the glottis region was automatically annotated during vocal fold vibrations. The network was then tested against manually segmented frames using different metrics, intersection over union (IoU), and Boundary F1 (BF) score, and its performance was assessed on various phonatory events on the HSV sequence. The designed network was successfully trained using the hybrid approach, without the need for manual labeling, and tested on the manually labeled data. The performance metrics showed a mean IoU of 0.82 and a mean BF score of 0.96. In addition, the evaluation assessment of the network's performance demonstrated an accurate segmentation of the glottal edges/area even during complex nonstationary phonatory events and when vocal folds were not vibrating, thus overcoming the limitations of the previous hybrid approach that could only be applied to the vibrating vocal folds. The introduced automated scheme guarantees accurate glottis representation in challenging color HSV data with lower image quality and excessive laryngeal maneuvers during all instances of connected speech. This facilitates the future development of HSV-based measures to assess the running vibratory characteristics of the vocal folds in speakers with and without voice disorder. https://doi.org/10.23641/asha.19798864.

Acoustic Voice Analysis during Phonation Onset, Measured from High-Speed Videoendoscopy in Connected Speech

Adductor spasmodic dysphonia (AdSD) is a neurological voice disorder, which affects the control of intrinsic muscles of the larynx during speech. Our previous research has shown that high-speed videoendoscopy (HSV) can be used to measure the glottal attack time for patients with AdSD during connected speech. This study builds upon that to analyze the acoustic signal during the phonation onset and prephonatory adjustments. The phonation onset events are quantified using the HSV data that were obtained simultaneously with the acoustic data. We analyze the power and energy waveform of the acoustic signal during vocal folds’ first contact, first oscillation, and transitory behaviors, extracted from high-temporal resolution HSV data. The data were obtained from five patients with AdSD and five vocally normal participants. The data were recorded while the participant produced: 2 productions of vowel /i/ at habitual pitch and loudness, 2 with a soft glottal attack and 2 with a hard glottal attack, six CAPE-V sentences, and the “Rainbow Passage.” The HSV system included a monochrome high-speed camera coupled with a flexible nasolaryngoscope. The results of this study could assist for more accurate clinical management of adductor spasmodic dysphonia.

Detection of Vocal Fold Image Obstructions in High-Speed Videoendoscopy During Connected Speech in Adductor Spasmodic Dysphonia: A Convolutional Neural Networks Approach

Adductor spasmodic dysphonia (AdSD) is a neurogenic voice disorder, affecting the intrinsic laryngeal muscle control. AdSD leads to involuntary laryngeal spasms and only reveals during connected speech. Laryngeal high-speed videoendoscopy (HSV) coupled with a flexible fiberoptic endoscope provides a unique opportunity to study voice production and visualize the vocal fold vibrations in AdSD during speech. The goal of this study is to automatically detect instances during which the image of the vocal folds is optically obstructed in HSV recordings obtained during connected speech. HSV data were recorded from vocally normal adults and patients with AdSD during reading of the "Rainbow Passage", six CAPE-V sentences, and production of the vowel /i/. A convolutional neural network was developed and trained as a classifier to detect obstructed/unobstructed vocal folds in HSV frames. Manually labelled data were used for training, validating, and testing of the network. Moreover, a comprehensive robustness evaluation was conducted to compare the performance of the developed classifier and visual analysis of HSV data. The developed convolutional neural network was able to automatically detect the vocal fold obstructions in HSV data in vocally normal participants and AdSD patients. The trained network was tested successfully and showed an overall classification accuracy of 94.18% on the testing dataset. The robustness evaluation showed an average overall accuracy of 94.81% on a massive number of HSV frames demonstrating the high robustness of the introduced technique while keeping a high level of accuracy. The proposed approach can be used for efficient analysis of HSV data to study laryngeal maneuvers in patients with AdSD during connected speech. Additionally, this method will facilitate development of vocal fold vibratory measures for HSV frames with an unobstructed view of the vocal folds. Indicating parts of connected speech that provide an unobstructed view of the vocal folds can be used for developing optimal passages for precise HSV examination during connected speech and subject-specific clinical voice assessment protocols.

Comparative analysis of high-speed videolaryngoscopy images and sound data simultaneously acquired from rigid and flexible laryngoscope: a pilot study

High-Speed Videoendoscopy (HSV) is becoming a robust tool for the assessment of vocal fold vibration in laboratory investigation and clinical practice. We describe the first successful application of flexible High Speed Videoendoscopy with innovative laser light source conducted in clinical settings. The acquired image and simultaneously recorded audio data are compared to the results obtained by means of a rigid endoscope. We demonstrated that the HSV recordings with fiber-optic laryngoscope have enabled obtaining consistently bright, color images suitable for parametrization of vocal fold oscillation similarly as in the case of the HSV data obtained from a rigid laryngoscope. The comparison of period and amplitude perturbation parameters calculated on the basis of image and audio data acquired from flexible and rigid HSV recording objectively confirm that flexible High-Speed Videoendoscopy is a more suitable method for examination of natural phonation. The HSV-based measures generated from this kymographic analysis are arguably a superior representation of the vocal fold vibrations than the acoustic analysis because their quantification is independent of the vocal tract influences. This experimental study has several implications for further research in the field of HSV application in clinical assessment of glottal pathologies nature and its effect on vocal folds vibrations.

A Hybrid Machine-Learning-Based Method for Analytic Representation of the Vocal Fold Edges during Connected Speech.

Investigating the phonatory processes in connected speech from high-speed videoendoscopy (HSV) demands the accurate detection of the vocal fold edges during vibration. The present paper proposes a new spatio-temporal technique to automatically segment vocal fold edges in HSV data during running speech. The HSV data were recorded from a vocally normal adult during a reading of the “Rainbow Passage.” The introduced technique was based on an unsupervised machine-learning (ML) approach combined with an active contour modeling (ACM) technique (also known as a hybrid approach). The hybrid method was implemented to capture the edges of vocal folds on different HSV kymograms, extracted at various cross-sections of vocal folds during vibration. The k-means clustering method, an ML approach, was first applied to cluster the kymograms to identify the clustered glottal area and consequently provided an initialized contour for the ACM. The ACM algorithm was then used to precisely detect the glottal edges of the vibrating vocal folds. The developed algorithm was able to accurately track the vocal fold edges across frames with low computational cost and high robustness against image noise. This algorithm offers a fully automated tool for analyzing the vibratory features of vocal folds in connected speech.

Spatial Segmentation for Laryngeal High-Speed Videoendoscopy in Connected Speech

This study proposes a new computational framework for automated spatial segmentation of the vocal fold edges in high-speed videoendoscopy (HSV) data during connected speech. This spatio-temporal analytic representation of the vocal folds enables the HSV-based measurement of the glottal area waveform and other vibratory characteristics in the context of running speech. HSV data were obtained from a vocally normal adult during production of the "Rainbow Passage." An algorithm based on an active contour modeling approach was developed for the analysis of HSV data. The algorithm was applied on a series of HSV kymograms at different intersections of the vocal folds to detect the edges of the vibrating vocal folds across the frames. This edge detection method follows a set of deformation rules for the active contours to capture the edges of the vocal folds through an energy optimization procedure. The detected edges in the kymograms were then registered back to the HSV frames. Subsequently, the glottal area waveform was calculated based on the area of the glottis enclosed by the vocal fold edges in each frame. The developed algorithm successfully captured the edges of the vocal folds in the HSV kymograms. This method led to an automated measurement of the glottal area waveform from the HSV frames during vocalizations in connected speech. The proposed algorithm serves as an automated method for spatial segmentation of the vocal folds in HSV data in connected speech. This study is one of the initial steps toward developing HSV-based measures to study vocal fold vibratory characteristics and voice production mechanisms in norm and disorder in the context of connected speech.

Influence of spatial camera resolution in high-speed videoendoscopy on laryngeal parameters.

In laryngeal high-speed videoendoscopy (HSV) the area between the vibrating vocal folds during phonation is of interest, being referred to as glottal area waveform (GAW). Varying camera resolution may influence parameters computed on the GAW and hence hinder the comparability between examinations. This study investigates the influence of spatial camera resolution on quantitative vocal fold vibratory function parameters obtained from the GAW. In total 40 HSV recordings during sustained phonation (20 healthy males and 20 healthy females) were investigated. A clinically used Photron Fastcam MC2 camera with a frame rate of 4000 fps and a spatial resolution of 512×256 pixels was applied. This initial resolution was reduced by pixel averaging to (1) a resolution of 256×128 and (2) to a resolution of 128×64 pixels, yielding three sets of recordings. The GAW was extracted and in total 50 vocal fold vibratory parameters representing different features of the GAW were computed. Statistical analyses using SPSS Statistics, version 21, was performed. 15 Parameters showing strong mathematical dependencies with other parameters were excluded from the main analysis but are given in the Supporting Information. Data analysis revealed clear influence of spatial resolution on GAW parameters. Fundamental period measures and period perturbation measures were the least affected. Amplitude perturbation measures and mechanical measures were most strongly influenced. Most glottal dynamic characteristics and symmetry measures deviated significantly. Most energy perturbation measures changed significantly in males but were mostly unaffected in females. In females 18 of 35 remaining parameters (51%) and in males 22 parameters (63%) changed significantly between spatial resolutions. This work represents the first step in studying the impact of video resolution on quantitative HSV parameters. Clear influences of spatial camera resolution on computed parameters were found. The study results suggest avoiding the use of the most strongly affected parameters. Further, the use of cameras with high resolution is recommended to analyze GAW measures in HSV data.

Influence of Analyzed Sequence Length on Parameters in Laryngeal High-Speed Videoendoscopy

Laryngeal high-speed videoendoscopy (HSV) allows objective quantification of vocal fold vibratory characteristics. However, it is unknown how the analyzed sequence length affects some of the computed parameters. To examine if varying sequence lengths influence parameter calculation, 20 HSV recordings of healthy females during sustained phonation were investigated. The clinical prevalent Photron Fastcam MC2 camera with a frame rate of 4000 fps and a spatial resolution of 512 × 256 pixels was used to collect HSV data. The glottal area waveform (GAW), describing the increase and decrease of the area between the vocal folds during phonation, was extracted. Based on the GAW, 16 perturbation parameters were computed for sequences of 5, 10, 20, 50 and 100 consecutive cycles. Statistical analysis was performed using SPSS Statistics, version 21. Only three parameters (18.8%) were statistically significantly influenced by changing sequence lengths. Of these parameters, one changed until 10 cycles were reached, one until 20 cycles were reached and one, namely Amplitude Variability Index (AVI), changed between almost all groups of different sequence lengths. Moreover, visually observable, but not statistically significant, changes within parameters were observed. These changes were often most prominent between shorter sequence lengths. Hence, we suggest using a minimum sequence length of at least 20 cycles and discarding the parameter AVI.

Temporal Segmentation for Laryngeal High-Speed Videoendoscopy in Connected Speech

This study proposes a gradient-based method for temporal segmentation of laryngeal high-speed videoendoscopy (HSV) data obtained during connected speech. A custom-developed HSV system coupled with a flexible fiberoptic nasolaryngoscope was used to record one vocally normal female participant during reading of the "Rainbow Passage." A gradient-based algorithm was developed to generate a motion window. When applied to the HSV data, the motion window acted as a filter tracking the location of the vibrating vocal folds. The glottal area waveform was estimated using a statistical-based image-processing approach. The vocal fold vibratory frequency was computed by an autocorrelation-based extraction of the fundamental frequency (f0) from the glottal area waveform. Temporal segmentation was then performed based on the f0 contour and automatic detection of the epiglottic obstructions. Additionally, visual temporal segmentation was performed by viewing the HSV images frame by frame to determine the time points of the vocalization onsets and offsets, and the epiglottic obstructions of the glottis. The time points resulting from the automatic and visual temporal segmentation methods were cross-validated. The f0-contour patterns of rise and fall resulting from the automatic algorithm were found to be in agreement with the visual inspection of the vibratory frequency change in the HSV data. This study demonstrated the feasibility of automatic temporal segmentation of HSV imaging of connected speech, which allows for mapping the video content into onsets, offsets, and epiglottic obstructions for each vocalization. Automated analysis of HSV imaging of connected speech has significant clinical potential for advancing instrumental voice assessment protocols.

Advanced Waveform Decomposition for High-Speed Videoendoscopy Analysis

This article presents a novel approach to analyze nonperiodic vocal fold behavior of high-speed videoendoscopy (HSV) data. Although HSV can capture true vibrational motions of the vocal folds, its clinical advantage over the videostroboscopy has not widely been accepted. One of the key advantages of the HSV over the videostroboscopy is its ability to capture vocal folds' nonperiodic behavior, which is more prominent in pathological vocal folds. However, such nonperiodicity in the HSV data has not been fully explored quantitatively beyond simple perturbation analysis. This article presents an advanced waveform modeling and decomposition technique for HSV-based waveforms. Waveforms are modeled to have three components: harmonic signal, deterministic nonharmonic signal, and random nonharmonic signal. This decomposition is motivated by the fact that voice disorders introduce signal content that is nonharmonic but carries deterministic quality such as subharmonic or modulating content. The proposed model is aimed to isolate such disordered behaviors as deterministic nonharmonic signal and quantify them. In addition to the model, the article outlines model parameter estimation procedures and a family of harmonics-to-noise ratio (HNR) parameters. The proposed HNR parameters include harmonics-to-deterministic-noise ratio (HDNR) and harmonics-to-random-noise ratio. A preliminary study demonstrates the effectiveness of the extended model and its HNR parameters. Vocal folds with and without benign lesions (Nwith=13; Nwithout=20) were studied with HSV glottal area waveforms. All three HNR parameters significantly distinguished the disordered condition, and the HDNR reported the largest effect size (Cohen's d=2.04).

Preprocessing Techniques for High-Speed Videoendoscopy Analysis

One of the critical requirements for high-speed videoendoscopy (HSV) to become a clinically useful tool is to pair it with a technique, which provides a quick overview of the vast amount of HSV data and rapidly identifies the best video segments for subjective and objective analyses. This article proposes intensity-based representations that are easily computed from the HSV data and can be used to identify the HSV features quickly. The first representation-termed the Quick Vibratory Profile (QVP)-is an HSV-based one-dimensional waveform that captures the vocal fold vibration as well as nonglottic activities. The QVP can be used in a wide range of experimental and clinical studies to select appropriate HSV recording segments quickly without extensive review of the actual video frames. Moreover, this article proposes a pair of spatial profiles to locate the vibrating vocal folds within the HSV frames. These profiles are useful in automation of objective assessments as their use together with the QVP are demonstrated in a proposed cyclewise three-dimensional glottal area segmentation. The article illustrates the usefulness of these proposed representations with examples.

Mitigation of temporal aliasing via harmonic modeling of laryngeal waveforms in high-speed videoendoscopy

High-speed videoendoscopy (HSV) enables the observation and measurement of vibratory behaviors of vocal folds by capturing the laryngeal imagery at high frame rates. The frame rates of commercially available HSVs, however, are still limited to carry out sample-based time-domain objective analyses. To mitigate the resulting lack of temporal resolution, existing studies have employed sum-of-harmonics parametric models to evaluate temporal vocal-fold behaviors. This paper focuses on the other weakness of HSV: its inherent susceptibility to temporal aliasing. Aliasing occurs when there are substantial harmonics above the Nyquist frequency of the HSV camera, and video cameras offer very little means to filter out these harmonics. Although the aliasing effect in HSV data minimally affects many of the laryngeal objective parameter measurements, some parameters, such as harmonics-to-noise ratio and derivative-based parameters, are sensitive to the aliased content. The use of a parametric model with a careful selection of the number of harmonics enables classification of the aliased harmonics as a part of the harmonic signal. Glottal area waveform examples are included to illustrate the modeling capability for cases of normal and disordered vocal folds.

Endoscope Motion Compensation for Laryngeal High-Speed Videoendoscopy

The purpose of this study was to develop and evaluate a method for automatic endoscopic motion compensation (MC) specialized for laryngeal high-speed videoendoscopy (HSV) of sustained phonation. Specifically, the left-right and posterior-anterior endoscopic shifts were addressed. The method is based on the hypothesis that the difference between endoscopic and vocal fold dynamics is sufficient for accurate estimation of the endoscopic motion relative to the vocal folds. The research questions of particular interest were as follows: is MC of laryngeal HSV possible at the subpixel level, and what are the validity, reliability, and accuracy of the proposed MC method? First, the MC accuracy was assessed on simulated HSV data with exactly known motion trajectories. Then, the accuracy and reliability of MC were evaluated on real HSV data. Finally, the validity of MC was established by correlating objective to perceptual ratings. The method demonstrated subpixel accuracy and reliable performance when tested on both simulated and real data. Perceptual ratings of the endoscopic motion demonstrated high intrarater and interrater reliabilities. Perceptual and quantitative assessments of endoscopic motion were in agreement. The reported result is encouraging for resolving a persistent problem toward building better methods for visual and quantitative voice assessment.