Pitch Shift Research Articles

The Accompanying Effect in Responses to Auditory Perturbations: Unconscious Vocal Adjustments to Unperturbed Parameters.

The present study examined whether participants respond to unperturbed parameters while experiencing specific perturbations in auditory feedback. For instance, we aim to determine if speakers adjust voice loudness when only pitch is artificially altered in auditory feedback. This phenomenon is referred to as the "accompanying effect" in the present study. Thirty native Mandarin speakers were asked to sustain the vowel /ɛ/ for 3 s while their auditory feedback underwent single shifts in one of the three distinct ways: pitch shift (±100 cents; coded as PT), loudness shift (±6 dB; coded as LD), or first formant (F1) shift (±100 Hz; coded as FM). Participants were instructed to ignore the perturbations in their auditory feedback. Response types were categorized based on pitch, loudness, and F1 for each individual trial, such as Popp_Lopp_Fopp indicating opposing responses in all three domains. The accompanying effect appeared 93% of the time. Bayesian Poisson regression models indicate that opposing responses in all three domains (Popp_Lopp_Fopp) were the most prevalent response type across the conditions (PT, LD, and FM). The more frequently used response types exhibited opposing responses and significantly larger response curves than the less frequently used response types. Following responses became more prevalent only when the perturbed stimuli were perceived as voices from someone else (external references), particularly in the FM condition. In terms of isotropy, loudness and F1 tended to change in the same direction rather than loudness and pitch. The presence of the accompanying effect suggests that the motor systems responsible for regulating pitch, loudness, and formants are not entirely independent but rather interconnected to some degree.

Immersive auditory awareness: A smart earphones platform for education on noise-induced hearing risks

The Audio Research Platform developed by the ÉTS-EERS Industrial Research Chair in In-Ear technologies is a digital audio processing wearable device that features a powerful Digital Signal Processor and a pair of wired earplugs featuring outer- and inner-ear microphones as well as miniaturize loudspeaker. ARP has been used since over the last 2 decades by CRITIAS to develop new algorithms for advanced hearing protection and communication in noise, as well as in-ear audio sensing. In this adaptation, the ARP enables transparent hearing of ambient sounds, simulates varying degrees of hearing loss, and includes a tinnitus simulator. ARP also functions as a personal music player, monitoring playback levels and displaying the “Age of your ears”) a metric recently proposed by CRITIAS), predicting accelerated auditory aging from excessive music playback. In the playback mode, wearers experience pre-recorded noise, prompting vocal adjustments known as the “Lombard Effect.” ARP analyzes the wearer's voice using the outer-ear microphone, illustrating level and pitch shifts in speech due to a noisy environment. This paper details ARP's technical aspects, positioning it as a powerful tool to inform the public about noise-induced hearing loss and promote awareness and prevention through enjoyable real-time demonstrations.

Auditory imagery ability influences accuracy when singing with altered auditory feedback.

In this preliminary study, we explored the relationship between auditory imagery ability and the maintenance of tonal and temporal accuracy when singing and audiating with altered auditory feedback (AAF). Actively performing participants sang and audiated (sang mentally but not aloud) a self-selected piece in AAF conditions, including upward pitch-shifts and delayed auditory feedback (DAF), and with speech distraction. Participants with higher self-reported scores on the Bucknell Auditory Imagery Scale (BAIS) produced a tonal reference that was less disrupted by pitch shifts and speech distraction than musicians with lower scores. However, there was no observed effect of BAIS score on temporal deviation when singing with DAF. Auditory imagery ability was not related to the experience of having studied music theory formally, but was significantly related to the experience of performing. The significant effect of auditory imagery ability on tonal reference deviation remained even after partialling out the effect of experience of performing. The results indicate that auditory imagery ability plays a key role in maintaining an internal tonal center during singing but has at most a weak effect on temporal consistency. In this article, we outline future directions in understanding the multifaceted role of auditory imagery ability in singers' accuracy and expression.

Conventional Remedy to Lou Gehrig’s Disease - Amyotrophic Lateral Sclerosis (ALS): A Rare Clinical Entity

Lou Gehrig’s disease, commonly known as amyotrophic lateral sclerosis, is an incredibly rare clinical condition that affects about 5 in 100,000 people annually in India, making it one of the rarest diseases in the entire world. With an average age of 55, this disease typically impacts people between the ages of 40 and 70. Males are more prone to develop the condition. Amyotrophic Lateral Sclerosis (ALS) is a neurodegenerative disorder affecting primarily the motor system, but in which extramotor manifestations are increasingly recognized. The loss of upper and lower motor neurons in the motor cortex, the brain stem nuclei and the anterior horn of the spinal cord gives rise to progressive muscle weakness and wasting. Typically, patients report difficulties swallowing or a shift in voice pitch while speaking due to tongue involvement. Initial ALS symptoms can resemble those of other neuromuscular illnesses and are completely non-specific. In this case report, we described a 44-year-old gentleman who has been diagnosed with Amyotrophic Lateral Sclerosis in accordance to an MRI scan describing symptoms like unable to speak, reduced tongue movements, trouble swallowing, excessive salivation, swelling over the tongue and loss of tongue tone for the past three to four months. The present study discusses the clinical presentation, diagnosis, therapeutic regimen and prognosis. Both allopathic and ayurvedic oral medications with local therapeutic regimens like Jivha Pratisarana (Rubbing of dry powder on tongue) and Nasya (nasal drops of medicated oil) were employed in the patient which have satisfactory results enhancing for betterment of lifestyle.

Development of efficient techniques for ASR System for Speech Detection and Recognization system using Gaussian Mixture Model- Universal Background Model

Some practical uses of ASR have been implemented, including the transcription of meetings and the usage of smart speakers. It is the process by which speech waves are transformed into text that allows computers to interpret and act upon human speech. Scalable strategies for developing ASR systems in languages where no voice transcriptions or pronunciation dictionaries exist are the primary focus of this work. We first show that the necessity for voice transcription into the target language can be greatly reduced through cross-lingual acoustic model transfer when phonemic pronunciation lexicons exist in the new language. Afterwards, we investigate three approaches to dealing with languages that lack a pronunciation lexicon. Secondly, we have a look at the efficiency of graphemic acoustic model transfer, which makes it easy to build pronunciation dictionaries. Thesis problems can be solved, in part, by investigating optimization strategies for training on huge corpora (such as GA+HMM and DE+HMM). In the training phase of acoustic modelling, the suggested method is applied to traditional methods. Read speech and HMI voice experiments indicated that while each data augmentation strategy alone did not always increase recognition performance, using all three techniques together did. Power normalised cepstral coefficient (PNCC) features are tweaked somewhat in this work to enhance verification accuracy. To increase speaker verification accuracy, we suggest employing multiple “Gaussian Mixture Model-Universal Background Model (GMM-UBM) and SVM classifiers”. Importantly, pitch shift data augmentation and multi-task training reduced bias by more than 18% absolute compared to the baseline system for read speech, and applying all three data augmentation techniques during fine tuning reduced bias by more than 7% for HMI speech, while increasing recognition accuracy of both native and non-native Dutch speech.

Development of a Filtimator for Pediatric Image-Guided Radiation Therapy with Low Imaging Dose

We propose to use the filtimator (a filter and a collimator) for CBCT-based image-guided radiation therapy (IGRT) optimized for pediatric patients to minimize the imaging dose but not to sacrifice the imaging quality required for bony-based image fusion. The filtimator was made of Tin (0.6 - 1 mm) and Cu sheets (0.6 to 1.2mm) for filter and collimation with adjustable collimation with 4 to 6 cm fields at the isocenter. The thickness of the filter in the central and the peripheral regions were determined using vendor-provided image registration software and to ensure that the rigid image registration results from the CBCT were with and without the filtimator. The image quality of the filtered CBCT was compared with the regular CBCTs. Image registration accuracy was investigated by creating a < 2° shift in pitch, roll, and rotation and < 3 cm shifts in the lateral, vertical, and longitudinal directions, using commercial head and house-made phantoms. The imaging dose reduction factor of the filtimator was measured using a CT dose index phantom. The contrast-to-noise ratio substantially improved in the opening region of the filter, which provided better visualization of normal anatomy and target volumes. The slim filter reduced the imaging dose by > 98% in the filtered region, and the visualization of bony structures was well preserved, allowing for accurate rigid image registration with the filter. The imaging registration difference was < 0.2° shift in pitch, roll, and rotation and < 0.5 mm shift in the lateral, vertical, and longitudinal directions compared to the regular CBCT. The proposed dose reduction with the filtimator was demonstrated to be efficient and effective in considerably reducing the patient imaging dose while yielding accurate registration results. This novel technique can become a valuable tool for generating 3- and 4-dimensional images with a much-reduced dose to improve the precision of target localization in radiation therapy.

Simulation of hearing loss can induce pitch shifts for complex tones.

Most studies on pitch shift provoked by hearing loss have been conducted using pure tones. However, many sounds encountered in everyday life are harmonic complex tones. In the present study, psychoacoustic experiments using complex tones were performed on healthy participants, and the possible mechanisms that cause pitch shift due to hearing loss are discussed. Two experiments were performed in this study. In experiment 1, two tones were presented, and the participants were asked to select the tone that was higher in pitch. Partials with frequencies less than 250, 500, 750, or 1,000 Hz were eliminated from the harmonic complex tones and used as test tones to simulate low-tone hearing loss. Each tone pair was constructed such that the tone with a lower fundamental frequency (F0) was higher in terms of the frequency of the lowest partial. Furthermore, partials whose frequencies were greater than 1,300 or 1,600 Hz were also eliminated from these test tones to simulate high-tone hearing loss or modified sounds that patients may hear in everyday life. When a tone with a lower F0 was perceived as higher in pitch, it was considered a pitch shift from the expected tone. In experiment 2, tonal sequences were constructed to create a passage of the song "Lightly Row." Similar to experiment 1, partials of harmonic complex tones were eliminated from the tones. After listening to these tonal sequences, the participants were asked if the sequences sounded correct based on the melody or off-key. The results showed that pitch shifts and the melody sound off-key when lower partials are eliminated from complex tones, especially when a greater number of high-frequency components are eliminated. Considering that these experiments were performed on healthy participants, the results suggest that the pitch shifts from the expected tone when patients with hearing loss hear certain complex tones, regardless of the underlying etiology of the hearing loss.

Salient sounds distort time perception and production

The auditory world is often cacophonous, with some sounds capturing attention and distracting us from our goals. Despite the universality of this experience, many questions remain about how and why sound captures attention, how rapidly behavior is disrupted, and how long this interference lasts. Here, we use a novel measure of behavioral disruption to test predictions made by models of auditory salience. Models predict that goal-directed behavior is disrupted immediately after points in time that feature a high degree of spectrotemporal change. We find that behavioral disruption is precisely time-locked to the onset of distracting sound events: Participants who tap to a metronome temporarily increase their tapping speed 750 ms after the onset of distractors. Moreover, this response is greater for more salient sounds (larger amplitude) and sound changes (greater pitch shift). We find that the time course of behavioral disruption is highly similar after acoustically disparate sound events: Both sound onsets and pitch shifts of continuous background sounds speed responses at 750 ms, with these effects dying out by 1,750 ms. These temporal distortions can be observed using only data from the first trial across participants. A potential mechanism underlying these results is that arousal increases after distracting sound events, leading to an expansion of time perception, and causing participants to misjudge when their next movement should begin.

Data augmentation for speech separation

Deep learning models have advanced the state of the art of monaural speech separation. However, the performance of a separation model considerably decreases when tested on unseen speakers and noisy conditions. Separation models trained with data augmentation generalize better to unseen conditions. In this paper, we conduct a comprehensive survey of data augmentation techniques and apply them to improve the generalization of time-domain speech separation models. The augmentation techniques include seven source-preserving approaches (Gaussian noise, Gain, Time masking, frequency masking, Short noise, Time stretch, and Pitch shift) and three non-source preserving approaches (Dynamix mixing, Mixup, and Cutmix). After hyperparameter search for each augmentation method, we test the generalization of the augmented model by cross-corpus testing on three datasets (LibriMix, TIMIT, and VCTK), and identify the best augmentation combination that enhances generalization. Experimental results indicate that a combination of several non-source preserving strategies (CutMix, Mixup, and Dynamic mixing) resulted in the best generalization performance. Finally, the augmentation combinations also improved the performance of the speech separation model even when fewer training data are available.

Klasifikasi Suara Untuk Memonitori Hutan Berbasis Convolutional Neural Network

Forest has an important role on earth. The need to monitor forest from illegal activities and the types of animals in there is needed to keep the forest in good condition. However, the condition of the vast forest and limited resource make direct forest monitoring by officer (human) is limited. In this case, sound with digital signal processing can be used as a tool for forest monitoring. In this study, a system was implemented to classify sound on the Raspberry Pi 3B+ using mel-spectrogram. Sounds that classified are the sound of chainsaw, gunshot, and the sound of 8 species of bird. This study also compared pretrained VGG-16 and MobileNetV3 as feature extractor, and several classification methods, namely Random Forest, SVM, KNN, and MLP. To vary and increase the number of training data, we used several types of data augmentation, namely add noise, time stretch, time shift, and pitch shift. Based on the result of this study, it was found that the MobileNetV3-Small + MLP model with combined training data from time stretch and time shift augmentation provide the best performance to be implemented in this system, with an inference duration of 0.8 seconds; 93.96% accuracy; and 94.1% precision.

Acoustic Classification of Bird Species Using an Early Fusion of Deep Features

Bird sound classification plays an important role in large-scale temporal and spatial environmental monitoring. In this paper, we investigate both transfer learning and training from scratch for bird sound classification, where pre-trained models are used as feature extractors. Specifically, deep cascade features are extracted from various layers of different pre-trained models, which are then fused to classify bird sounds. A multi-view spectrogram is constructed to characterize bird sounds by simply repeating the spectrogram to make it suitable for pre-trained models. Furthermore, both mixup and pitch shift are applied for augmenting bird sounds to improve the classification performance. Experimental classification on 43 bird species using linear SVM indicates that deep cascade features can achieve the highest balanced accuracy of 90.94% ± 1.53%. To further improve the classification performance, an early fusion method is used by combining deep cascaded features extracted from different pre-trained models. The final best classification balanced accuracy is 94.89% ± 1.35%.

Identifying distinct latent classes of pitch-shift response consistency: Evidence from manipulating the predictability of shift direction.

Auditory feedback plays an important role in regulating our vocal pitch. When pitch shifts suddenly appear in auditory feedback, the majority of the responses are opposing, correcting for the mismatch between perceived pitch and actual pitch. However, research has indicated that following responses to auditory perturbation could be common. This study attempts to explore the ways individual speakers would respond to pitch perturbation (using an opposing response or a following response) from trial to trial. Thirty-six native speakers of Mandarin produced the vowel /a/ while receiving perturbed pitch at a random time (500 ~ 700 ms) after vocal onset for a duration of 200 ms. Three blocks of 30 trials that differed in the pitch-shift stimulus direction were recorded in a randomized order: (a) the down-only condition where pitch was shifted downwards 250 cents; (b) the up-only condition where pitch was shifted upwards 250 cents; and (c) the random condition where downshifts and upshifts occurred randomly and were equally likely. The participants were instructed to ignore the pitch shifts. Results from the latent class analysis show that at the individual level across trials, 57% of participants were switchers, 28% were opposers, and 15% were followers. Our results support that speakers produce a mix of opposing and following responses when they respond to perturbed pitch. Specifically, the proportion of followers was conditional on the expectancy of pitch-shift stimulus direction: More followers were observed when the pitch-shift stimulus direction was predictable. Closer inspection of the levels of response consistency in different time phases shows that a particular mechanism (opposing or following) was initially implemented; the two mechanisms may alternate in the middle phase; and then finally, the pitch-shift response was featured as a particular mechanism near the end phase.

Data Augmentation Based on Contrastive Learning in Audio Anomaly Detection

Based on the contrastive supervised learning algorithm, this paper studies the impact of the linear combination of data augmentation in audio anomaly detection. Previous studies have shown that the linear combination of data enhancement performs well in image detection. However, there are few studies on similar methods in sound anomaly detection, and this study makes up for this gap. We apply transformations to one branch of the algorithm with the audio augmentation module of the AugLy library, and the other branch is directly transformed into the Mel spectrogram. We find that more data augmentation patterns can help neural networks learn more high-quality representations, and a suitable linear data combination augmentation can improve the performance of model generalization at a low cost. The combination effect of Add Background Noise + Pitch Shift is the best. The AUC index of the fan dataset is increased by 10.06% on average, and other datasets are also improved to varying degrees.

Heritability of Sound Processing Deficits in Autism: Neural Insights

Heritability of Sound Processing Deficits in Autism: Neural Insights

Effect of stimulation parameters on sequential current-steered stimuli in cochlear implants.

Manipulation of cochlear implant (CI) place pitch was carried out with current steering by stimulating two CI electrodes sequentially. The objective was to investigate whether shifts in activated neural populations could be achieved to produce salient pitch differences and to determine which stimulation parameters would be more effective in steering of current. These were the pulse rate and pulse width of electrical stimuli and the distance between the two current-steering electrodes. Nine CI users participated, and ten ears were tested. The pattern of pitch changes was not consistent across listeners, but the data suggest that individualized selection of stimulation parameters may be used to effect place pitch changes with sequential current steering. Individual analyses showed that pulse width generally had little influence on the effectiveness of current steering with sequential stimuli, while more salient place pitch shifts were often achieved at wider electrode spacing or when the stimulation pulse rate was the same as that indicated on the clinical MAP (the set of stimulation parameters) of the listener. Results imply that current steering may be used in CIs that allow only sequential stimulation to achieve place pitch manipulation.

Comparison of Involuntary and Volitional Responses to Pitch-Shifted Auditory Feedback: Evidence for Tone Speakers' Flexibility to Switch Between Opposing and Following Responses.

Our audio-vocal system involves a negative feedback system that functions to correct for fundamental frequency (f 0) errors in production. Therefore, automatic and opposing responses appear when an unexpected change in voice pitch is present in auditory feedback. This study explores following responses to pitch perturbation in auditory feedback in tonal language speakers, which have been commonly overlooked or discarded by past research. We examine whether the number of response types (opposing vs. following) and their dynamic f 0 contours in tone word production vary as a function of instruction (involuntary ["to ignore"] vs. volitional ["to compensate"]). Twenty-four native speakers of Taiwanese Southern Min (TSM) produced three disyllabic TSM words while receiving pitch perturbation through headphones. The three disyllabic words were tsau55-im55 ("out of tune"; HH), kau33-uann33 ("exchange"; MM), and pan11-an11 ("handle a case"; LL) that carry an identical high-, mid-, or low-level tone. The participants were instructed either "to ignore" or "to compensate" for the pitch shifts. Results from a Bayesian Poisson regression show that the number of opposing and following responses were split nearly 50-50 for the "ignore" condition and 55%-35% for the "compensate" condition. The simulation results indicate that the speakers were able to switch between the feedback and the feedforward mode during the testing. On the other hand, f 0 contour analyses using generalized additive models show that pitch-increasing responses (i.e., oppose to downshifts or follow upshifts) were significantly larger than pitch-decreasing responses (i.e., oppose to upshifts or follow downshifts) for the MM and LL words, but not for the HH word. Our results provide support for the view that, for tone speakers, following responses are not uncommon. The magnitudes of pitch shift response trajectories have to do with the available pitch range for moving up or down in tone word production.

Carbamazepine induces upward frequency shifts of spontaneous otoacoustic emissions

In three subjects, we measured spontaneous otoacoustic emissions (SOAEs) when they were using carbamazepine (CBZ), and compared this to the SOAE measurement when they were not using CBZ. We observed 14 SOAEs showing a consistent upward shift of center frequency, related to CBZ intake. On average, the magnitude of the frequency shift increased with increasing frequency. The magnitude of the shift was 30–104 Hz, at frequencies ranging from 1.3 to 2.3 kHz, corresponding to a shift between 2.3 and 4.5%. Compared to other causes and manipulations known to change SOAE frequency, these shifts are relatively large. The underlying mechanism is most likely an increased stiffness of the cochlear partition. This would also explain the downward pitch shift due to CBZ, which has been reported by subjects with absolute pitch.

Feature-Based Fusion Using CNN for Lung and Heart Sound Classification.

Lung or heart sound classification is challenging due to the complex nature of audio data, its dynamic properties of time, and frequency domains. It is also very difficult to detect lung or heart conditions with small amounts of data or unbalanced and high noise in data. Furthermore, the quality of data is a considerable pitfall for improving the performance of deep learning. In this paper, we propose a novel feature-based fusion network called FDC-FS for classifying heart and lung sounds. The FDC-FS framework aims to effectively transfer learning from three different deep neural network models built from audio datasets. The innovation of the proposed transfer learning relies on the transformation from audio data to image vectors and from three specific models to one fused model that would be more suitable for deep learning. We used two publicly available datasets for this study, i.e., lung sound data from ICHBI 2017 challenge and heart challenge data. We applied data augmentation techniques, such as noise distortion, pitch shift, and time stretching, dealing with some data issues in these datasets. Importantly, we extracted three unique features from the audio samples, i.e., Spectrogram, MFCC, and Chromagram. Finally, we built a fusion of three optimal convolutional neural network models by feeding the image feature vectors transformed from audio features. We confirmed the superiority of the proposed fusion model compared to the state-of-the-art works. The highest accuracy we achieved with FDC-FS is 99.1% with Spectrogram-based lung sound classification while 97% for Spectrogram and Chromagram based heart sound classification.

Don't blame yourself: Conscious source monitoring modulates feedback control during speech production.

Sensory feedback plays an important role in speech motor control. One of the main sources of evidence for this is studies in which online auditory feedback is perturbed during ongoing speech. In motor control, it is therefore crucial to distinguish between sensory feedback and externally generated sensory events. This is called source monitoring. Previous altered feedback studies have taken non-conscious source monitoring for granted, as automatic responses to altered sensory feedback imply that the feedback changes are processed as self-caused. However, the role of conscious source monitoring is unclear. The current study investigated whether conscious source monitoring modulates responses to unexpected pitch changes in auditory feedback. During the first block, some participants spontaneously attributed the pitch shifts to themselves (self-blamers) while others attributed them to an external source (other-blamers). Before Block 2, all participants were informed that the pitch shifts were experimentally induced. The self-blamers then showed a reduction in response magnitude in Block 2 compared with Block 1, while the other-blamers did not. This suggests that conscious source monitoring modulates responses to altered auditory feedback, such that consciously ascribing feedback to oneself leads to larger compensation responses. These results can be accounted for within the dominant comparator framework, where conscious source monitoring could modulate the gain on sensory feedback. Alternatively, the results can be naturally explained from an inferential framework, where conscious knowledge may bias the priors in a Bayesian process to determine the most likely source of a sensory event.

Reliability and Efficiency of Pitch-Shifting Plug-Ins in Voice and Hearing Research.

Auditory feedback perturbation with voice pitch manipulation has been widely used in previous studies. There are several hardware and software tools for such manipulations, but audio plug-ins developed for music, movies, and radio applications that operate in digital audio workstations may be extremely beneficial and are easy to use, accessible, and cost effective. However, it is unknown whether these plug-ins can perform similarly to tools that have been described in previous literature. Hence, this study aimed to evaluate the reliability and efficiency of these plug-ins. Six different plug-ins were used at +1 and -1 st pitch shifting with formant correction on and off to pitch shift the sustained /ɑ/ voice recording sample of 12 healthy participants (six cisgender males and six cisgender females). Pitch-shifting accuracy, formant shifting amount, intensity changes, and total latency values were reported. Some variability was observed between different plug-ins and pitch shift settings. One plug-in managed to perform similarly in all four measured aspects with well-known hardware and software units with 1-cent pitch-shifting accuracy, low latency values, negligible intensity difference, and preserved formants. Other plug-ins performed similarly in some respects. Audio plug-ins may be used effectively in pitch-shifting applications. Researchers and clinicians can access these plug-ins easily and test whether the features also fit their aims.