Speech Encoding Research Articles

Non-native speech sound learning as a window to individual differences in the neural encoding of speech

Non-native speech sound learning in adulthood is a skill marked by enormous individual differences, especially when compared to acquisition of speech sounds in childhood. This has led to the proposal that adult speech sound acquisition might be achieved through different mechanisms (or relying on fundamentally altered neural circuitry) compared to sound learning in infancy and childhood. One possibility we explore in a recent line of work is that individual differences in non-native speech learning are attributable to differences in the structure of brain areas responsible for processing speech, especially the transverse temporal gyrus. Of interest, the morphology of these same regions also predicts individual differences in native language speech perception, suggesting that sound acquisition (whether in childhood or adulthood) is limited by (likely innate) differences in neural architecture supporting sound processing. However, evidence that the best non-native perceivers are also the most finely-tuned native-sound perceivers is limited. Taken together, these results suggest that differences in brain structure are a soft constraint on the precision of acoustic-phonetic encoding, but that native language speech category structure is not the explanation for the difficulty of non-native sound learning.

Brainstem speech encoding is dynamically shaped online by fluctuations in cortical [formula omitted] state

Experimental evidence in animals demonstrates cortical neurons innervate subcortex bilaterally to tune brainstem auditory coding. Yet, the role of the descending (corticofugal) auditory system in modulating earlier sound processing in humans during speech perception remains unclear. Here, we measured EEG activity as listeners performed speech identification tasks in different noise backgrounds designed to tax perceptual and attentional processing. We hypothesized brainstem speech coding might be tied to attention and arousal states (indexed by cortical α power) that actively modulate the interplay of brainstem-cortical signal processing. When speech-evoked brainstem frequency-following responses (FFRs) were categorized according to cortical α states, we found low α FFRs in noise were weaker, correlated positively with behavioral response times, and were more ”decodable” via neural classifiers. Our data provide new evidence for online corticofugal interplay in humans and establish that brainstem sensory representations are continuously yoked to (i.e., modulated by) the ebb and flow of cortical states to dynamically update perceptual processing.

The effect of child development on the components of the Frequency Following Response: Child development and the Frequency Following Response.

During childhood, neuronal modifications occur so that typical childhood communicative development occurs. This work aims to contribute to the understanding of differences in the speech encoding of infants and school-age children by assessing the effects of child development, in different phases of early childhood, on the encoding of speech sounds. There were 98 subjects of both sexes, aged from 1 day to 8 years and 9 months who participated in the study. All subjects underwent a Frequency Following Response (FFR) assessment. A regression and linear correlation showed the effects of age in the FFR components, i.e., significant decrease in the latency and increased amplitude of all FFR waves with age. An increase in the slope measure was also observed. Younger infants require more time and show less robust responses when encoding speech than their older counterparts, which were shown to have more stable and well-organized FFR responses.

Interaction of bottom-up and top-down neural mechanisms in spatial multi-talker speech perception

How the human auditory cortex represents spatially separated simultaneous talkers and how talkers' locations and voices modulate the neural representations of attended and unattended speech are unclear. Here, we measured the neural responses from electrodes implanted in neurosurgical patients as they performed single-talker and multi-talker speech perception tasks. We found that spatial separation between talkers caused a preferential encoding of the contralateral speech in Heschl's gyrus (HG), planum temporale (PT), and superior temporal gyrus (STG). Location and spectrotemporal features were encoded in different aspects of the neural response. Specifically, the talker's location changed the mean response level, whereas the talker's spectrotemporal features altered the variation of response around response's baseline. These components were differentially modulated by the attended talker's voice or location, which improved the population decoding of attended speech features. Attentional modulation due to the talker's voice only appeared in the auditory areas with longer latencies, but attentional modulation due to location was present throughout. Our results show that spatial multi-talker speech perception relies upon a separable pre-attentive neural representation, which could be further tuned by top-down attention to the location and voice of the talker.

Frequency-Following Response to Steady-State Vowel in Quiet and Background Noise Among Marching Band Participants With Normal Hearing

Human studies enrolling individuals at high risk for cochlear synaptopathy (CS) have reported difficulties in speech perception in adverse listening conditions. The aim of this study is to determine if these individuals show a degradation in the neural encoding of speech in quiet and in the presence of background noise as reflected in neural phase-locking to both envelope periodicity and temporal fine structure (TFS). To our knowledge, there are no published reports that have specifically examined the neural encoding of both envelope periodicity and TFS of speech stimuli (in quiet and in adverse listening conditions) among a sample with loud-sound exposure history who are at risk for CS. Using scalp-recorded frequency-following response (FFR), the authors evaluated the neural encoding of envelope periodicity (FFRENV) and TFS (FFRTFS) for a steady-state vowel (English back vowel /u/) in quiet and in the presence of speech-shaped noise presented at +5- and 0 dB SNR. Participants were young individuals with normal hearing who participated in the marching band for at least 5 years (high-risk group) and non-marching band group with low-noise exposure history (low-risk group). The results showed no group differences in the neural encoding of either the FFRENV or the first formant (F1) in the FFRTFS in quiet and in noise. Paradoxically, the high-risk group demonstrated enhanced representation of F2 harmonics across all stimulus conditions. These results appear to be in line with a music experience-dependent enhancement of F2 harmonics. However, due to sound overexposure in the high-risk group, the role of homeostatic central compensation cannot be ruled out. A larger scale data set with different noise exposure background, longitudinal measurements with an array of behavioral and electrophysiological tests is needed to disentangle the nature of the complex interaction between the effects of central compensatory gain and experience-dependent enhancement.

The Role of Efferent Reflexes in the Efficient Encoding of Speech by the Auditory Nerve.

To avoid information loss, the auditory system must adapt the broad dynamic range of natural sounds to the restricted dynamic range of auditory nerve fibers. How it solves this dynamic range problem is not fully understood. Recent electrophysiological studies showed that dynamic-range adaptation occurs at the auditory nerve level, but the amount of adaptation found was insufficient to prevent information loss. We used the physiological MATLAB Auditory Periphery model to study the contribution of efferent reflexes to dynamic range adaptation. Simulating the healthy human auditory periphery provided adaptation predictions that suggest that the acoustic reflex shifts rate-level functions toward a given context level and the medial olivocochlear reflex sharpens the response of nerve fibers around that context level. A simulator of hearing was created to decode model-predicted firing of the auditory nerve back into an acoustic signal, for use in psychophysical tasks. Speech reception thresholds in noise obtained with a normal-hearing implementation of the simulator were just 1 dB above those measured with unprocessed stimuli. This result validates the simulator for speech stimuli. Disabling efferent reflexes elevated thresholds by 4 dB, reaching thresholds found in mild-to-moderately hearing-impaired individuals. Overall, our studies suggest that efferent reflexes may contribute to overcoming the dynamic range problem. Because specific sensorineural pathologies can be inserted in the model, the simulator can be used to obtain the psychophysical signatures of each pathology, thereby laying a path to differential diagnosis.SIGNIFICANCE STATEMENT The saturation of auditory nerve fibers at moderate sound levels seen in rate-level functions challenges our understanding of how sounds of wide dynamic range are encoded. Our physiologically inspired simulations suggest that efferent reflexes may play a major role in dynamic range adaptation, with the acoustic reflex moving auditory nerve rate-level function toward a given context level and the medial olivocochlear reflex increasing fiber sensitivity around that context level. A psychophysical task using advanced simulations showed how the existence of the efferent system could prevent unrecoverable information loss and severe impairment of speech-in-noise intelligibility. These findings illustrate how important the precise modeling of peripheral compression is to both simulations and the understanding of normal and impaired hearing.

Distributed storage scheme for encryption speech data based on blockchain and IPFS

Distributed storage scheme for encryption speech data based on blockchain and IPFS

Language experience during the sensitive period narrows infants' sensory encoding of lexical tones-Music intervention reverses it.

The sensitive period for phonetic learning (6∼12 months), evidenced by improved native speech processing and declined non-native speech processing, represents an early milestone in language acquisition. We examined the extent that sensory encoding of speech is altered by experience during this period by testing two hypotheses: (1) early sensory encoding of non-native speech declines as infants gain native-language experience, and (2) music intervention reverses this decline. We longitudinally measured the frequency-following response (FFR), a robust indicator of early sensory encoding along the auditory pathway, to a Mandarin lexical tone in 7- and 11-months-old monolingual English-learning infants. Infants received either no intervention (language-experience group) or music intervention (music-intervention group) randomly between FFR recordings. The language-experience group exhibited the expected decline in FFR pitch-tracking accuracy to the Mandarin tone, while the music-intervention group did not. Our results support both hypotheses and demonstrate that both language and music experiences alter infants’ speech encoding.

A novel distortion-tolerant speech encryption scheme for secure voice communication

This paper details a novel distortion-tolerant speech encryption scheme for securing voice communications over digital voice-specific channels such as 3G/4G or VoIP voice calls. The scheme relies on the usual “loudness, pitch and timbre” parametric representation of speech signals but seen as points on various perceptually-continuous manifolds linked by quasi-isometric continuous mappings. The ciphering of the vocal parameters of the initial speech is done by provably-secure, pseudo-random continuous transforms on these manifolds and hence ensures resilience against moderate errors. The enciphered parameters are then re-encoded into a pseudo-speech signal and transmitted over lossy digital voice channels with compression. Upon reception of the distorted pseudo-speech signal, the legitimate receiver retrieves corrupted versions of the initial vocal parameters and reconstructs an intelligible speech signal, perceptually close to the original, using a machine-learning vocoder derived from the LPCNet.The important parts of our scheme are extensively tested through simulations and experiments with encrypted pseudo-speech signals between different mobile phones. Furthermore, we detail a thorough speech quality subjective assessment with a group of 44 participants that demonstrates how the proposed scheme maintains intelligibility with tolerable progressive quality degradation linked to channel distortion. Finally, a preliminary computational analysis is done to justify the feasibility of real-time implementation of our scheme on portable devices.

Transcranial Direct Current Stimulation Combined With Listening to Preferred Music Alters Cortical Speech Processing in Older Adults.

Emerging evidence suggests transcranial direct current stimulation (tDCS) can improve cognitive performance in older adults. Similarly, music listening may improve arousal and stimulate subsequent performance on memory-related tasks. We examined the synergistic effects of tDCS paired with music listening on auditory neurobehavioral measures to investigate causal evidence of short-term plasticity in speech processing among older adults. In a randomized sham-controlled crossover study, we measured how combined anodal tDCS over dorsolateral prefrontal cortex (DLPFC) paired with listening to autobiographically salient music alters neural speech processing in older adults compared to either music listening (sham stimulation) or tDCS alone. EEG assays included both frequency-following responses (FFRs) and auditory event-related potentials (ERPs) to trace neuromodulation-related changes at brainstem and cortical levels. Relative to music without tDCS (sham), we found tDCS alone (without music) modulates the early cortical neural encoding of speech in the time frame of ∼100–150 ms. Whereas tDCS by itself appeared to largely produce suppressive effects (i.e., reducing ERP amplitude), concurrent music with tDCS restored responses to those of the music+sham levels. However, the interpretation of this effect is somewhat ambiguous as this neural modulation could be attributable to a true effect of tDCS or presence/absence music. Still, the combined benefit of tDCS+music (above tDCS alone) was correlated with listeners’ education level suggesting the benefit of neurostimulation paired with music might depend on listener demographics. tDCS changes in speech-FFRs were not observed with DLPFC stimulation. Improvements in working memory pre to post session were also associated with better speech-in-noise listening skills. Our findings provide new causal evidence that combined tDCS+music relative to tDCS-alone (i) modulates the early (100–150 ms) cortical encoding of speech and (ii) improves working memory, a cognitive skill which may indirectly bolster noise-degraded speech perception in older listeners.

A Combined Chaotic System for Speech Encryption

This paper presents a speech encryption scheme by performing a combination of modified chaotic maps inspired by classic logistic and cubic maps. The main idea was to enhance the performance of classical chaotic maps by extending the range of the chaotic parameter. The resulted combining map was applied to a speech encryption scheme by using the confusion and diffusion architecture. The evaluation results showed a good performance regarding the chaotic behaviors such as initial value, control parameter, Lyapunov exponent, and bifurcation diagram. Simulations and computer evaluations with security analysis showed that the proposed chaotic system exhibits excellent performance in speech encryption against various attacks. The results obtained demonstrated the efficiency of the proposed scheme compared to an existing valuable method for static and differential cryptographic attacks.

The Relationship between Contralateral Suppression of Transient Evoked Otoacoustic Emission and Unmasking of Speech Evoked Auditory Brainstem Response

Introduction Several studies have shown that efferent pathways of the auditory system improve perception of speech-in-noise. But, the majority of investigations assessing the role of efferent pathways on speech perception have used contralateral suppression of otoacoustic emissions as a measure of efferent activity. By studying the effect of efferent activity on the speech-evoked auditory brainstem response (ABR), some more light could be shed on the effect of efferent pathways on the encoding of speech in the auditory pathway. Objectives To investigate the relationship between contralateral suppression of transient evoked otoacoustic emission (CSTEOAE) and unmasking of speech ABR. Methods A total of 23 young adults participated in the study. The CSTEOAE was measured using linear clicks at 60 dB peSPL and white noise at 60 dB sound pressure level (SPL). The speech ABR was recorded using the syllable /da/ at 80 dB SPL in quiet, ipsilateral noise, and binaural noise conditions. In the ipsilateral noise condition, white noise was presented to the test ear at 60 dB SPL, and, in the binaural noise condition, two separate white noises were presented to both ears. Results The F0 amplitude of speech ABR was higher in quiet condition; however, the mean amplitude of F0 was not significantly different across conditions. Correlation analysis showed a significant positive correlation between the CSTEOAE and the magnitude of unmasking of F0 amplitude of speech ABR. Conclusions The findings of the present study suggests that the efferent pathways are involved in speech-in-noise processing.

Research on the effectiveness of noise reduction when encoding a lossless speech signal

In the meantime, speech coding is one of the methods to represent the digital speech signal as in possible fewer bits value and to maintain the quality and its clearness. In omnipresent situations, encryption and examination of speech maintain a crucial role in various acoustic-based coding systems. This paper, using subband and Huffman coding technique, has been used for speech signals description to reduce the occupied by the speech data memory. The amplitude values of the taken speech are segregated after pre-processing, windowing and decomposition techniques. These data are converted into the frequency domain using discrete cosine transform (DCT). Then 90 foremost coefficients have been coded by Huffman method, they contain the most valuable information of speech signals. Signals are segregated then and subband coding techniques applied. To reconstruct the input speech, the taken speech is re-transformed in the form of time-domain applying through inverse discrete cosine transform (IDCT). This experiment is carried out by speech data at 8 kHz with 16 bits/per sample. The SNR (Signal to Noise Ratio) shows the efficiency of this applied technique.

Effects of Spectral Shaping on Speech Auditory Brainstem Responses to Stop Consonant-Vowel Syllables.

Spectral shaping is employed by hearing aids to make consonantal information, such as formant transitions, audible for listeners with hearing loss. How manipulations of the stimuli, such as spectral shaping, may alter encoding in the auditory brainstem has not been thoroughly studied. The aim of this study was to determine how spectral shaping of synthetic consonant-vowel (CV) syllables, varying in their second formant (F2) onset frequency, may affect encoding of the syllables in the auditory brainstem. We employed a repeated measure design. Sixteen young adults (mean = 20.94 years, 6 males) and 11 older adults (mean = 58.60 years, 4 males) participated in this study. Speech-evoked auditory brainstem responses (speech-ABRs) were obtained from each participant using three CV exemplars selected from synthetic stimuli generated for a /ba-da-ga/ continuum. Brainstem responses were also recorded to corresponding three CV exemplars that were spectrally shaped to decrease low-frequency information and provide gain for middle and high frequencies according to a Desired Sensation Level function. In total, six grand average waveforms (3 phonemes [/ba/, /da/, /ga/] X 2 shaping conditions [unshaped, shaped]) were produced for each participant. Peak latencies and amplitudes, referenced to prestimulus baseline, were identified for 15 speech-ABR peaks. Peaks were marked manually using the program cursor on each individual waveform. Repeated-measures analysis of variances were used to determine the effects of shaping on the latencies and amplitudes of the speech-ABR peaks. Shaping effects produced changes within participants in ABR latencies and amplitudes involving onset and major peaks of the speech-ABR waveform for certain phonemes. Specifically, data from onset peaks showed that shaping decreased latency for /ga/ in older listeners, and decreased amplitude onset for /ba/ in younger listeners. Shaping also increased the amplitudes of major peaks for /ga/ stimuli in both groups. Encoding of speech in the ABR waveform may be more complex and multidimensional than a simple demarcation of source and filter information, and may also be influenced by cue intensity and age. These results suggest a more complex subcortical encoding of vocal tract filter information in the ABR waveform, which may also be influenced by cue intensity and age.

Encrypted Speech Retrieval Scheme Based on Multiuser Searchable Encryption in Cloud Storage

In order to solve the problem of multiuser security sharing and privacy protection of the speech data in cloud storage and realize efficient encrypted speech retrieval, an encrypted speech retrieval scheme based on multiuser searchable encryption was proposed. Firstly, the ciphertext-policy attribute-based encryption (CP-ABE) and searchable encryption (SE) are combined to support a multiuser searchable speech encryption scheme, which achieves the encryption and fine-grained access control of the speech data. Secondly, the Mel frequency cepstral coefficient (MFCC) feature of the original speech that is used as the input of the long- and short-term memory network (LSTM) is extracted to perform the deep semantic feature extraction as the speech keywords. Finally, the speech keywords are encrypted to generate a secure index, bound together with the encrypted speech, and then stored in the cloud. During the user retrieving, the keywords of query speech are extracted by utilizing the trained LSTM to generate the search trapdoor of the user and then uploaded to the cloud server, and the Euclidean distance is used for matching the security index with the search trapdoor. In addition, a proxy server is introduced to execute the partial ciphertext decryption operations to reduce computation overhead and storage space. The theoretical analysis and experimental results show that the proposed scheme has higher security and retrieval accuracy and can realize the secure storage of the massive speech data and multiuser data sharing.

Firing Rate Adaptation of the Human Auditory Nerve Optimizes Neural Signal-to-Noise Ratios.

Several physiological mechanisms act on the response of the auditory nerve (AN) during acoustic stimulation, resulting in an adjustment in auditory gain. These mechanisms include-but are not limited to-firing rate adaptation, dynamic range adaptation, the middle ear muscle reflex, and the medial olivocochlear reflex. A potential role of these mechanisms is to improve the neural signal-to-noise ratio (SNR) at the output of the AN in real time. This study tested the hypothesis that neural SNRs, inferred from non-invasive assessment of the human AN, improve over the duration of acoustic stimulation. Cochlear potentials were measured in response to a series of six high-level clicks embedded in a series of six lower-level broadband noise bursts. This paradigm elicited a compound action potential (CAP) in response to each click and to the onset of each noise burst. The ratio of CAP amplitudes elicited by each click and noise burst pair (i.e., neural SNR) was tracked over the six click/noise bursts. The main finding was a rapid (< 24ms) increase in neural SNR from the first to the second click/noise burst, consistent with a real-time adjustment in the response of the auditory periphery toward improving the SNR of the signal transmitted to the brainstem. Analysis of cochlear microphonic and ear canal sound pressure recordings, as well as the time course for this improvement in neural SNR, supports the conclusion that firing rate adaptation is likely the primary mechanism responsible for improving neural SNR, while dynamic range adaptation, the middle ear muscle reflex, and the medial olivocochlear reflex played a secondary role on the effects observed in this study. Real-time improvements in neural SNR are significant because they may be essential for robust encoding of speech and other relevant stimuli in the presence of background noise.

Digital Speech Files Encryption based on Hénon and Gingerbread Chaotic Maps

Speech encryption approaches are used to prevent eavesdropping, tracking, and other security concerns in speech communication. In this paper, a new cryptography algorithm is proposed to encrypt digital speech files. Initially, the digital speech files are rearranged as a cubic model with six sides to scatter speech data. Furthermore, each side is encrypted by random keys that are created by using two chaotic maps (Hénon and Gingerbread chaotic maps). Encryption for each side of the cube is achieved, using the based map vector that is generated randomly by using a simple random function. Map vector that consists of six bits, each bit refers to one of the specific chaotic maps that generate a random key to encrypt each face of the cube. Results show that the pseudo-random keys created by using chaotic maps for cryptographic speech file have an acceptable characteristic concerning randomness tests, which is confirmed in this paper by using five statistical tests. The final evaluation of the speech encryption algorithm is measured by using different quality metrics, and the results show that the algorithm can achieve resist encryption.

Content-based encrypted speech retrieval scheme with deep hashing

Content-based encrypted speech retrieval scheme with deep hashing

Chaotic elliptic map for speech encryption

Using a new key management system and Jacobian elliptic map, a new speech encryption scheme has been developed for secure speech communication data. Jacobian elliptic map-based speech encryption has been developed as a novel method to improve the existing speech encryption methods' drawbacks, such as poor quality in decrypted signals, residual intelligibility, high computational complexity, and low-key space. Using the Jacobian elliptic map as a key management solution, a new cryptosystem was created. The proposed scheme's performance is evaluated using spectrogram analysis, histogram analysis, key space analysis, correlation analysis, key sensitivity analysis and randomness test analysis. Using the results, we can conclude that the proposed speech encryption scheme provides a better security system with robust decryption quality.

Speech encryption using hybrid-hyper chaotic system and binary masking technique

Speech encryption using hybrid-hyper chaotic system and binary masking technique