Spectro-temporal Patterns Research Articles

Musician Advantage for Segregation of Competing Speech in Native Tonal Language Speakers

The aim of this study was to replicate previous English-language musician advantage studies in Mandarin-speaking musicians and nonmusicians. Segregation of competing speech, melodic pitch perception, and spectro-temporal pattern perception were measured in normal-hearing native Mandarin-speaking musicians and nonmusicians. Speech recognition thresholds were measured in the presence of two-talker masker speech. The masker sex was either the same as or different from the target; target and masker speech were either co-located or spatially separated. Melodic pitch perception was tested using a melodic contour identification task. Spectro-temporal resolution was measured using a modified spectral ripple detection task. We hypothesized that, given musician advantages in pitch perception, musician effects would be larger when the target and masker sex was the same than when different. For all tests, performance was significantly better for musicians than for nonmusicians. Contrary to our expectation, larger musician effects were observed for segregation of competing speech when the target and masker sex was different. The results show that musician effects observed for non-tonal language speakers extend to tonal language speakers. The data also suggest that musician effects may depend on the difficulty of the listening task and may be reduced when listening tasks are too easy or too difficult.

Experience with the cochlear implant enhances the neural tracking of spectrotemporal patterns in the Alberti bass

Cochlear implant (CI) users experience diminished music enjoyment due to the technical limitations of the CI. Nonetheless, behavioral studies have reported that rhythmic features are well-transmitted through the CI. Still, the gradual improvement of rhythm perception after the CI switch-on has not yet been determined using neurophysiological measures. To fill this gap, we here reanalyzed the electroencephalographic responses of participants from two previous mismatch negativity studies. These studies included eight recently implanted CI users measured twice, within the first six weeks after CI switch-on and approximately three months later; thirteen experienced CI users with a median experience of 7 years; and fourteen normally hearing (NH) controls. All participants listened to a repetitive four-tone pattern (known in music as Alberti bass) for 35 min. Applying frequency tagging, we aimed to estimate the neural activity synchronized to the periodicities of the Alberti bass. We hypothesized that longer experience with the CI would be reflected in stronger frequency-tagged neural responses approaching the responses of NH controls. We found an increase in the frequency-tagged amplitudes after only 3 months of CI use. This increase in neural synchronization may reflect an early adaptation to the CI stimulation. Moreover, the frequency-tagged amplitudes of experienced CI users were significantly greater than those of recently implanted CI users, but still smaller than those of NH controls. The frequency-tagged neural responses did not just reflect spectrotemporal changes in the stimuli (i.e., intensity or spectral content fluctuating over time), but also showed non-linear transformations that seemed to enhance relevant periodicities of the Alberti bass. Our findings provide neurophysiological evidence indicating a gradual adaptation to the CI, which is noticeable already after three months, resulting in close to NH brain processing of spectrotemporal features of musical rhythms after extended CI use.

Frequency detection, frequency discrimination, and spectro-temporal pattern perception in older and younger typically hearing adults

Elderly adults often experience difficulties in speech understanding, possibly due to age-related deficits in frequency perception. It is unclear whether age-related deficits in frequency perception differ between the apical or basal regions of the cochlea. It is also unclear how aging might differently affect frequency discrimination or detection of a change in frequency within a stimulus. In the present study, pure-tone frequency thresholds were measured in 19 older (61–74 years) and 20 younger (22–28 years) typically hearing adults. Participants were asked to discriminate between reference and probe frequencies or to detect changes in frequency within a probe stimulus. Broadband spectro-temporal pattern perception was also measured using the spectro-temporal modulated ripple test (SMRT). Frequency thresholds were significantly poorer in the basal than in the apical region of the cochlea; the deficit in the basal region was 2 times larger for the older than for the younger group. Frequency thresholds were significantly poorer in the older group, especially in the basal region where frequency detection thresholds were 3.9 times poorer for the older than for the younger group. SMRT thresholds were 1.5 times better for the younger than for the older group. Significant age effects were observed for SMRT thresholds and for frequency thresholds only in the basal region. SMRT thresholds were significantly correlated with frequency thresholds only in the older group. The poorer frequency and spectro-temporal pattern perception may contribute to age-related deficits in speech perception, even when audiometric thresholds are nearly normal.

Few-Shot Learning in Emotion Recognition of Spontaneous Speech Using a Siamese Neural Network With Adaptive Sample Pair Formation

Speech-based machine learning (ML) has been heralded as a promising solution for tracking prosodic and spectrotemporal patterns in real-life that are indicative of emotional changes, providing a valuable window into one's cognitive and mental state. Yet, the scarcity of labelled data in ambulatory studies prevents the reliable training of ML models, which usually rely on "data-hungry" distribution-based learning. Leveraging the abundance of labelled speech data from acted emotions, this paper proposes a few-shot learning approach for automatically recognizing emotion in spontaneous speech from a small number of labelled samples. Few-shot learning is implemented via a metric learning approach through a siamese neural network, which models the relative distance between samples rather than relying on learning absolute patterns of the corresponding distributions of each emotion. Results indicate the feasibility of the proposed metric learning in recognizing emotions from spontaneous speech in four datasets, even with a small amount of labelled samples. They further demonstrate superior performance of the proposed metric learning compared to commonly used adaptation methods, including network fine-tuning and adversarial learning. Findings from this work provide a foundation for the ambulatory tracking of human emotion in spontaneous speech contributing to the real-life assessment of mental health degradation.

Benefits of long-term music training for segregation of competing speech by tonal language speakers

Extended experience with meaningful pitch information has been shown to benefit music perception as well as speech perception where pitch cues are important, such as segregation of competing speech and tonal language perception. Interestingly, pitch perception has been shown to be similar between non-musicians who speak a tonal language and musicians who speak a non-tonal language, both of which outperform non-musicians who speak a non-tonal language. However, it is unknown whether extensive music training can further benefit pitch perception in tonal language speakers. In this study, melodic contour identification, spectro-temporal pattern perception, and masked speech recognition was measured in 16 adult normal-hearing musicians and 16 non-musicians; all were Chinese native speakers of Mandarin. Melodic contour identification, spectro-temporal pattern perception, and masked speech recognition all were significantly better for musicians than for non-musicians. Compared to non-musicians, musicians better utilized talker sex cues to segregate competing speech; utilization of talker sex cues by musicians was associated with the onset and extent of music training. Across all participants, spectro-temporal pattern perception was associated with better masked speech understanding. The data suggest that early and extensive music training may further benefit tonal language speakers’ perception and utilization of pitch cues.

WASN-Based Spectro-Temporal Analysis and Clustering of Road Traffic Noise in Urban and Suburban Areas

Environmental noise has become one of the principal health risks for urban dwellers and road traffic noise (RTN) is considered to be the main source of these adverse effects. To address this problem, strategic noise maps and corresponding action plans have been developed throughout Europe in recent years in response to the European Noise Directive 2002/49/EC (END), especially in populated cities. Recently, wireless acoustic sensor networks (WASNs) have started to serve as an alternative to static noise maps to monitor urban areas by gathering environmental noise data in real time. Several studies have analysed and categorized the different acoustic environments described in the END (e.g., traffic, industrial, leisure, etc.). However, most of them have only considered the dynamic evolution of the A-weighted equivalent noise levels LAeq over different periods of time. In order to focus on the analysis of RTN solely, this paper introduces a clustering methodology to analyse and group spectro-temporal profiles of RTN collected simultaneously across an area of interest. The experiments were conducted on two acoustic databases collected during a weekday and a weekend day through WASNs deployed in the pilot areas of the LIFE+ DYNAMAP project. The results obtained show that the clustering of RTN, based on its spectro-temporal patterns, yields different solutions on weekdays and at weekends in both environments, being larger than those found in the suburban environment and lower than the number of clusters in the urban scenario.

Predictability-Based Source Segregation and Sensory Deviance Detection in Auditory Aging.

When multiple sound sources are present at the same time, auditory perception is often challenged with disentangling the resulting mixture and focusing attention on the target source. It has been repeatedly demonstrated that background (distractor) sound sources are easier to ignore when their spectrotemporal signature is predictable. Prior evidence suggests that this ability to exploit predictability for foreground-background segregation degrades with age. On a theoretical level, this has been related with an impairment in elderly adults’ capabilities to detect certain types of sensory deviance in unattended sound sequences. Yet the link between those two capacities, deviance detection and predictability-based sound source segregation, has not been empirically demonstrated. Here we report on a combined behavioral-EEG study investigating the ability of elderly listeners (60–75 years of age) to use predictability as a cue for sound source segregation, as well as their sensory deviance detection capacities. Listeners performed a detection task on a target stream that can only be solved when a concurrent distractor stream is successfully ignored. We contrast two conditions whose distractor streams differ in their predictability. The ability to benefit from predictability was operationalized as performance difference between the two conditions. Results show that elderly listeners can use predictability for sound source segregation at group level, yet with a high degree of inter-individual variation in this ability. In a further, passive-listening control condition, we measured correlates of deviance detection in the event-related brain potential (ERP) elicited by occasional deviations from the same spectrotemporal pattern as used for the predictable distractor sequence during the behavioral task. ERP results confirmed neural signatures of deviance detection in terms of mismatch negativity (MMN) at group level. Correlation analyses at single-subject level provide no evidence for the hypothesis that deviance detection ability (measured by MMN amplitude) is related to the ability to benefit from predictability for sound source segregation. These results are discussed in the frameworks of sensory deviance detection and predictive coding.

Application of psychoacoustic analyses according to ECMA-418-2

Assessing and assuring sound quality has become a very important task for product design. Customers expect product sounds without disturbing noises. This is a challenge because spectro-temporal noise patterns (such as tonal sounds or modulated signals that generate a roughness sensation) must be taken into account, in addition to frequency-weighted values like dB(A) and loudness. If the sound of a technical product exhibits these characteristics, it is most likely perceived as having poor quality. The new standard ECMA-418-2 describes methods for the automatic quantification of tonal sounds and modulated sounds, which generate a sensation of roughness. The methods are based on a psychoacoustic hearing model and thus emulate human perception very closely. This paper describes the application of these methods. Several examples show how these parameters can be used for sound engineering and how to interpret the results.

Learning Waveform-Based Acoustic Models Using Deep Variational Convolutional Neural Networks

We investigate the potential of stochastic neural networks for learning effective waveform-based acoustic models. The waveform-based setting, inherent to fully end-to-end speech recognition systems, is motivated by several comparative studies of automatic and human speech recognition that associate standard non-adaptive feature extraction techniques with information loss which can adversely affect robustness. Stochastic neural networks, on the other hand, are a class of models capable of incorporating rich regularization mechanisms into the learning process. We consider a deep convolutional neural network that first decomposes speech into frequency sub-bands via an adaptive parametric convolutional block where filters are specified by cosine modulations of compactly supported windows. The network then employs standard non-parametric 1D convolutions to extract relevant spectro-temporal patterns while gradually compressing the structured high dimensional representation generated by the parametric block. We rely on a probabilistic parametrization of the proposed neural architecture and learn the model using stochastic variational inference. This requires evaluation of an analytically intractable integral defining the Kullback-Leibler divergence term responsible for regularization, for which we propose an effective approximation based on the Gauss-Hermite quadrature. Our empirical results demonstrate a superior performance of the proposed approach over comparable waveform-based baselines and indicate that it could lead to robustness. Moreover, the approach outperforms a recently proposed deep convolutional neural network for learning of robust acoustic models with standard FBANK features.

Differential activation of the visual word form area during auditory phoneme perception in youth with dyslexia

Developmental dyslexia is a learning disorder characterized by difficulties reading words accurately and/or fluently. Several behavioral studies have suggested the presence of anomalies at an early stage of phoneme processing, when the complex spectrotemporal patterns in the speech signal are analyzed and assigned to phonemic categories. In this study, fMRI was used to compare brain responses associated with categorical discrimination of speech syllables (P) and acoustically matched nonphonemic stimuli (N) in children and adolescents with dyslexia and in typically developing (TD) controls, aged 8–17 years. The TD group showed significantly greater activation during the P condition relative to N in an area of the left ventral occipitotemporal cortex that corresponds well with the region referred to as the “visual word form area” (VWFA). Regression analyses using reading performance as a continuous variable across the full group of participants yielded similar results. Overall, the findings are consistent with those of previous neuroimaging studies using print stimuli in individuals with dyslexia that found reduced activation in left occipitotemporal regions; however, the current study shows that these activation differences seen during reading are apparent during auditory phoneme discrimination in youth with dyslexia, suggesting that the primary deficit in at least a subset of children may lie early in the speech processing stream and that categorical perception may be an important target of early intervention in children at risk for dyslexia.

Ship Type Classification by Convolutional Neural Networks with Auditory-like Mechanisms.

Ship type classification with radiated noise helps monitor the noise of shipping around the hydrophone deployment site. This paper introduces a convolutional neural network with several auditory-like mechanisms for ship type classification. The proposed model mainly includes a cochlea model and an auditory center model. In cochlea model, acoustic signal decomposition at basement membrane is implemented by time convolutional layer with auditory filters and dilated convolutions. The transformation of neural patterns at hair cells is modeled by a time frequency conversion layer to extract auditory features. In the auditory center model, auditory features are first selectively emphasized in a supervised manner. Then, spectro-temporal patterns are extracted by deep architecture with multistage auditory mechanisms. The whole model is optimized with an objective function of ship type classification to form the plasticity of the auditory system. The contributions compared with an auditory inspired convolutional neural network include the improvements in dilated convolutions, deep architecture and target layer. The proposed model can extract auditory features from a raw hydrophone signal and identify types of ships under different working conditions. The model achieved a classification accuracy of 87.2% on four ship types and ocean background noise.

Spectral tuning of adaptation supports coding of sensory context in auditory cortex.

Perception of vocalizations and other behaviorally relevant sounds requires integrating acoustic information over hundreds of milliseconds. Sound-evoked activity in auditory cortex typically has much shorter latency, but the acoustic context, i.e., sound history, can modulate sound evoked activity over longer periods. Contextual effects are attributed to modulatory phenomena, such as stimulus-specific adaption and contrast gain control. However, an encoding model that links context to natural sound processing has yet to be established. We tested whether a model in which spectrally tuned inputs undergo adaptation mimicking short-term synaptic plasticity (STP) can account for contextual effects during natural sound processing. Single-unit activity was recorded from primary auditory cortex of awake ferrets during presentation of noise with natural temporal dynamics and fully natural sounds. Encoding properties were characterized by a standard linear-nonlinear spectro-temporal receptive field (LN) model and variants that incorporated STP-like adaptation. In the adapting models, STP was applied either globally across all input spectral channels or locally to subsets of channels. For most neurons, models incorporating local STP predicted neural activity as well or better than LN and global STP models. The strength of nonlinear adaptation varied across neurons. Within neurons, adaptation was generally stronger for spectral channels with excitatory than inhibitory gain. Neurons showing improved STP model performance also tended to undergo stimulus-specific adaptation, suggesting a common mechanism for these phenomena. When STP models were compared between passive and active behavior conditions, response gain often changed, but average STP parameters were stable. Thus, spectrally and temporally heterogeneous adaptation, subserved by a mechanism with STP-like dynamics, may support representation of the complex spectro-temporal patterns that comprise natural sounds across wide-ranging sensory contexts.

Serial recall of spectro-temporal patterns under informational masking

This study investigated the memory costs associated with informational masking (IM) and the memory benefits associated with its reduction. A serial recall task was used wherein the stimuli to be recalled were random sequences of six possible learned spectro-temporal patterns masked by other random sequences of spectro-temporal patterns. Sequence lengths of 1, 3, and 5 were tested. In all conditions, a target sequence and a masker sequence were assigned to mutually exclusive frequency bands. In the high-IM reference condition, the frequency separation of the two bands was chosen so as to be just discriminable and the onsets of corresponding target and masker patterns roughly overlapped in time. In the comparison conditions, IM was reduced systematically by increasing the frequency separation of the two bands and staggering the onsets of corresponding patterns. When IM was high, recall was poor and memory costs were large. As IM was reduced, recall improved, revealing memory benefits typically not discernable in standard detection, discrimination, or recognition-based studies of IM. Large individual differences were found and a number of observers were unable to perform the task at the pre-established criterion level even after training in unmasked conditions. The interaction between IM and memory will be discussed.This study investigated the memory costs associated with informational masking (IM) and the memory benefits associated with its reduction. A serial recall task was used wherein the stimuli to be recalled were random sequences of six possible learned spectro-temporal patterns masked by other random sequences of spectro-temporal patterns. Sequence lengths of 1, 3, and 5 were tested. In all conditions, a target sequence and a masker sequence were assigned to mutually exclusive frequency bands. In the high-IM reference condition, the frequency separation of the two bands was chosen so as to be just discriminable and the onsets of corresponding target and masker patterns roughly overlapped in time. In the comparison conditions, IM was reduced systematically by increasing the frequency separation of the two bands and staggering the onsets of corresponding patterns. When IM was high, recall was poor and memory costs were large. As IM was reduced, recall improved, revealing memory benefits typically not discernab...

Biometric identification of listener identity from frequency following responses to speech

Objective. We investigate the biometric specificity of the frequency following response (FFR), an EEG marker of early auditory processing that reflects phase-locked activity from neural ensembles in the auditory cortex and subcortex (Chandrasekaran and Kraus , Bidelman, , , Coffey et al ). Our objective is two-fold: demonstrate that the FFR contains information beyond stimulus properties and broad group-level markers, and to assess the practical viability of the FFR as a biometric across different sounds, auditory experiences, and recording days. Approach. We trained the hidden Markov model (HMM) to decode listener identity from FFR spectro-temporal patterns across multiple frequency bands. Our dataset included FFRs from twenty native speakers of English or Mandarin Chinese (10 per group) listening to Mandarin Chinese tones across three EEG sessions separated by days. We decoded subject identity within the same auditory context (same tone and session) and across different stimuli and recording sessions. Main results. The HMM decoded listeners for averaging sizes as small as one single FFR. However, model performance improved for larger averaging sizes (e.g. 25 FFRs), similarity in auditory context (same tone and day), and lack of familiarity with the sounds (i.e. native English relative to native Chinese listeners). Our results also revealed important biometric contributions from frequency bands in the cortical and subcortical EEG. Significance. Our study provides the first deep and systematic biometric characterization of the FFR and provides the basis for biometric identification systems incorporating this neural signal.

Auditory Inspired Convolutional Neural Networks for Ship Type Classification with Raw Hydrophone Data.

Detecting and classifying ships based on radiated noise provide practical guidelines for the reduction of underwater noise footprint of shipping. In this paper, the detection and classification are implemented by auditory inspired convolutional neural networks trained from raw underwater acoustic signal. The proposed model includes three parts. The first part is performed by a multi-scale 1D time convolutional layer initialized by auditory filter banks. Signals are decomposed into frequency components by convolution operation. In the second part, the decomposed signals are converted into frequency domain by permute layer and energy pooling layer to form frequency distribution in auditory cortex. Then, 2D frequency convolutional layers are applied to discover spectro-temporal patterns, as well as preserve locality and reduce spectral variations in ship noise. In the third part, the whole model is optimized with an objective function of classification to obtain appropriate auditory filters and feature representations that are correlative with ship categories. The optimization reflects the plasticity of auditory system. Experiments on five ship types and background noise show that the proposed approach achieved an overall classification accuracy of 79.2%, which improved by 6% compared to conventional approaches. Auditory filter banks were adaptive in shape to improve accuracy of classification.

Hierarchy of speech-driven spectrotemporal receptive fields in human auditory cortex

Existing data indicate that cortical speech processing is hierarchically organized. Numerous studies have shown that early auditory areas encode fine acoustic details while later areas encode abstracted speech patterns. However, it remains unclear precisely what speech information is encoded across these hierarchical levels. Estimation of speech-driven spectrotemporal receptive fields (STRFs) provides a means to explore cortical speech processing in terms of acoustic or linguistic information associated with characteristic spectrotemporal patterns. Here, we estimate STRFs from cortical responses to continuous speech in fMRI. Using a novel approach based on filtering randomly-selected spectrotemporal modulations (STMs) from aurally-presented sentences, STRFs were estimated for a group of listeners and categorized using a data-driven clustering algorithm. ‘Behavioral STRFs’ highlighting STMs crucial for speech recognition were derived from intelligibility judgments. Clustering revealed that STRFs in the supratemporal plane represented a broad range of STMs, while STRFs in the lateral temporal lobe represented circumscribed STM patterns important to intelligibility. Detailed analysis recovered a bilateral organization with posterior-lateral regions preferentially processing STMs associated with phonological information and anterior-lateral regions preferentially processing STMs associated with word- and phrase-level information. Regions in lateral Heschl's gyrus preferentially processed STMs associated with vocalic information (pitch).

Gliding ripple speed thresholds for rippled spectra

Rippled noise is a productive model of natural signals with complex spectrum patterns. It was used as a test signal to measure spectrum-pattern resolution both in normal-hearing listeners and in hearing-impaired listeners and users of cochlear implants. However, a variety of natural auditory signals feature combined spectro-temporal patterns. These signals may be modeled by rippled noise with ripples. In the present study, ripple gliding speed thresholds were measured in normal-hearing listeners. The highest gliding speed (expressed in oct/s or ripples/s) at which the gliding ripple pattern could be distinguished from a non-rippled noise was determined. The ripple gliding speed threshold expressed in ripple/s was inversely-proportional to the ripple density (ripples per octave, RPO); it decreased from approximately 400–500 ripple/s at RPO of 1/oct to approximately 50 ripple/s at RPO of 10/oct. Expressed in oct/s, the threshold decreased from 400 to 500 oct/s at RPO of 1/oct to 5 oct/s at RPO of 10/oct. Thus, the ripples fused into non-rippled spectrum at gliding speed more than 400 to 500 Hz/oct. At this speed, 400 to 500 ripples per second glide through an octave frequency band. [Work supported by The Russian Science Foundation, Grant 16-15-10046.]Rippled noise is a productive model of natural signals with complex spectrum patterns. It was used as a test signal to measure spectrum-pattern resolution both in normal-hearing listeners and in hearing-impaired listeners and users of cochlear implants. However, a variety of natural auditory signals feature combined spectro-temporal patterns. These signals may be modeled by rippled noise with ripples. In the present study, ripple gliding speed thresholds were measured in normal-hearing listeners. The highest gliding speed (expressed in oct/s or ripples/s) at which the gliding ripple pattern could be distinguished from a non-rippled noise was determined. The ripple gliding speed threshold expressed in ripple/s was inversely-proportional to the ripple density (ripples per octave, RPO); it decreased from approximately 400–500 ripple/s at RPO of 1/oct to approximately 50 ripple/s at RPO of 10/oct. Expressed in oct/s, the threshold decreased from 400 to 500 oct/s at RPO of 1/oct to 5 oct/s at RPO of ...

Underwater sound pressure and particle motion (acceleration, velocity, and displacement) from recreational swimmers, divers, surfers, and kayakers

Naturally constrained environments like bays and lakes are frequently used for water sports activities. While the sound of motorized vessels is rather well understood, non-motorized activities have received less investigation. Ten water sports activities (swimming backstroke, breaststroke, butterfly, and freestyle; snorkelling with fins; kicking a boogie board with fins; paddling with alternating or simultaneous arms while lying on a surfboard; scuba-diving; kayaking and jumping into the water) were recorded in a controlled yet even more constrained environment: an Olympic-sized pool. Activities that occurred at the surface involved repeatedly piercing the surface and hence creating the bubble clouds which were the strongest sound generators. Received levels were 110–131 dB re 1μPa (10–16,000 Hz) for all of the activities at the closest point of approach (1 m). All activities produced a characteristic spectro-temporal pattern in all the quantities measured (sound pressure, particle displacement, velocity, and acceleration) by which they could be identified. Applicability of the results to security monitoring of pools, performance assessments of professional or competitive swimmers, and studies of the distances at which humans may be detectable by marine animals in the sea will be discussed.

Identifying Missing and Extra Notes in Piano Recordings Using Score-Informed Dictionary Learning

The goal of automatic music transcription (AMT) is to obtain a high-level symbolic representation of the notes played in a given audio recording. Despite being researched for several decades, current methods are still inadequate for many applications. To boost the accuracy in a music tutoring scenario, we exploit that the score to be played is specified and we only need to detect the differences to the actual performance. In contrast to previous work that uses score information for postprocessing, we employ the score to construct a transcription method that is tailored to the given audio recording. By adapting a score-informed dictionary learning technique as used for source separation, we learn for each score pitch a spectral pattern describing the energy distribution of associated notes in the recording. In this paper, we identify several systematic weaknesses in our previous approach and introduce three extensions to improve its performance. First, we extend our dictionary of spectral templates to a dictionary of variable-length spectrotemporal patterns. Second, we integrate the score information using soft rather than hard constraints, to better take into account that differences from the score indeed occur. Third, we introduce new regularizers to guide the learning process. Our experiments show that these extensions particularly improve the accuracy for identifying extra notes, while the accuracy for correct and missing notes remains at a similar level. The influence of each extension is demonstrated with further experiments.

Deep Convolutional Neural Networks and Data Augmentation for Environmental Sound Classification

The ability of deep convolutional neural networks (CNNs) to learn discriminative spectro-temporal patterns makes them well suited to environmental sound classification. However, the relative scarcity of labeled data has impeded the exploitation of this family of high-capacity models. This study has two primary contributions: first, we propose a deep CNN architecture for environmental sound classification. Second, we propose the use of audio data augmentation for overcoming the problem of data scarcity and explore the influence of different augmentations on the performance of the proposed CNN architecture. Combined with data augmentation, the proposed model produces state-of-the-art results for environmental sound classification. We show that the improved performance stems from the combination of a deep, high-capacity model and an augmented training set: this combination outperforms both the proposed CNN without augmentation and a “shallow” dictionary learning model with augmentation. Finally, we examine the influence of each augmentation on the model's classification accuracy for each class, and observe that the accuracy for each class is influenced differently by each augmentation, suggesting that the performance of the model could be improved further by applying class-conditional data augmentation.