Cocktail Party Environment Research Articles

Sound masking by a low-pitch speech-shaped noise improves a social robot's talk in noisy environments.

Introduction: There has been a surge in the use of social robots for providing information, persuasion, and entertainment in noisy public spaces in recent years. Considering the well-documented negative effect of noise on human cognition, masking sounds have been introduced. Masking sounds work, in principle, by making the intrusive background speeches less intelligible, and hence, less distracting. However, this reduced distraction comes with the cost of increasing annoyance and reduced cognitive performance in the users of masking sounds. Methods: In a previous study, it was shown that reducing the fundamental frequency of the speech-shaped noise as a masking sound significantly contributes to its being less annoying and more efficient. In this study, the effectiveness of the proposed masking sound was tested on the performance of subjects listening to a lecture given by a social robot in a noisy cocktail party environment. Results: The results indicate that the presence of the masking sound significantly increased speech comprehension, perceived understandability, acoustic satisfaction, and sound privacy of the individuals listening to the robot in an adverse listening condition. Discussion: To the knowledge of the authors, no previous work has investigated the application of sound masking technology in human-robot interaction designs. The future directions of this trend are discussed.

Statistical learning at a virtual cocktail party

Statistical learning – the ability to extract distributional regularities from input – is suggested to be key to language acquisition. Yet, evidence for the human capacity for statistical learning comes mainly from studies conducted in carefully controlled settings without auditory distraction. While such conditions permit careful examination of learning, they do not reflect the naturalistic language learning experience, which is replete with auditory distraction – including competing talkers. Here, we examine how statistical language learning proceeds in a virtual cocktail party environment, where the to-be-learned input is presented alongside a competing speech stream with its own distributional regularities. During exposure, participants in the Dual Talker group concurrently heard two novel languages, one produced by a female talker and one by a male talker, with each talker virtually positioned at opposite sides of the listener (left/right) using binaural acoustic manipulations. Selective attention was manipulated by instructing participants to attend to only one of the two talkers. At test, participants were asked to distinguish words from part-words for both the attended and the unattended languages. Results indicated that participants’ accuracy was significantly higher for trials from the attended vs. unattended language. Further, the performance of this Dual Talker group was no different compared to a control group who heard only one language from a single talker (Single Talker group). We thus conclude that statistical learning is modulated by selective attention, being relatively robust against the additional cognitive load provided by competing speech, emphasizing its efficiency in naturalistic language learning situations.

Solving the cocktail party problem using Multi-modal Hearing Assistive Technology Prototype

Hearing loss is a major global health problem, affecting over 1.5 billion people. According to estimations by the World Health Organization, 83% of those who could benefit from hearing assistive devices do not use them. The limited adoption of hearing aids can be attributed to the suboptimal performance in acoustically challenging environments, such as cocktail parties, where there are multiple competing speakers and noise sources leading to poor speech intelligibility for hearing impaired listeners. This work presents a first-of-its-kind real-time, multi-modal speech enhancement system that can effectively enhance speech in challenging real noisy environments. The system exploits both audio and visual cues to isolate the target speaker's voice from interfering background noises. The system is developed using deep neural networks and is integrated with the Open Master Hearing Aid (openMHA) platform. The integrated prototype was evaluated in a cocktail party setting to isolate the target speaker's voice from interfering speakers, music, and other non-speech noise sources and was able to significantly improve the speech intelligibility of hearing impaired listeners. This work has the potential to improve the quality of life for hearing impaired listeners by facilitating effective communication in cocktail party environments.

The integration of continuous audio and visual speech in a cocktail-party environment depends on attention

In noisy environments, our ability to understand speech benefits greatly from seeing the speaker's face. This is attributed to the brain's ability to integrate audio and visual information, a process known as multisensory integration. In addition, selective attention plays an enormous role in what we understand, the so-called cocktail-party phenomenon. But how attention and multisensory integration interact remains incompletely understood, particularly in the case of natural, continuous speech. Here, we addressed this issue by analyzing EEG data recorded from participants who undertook a multisensory cocktail-party task using natural speech. To assess multisensory integration, we modeled the EEG responses to the speech in two ways. The first assumed that audiovisual speech processing is simply a linear combination of audio speech processing and visual speech processing (i.e., an A + V model), while the second allows for the possibility of audiovisual interactions (i.e., an AV model). Applying these models to the data revealed that EEG responses to attended audiovisual speech were better explained by an AV model, providing evidence for multisensory integration. In contrast, unattended audiovisual speech responses were best captured using an A + V model, suggesting that multisensory integration is suppressed for unattended speech. Follow up analyses revealed some limited evidence for early multisensory integration of unattended AV speech, with no integration occurring at later levels of processing. We take these findings as evidence that the integration of natural audio and visual speech occurs at multiple levels of processing in the brain, each of which can be differentially affected by attention.

A review on speech separation in cocktail party environment: challenges and approaches

A review on speech separation in cocktail party environment: challenges and approaches

Robot Hearing Through Optical Channel in a Cocktail Party Environment

The cocktail party problem refers to a challenging process when the human sensory system tries to separate a specific voice from a loud mixture of background sound sources. The problem is much more demanding for machines and has become the holy grail in robotic hearing. Despite many advances in noise suppression, the intrinsic information from the contaminated acoustic channel remains difficult to recover. Herein, a simple‐yet‐powerful laser‐assisted audio system termed robot ear accomplished by laser (REAL) is shown to probe the vibrations of sound‐carrying surfaces (mask, throat, and other nearby surfaces) in optical channel, which is intrinsically immune to acoustic background noises. The results demonstrate that REAL can directly obtain the audio‐frequency content from the laser without acoustic channel interference. The signals can be further transcribed into human‐recognizable audios by exploiting the internal time and frequency correlations through memory‐enabled neural networks. The REAL system would enable a new way in human–robot interaction. An interactive preprint version of the article can be found at: https://www.authorea.com/doi/full/10.1002/aisy.202200143.

Temporal and Directional Cue Effects on the Cocktail Party Problem for Patients With Listening Difficulties Without Clinical Hearing Loss.

To evaluate the listening difficulty in a cocktail party environment in the sound field in order to better demonstrate patients' difficulties listening in noise, and to examine temporal and directional cue effects on the speech intelligibility in patients with listening difficulties in noise in comparison with control subjects. This study examined and analyzed 16 control subjects without any complaints of listening difficulties and 16 patients who had visited the outpatient clinic of the Department of Otolaryngology-Head and Neck Surgery, Tohoku University Hospital, with complaints of listening difficulties, especially in background crowded conditions, despite having relatively good hearing on routine audiograms and speech audiometry. Using five loudspeakers located in front of the subject and at 30° and 60° to the left and right from the front, word intelligibility for the target voice (female talker) presented from one of the loudspeakers in random order with four distractor voices (male talker) was assessed under the following cue conditions: (1) "no additional temporal/directional cue (only talker sex as a cue)"; (2) "fixed temporal cue without directional cue" (white noise bursts [cue sounds] were presented from the five loudspeakers just before word presentation at 500-ms intervals); (3) "directional + variable temporal cues" [cue sounds were presented from the loudspeaker where the next target word would be presented with a variable inter-stimulus interval [ISI] of 500, 1000, 1500, or 2000 ms between the cue sound and word presentation); and (4) "directional + fixed temporal cues" (cue sounds were presented from the loudspeaker where the next target word would be presented with a fixed ISI of 500 ms). The results indicated the following: (1) word intelligibility under distractors was significantly deteriorated in patients with listening difficulties compared with control subjects, although the clinical speech in noise test using the headphone system did not show any significant differences between the two groups; (2) word intelligibility under distractors for patients with listening difficulties was significantly improved with directional cues presented in advance; and (3) under most cue conditions, individual differences in word intelligibility among patients with listening difficulties were significantly correlated with their dichotic listening ability, which is one of the indicators used to assess auditory selective attention ability. The results of this study indicate the usefulness of the presentation of directional cues for speech comprehension in the cocktail party situation in patients with listening difficulties, as well as the importance of evaluating the degree of listening difficulties spatially in the cocktail party situation.

Robust speech separation in underdetermined conditions by estimating Green's functions

Research results from near and far field speech separation in cocktail party environments using compact arrays of four or more microphones to robustly estimate Green's Functions of real rooms with reverberation, co-speech, and ambient noise.

Musicians Show Improved Speech Segregation in Competitive, Multi-Talker Cocktail Party Scenarios.

Studies suggest that long-term music experience enhances the brain’s ability to segregate speech from noise. Musicians’ “speech-in-noise (SIN) benefit” is based largely on perception from simple figure-ground tasks rather than competitive, multi-talker scenarios that offer realistic spatial cues for segregation and engage binaural processing. We aimed to investigate whether musicians show perceptual advantages in cocktail party speech segregation in a competitive, multi-talker environment. We used the coordinate response measure (CRM) paradigm to measure speech recognition and localization performance in musicians vs. non-musicians in a simulated 3D cocktail party environment conducted in an anechoic chamber. Speech was delivered through a 16-channel speaker array distributed around the horizontal soundfield surrounding the listener. Participants recalled the color, number, and perceived location of target callsign sentences. We manipulated task difficulty by varying the number of additional maskers presented at other spatial locations in the horizontal soundfield (0–1–2–3–4–6–8 multi-talkers). Musicians obtained faster and better speech recognition amidst up to around eight simultaneous talkers and showed less noise-related decline in performance with increasing interferers than their non-musician peers. Correlations revealed associations between listeners’ years of musical training and CRM recognition and working memory. However, better working memory correlated with better speech streaming. Basic (QuickSIN) but not more complex (speech streaming) SIN processing was still predicted by music training after controlling for working memory. Our findings confirm a relationship between musicianship and naturalistic cocktail party speech streaming but also suggest that cognitive factors at least partially drive musicians’ SIN advantage.

Auditory attention tracking states in a cocktail party environment can be decoded by deep convolutional neural networks

Objective. A deep convolutional neural network (CNN) is a method for deep learning (DL). It has a powerful ability to automatically extract features and is widely used in classification tasks with scalp electroencephalogram (EEG) signals. However, the small number of samples and low signal-to-noise ratio involved in scalp EEG with low spatial resolution constitute a limitation that might restrict potential brain-computer interface (BCI) applications that are based on the CNN model. In the present study, a novel CNN model with source-spatial feature images (SSFIs) as the input is proposed to decode auditory attention tracking states in a cocktail party environment. Approach. We first extract SSFIs using rhythm entropy and weighted minimum norm estimation. Next, we develop a CNN model with three convolutional layers. Furthermore, we estimate the performance of the proposed model via generalized performance, alternative models that deleted or replaced a model’s component, and loss curves. Finally, we use a deep transfer model with fine-tuning for a low (poor) behavioral performance group (L-group). Main results. Based on cortical activity reconstructions from the scalp EEGs, the classification accuracy (CA) of the proposed model is 80.4% (chance level: 52.5%), which is superior to that achieved by scalp EEG. Additionally, the performance of the proposed model is more stable when compared to alternative models that delete or replace specific model components. The proposed model identifies the difference between two auditory attention tracking states (successful versus unsuccessful) at an early stage with a short time window (250 ms after target offset). Furthermore, we propose a deep transfer learning model to improve the classification for the L-group. With this model, the CA of the L-group significantly increase by 5.3%. Significance. Our proposed model improves the performance of a decoder for auditory attention tracking, which could be suitable for relieving the difficulty with the attentional modulation of individual’s neural responses. It provides a novel communication channel with auditory cognitive BCI for patients with attention and hearing impairment.

Stochastic allocation strategy for irregular arrays based on geometric feature control

Irregularities in microphone distribution enrich the diversity of spatial differences to decorrelate interferences from the beamforming target. However, the large degrees of freedom of irregular placements make it difficult to analyse and optimize array performance. This article proposes fast and feasible optimal irregular array design methods with improved beamforming performance for human speech. Important geometric features are extracted to be used as the input vector of the neural network structure to determine the optimal irregular arrangements of sensors. In addition, a hyperbola design method is proposed to directly cluster microphones in the hyperbola areas to produce rich differential distance entropies and yield significant signal-to-noise ratio improvements. These methods can be easily applied to guide non-computer-aided optimal irregular array designs for human speech in acoustic scenes such as immersive cocktail party environments.

Sleepers Selectively Suppress Informative Inputs During Rapid Eye Movements

Sleep leads to a disconnection from the external world. Even when sleepers regain consciousness during Rapid Eye Movement (REM) sleep, little, if any, external information is incorporated into dream content. While gating mechanisms might be at play to avoid interference on dreaming activity, a total disconnection would prevent the sleeper from promptly responding to informative events (e.g., threat signals). In fact, a whole range of neural responses to external events are preserved, at least partially, during REM sleep. Thus, it remains unsettled whether external inputs are either integrated or conversely gated during REM sleep. Here, we sought to disentangle this issue by studying the impact of eye movements (EMs) on the processing of informative signals. EMs constitute a reliable predictor of reporting a dream upon awakening and induce a higher arousal threshold to external stimuli. We hypothesized that although sleepers might continue responding to external events, periods of EMs would be associated with the gating of informative signals in order to reduce interference with dreaming activity. By reconstructing perceived speech in a multi-talker, cocktail-party environment from electrophysiological responses, we show that informative speech is selectively amplified over meaningless speech during REM sleep. However, we show that at the precise timing of EMs, informative speech is, on the contrary, selectively suppressed. Importantly, this suppression did not affect the processing of non-informative signals, revealing a selective mechanism of gating for informative contents rather than a general decline in sensory encoding. These results reveal flexible mechanisms of amplification/suppression of external inputs during REM sleep, and demonstrate the impact of EMs on the selective gating of informative stimuli.

Look at me when I'm talking to you: Selective attention at a multisensory cocktail party can be decoded using stimulus reconstruction and alpha power modulations.

Recent work using electroencephalography has applied stimulus reconstruction techniques to identify the attended speaker in a cocktail party environment. The success of these approaches has been primarily based on the ability to detect cortical tracking of the acoustic envelope at the scalp level. However, most studies have ignored the effects of visual input, which is almost always present in naturalistic scenarios. In this study, we investigated the effects of visual input on envelope-based cocktail party decoding in two multisensory cocktail party situations: (a) Congruent AV-facing the attended speaker while ignoring another speaker represented by the audio-only stream and (b) Incongruent AV (eavesdropping)-attending the audio-only speaker while looking at the unattended speaker. We trained and tested decoders for each condition separately and found that we can successfully decode attention to congruent audiovisual speech and can also decode attention when listeners were eavesdropping, i.e., looking at the face of the unattended talker. In addition to this, we found alpha power to be a reliable measure of attention to the visual speech. Using parieto-occipital alpha power, we found that we can distinguish whether subjects are attending or ignoring the speaker's face. Considering the practical applications of these methods, we demonstrate that with only six near-ear electrodes we can successfully determine the attended speech. This work extends the current framework for decoding attention to speech to more naturalistic scenarios, and in doing so provides additional neural measures which may be incorporated to improve decoding accuracy.

A Tutorial on Auditory Attention Identification Methods.

Auditory attention identification methods attempt to identify the sound source of a listener's interest by analyzing measurements of electrophysiological data. We present a tutorial on the numerous techniques that have been developed in recent decades, and we present an overview of current trends in multivariate correlation-based and model-based learning frameworks. The focus is on the use of linear relations between electrophysiological and audio data. The way in which these relations are computed differs. For example, canonical correlation analysis (CCA) finds a linear subset of electrophysiological data that best correlates to audio data and a similar subset of audio data that best correlates to electrophysiological data. Model-based (encoding and decoding) approaches focus on either of these two sets. We investigate the similarities and differences between these linear model philosophies. We focus on (1) correlation-based approaches (CCA), (2) encoding/decoding models based on dense estimation, and (3) (adaptive) encoding/decoding models based on sparse estimation. The specific focus is on sparsity-driven adaptive encoding models and comparing the methodology in state-of-the-art models found in the auditory literature. Furthermore, we outline the main signal processing pipeline for how to identify the attended sound source in a cocktail party environment from the raw electrophysiological data with all the necessary steps, complemented with the necessary MATLAB code and the relevant references for each step. Our main aim is to compare the methodology of the available methods, and provide numerical illustrations to some of them to get a feeling for their potential. A thorough performance comparison is outside the scope of this tutorial.

EEG-based auditory attention detection: boundary conditions for background noise and speaker positions**The work is funded by KU Leuven Special Research Fund C14/16/057 and OT/14/119, FWO project nrs. 1.5.123.16N and G0A4918N, the ERC (637424) under the European Union’s Horizon 2020 research and innovation programme, and a research gift of Starkey Hearing Technologies. The scientific responsibility is assumed by its authors.

Objective. A listener’s neural responses can be decoded to identify the speaker the person is attending to in a cocktail party environment. Such auditory attention detection methods have the potential to provide noise suppression algorithms in hearing devices with information about the listener’s attention. A challenge is the effect of noise and other acoustic conditions that can reduce the attention detection accuracy. Specifically, noise can impact the ability of the person to segregate the sound sources and perform selective attention, as well as the external signal processing necessary to decode the attention effectively. The aim of this work is to systematically analyze the effect of noise level and speaker position on attention decoding accuracy. Approach. 28 subjects participated in the experiment. Auditory stimuli consisted of stories narrated by different speakers from two different locations, along with surrounding multi-talker background babble. EEG signals of the subjects were recorded while they focused on one story and ignored the other. The strength of the babble noise as well as the spatial separation between the two speakers were varied between presentations. Spatio-temporal decoders were trained for each subject, and applied to decode attention of the subjects from every 30 s segment of data. Behavioral speech recognition thresholds were obtained for the different speaker separations. Main results. Both the background noise level and the angular separation between speakers affected attention decoding accuracy. Remarkably, attention decoding performance was seen to increase with the inclusion of moderate background noise (versus no noise), while across the different noise conditions performance dropped significantly with increasing noise level. We also observed that decoding accuracy improved with increasing speaker separation, exhibiting the advantage of spatial release from masking. Furthermore, the effect of speaker separation on the decoding accuracy became stronger when the background noise level increased. A significant correlation between speech intelligibility and attention decoding accuracy was found across conditions. Significance. This work shows how the background noise level and relative positions of competing talkers impact attention decoding accuracy. It indicates in which circumstances a neuro-steered noise suppression system may need to operate, in function of acoustic conditions. It also indicates the boundary conditions for the operation of EEG-based attention detection systems in neuro-steered hearing prostheses.

Modeling Attention and Memory for Auditory Selection in a Cocktail Party Environment

Developing a computational auditory model to solve the cocktail party problem has long bedeviled scientists, especially for a single microphone recording. Although recent deep learning based frameworks have made significant progress in multi-talker mixed speech separation, most existing deep learning based methods, focusing on separating all the speech channels rather than selectively attending the target speech and ignoring other sounds, may fail to offer a satisfactory solution in a complex auditory scene where the number of input sounds is usually uncertain and even dynamic. In this work, we employ ideas from auditory selective attention of behavioral and cognitive neurosciences and from recent advances of memory-augmented neural networks. Specifically, a unified Auditory Selection framework with Attention and Memory (dubbed ASAM) is proposed. Our ASAM first accumulates the prior knowledge (that is the acoustic feature to one specific speaker) into a life-long memory during the training phase, meanwhile a speech perceptor is trained to extract the temporal acoustic feature and update the memory online when a salient speech is given. Then, the learned memory is utilized to interact with the mixture input to attend and filter the target frequency out from the mixture stream. Finally, the network is trained to minimize the reconstruction error of the attended speech. We evaluate the proposed approach on WSJ0 and THCHS-30 datasets and the experimental results demonstrate that our approach successfully conducts two auditory selection tasks: the top-down task-specific attention (e.g. to follow a conversation with friend) and the bottom-up stimulus-driven attention (e.g. be attracted by a salient speech). Compared with deep clustering based methods, our method conducts competitive advantages especially in a real noise environment (e.g. street junction). Our code is available at https://github.com/jacoxu/ASAM.

Cortical Classification with Rhythm Entropy for Error Processing in Cocktail Party Environment Based on Scalp EEG Recording

Using single-trial cortical signals calculated by weighted minimum norm solution estimation (WMNE), the present study explored a feature extraction method based on rhythm entropy to classify the scalp electroencephalography (EEG) signals of error response from that of correct response during performing auditory-track tasks in cocktail party environment. The classification rate achieved 89.7% with single-trial (≈700 ms) when using support vector machine(SVM) with the leave-one-out-cross-validation (LOOCV). And high discriminative regions mainly distributed at the medial frontal cortex (MFC), the left supplementary motor area (lSMA) and the right supplementary motor area (rSMA). The mean entropy value for error trials was significantly lower than that for correct trials in the discriminative cortices. By time-varying network analysis, different information flows changed among these discriminative regions with time, i.e. error processing showed a left-bias information flow, and correct processing presented a right-bias information flow. These findings revealed that the rhythm information based on single cortical signals could be well used to describe characteristics of error-related EEG signals and further provided a novel application about auditory attention for brain computer interfaces (BCIs).

Speech Understanding and Sound Source Localization by Cochlear Implant Listeners Using a Pinna-Effect Imitating Microphone and an Adaptive Beamformer.

Sentence understanding scores for patients with cochlear implants (CIs) when tested in quiet are relatively high. However, sentence understanding scores for patients with CIs plummet with the addition of noise. To assess, for patients with CIs (MED-EL), (1) the value to speech understanding of two new, noise-reducing microphone settings and (2) the effect of the microphone settings on sound source localization. Single-subject, repeated measures design. For tests of speech understanding, repeated measures on (1) number of CIs (one, two), (2) microphone type (omni, natural, adaptive beamformer), and (3) type of noise (restaurant, cocktail party). For sound source localization, repeated measures on type of signal (low-pass [LP], high-pass [HP], broadband noise). Ten listeners, ranging in age from 48 to 83 yr (mean = 57 yr), participated in this prospective study. Speech understanding was assessed in two noise environments using monaural and bilateral CIs fit with three microphone types. Sound source localization was assessed using three microphone types. In Experiment 1, sentence understanding scores (in terms of percent words correct) were obtained in quiet and in noise. For each patient, noise was first added to the signal to drive performance off of the ceiling in the bilateral CI-omni microphone condition. The other conditions were then administered at that signal-to-noise ratio in quasi-random order. In Experiment 2, sound source localization accuracy was assessed for three signal types using a 13-loudspeaker array over a 180° arc. The dependent measure was root-mean-score error. Both the natural and adaptive microphone settings significantly improved speech understanding in the two noise environments. The magnitude of the improvement varied between 16 and 19 percentage points for tests conducted in the restaurant environment and between 19 and 36 percentage points for tests conducted in the cocktail party environment. In the restaurant and cocktail party environments, both the natural and adaptive settings, when implemented on a single CI, allowed scores that were as good as, or better, than scores in the bilateral omni test condition. Sound source localization accuracy was unaltered by either the natural or adaptive settings for LP, HP, or wideband noise stimuli. The data support the use of the natural microphone setting as a default setting. The natural setting (1) provides better speech understanding in noise than the omni setting, (2) does not impair sound source localization, and (3) retains low-frequency sensitivity to signals from the rear. Moreover, bilateral CIs equipped with adaptive beamforming technology can engender speech understanding scores in noise that fall only a little short of scores for a single CI in quiet.

Past review, current progress, and challenges ahead on the cocktail party problem

The cocktail party problem, i.e., tracing and recognizing the speech of a specific speaker when multiple speakers talk simultaneously, is one of the critical problems yet to be solved to enable the wide application of automatic speech recognition (ASR) systems. In this overview paper, we review the techniques proposed in the last two decades in attacking this problem. We focus our discussions on the speech separation problem given its central role in the cocktail party environment, and describe the conventional single-channel techniques such as computational auditory scene analysis (CASA), non-negative matrix factorization (NMF) and generative models, the conventional multi-channel techniques such as beamforming and multi-channel blind source separation, and the newly developed deep learning-based techniques, such as deep clustering (DPCL), the deep attractor network (DANet), and permutation invariant training (PIT). We also present techniques developed to improve ASR accuracy and speaker identification in the cocktail party environment. We argue effectively exploiting information in the microphone array, the acoustic training set, and the language itself using a more powerful model. Better optimization objective and techniques will be the approach to solving the cocktail party problem.

Cortical Representations of Speech in a Multitalker Auditory Scene.

The ability to parse a complex auditory scene into perceptual objects is facilitated by a hierarchical auditory system. Successive stages in the hierarchy transform an auditory scene of multiple overlapping sources, from peripheral tonotopically based representations in the auditory nerve, into perceptually distinct auditory-object-based representations in the auditory cortex. Here, using magnetoencephalography recordings from men and women, we investigate how a complex acoustic scene consisting of multiple speech sources is represented in distinct hierarchical stages of the auditory cortex. Using systems-theoretic methods of stimulus reconstruction, we show that the primary-like areas in the auditory cortex contain dominantly spectrotemporal-based representations of the entire auditory scene. Here, both attended and ignored speech streams are represented with almost equal fidelity, and a global representation of the full auditory scene with all its streams is a better candidate neural representation than that of individual streams being represented separately. We also show that higher-order auditory cortical areas, by contrast, represent the attended stream separately and with significantly higher fidelity than unattended streams. Furthermore, the unattended background streams are more faithfully represented as a single unsegregated background object rather than as separated objects. Together, these findings demonstrate the progression of the representations and processing of a complex acoustic scene up through the hierarchy of the human auditory cortex.SIGNIFICANCE STATEMENT Using magnetoencephalography recordings from human listeners in a simulated cocktail party environment, we investigate how a complex acoustic scene consisting of multiple speech sources is represented in separate hierarchical stages of the auditory cortex. We show that the primary-like areas in the auditory cortex use a dominantly spectrotemporal-based representation of the entire auditory scene, with both attended and unattended speech streams represented with almost equal fidelity. We also show that higher-order auditory cortical areas, by contrast, represent an attended speech stream separately from, and with significantly higher fidelity than, unattended speech streams. Furthermore, the unattended background streams are represented as a single undivided background object rather than as distinct background objects.