Real Room Environment Research Articles

The study of self-organised behaviours and movement pattern of pedestrians during fire evacuations: virtual experiments and survey

In this study, Minecraft was adopted to investigate pedestrian behaviours and evacuation processes in non-fire (normal) and fire conditions involving multiple simultaneous players. We built a PaddlePaddle-based deep learning platform for the automatic recognition of pedestrian coordinates from the Minecraft background database. To verify the virtual experiments in Minecraft, we conducted a series of experiments under identical conditions in a real (physical) room and virtual environment on the Minecraft platform. The pedestrian behaviour, flow patterns and evacuation time in the Minecraft experiments fit well with the results of the real-life experiments, indicating the potential of Minecraft for realistically simulating evacuation processes. To further demonstrate the feasibility of using Minecraft for evacuation studies, we conducted a series of virtual evacuation experiments with different fire configurations and room geometries in Minecraft. The pedestrians were found to keep higher distance towards fire and greater detour angle during the fire emergency in a square room. Furthermore, in the scenario of a fire in a corridor, in addition to the aforementioned fire avoidance behaviours, the pedestrians exhibited orderly queuing and exited the room one by one, resulting in increased walking speeds and thus a faster evacuation for most pedestrians. Finally, one post-experiment survey was designed to assess participants' perceptions of the Minecraft environment and the pedestrian behaviours and related psychological factors. The results show that the Minecraft can generate considerate realistic real-life situations, and pedestrians chose self-organised behaviours including away from fire, along the wall and detour towards fire in both survey and evacuation experiments.

How do You Feel in Virtual Environments? The Role of Emotions and Openness Trait Over Creative Performance

Abstract In the Dynamic Creativity Framework creativity is defined as a context-embedded phenomenon requiring potential originality and effectiveness. This definition indicates that the environmental conditions embedding the creative process have fundamental impact on the process itself and its outcomes. In particular, Virtual environments (VEs) are emerging as everyday contexts for a large part of the world population, affecting behaviors and feelings. VEs have been demonstrated to affect creative performance in several ways, even if the psychological mechanisms at the basis of the different modifications in the creative behavior are far from being completely explained. The aim of this study was to explore the influence of different types of VEs on creative performance, with a specific focus on participants’ emotional reactions and on their individual differences in the Openness personality trait. A total of 22 participants were exposed to four different types of environments: a real room environment (RE), a control virtual environment (CVE) resembling the physical characteristics of the RE, a positive virtual environment (PVE) and a negative virtual environment (NVE). Participants were free to explore each environment for two minutes, then they were asked to perform an Alternative Uses Task for five minutes, to measure divergent thinking performance. Openness and affective reactions in each environment were measured in all participants. Results showed that Openness was associated with higher originality of responses and that this effect was particularly significant in PVE. Importantly, the type of environment interacted significantly with participants’ affective reactions in explaining their creative performance, revealing that an increase of ideas originality was associated with an increase of positive affect, emerging as a consequence of experiencing a PVE. Affective reactions to VEs, in combination with individual differences in term of Openness, thus emerge as one of the possible explicatory mechanisms of the impact of virtual reality on creative performance.

Multiple Speech Sources Localization in Room Reverberant Environment Using Spherical Harmonic Sparse Bayesian Learning

In recent years, sparse representation techniques have been proposed for source localization using spherical microphone arrays (SMAs). However, the performance of these sparse representation techniques for SMAs degrades for speech source localization in the room environment due to sound reverberation. This article proposes a robust sparse presentation method for localization of multiple speech sources in the room environment using an SMA, which employs the spherical harmonic temporal extension of multiple response model sparse Bayesian learning. Real-world experimental results demonstrate that the proposed method outperforms its existing counterparts for speech source localization in the real room environment.

A Bayesian Hierarchical Model for Speech Enhancement With Time-Varying Audio Channel

We present a fully Bayesian hierarchical approach for multichannel speech enhancement with time-varying audio channel. Our probabilistic approach relies on a Gaussian prior for the speech signal and a Gamma hyperprior for the speech precision, combined with a multichannel linear-Gaussian state-space model for the acoustic channel. Furthermore, we assume a Wishart prior for the noise precision matrix. We derive a variational expectation-maximization VEM algorithm that uses a variant of a multichannel Wiener filter MCWF to infer the sound source and a Kalman smoother to infer the acoustic channel. It is further shown that the VEM speech estimator can be recasted as a multichannel minimum variance distortionless response MVDR beamformer followed by a single-channel variational postfilter. The proposed algorithm was evaluated using both simulated and real room environments with several noise types and reverberation levels. Both static and dynamic scenarios are considered. In terms of speech quality, it is shown that a significant improvement is obtained with respect to the noisy signal, and that the proposed method outperforms a baseline algorithm. In terms of channel alignment and tracking ability, a superior channel estimate is demonstrated.

Bidirectional microphone array with adaptation controlled by voice activity detector based on multiple beamformers

Ambient noise suppression in a reverberant room is usually performed by the microphone array. The adaptive beamforming, whose typical representative is minimum variance distortionless (MVDR) beamformer, is an effective method for noise suppression. However, MVDR beamformer gives poor results in the real room because of its sensitivity to the steering error and the multipath wave propagation. In this paper we propose a noise suppression method based on assumption that the positions of the speakers in the reverberant room are roughly known. Noise reduction is realized by two MVDR beamformers directed toward each of the speakers. Adaptation of the MVDR beamformers are controlled by a speaker activity detector which decision is based on power transfer model of the multiple superdirective beamformers in combined diffuse and coherent noise field. The proposed voice activity detector also provides residual noise reduction. The proposed method and its robustness to steering error were tested on the model of simulated room as well as in real room environment. The improvement of the restored speech signal was evaluated by Signal to Noise Ratio Enhancement (SNRE) and by Perceptual evaluation of speech quality (PESQ) measure.

Frequency-Dependent Amplitude Panning for the Stereophonic Image Enhancement of Audio Recorded Using Two Closely Spaced Microphones

In this paper, we propose a new frequency-dependent amplitude panning method for stereophonic image enhancement applied to a sound source recorded using two closely spaced omni-directional microphones. The ability to detect the direction of such a sound source is limited due to weak spatial information, such as the inter-channel time difference (ICTD) and inter-channel level difference (ICLD). Moreover, when sound sources are recorded in a convolutive or a real room environment, the detection of sources is affected by reverberation effects. Thus, the proposed method first tries to estimate the source direction depending on the frequency using azimuth-frequency analysis. Then, a frequency-dependent amplitude panning technique is proposed to enhance the stereophonic image by modifying the stereophonic law of sines. To demonstrate the effectiveness of the proposed method, we compare its performance with that of a conventional method based on the beamforming technique in terms of directivity pattern, perceived direction, and quality degradation under three different recording conditions (anechoic, convolutive, and real reverberant). The comparison shows that the proposed method gives us better stereophonic images in a stereo loudspeaker reproduction than the conventional method without any annoying effects.

Underdetermined Convolutive BSS: Bayes Risk Minimization Based on a Mixture of Super-Gaussian Posterior Approximation

This paper considers the underdetermined blind source separation (BSS) of convolutively mixed super-Gaussian signals that include speech, audio, and various other sparse signals. Here, the separation is performed in three steps. In the first and second steps, the mixing matrix and the sources at each time-frequency location are estimated by minimizing the Bayes risk (or the posterior risk) with squared loss. In the final third step, the permutation alignment is conducted by considering the correlation between adjacent spectral bins as in many conventional algorithms. To overcome any computationally intractable integrations involving a complex-valued super-Gaussian source prior, the posterior distribution of the sources is approximated as a mixture of super-Gaussians. The posterior means of the mixing matrix and the sources are obtained with Metropolis-Hastings within Gibbs sampling and the weighted sum of individual super-Gaussians, respectively. Overall, this approximation leads to a separation that is computationally lighter than and as accurate as the algorithm without the approximation. The simulation results of the synthetically generated data in a virtual room with reverberation show that the estimates of the mixing matrix in the first step and the sources in the second step are more accurate than the estimates from the state-of-the-art algorithms in terms of the mixing error ratio (MER) and the signal-to-distortion ratio (SDR). The experiment was also conducted with recorded data in a real room environment using a public benchmark dataset. Results show that the proposed algorithm gives a better performance compared to the state-of-the-art algorithms in terms of the SDR.

A hybrid algorithm for blind source separation of a convolutive mixture of three speech sources

In this paper we present a novel hybrid algorithm for blind source separation of three speech signals in a real room environment. The algorithm in addition to using second-order statistics also exploits an information-theoretic approach, based on higher order statistics, to achieve source separation and is well suited for real-time implementation due to its fast adaptive methodology. It does not require any prior information or parameter estimation. The algorithm also uses a novel post-separation speech harmonic alignment that results in an improved performance. Experimental results in simulated and real environments verify the effectiveness of the proposed method, and analysis demonstrates that the algorithm is computationally efficient.

Online Speech Dereverberation Algorithm Based on Adaptive Multichannel Linear Prediction

This paper proposes a real-time acoustic channel equalization method that uses an adaptive multichannel linear prediction technique. In general, multichannel equalization algorithms can eliminate reverberation if they meet the following specific conditions including: the co-primeness between channels and sufficient filter length. It also requires the characteristic of correct channel information, however, it is difficult to estimate accurate acoustic channels in a practical system. The proposed method utilizes a theoretically perfect channel equalization algorithm and considers problems that may arise in the actual system. Linear-predictive multi-input equalization (LIME) is also an appropriate attempt at blind dereverberation by assuring the theoretical basis. However, a huge computational cost is incurred by calculating the large dimensions of a covariance matrix and its inversion. The proposed equalizer is developed as a multichannel linear prediction (MLP) oriented structure with a new formula that is optimized to time-varying acoustical room environments. Moreover, experimental results show that the proposed method works well even if the channel characteristics of each microphone are similar. The results of experiments using various room impulse response (RIR) models, including both the synthesized and real room environments, show that the proposed method is superior to conventional methods.

Localization of multiple acoustic sources in a room environment

Combustion systems results in high pollution and low performance due to lack of uniform thermal and chemical behavior in the entire reactor zone. Localization of multiple acoustic sources using an array of microphones and time difference of arrival (TDOA) method is examined. Adaptations of such a scheme into high speed jet flows, combustors or reactors can help assist in the identification of local acoustic sources in a system to seek improved performance of the system. Linear microphones can be used for acoustic source localization in a practical room environment using time delay estimation. Hyperbolic position algorithm based on TDOA provides good localization of a single acoustic in both two dimensional and three dimensional space. The time delay between the signals acquired by two microphones is obtained from maximum of cross-correlation between the two signals. Noise and reverberation effects can introduce ambiguity in the time delay estimation from multiple peaks in the delay estimator. In addition directional incidence from multiple sources can result in cross talk between different sources to degrade localization accuracy. CPSP (Cross-power spectral phase analysis) is examined here for localization of multiple sound sources present in a room environment. The CPSP method localizes a sound source as a crossing point of sound directions estimated using different microphone pairs but it degrades due to cross-correlation between different sound sources. A new method based on averaging of CPSP coefficients has been examined here to reduce the directional ambiguity of the wave front direction angle of multiple sound sources. Experiment results in a real room environment validated the accuracy and precision of the above method for the estimation of directions of multiple sound sources. Experiment results also showed that noise and reverberation effects can be incorporated by averaging CPSP data over larger pairs of microphones. Subsequently localization of multiple sources was accomplished in a room environment by using hyperbolic position estimation. These results provide a foundation to develop methodologies for smart, low noise, low pollution and highly efficient combustors and gasifiers through seeking uniform reaction field in the entire reaction zone.

Objective Assessment of Laparoscopic Skills

Assessment of surgical performance is often accomplished with traditional methods that often provide only subjective data. Trainees who perform well on a simulator in a controlled environment may not perform well in a real operating room environment with distractions. This project uses the ideas of dual-task methodology and applies them to the assessment of performance of laparoscopic surgical skills. The level of performance on distracting secondary tasks while trying to perform a primary task becomes an indirect but objective measure of the surgical skill of the trainee. Nine surgery residents and 6 experienced laparoscopic surgeons performed 3 primary tasks on a laparoscopic virtual reality simulator (camera position, grasping, and cholecystectomy) while being distracted by 3 secondary tasks (counting beeps, selective responses, and mental arithmetic). Completion time and error rates were recorded for each combination of tasks. When performed separately, time to completion and error rates for primary and secondary tasks were similar for learners and experts. When performing the tasks simultaneously, learners had more errors than experts. Error rates increased for learners when distracting tasks became more difficult or required more attention. Expert surgeons maintained consistent error rates despite the increasing difficulty of task combinations. The use of dual-task methodology may help trainers to identify which surgical trainees require more preparation before entering the real operating room environment. Expert surgeons are capable of maintaining performance levels on a primary task in the face of distractions that may occur in the operating room.

Implementation of target sound extraction system in frequency domain and its performance evaluation in actual room environments

AbstractIn this paper, we present a real‐time implementation of target sound extraction operating in a frequency domain using a microphone array, and describe its performance evaluation in actual room environments. In the realized system, the target sound extraction can be achieved by using the FDGSC (Frequency Domain Generalized Sidelobe Canceler) which is one of GSCS. In each frequency band, the FDGSC is composed of a fixed beamformer part for target sound enhancement and an adaptive filter part for noisy sound reduction. Since the FDGSC is required to apply the DFT (Discrete Fourier Transform) in every sound data block, the processing for a series of block data must be completed within the transfer time of block data in the real‐time system. Thus, we implemented the FDGSC using a general‐purpose DSP (Digital Signal Processor) evaluation board which enabled us to perform high‐speed signal processing. Superior performance of real‐time processing and target sound extraction are shown by several experimental results in real room environments. © 2011 Wiley Periodicals, Inc. Electron Comm Jpn, 94(7): 9–16, 2011; Published online in Wiley Online Library (wileyonlinelibrary.com). DOI 10.1002/ecj.10334

Stable sound source tracking based on two updating algorithms

AbstractIn this paper, we present a stable sound source tracking method based on two updating algorithm. In the method, an eigenvector spanning the signal subspace is updated by using the Projection Approximation Subspace Tracking (PAST) algorithm without the eigen‐decomposition of a correlation matrix. Then, a constrained minimization problem is formulated taking into account the stability, and the solution is determined successively by applying the Interior Point Least Square (IPLS) algorithm. As a result, stable tracking can be achieved without the peak search of a MUltiple SIgnal Classification (MUSIC) spectrum, which often requires enormous computational costs because many complex multiplying operations are involved. Several experimental results in real room environments have demonstrated the high accuracy and the low computational costs of the proposed method. © Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

Underdetermined Convolutive Blind Source Separation via Time–Frequency Masking

In this paper, we consider the problem of separation of unknown number of sources from their underdetermined convolutive mixtures via time-frequency (TF) masking. We propose two algorithms, one for the estimation of the masks which are to be applied to the mixture in the TF domain for the separation of signals in the frequency domain, and the other for solving the permutation problem. The algorithm for mask estimation is based on the concept of angles in complex vector space. Unlike the previously reported methods, the algorithm does not require any estimation of the mixing matrix or the source positions for mask estimation. The algorithm clusters the mixture samples in the TF domain based on the Hermitian angle between the sample vector and a reference vector using the well known <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</i> -means or fuzzy <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">c</i> -means clustering algorithms. The membership functions so obtained from the clustering algorithms are directly used as the masks. The algorithm for solving the permutation problem clusters the estimated masks by using <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</i> -means clustering of small groups of nearby masks with overlap. The effectiveness of the algorithm in separating the sources, including collinear sources, from their underdetermined convolutive mixtures obtained in a real room environment, is demonstrated.

Determining Mixing Parameters From Multispeaker Data Using Speech-Specific Information

In this paper, we propose an approach for processing multispeaker speech signals collected simultaneously using a pair of spatially separated microphones in a real room environment. Spatial separation of microphones results in a fixed time-delay of arrival of speech signals from a given speaker at the pair of microphones. These time-delays are estimated by exploiting the impulse-like characteristic of excitation during speech production. The differences in the time-delays for different speakers are used to determine the number of speakers from the mixed multispeaker speech signals. There is difference in the signal levels due to differences in the distances between the speaker and each of the microphones. The differences in the signal levels dictate the values of the mixing parameters. Knowledge of speech production, especially the excitation source characteristics, is used to derive an approximate weight function for locating the regions specific to a given speaker. The scatter plots of the weighted and delay-compensated mixed speech signals are used to estimate the mixing parameters. The proposed method is applied on the data collected in actual laboratory environment for an underdetermined case, where the number of speakers is more than the number of microphones. Enhancement of speech due to a speaker is also examined using the information of the time-delays and the mixing parameters, and is evaluated using objective measures proposed in the literature.

周波数領域目的音抽出システムの試作と実環境性能評価

In this paper, we present a real-time implementation of target sound extraction operating in a frequency domain using a microphone array, and describe its performance evaluation in actual room environments. In the realized system, the target sound extraction can be achieved by using the FDGSC (Frequency Domain Generalized Sidelobe Canceler) which is one of GSCS. In each frequency band, the FDGSC is composed of a fixed beamformer part for target sound enhancement and an adaptive filter part for noisy sound reduction. Since the FDGSC is required to apply the DFT (Discrete Fourier Transform) in every sound data block, the processing for a series of block data must be completed within the transfer time of block data in the real-time system. Thus, we implemented the FDGSC using a general-purpose DSP (Digital Signal Processor) evaluation board which enabled us to perform high-speed signal processing. Superior performance of real-time processing and target sound extraction are shown by several experimental results in real room environments. © 2011 Wiley Periodicals, Inc. Electron Comm Jpn, 94(7): 9–16, 2011; Published online in Wiley Online Library (wileyonlinelibrary.com). DOI 10.1002/ecj.10334

Blind Signal Separation Using Steepest Descent Method

A method that significantly improves the convergence rate of the gradient-based blind signal separation (BSS) algorithm for convolutive mixtures is proposed. The proposed approach is based on the steepest descent algorithm suitable for constrained BSS problems, where the constraints are included to ease the permutation effects associated with the convolutive mixtures. In addition, the method is realized using a modified golden search method plus parabolic interpolation, and this allows the optimum step size to be determined with only a few calculations of the cost function. Evaluation of the proposed procedure in simulated environments and in a real room environment shows that the proposed method results in significantly faster convergence for the BSS when compared with a fixed step-size gradient-based algorithm. In addition, for blind signal extraction where only a main speech source is desired, a combined scheme consisting of the proposed BSS and a postprocessor, such as an adaptive noise canceller, offers impressive noise suppression levels while maintaining low-target signal distortion levels.

Postfiltering Using Multichannel Spectral Estimation in Multispeaker Environments

This paper investigates the problem of enhancing a single desired speech source from a mixture of signals in multispeaker environments. A beamformer structure is proposed which combines a fixed beamformer with postfiltering. In the first stage, the fixed multiobjective optimal beamformer is designed to spatially extract the desired source by suppressing all other undesired sources. In the second stage, a multichannel power spectral estimator is proposed and incorporated in the postfilter, thus enabling further suppression capability. The combined scheme exploits both spatial and spectral characteristics of the signals. Two new multichannel spectral estimation methods are proposed for the postfiltering using, respectively, inner product and joint diagonalization. Evaluations using recordings from a real-room environment show that the proposed beamformer offers a good interference suppression level whilst maintaining a low-distortion level of the desired source.

Speedup convergence and reduce noise for enhanced speech separation and recognition

Novel techniques are proposed to enhance time-domain adaptive decorrelation filtering (ADF) for separation and recognition of cochannel speech in reverberant room conditions. The enhancement techniques include whitening filtering on cochannel speech to improve condition of adaptive estimation, block-iterative formulation of ADF to speed up convergence, and integration of multiple ADF outputs through post filtering to reduce reverberation noise. Experimental data were generated by convolving TIMIT speech with acoustic path impulse responses measured in real room environment, with approximately 2 m microphone-source distance and initial target-to-interference ratio of about 0 dB. The proposed techniques significantly improved ADF convergence rate, target-to-interference ratio, and accuracy of phone recognition

Audio source separation: solutions and problems

AbstractThe problem of separating out a number of audio sources observed from an array of microphones in a real room environment has received a great deal of attention in the past decade. While there are now a number of workable methods that can even deal with relatively high reverberation (IEEE Trans Audio Speech Process, 2003; 11:489–497), a number of interesting problems still remain. In this paper, the authors review the methods based around independent component analysis, discussing the various choices available in algorithm design. We then explore the issue of sensitivity to speaker movement which appears to impose fundamental limitations on BSS performance. Copyright © 2004 John Wiley &amp; Sons, Ltd.