Speech Extraction Research Articles

Research of Feature Extraction in Mongolian Speech Based on an Improved Algorithm of MFCC Parameter

The extraction of characteristic parameters is extremely important front-end of the speech recognition system, accurate parameters extraction of Mongolian speech have the important meaning to its recognition technology development. This paper clarified Mel frequency cepstrum coefficients(MFCC) features and extraction method at first, then it will treated with the first-order and second-order difference, after that, combining these parameters will get a new feature parameter vector. Analyze and extract this eigenvector from Mongolian speech signal by means of MATLAB. Experimental results reveal that the more the feature parameter is, the more precise the description for static and dynamic characteristics of ear to speech is.

SPEECH SHOT EXTRACTION FROM BROADCAST NEWS VIDEOS

We propose a method for discriminating between a speech shot and a narrated shot to extract genuine speech shots from a broadcast news video. Speech shots in news videos contain a wealth of multimedia information of the speaker, and could thus be considered valuable as archived material. In order to extract speech shots from news videos, there is an approach that uses the position and size of a face region. However, it is difficult to extract them with only such an approach, since news videos contain non-speech shots where the speaker is not the subject that appears in the screen, namely, narrated shots. To solve this problem, we propose a method to discriminate between a speech shot and a narrated shot in two stages. The first stage of the proposed method directly evaluates the inconsistency between a subject and a speaker based on the co-occurrence between lip motion and voice. The second stage of the proposed method evaluates based on the intra- and inter-shot features that focus on the tendency of speech shots. With the combination of both stages, the proposed method accurately discriminates between a speech shot and a narrated shot. In the experiments, the overall accuracy of speech shots extraction by the proposed method was 0.871. Therefore, we confirmed the effectiveness of the proposed method.

A New Framework for Underdetermined Speech Extraction Using Mixture of Beamformers

This paper describes frequency-domain nonlinear mixture of beamformers that can extract a speech source from a known direction when there are fewer microphones than sources (the underdetermined case). Our approach models the data in each frequency bin via Gaussian mixture distributions, which can be learned using the expectation maximization algorithm. The model learning is performed using the observed mixture signals only, and no prior training is required. Nonlinear beamformers are then developed based on this model. The proposed estimators are a nonlinear weighted sum of linear minimum mean square error or minimum variance distortionless response beamformers. The resulting nonlinear beamformers do not need to know or estimate the number of sources, and can be applied to microphone arrays with two or more microphones. We test and evaluate the described methods on underdetermined speech mixtures.

Realization of Sensitive Speech Extraction Under GUI

Under condition that speech is pure, this paper applies short time Fourier transform to obtain the frequency distribution of pure speech and analyse it, then the needed frequency zone is extracted by FIR digital filter, so different sound effect can be produced. What’more, the sound form isn’t limited. The transform and extraction of speech are realized by Graphic User Interface (GUI) of Matlab, which makes data modification convenient, operation simple and flexible, effect intuitive and clear.

Relaxing the WDO Assumption in Blind Extraction of Speakers from Speech Mixtures

The time-frequency masking approach in blind speech extraction consists of two main steps: feature clustering in a space spanned over delay-time and attenuationrate, and spectrogram masking in order to reconstruct the sources. Usually a binary mask is generated under the strong W-disjoint orthogonal (WDO) assumption (disjoint orthogonal representations in the frequency domain). In practice, this assumption is most often violated leading to weak quality of reconstructed sources. In this paper we propose the WDO to be relaxed by allowing some frequency bins to be shared by both sources. As we detect instantaneous fundamental frequencies the mask creation is supported by exploring a harmonic structure of speech. The proposed method is proved to be effective and reliable in experiments with both simulated and real acquired mixtures.

Automatic Beamforming for Blind Extraction of Speech From Music Environment Using Variance of Spectral Flux-Inspired Criterion

This paper addresses the problem of automatic beamforming for blind extraction of speech in a music environment, using multiple microphones. A new criterion is proposed based on the variance of the spectral flux (VSF), which is shown to be a compound measure of the Kurtosis and across-time correlation for the time-frequency domain signals. Spectral flux (SF) had been adopted as a feature that distinguishes speech from other acoustic noises and the VSF of speech tends to be larger than that of other acoustic sounds. Henceforth, maximization of VSF can be employed as one potential criterion to identify the speech direction-of-arrival (DOA), in order to extract speech from the noisy observations. We construct a VSF-inspired cost function and develop a complex-value fixed-point algorithm for the optimization. Then, the stability of the proposed algorithm is analyzed based on the second-order Taylor series expansion. Rather than the DOA identification ambiguity caused by subspace decomposition-based methods or maximization of non-Gaussianity-based approaches, both real and simulated evaluations indicate that the VSF-inspired criterion can effectively extract speech from a music diffuse noise field or a musical interference noise field. A key feature of the proposed approach is that it can operate blindly, i.e., it does not require a priori knowledge about the array geometry, the noise covariance matrix, or the geometrical knowledge of the location of desired speech. Therefore, this study offers a potential perspective for blindly extracting speech from a music environment.

Improving speech intelligibility for hearing aid users using a speech segregation algorithm.

Users of hearing aid devices encounter great difficulty in speech recognition in noisy environments, and show inferior performance in situations with interfering background noise or competing speakers as compared to the case of a quiet background. In such devices, a pre‐processing stage, which first removes the background interference, should greatly boost the speech recognition performance in noisy environments. In this work, the performance of a previously reported speech extraction algorithm is investigated for the purpose of improving the speech intelligibility of hearing aid users. Recently, we performed segregation of speech mixtures using our speech extraction algorithm and showed improvement in the speech recognition performance of normal hearing listeners. In this work, we investigate the performance of our algorithm for users of hearing aids on the same perceptual task of recognizing speech in the presence of either interfering speech or noise. We focus on tuning the segregation algorithm for hearing aid users, and emphasize on the preservation of the appropriate envelope and fine structure cues which will prove most helpful in the recognition of the target speech signal.

Blind Spatial Subtraction Array for Speech Enhancement in Noisy Environment

We propose a new blind spatial subtraction array (BSSA) consisting of a noise estimator based on independent component analysis (ICA) for efficient speech enhancement. In this paper, first, we theoretically and experimentally point out that ICA is proficient in noise estimation under a non-point-source noise condition rather than in speech estimation. Therefore, we propose BSSA that utilizes ICA as a noise estimator. In BSSA, speech extraction is achieved by subtracting the power spectrum of noise signals estimated using ICA from the power spectrum of the partly enhanced target speech signal with a delay-and-sum beamformer. This ldquopower-spectrum-domain subtractionrdquo procedure enables better noise reduction than the conventional ICA with estimation-error robustness. Another benefit of BSSA architecture is ldquopermutation robustness". Although the ICA part in BSSA suffers from a source permutation problem, the BSSA architecture can reduce the negative affection when permutation arises. The results of various speech enhancement test reveal that the noise reduction and speech recognition performance of the proposed BSSA are superior to those of conventional methods.

A speech feature extraction method using complexity measure for voice activity detection in WGN

A novel speech extraction algorithm is proposed in this paper for Voice Activity Detection (VAD). Signal complexity analysis with definition of Kolmogorov complexity is adopted, which explores model characteristics of speech production to differentiate speech and white Gaussian noise (WGN). In the view of speech signal processing, properties of speech’s source and vocal tract are explored by complexity analysis. Also, some interesting properties of signal complexity are presented with experimental study, including complexity analysis of general noise-corrupted signal. Moreover, some enhanced features with complexity and a feature incorporation method are presented. These features incorporate some unique characteristics of speech, like pitch information, vocal organ information, and so on. With a large database of speech signals and synthetic/real Gaussian noise, distributions of novel features and receiver operating characteristics (ROC) curves are shown, which are proved as potential features for voice activity detection.

Statistical Analysis of a Local Quadratic Criterion for Blind Speech Extraction

This letter aims at complementing previous empirical work regarding a certain beamforming technique for blind speech extraction that uses a local quadratic approximation of a Kurtosis expression. It is shown here that the proposed method possesses a fixed-point property which means that it remains at an optimal solution once this solution has been reached. The proposed method's fixed-point property is valid for a range of source signals including Gaussian sources. This is an improvement over the FastICA method which diverges at the optimal points that correspond to a Gaussian source. In a real application, it cannot be assured that non-Gaussian mixtures are constantly observed; hence, the proposed method is a viable alternative in that case. The fixed-point property further implies that the approximative Kurtosis expression is identical to the true Kurtosis value at an optimal point which, in turn, means that the approximation error is zero. In addition, the convergence towards an optimal solution is always in the direction of a local minimum point even though the optimal solution that correspond to a super-Gaussian source is always a maximum solution which harmonizes with the concept of Kurtosis maximization.

Noise Statistics Update Adaptive Beamformer With PSD Estimation for Speech Extraction in Noisy Environment

This paper addresses the problem of extracting a desired speech source from a multispeaker environment in the presence of background noise. A new adaptive beamforming structure is proposed for this speech enhancement problem. This structure incorporates power spectral density (PSD) estimation of the speech sources together with a noise statistics update. An inactive-source detector based on minimum statistics is developed to detect the speech presence and to track the noise statistics. Performance of the proposed beamformer is investigated and compared to the minimum variance distortionless response (MVDR) beamformer with or without a postfilter in a real hands-free communication environment. Evaluations show that the proposed beamformer offers good interference and noise suppression levels while maintaining low distortion of the desired source.

Complex-Valued Independent Component Analysis for Online Blind Speech Extraction

This paper presents a theoretical analysis of a certain criterion for complex-valued independent component analysis (ICA) with a focus on blind speech extraction (BSE) of a spatio-temporally nonstationary speech source. In the paper, the proposed criteria denoted KSICA is related to the well-known FastICA method with the Kurtosis contrast function. The proposed method is shown to share the important fixed-point feature with the FastICA method, although an improvement with the proposed method is that it does not exhibit the divergent behavior for a of Gaussian-only sources that the FastICA method tends to do, and it shows better performance in online implementations. Compared to the FastICA, the KSICA method provides a 10 dB higher source extraction performance and a 10 dB lower standard deviation in a data batch approach when the data batch size is less than 100 samples. For larger batch sizes, the KSICA metod performs equally well. In an online application with spatially stationary sources the KSICA method provides around 10 dB higher interference suppression, and 1 MOS-unit lower speech distortion compared to the FastICA for 0.15 s time constant in the algorithm update parameter. Thus, the FastICA performance matches the KSICA performance for a time constant above 1 s. Finally, in an online application with a moving speech source, the KSICA method provides 10 dB higher interference suppression, compared to the FastICA for the same algorithm settings. All in all, the proposed KSICA method is shown to be a viable alternative for online BSE of complex-valued signal mixtures.

Low-level information and high-level perception: the case of speech in noise.

Auditory information is processed in a fine-to-crude hierarchical scheme, from low-level acoustic information to high-level abstract representations, such as phonological labels. We now ask whether fine acoustic information, which is not retained at high levels, can still be used to extract speech from noise. Previous theories suggested either full availability of low-level information or availability that is limited by task difficulty. We propose a third alternative, based on the Reverse Hierarchy Theory (RHT), originally derived to describe the relations between the processing hierarchy and visual perception. RHT asserts that only the higher levels of the hierarchy are immediately available for perception. Direct access to low-level information requires specific conditions, and can be achieved only at the cost of concurrent comprehension. We tested the predictions of these three views in a series of experiments in which we measured the benefits from utilizing low-level binaural information for speech perception, and compared it to that predicted from a model of the early auditory system. Only auditory RHT could account for the full pattern of the results, suggesting that similar defaults and tradeoffs underlie the relations between hierarchical processing and perception in the visual and auditory modalities.

Audio input device and method of controlling the same

An audio input device calculates respective degrees of coincidence between a user's vocalizations and recognition terms in a voice dictionary by speech recognition and extracts a recognition term having the highest degree of coincidence from the voice dictionary. Further, the audio input device outputs a command to operate one of installations, such as radio, CD player, air conditioner, etc. The audio input device includes a remote controller for transmitting signal waves to the installations equipped with receivers to carry out remote control of the installations. In operation, the larger the installation which the remote controller is directed to has a receiving intensity of the signal wave detected byte receiver, the larger the audio input device makes an extraction weight of the recognition terms concerned with the installation, in the voice dictionary.

Speech Extraction of a Target Speaker from One Mixed Speech Signal

This paper proposes a method of speech extraction of a target speaker from one mixed speech signal of some speakers. The proposed method uses a dictionary consisted of feature vectors of a speech signal. The feature vectors are spectra of a segment of the speech signal which contains all phonemes. The dictionary is made in advance from a clear speech signal of the target speaker, and is employed in extracting the target speech signal from one mixed speech signal. The effectiveness of the proposed method has been verified by computer simulations.

Speech Segregation Using an Auditory Vocoder With Event-Synchronous Enhancements.

We propose a new method to segregate concurrent speech sounds using an auditory version of a channel vocoder. The auditory representation of sound, referred to as an "auditory image," preserves fine temporal information, unlike conventional window-based processing systems. This makes it possible to segregate speech sources with an event synchronous procedure. Fundamental frequency information is used to estimate the sequence of glottal pulse times for a target speaker, and to repress the glottal events of other speakers. The procedure leads to robust extraction of the target speech and effective segregation even when the signal-to-noise ratio is as low as 0 dB. Moreover, the segregation performance remains high when the speech contains jitter, or when the estimate of the fundamental frequency F0 is inaccurate. This contrasts with conventional comb-filter methods where errors in F0 estimation produce a marked reduction in performance. We compared the new method to a comb-filter method using a cross-correlation measure and perceptual recognition experiments. The results suggest that the new method has the potential to supplant comb-filter and harmonic-selection methods for speech enhancement.

Audio-visual speech perception off the top of the head

The study examined whether people can extract speech related information from the talker's upper face that was presented using either normally textured videos (Experiments 1 and 3) or videos showing only the outlined of the head (Experiments 2 and 4). Experiments 1 and 2 used within- and cross-modal matching tasks. In the within-modal task, observers were presented two pairs of short silent video clips that showed the top part of a talker's head. In the cross-modal task, pairs of audio and silent video clips were presented. The task was to determine the pair in which the talker said the same sentence. Performance on both tasks was better than chance for the outline as well as textured presentation suggesting that judgments were primarily based on head movements. Experiments 3 and 4 tested if observing the talker's upper face would help identify speech in noise. The results showed the viewing the talker's moving upper head produced a small but reliable improvement in speech intelligibility, however, this effect was only secure for the expressive sentences that involved greater head movements. The results suggest that people are sensitive to speech related head movements that extend beyond the mouth area and can use these to assist in language processing.

Improved BTE hearing-aid directivity using a directional microphone array

Extraction of speech in noise is of great importance to hearing-impaired listeners. Directional microphones are incorporated in some hearing aids to improve noise rejection through increased directivity. A symmetric cardioid response is created either with a single directional microphone, or using beamforming with two omnidirectional microphones. The head-related transfer function (HRTF), however, introduces an asymmetry that cannot be exploited by a linear array of omnidirectional microphones. A new BTE array consisting of a gradient directional microphone with nulls in the front–back vertical plane and two omnidirectional microphones exploits the asymmetry of the HRTF to obtain almost 2 dB better directivity than the best cardioid. HRTFs measured on KEMAR with this array were transformed to the frequency domain, where directivity-maximizing coefficients in each band were derived. The Articulation-Index (AI) weighted directivity gain of this optimal three-element directional array was 6.4 dB greater than a single omnidirectional microphone on the BTE, whereas the directivity gain of the HRTF-optimized two-omni beamformer was 4.6 dB, and the optimal free-field cardioid placed on the head yielded 4.4 dB. [Work partially supported by NIH (NIDCD) under Grant No. 1 R01 DC005762-01A1.]

Development of context dependent sequential K-nearest neighbor classifier for usable speech classification

The accuracy of speech processing applications degrades when operating in co-channel environment. Co-channel speech occurs when more than one person is talking at the same time. The idea of usable speech segmentation is to identify and extract those portions of co-channel speech that are minimally degraded but still useful for speech processing applications (such as speaker identification or speech recognition) which do not work in co-channel environments. Usable speech measures are features that are extracted from the co-channel signal to distinguish between usable and unusable speech. Several usable speech extraction methods have recently been developed based on a single feature of the speech signal being considered. In this paper, however, a new usable speech extraction technique, which sequentially and contextually selects several features of the given signal using the K-nearest neighbor classifier, is being investigated. This new approach considers periodicity and structure based features simultaneously in order to achieve the maximum classification rate, and by observing all the incoming frames, avoids the problem of deciding the amount of data needed to make accurate decisions. A 100% accuracy can be achieved in speech processing applications by using this extracted usable speech segment.

The relationship between auditory temporal resolution and auditory object formation

The goal of the present experiment was to determine whether a relationship exists between auditory temporal resolution and the ability to use amplitude modulation for auditory object formation in sentence perception. A standard gap detection task was used to evaluate temporal resolution in listeners [D. P. Phillips, T. L. Taylor, S. E. Hall, M. M. Carr, and J. E. Mossop, J. Acoust. Soc. Am. 101, 3694–3705 (1997)]. Each stimulus consisted of a leading element, a silent duration, and a trailing element. Both within- and between-channel detection was measured. An adaptive tracking procedure (PEST) was used to determine the 70%-correct gap durations for each subject. Next, a sentence identification task was used to evaluate listeners’ abilities to use amplitude modulation to extract speech from noise. Time Varying Sinusoidal (TVS) sentences were created for this purpose [T. D. Carrell and J. M. Opie, Percept. Psychophys. 52, 437–445 (1992)]. Half of the TVS sentences were amplitude modulated at 100 Hz and half remained unmodulated. The difference in intelligibility between the modulated and unmodulated sentences was used as a measure of auditory object formation. No correlation was found between gap discriminability and auditory object improvement among the subjects we tested. A wider range of subjects is currently being investigated.