Non-audible Murmur Microphone Research Articles

Embedded wavelet analysis of non-audible signals

The analysis of non-audible signals has gained a significant importance due to their many fields of application, among them, speech synthesis for people with speech disabilities. This analysis can be used to acquire information from the vocal apparatus without the need of speaking in order to produce a phonetic expression. The analysis of a Wavelet transformation of Spanish words recorded through a non-audible murmur microphone in order to achieve an embedded silent speech recognition system of Spanish language is proposed. A non-audible murmur microphone is used as sensor of non-vocal speech. Coding of the input data is done through a Wavelet transform using a fourth-order Daubechies function. The acquisition, processing and transmission system is applied through a STM32F4-Discovery evaluation board. The used vocabulary consists of command words aimed to control mobile robots or human-machine interfaces. The Wavelet transformation of four Spanish words, each of them having five independent samples, was accomplished. An analysis of the resulting data was performed, and features as average, peaks and frequency were distinguished. The processing of the signals is performed successfully and further work in speech activity detection and features classifiers is proposed.

Using Body-Conducted Acoustic Sensors for Human-Robot Communication in Noisy Environments

In this study, the use of alternative acoustic sensors in human-robot communication is investigated. In particular, a Non-Audible Murmur (NAM) microphone was applied in tele-operating the Geminoid HI-1 robot in noisy environments. The current study introduces the methodology and the results of speech intelligibility subjective tests when a NAM microphone was used in comparison with using a standard microphone. The results show the advantage of using a NAM microphone when the operation takes place in adverse environmental conditions. In addition, the effect of Geminoid's lip movements on speech intelligibility is also investigated. Subjective speech intelligibility tests show that the operator's speech can be perceived with higher intelligibility scores when the operator's audio speech is perceived along with the lip movements of robots.

Numerical Simulation of Air Flow through Glottis during Very Weak Whisper Sound Production

A non-audible murmur (NAM), a very weak whisper sound produced without vocal fold vibration, has been researched in the development of a silent-speech communication tool for functional speech disorders as well as human-to-human/machine interfaces with inaudible voice input. The NAM can be detected using a specially designed microphone, called a NAM microphone, attached to the neck. However, the detected NAM signal has a low signal-to-noise ratio and severely suppressed high-frequency component. To improve NAM clarity, the mechanism of a NAM production must be clarified. In this work, an air flow through a glottis in the vocal tract was numerically simulated using computational fluid dynamics and vocal tract shape models that are obtained by a magnetic resonance imaging (MRI) scan for whispered voice production with various strengths, i.e. strong, weak, and very weak. For a very weak whispering during the MRI scan, subjects were trained, just before the scanning, to produce the very weak whispered voice, or the NAM. The numerical results show that a weak vorticity flow occurs in the supraglottal region even during a very weak whisper production; such vorticity flow provide aeroacoustic sources for a very weak whispering, i.e. NAM, as in an ordinary whispering.

Analysis and Recognition of NAM Speech Using HMM Distances and Visual Information

Non-audible murmur (NAM) is an unvoiced speech signal that can be received through the body tissue with the use of special acoustic sensors (i.e., NAM microphones) attached behind the talker's ear. The authors had previously reported experimental results for NAM recognition using a stethoscopic and a silicon NAM microphone. Using a small amount of training data from a single speaker and adaptation approaches, 93.9% of word accuracy was achieved for a 20 k Japanese vocabulary dictation task. In this paper, further analysis of NAM speech is made using distance measures between hidden Markov models (HMMs). It has been shown that owing to the reduced spectral space of NAM speech, the HMM distances are also reduced when compared with those of normal speech. In the case of Japanese vowels and fricatives, the distance measures in NAM speech follow the same relative inter-phoneme relationship as that in normal speech without significant differences. However, significant differences have been found in the case of Japanese plosives. More specifically, in NAM speech, the distances between voiced/unvoiced consonant pairs articulated in the same place drastically decreased. As a result, the inter-phoneme relationship as compared to normal-speech changed significantly, causing a substantial decrease in the recognition accuracy. A speaker-dependent phoneme recognition experiment has been conducted, obtained 81.5% NAM phoneme correct, showing a relationship between HMM distance measures and phoneme accuracy. In a NAM microphone, body transmission and loss of lip radiation act as a low-pass filter. As a result, higher frequency components are attenuated in a NAM signal. Because of spectral reduction, NAM's unvoiced nature, and the type of articulation, NAM sounds become similar, causing a larger number of confusions when compared with normal speech. Yet many of those sounds are visually different on face/mouth/lips, and the integration of visual information increases their discrimination. As a result, recognition accuracy increases as well. In this article, the visual information extracted from the talkers' facial movements is fused with NAM speech. The experimental results reveal a relative improvement of 10.5% on average when fused NAM speech and facial information were used compared with using only NAM speech.

Voice conversion for enhancing various types of body‐conducted speech detected with non‐audible murmur microphone.

Our proposed statistical voice conversion approach to enhancing various types of body‐conducted speech detected with Non‐Audible Murmur (NAM) microphone is presented in this talk. NAM microphone, one of the body‐conductive microphones [Nakajima et al., IEICE Trans. Inf. and Syst., E89‐D, 1–8 (2006)], enables us to detect various types of body‐conducted speech such as extremely soft whisper, normal speech, and so on. Moreover, it is robust against external noise due to its noise‐proof structure. To make speech communication more universal by effectively using these properties of NAM microphone, body‐conducted speech enhancement techniques have been developed with a state‐of‐the‐art statistical voice conversion algorithm [Toda et al., IEEE Trans. ASLP, 15, 2222–2235 (2007)]. The proposed techniques would bring a new paradigm to human‐to‐human speech communication: e.g., the use of body‐conducted voiced speech for noise robust speech communication, the use of body‐conducted unvoiced speech for silent speech ...

Silent-speech enhancement using body-conducted vocal-tract resonance signals

The physical characteristics of weak body-conducted vocal-tract resonance signals called non-audible murmur (NAM) and the acoustic characteristics of three sensors developed for detecting these signals have been investigated. NAM signals attenuate 50 dB at 1 kHz; this attenuation consists of 30-dB full-range attenuation due to air-to-body transmission loss and −10 dB/octave spectral decay due to a sound propagation loss within the body. These characteristics agree with the spectral characteristics of measured NAM signals. The sensors have a sensitivity of between −41 and −58 dB [V/Pa] at 1 kHz, and the mean signal-to-noise ratio of the detected signals was 15 dB. On the basis of these investigations, three types of silent-speech enhancement systems were developed: (1) simple, direct amplification of weak vocal-tract resonance signals using a wired urethane-elastomer NAM microphone, (2) simple, direct amplification using a wireless urethane-elastomer-duplex NAM microphone, and (3) transformation of the weak vocal-tract resonance signals sensed by a soft-silicone NAM microphone into whispered speech using statistical conversion. Field testing of the systems showed that they enable voice impaired people to communicate verbally using body-conducted vocal-tract resonance signals. Listening tests demonstrated that weak body-conducted vocal-tract resonance sounds can be transformed into intelligible whispered speech sounds. Using these systems, people with voice impairments can re-acquire speech communication with less effort.

Exploiting visual information for NAM recognition

Non-audible murmur (NAM) is an unvoiced speech received through body tissue using special acoustic sensors (i.e., NAM microphones) attached behind the talkers ear. Although NAM has different frequency characteristics compared to normal speech, it is possible to perform automatic speech recognition (ASR) using conventional methods. In using a NAM microphone, body transmission and the loss of lip radiation act as a low-pass filter; as a result, higher frequency components are attenuated in NAM signal. A decrease in NAM recognition performance is attributed to spectral reduction. To address the problem of loss of lip radiation, visual information extracted from the talker's facial movements is fused with NAM speech. Experimental results revealed a relative improvement of 39% when fused NAM speech and facial information were used as compared to using only NAM speech. Results also showed that improvements in the recognition rate depend on the place of articulation.

Numerical simulation of transfer and attenuation characteristics of soft-tissue conducted sound originating from vocal tract

A non-audible murmur (NAM), a very weak speech sound produced without vocal cord vibration, can be detected by a special NAM microphone attached to the neck, thereby providing a new speech communication tool for functional speech disorders as well as human-to-machine and human-to-human interfaces with inaudible voice input for use with unimpaired. The NAM microphone is a condenser microphone covered with soft-silicone impression material that provides good impedance matching with the soft tissues of the neck. Because higher-frequency components are suppressed severely, however, the NAM detected with this device can be insufficiently clear. To improve NAM clarity, the mechanism of NAM production as well as the transfer characteristics of the NAM in soft neck tissues must be clarified. We have investigated sound propagation from the vocal tract to the neck surface, using a finite difference time domain method and a head model based on magnetic resonance imaging scans. Numerical results show that, compared to air-conducted sound detected in front of a mouth, soft-tissue-conducted sound attenuates 50 dB at 1 kHz, which consists of 30 dB full-range attenuation due to air-to-soft-tissues transmission loss and −10 dB/octave spectral decay due to a propagation loss in soft tissues. The decay agrees well with the spectral characteristics of the measured NAM.

Improving quality of small body transmitted ordinary speech with statistical voice conversion

The explosive spread of cellular phones enables us to communicate with each other at any time or place. Although cellular phones are convenient, there are still some problems. For example, it is difficult to send intelligible speech under noisy conditions, which is a fatal problem especially when talking privately using small speech in crowds. To improve the quality of small speech under such situations, a new speech communication style is proposed using a nonaudible murmur (NAM) microphone [Nakajima et al., Eurospeech (2003), pp. IV‐2601–2604]. The NAM microphone is robust to eternal noise, although body transmission causes quality degradation. In this paper, to improve the sound quality of small body transmitted ordinary speech (SBTOS), which is small speech recorded with a NAM microphone, two conversion methods that reflect a statistical voice conversion method based on Gaussian mixture models (GMMs) [Toda et al. Interspeech (2005), pp. 1957–1960] are conducted. One conversion method is from SBTOS to o...

Methods of fitting a nonaudible murmur microphone for daily use and development of urethane elastmer duplex structure type nonaudible murmur microphone

Nonaudible murmur (NAM) microphones capturing vibrations directly from the skin without mediation by the air enable the sampling of non-audible murmur with almost the same amplification rate as normal speech. Because they are filtered through the human body, they are robust against air-conducted noises. Soft silicone has been used, which has acoustic impedance close to human flesh, as a sound medium between the electrode of the condenser microphone and the skin by fixing it in a neckband style. We replaced the soft silicone with urethane elastomer, which has self-adhesiveness, to develop a NAM microphone that is easily wearable in everyday life without expendable supplies or supporting devices. It can be fixed to the skin, and it also decreases body movement noises. But miniaturized or sticking-type NAM microphones increased the mixture of air-conducted noises from the back of the microphone and from a gap between the skin and the rim of the microphone. To solve this problem, we created a duplex structure inside the urethane elastomer wrapping condenser microphone and outside the urethane elastomer for fixation and noise proofing to develop a miniaturized sticking-type NAM microphone that was again robust against air-conducted noise.

Numerical simulation of sound originated from the vocal tract in soft neck tissues

A nonaudible murmur (NAM), a very weak speech sound produced without vocal vibration, can be detected by a special NAM microphone attached to the neck, thereby providing a new communication tool for use with functional speech disorders. The microphone is a condenser microphone covered with soft-silicone impression material that provides good impedance matching with soft tissues that are found in the neck. The NAM detected with the NAM microphone, however, is insufficiently clear because higher-frequency components are suppressed severely. A production mechanism of the NAM as well as transfer characteristics of the NAM in soft tissues of a neck should be clarified to improve the NAM clarity. As a first step, sound propagation from the vocal tract to the neck surface is simulated using a head model scanned with magnetic resonance imaging and finite difference method. Results show that soft-meat-tissue-conducted sound has fewer higher-frequency components than an air-conducted sound. The decay of transfer characteristics is approximately −10 dB/oct in the audible range, which is roughly parallel to the spectral characteristics of the measured NAM. [Work supported by SCOPE-S.]

Study on nonaudible murmur speaker verification using multiple session data

A study on speaker verification with nonaudible murmur (NAM) segments using multiple session data was conducted. NAM is different from normal speech and is difficult for other people to catch. Therefore, a text-dependent verification strategy can be used in which each user utters her/his own keyword phrase so that not only speaker-specific but also keyword-specific acoustic information is utilized. A special device called a NAM microphone worn on the surface of the skin below the mastoid bone is used to catch NAM because it is too low to be recorded using ordinary microphones. However, it is tolerant to exterior noise. This strategy is expected to yield relatively high performance. NAM segments, which consist of multiple short-term feature vectors, are used as input vectors to capture keyword-specific acoustic information well. To handle segments with a large number of dimensions, a support vector machine (SVM) is used. In experiments using NAM data uttered by 19 male and 10 female speakers in several different sessions, robustness against session-to-session data variation is examined. The effect of segment length is also investigated. The proposed approach achieves equal error rates of 0.04% (male) and 1.1% (female) when using 145-ms-long NAM segments.

Unvoiced Speech Recognition Using Tissue-Conductive Acoustic Sensor

We present the use of stethoscope and silicon NAM (nonaudible murmur) microphones in automatic speech recognition. NAM microphones are special acoustic sensors, which are attached behind the talker's ear and can capture not only normal (audible) speech, but also very quietly uttered speech (nonaudible murmur). As a result, NAM microphones can be applied in automatic speech recognition systems when privacy is desired in human-machine communication. Moreover, NAM microphones show robustness against noise and they might be used in special systems (speech recognition, speech transform, etc.) for sound-impaired people. Using adaptation techniques and a small amount of training data, we achieved for a 20 k dictation task a word accuracy for nonaudible murmur recognition in a clean environment. In this paper, we also investigate nonaudible murmur recognition in noisy environments and the effect of the Lombard reflex on nonaudible murmur recognition. We also propose three methods to integrate audible speech and nonaudible murmur recognition using a stethoscope NAM microphone with very promising results.