Abstract
Wearable flexible sensors attached on the neck have been developed to measure the vibration of vocal cords during speech. However, high-frequency attenuation caused by the frequency response of the flexible sensors and absorption of high-frequency sound by the skin are obstacles to the practical application of these sensors in speech capture based on bone conduction. In this paper, speech enhancement techniques for enhancing the intelligibility of sensor signals are developed and compared. Four kinds of speech enhancement algorithms based on a fully connected neural network (FCNN), a long short-term memory (LSTM), a bidirectional long short-term memory (BLSTM), and a convolutional-recurrent neural network (CRNN) are adopted to enhance the sensor signals, and their performance after deployment on four kinds of edge and cloud platforms is also investigated. Experimental results show that the BLSTM performs best in improving speech quality, but is poorest with regard to hardware deployment. It improves short-time objective intelligibility (STOI) by 0.18 to nearly 0.80, which corresponds to a good intelligibility level, but it introduces latency as well as being a large model. The CRNN, which improves STOI to about 0.75, ranks second among the four neural networks. It is also the only model that is able to achieves real-time processing with all four hardware platforms, demonstrating its great potential for deployment on mobile platforms. To the best of our knowledge, this is one of the first trials to systematically and specifically develop processing techniques for bone-conduction speed signals captured by flexible sensors. The results demonstrate the possibility of realizing a wearable lightweight speech collection system based on flexible vibration sensors and real-time speech enhancement to compensate for high-frequency attenuation.
Highlights
Voice is one of the most important biological signals involved in the process of communication between people as well as in human–computer interfaces.1,2 The human voice can be transmitted by air through acoustic waves and collected by microphones
Besides an fully connected neural network (FCNN), our research has demonstrated that using a long short-term memory (LSTM) and a bidirectional long short-term memory (BLSTM) to enhance the signal from rigid throat microphones gave better results than other methods based on a Gaussian mixed model (GMM) or FCNN,30,31 since both the LSTM and BLSTM can learn contextual information from series of features
Considering that the signal enhanced by the convolutional-recurrent neural network (CRNN) has a higher short-time objective intelligibility (STOI) value compared with the FCNN, the CRNN is worthy of further investigation in the quest to develop a complete mobile and flexible speech processing system
Summary
Voice is one of the most important biological signals involved in the process of communication between people as well as in human–computer interfaces. The human voice can be transmitted by air through acoustic waves and collected by microphones. Similar to other throat microphones, voice signals transmitted through soft tissues such as skin and muscles are subject to a low-pass effect due to damping caused by soft tissues, leading to high-frequency attenuation and loss of consonant syllables. These problems become more significant when flexible vibration sensors are involved, since soft vibrational structures in the sensors tend to have lower resonance frequencies. This paper is one of the first studies to systematically investigate speech processing techniques for flexible sensors It may benefit the development of enhancement algorithms and miniaturized skin-like electronic systems to facilitate communication between people and human–computer interfaces
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
