Bottleneck Feature Extraction Research Articles

IHearken: Chewing sound signal analysis based food intake recognition system using Bi-LSTM softmax network

Background and ObjectiveFood ingestion is an integral part of health and wellness. Continues monitoring of different food types and observing the amount being consumed prevents gastrointestinal diseases and weight-related issues. Food intake recognition (FIR) systems, thus have significant impact on everyday life. The purpose of this study is to develop an automatic approach for the FIR using a contemporary wearable hardware and machine learning technique. This will assist clinicians and concern person to manage health issues associated with food intake. MethodsIn this work, we present a novel hardware iHearken, a headphone-like wearable sensor-based system to monitor eating activities and recognize food intake type in the free-living condition. State-of-the-art hardware is designed for data acquisition where 16 subjects are recruited and 20 different food items are used for data collection. Further, chewing sound signals are analyzed for FIR using bottleneck features. The proposed model is divided into 4 distinct phases: data acquisition, event detection using a pre-trained model, bottleneck feature extraction, and classification based on bidirectional long short-term memory (Bi-LSTM) softmax model. The Bi-LSTM network with softmax function is applied to calculate the identification score for apiece chewing signal which further classifies the chewing signal data into liquid / solid food classes. ResultsThe results of proposed model performance is evaluated in (%) for accuracy, precision, recall and F-score as 97.422, 96.808, 98.0, and 97.512, respectively, and root mean square error (RMSE), and mean absolute percentage error (MAPE) as 0.160 1.030 respectively for numbers of correct food type recognized. Further, we also evaluated our model's performance for food classification into solid and liquid and achieved an accuracy (96.66%), precision (96.40%), recall (95.230%), F-score (95.79%), RMSE (0.182), and MAPE (2.22). We also demonstrated that the food recognition accuracy of different models with the proposed model differed statistically. ConclusionAn informatics complexity study of the proposed model was subsequently explored to review the effectiveness of the proposed wearable device and the methodology. The medical importance of this investigation is the reliable monitoring of the clinical development of the food intake classification methods via food chew event detection in the ambulatory environment has been justified.

COVID-19 detection in cough, breath and speech using deep transfer learning and bottleneck features

We present an experimental investigation into the effectiveness of transfer learning and bottleneck feature extraction in detecting COVID-19 from audio recordings of cough, breath and speech. This type of screening is non-contact, does not require specialist medical expertise or laboratory facilities and can be deployed on inexpensive consumer hardware such as a smartphone. We use datasets that contain cough, sneeze, speech and other noises, but do not contain COVID-19 labels, to pre-train three deep neural networks: a CNN, an LSTM and a Resnet50. These pre-trained networks are subsequently either fine-tuned using smaller datasets of coughing with COVID-19 labels in the process of transfer learning, or are used as bottleneck feature extractors. Results show that a Resnet50 classifier trained by this transfer learning process delivers optimal or near-optimal performance across all datasets achieving areas under the receiver operating characteristic (ROC AUC) of 0.98, 0.94 and 0.92 respectively for all three sound classes: coughs, breaths and speech.This indicates that coughs carry the strongest COVID-19 signature, followed by breath and speech. Our results also show that applying transfer learning and extracting bottleneck features using the larger datasets without COVID-19 labels led not only to improved performance, but also to a marked reduction in the standard deviation of the classifier AUCs measured over the outer folds during nested cross-validation, indicating better generalisation.We conclude that deep transfer learning and bottleneck feature extraction can improve COVID-19 cough, breath and speech audio classification, yielding automatic COVID-19 detection with a better and more consistent overall performance.

Transfer Learning Models for Land Cover and Land Use Classification in Remote Sensing Image

ABSTRACT Land Cover or Land Use (LCLU) classification is an important, challenging problem in remote sensing (RS) images. RS image classification is a recent technology used to extract hidden information from remotely sensed images in the observed earth environment. This classification is essential for sustainable development in agricultural decisions and urban planning using deep learning (DL) methods. DL gets more attention for accuracy and performance improvements in large datasets. This paper is aimed to apply one of the DL methods called transfer learning (TL). TL is the recent research problem in machine learning and DL approaches for image classification. DL consumes much time for training when starting from scratch. This problem could be overcome in the TL modeling technique, which uses pre-trained models to build deep TL models efficiently. We applied the TL model using bottleneck feature extraction from the pre-trained models: InceptionV3, Resnet50V2, and VGG19 to LCLU classification in the UC Merced dataset. With these experiments, the TL model has been built the outdate performance of 92.46, 94.38, and 99.64 in Resnet50V2, InceptionV3, and VGG19, respectively.

Detecting Unipolar and Bipolar Depressive Disorders from Elicited Speech Responses Using Latent Affective Structure Model

Mood disorders, including unipolar depression (UD) and bipolar disorder (BD) [1] , are reported to be one of the most common mental illnesses in recent years. In diagnostic evaluation on the outpatients with mood disorder, a large portion of BD patients are initially misdiagnosed as having UD [2] . As most previous research focused on long-term monitoring of mood disorders, short-term detection which could be used in early detection and intervention is thus desirable. This work proposes an approach to short-term detection of mood disorder based on the patterns in emotion of elicited speech responses. To the best of our knowledge, there is no database for short-term detection on the discrimination between BD and UD currently. This work collected two databases containing an emotional database (MHMC-EM) collected by the Multimedia Human Machine Communication (MHMC) lab and a mood disorder database (CHI-MEI) collected by the CHI-MEI Medical Center, Taiwan. As the collected CHI-MEI mood disorder database is quite small and emotion annotation is difficult, the MHMC-EM emotional database is selected as a reference database for data adaptation. For the CHI-MEI mood disorder data collection, six eliciting emotional videos are selected and used to elicit the participants' emotions. After watching each of the six eliciting emotional video clips, the participants answer the questions raised by the clinician. The speech responses are then used to construct the CHI-MEI mood disorder database. Hierarchical spectral clustering is used to adapt the collected MHMC-EM emotional database to fit the CHI-MEI mood disorder database for dealing with the data bias problem. The adapted MHMC-EM emotional data are then fed to a denoising autoencoder for bottleneck feature extraction. The bottleneck features are used to construct a long short term memory (LSTM)-based emotion detector for generation of emotion profiles from each speech response. The emotion profiles are then clustered into emotion codewords using the K-means algorithm. Finally, a class-specific latent affective structure model (LASM) is proposed to model the structural relationships among the emotion codewords with respect to six emotional videos for mood disorder detection. Leave-one-group-out cross validation scheme was employed for the evaluation of the proposed class-specific LASM-based approaches. Experimental results show that the proposed class-specific LASM-based method achieved an accuracy of 73.33 percent for mood disorder detection, outperforming the classifiers based on SVM and LSTM.

Robust feature extraction and uncertainty estimation based on attractor dynamics in cyclic deep denoising autoencoders

Because the input and the output values of the deep denoising autoencoders (DDAs) have the same representation space, the output values of a DDA can be used as its input values, which leads to a repeatable cycle for DDA. Using this fact, in this paper we proposed cyclic deep denoising autoencoders (CDDAs). Moreover, the proposed CDDA was exploited for noise robust bottleneck feature extraction. Considering the CDDA as an attractor network, we analyzed its attractor dynamics and proposed an uncertainty estimation method for the bottleneck features. The capabilities of the proposed feature extraction method are: eliminating the need for labeled training data, improving the robustness of bottleneck features of DDAs without excessive training, providing a new uncertainty estimation trend, and being applicable in other speech processing tasks like speaker recognition. Our proposed feature extraction method was evaluated on the Aurora-2 robust speech recognition task and compared it to conventional Mel frequency cepstral coefficients (MFCC) and autoencoder bottleneck features. The proposed method resulted in up to 8% improvement in word recognition accuracy relative to MFCC features.

Language/Dialect Recognition Based on Unsupervised Deep Learning

Over the past decade, bottleneck features within an i-Vector framework have been used for state-of-the-art language/dialect identification (LID/DID). However, traditional bottleneck feature extraction requires additional transcribed speech information. Alternatively, two types of unsupervised deep learning methods are introduced in this study. To address this limitation, an unsupervised bottleneck feature extraction approach is proposed, which is derived from the traditional bottleneck structure but trained with estimated phonetic labels. In addition, based on a generative modeling autoencoder, two types of latent variable learning algorithms are introduced for speech feature processing, which have been previous considered for image processing/reconstruction. Specifically, a variational autoencoder and adversarial autoencoder are utilized on alternative phase of speech processing. To demonstrate the effectiveness of the proposed methods, three corpora are evaluated: 1) a four Chinese dialect dataset, 2) a five Arabic dialect corpus, and 3) multigenre broadcast challenge corpus (MGB-3) for arabic DID. The proposed features are shown to outperform traditional acoustic feature MFCCs consistently across three corpora. Taken collectively, the proposed features achieve up to a relative +58% improvement in $C_{\text{avg}}$ for LID/DID without the need of any secondary speech corpora.

Speech bottleneck feature extraction method based on overlapping group lasso sparse deep neural network

In this paper, a novel speech bottleneck feature extraction method based on overlapping group lasso sparse deep neural network is proposed. This method extracts the speech bottleneck features which contain supervised class information and adjacent voice frames related information by changing the architecture of deep neural network (DNN), to improve of the performance speech recognition. Firstly, in order to construct the sparse deep neural network, the sparse regularization of the overlapping group lasso algorithm is added to the single DNN object function which is regarded as the penalty item. Second, the speech bottleneck features can be extracted from the bottleneck layer of the sparse bottleneck DNN (BN-DNN). Finally, the speech bottleneck features are used as the input features of the deep neural network-hidden Markov model (DNN-HMM) speech recognition system. The large vocabulary continuous speech recognition experiment results on the Switchboard database indicate that the algorithm proposed in this paper can extract the speech bottleneck features with priori information, and reduce the word error rate in continuous speech recognition.

Bottle-Neck Feature Extraction Structures for Multilingual Training and Porting

Stacked-Bottle-Neck (SBN) feature extraction is a crucial part of modern automatic speech recognition (ASR) systems. The SBN network traditionally contains a hidden layer between the BN and output layers. Recently, we have observed that an SBN architecture without this hidden layer (i.e. direct BN-layer – output-layer connection) performs better for a single language but fails in scenarios where a network pre-trained in multilingual fashion is ported to a target language. In this paper, we describe two strategies allowing the direct-connection SBN network to indeed benefit from pre-training with a multilingual net: (1) pre-training multilingual net with the hidden layer which is discarded before porting to the target language and (2) using only the the direct- connection SBN with triphone targets both in multilingual pre-training and porting to the target language. The results are reported on IARPA-BABEL limited language pack (LLP) data.

Combination of bottleneck feature extraction and dereverberation for distant-talking speech recognition

The performance of speech recognition in distant-talking environments is severely degraded by the reverberation that can occur in enclosed spaces (e.g., meeting rooms). To mitigate this degradation...