Segmentation Of Audio Signals Research Articles

Improving Dysarthric Speech Segmentation With Emulated and Synthetic Augmentation.

Acoustic features extracted from speech can help with the diagnosis of neurological diseases and monitoring of symptoms over time. Temporal segmentation of audio signals into individual words is an important pre-processing step needed prior to extracting acoustic features. Machine learning techniques could be used to automate speech segmentation via automatic speech recognition (ASR) and sequence to sequence alignment. While state-of-the-art ASR models achieve good performance on healthy speech, their performance significantly drops when evaluated on dysarthric speech. Fine-tuning ASR models on impaired speech can improve performance in dysarthric individuals, but it requires representative clinical data, which is difficult to collect and may raise privacy concerns. This study explores the feasibility of using two augmentation methods to increase ASR performance on dysarthric speech: 1) healthy individuals varying their speaking rate and loudness (as is often used in assessments of pathological speech); 2) synthetic speech with variations in speaking rate and accent (to ensure more diverse vocal representations and fairness). Experimental evaluations showed that fine-tuning a pre-trained ASR model with data from these two sources outperformed a model fine-tuned only on real clinical data and matched the performance of a model fine-tuned on the combination of real clinical data and synthetic speech. When evaluated on held-out acoustic data from 24 individuals with various neurological diseases, the best performing model achieved an average word error rate of 5.7% and a mean correct count accuracy of 94.4%. In segmenting the data into individual words, a mean intersection-over-union of 89.2% was obtained against manual parsing (ground truth). It can be concluded that emulated and synthetic augmentations can significantly reduce the need for real clinical data of dysarthric speech when fine-tuning ASR models and, in turn, for speech segmentation.

Underwater bolted flange looseness detection using percussion-induced sound and Feature-reduced Multi-ROCKET model

Recently, in the field of structural health monitoring, the detection of bolted connection looseness through percussion-based method and machine learning technology has received much attention due to the advantages of removing the requirement of sensor installation and potential for automation. However, there are few such research which are performed in the underwater environment. The paper proposes a new method, Feature-reduced Multiple Random Convolution Kernel Transform (FM-ROCKET), to identify the looseness level of the underwater bolted connections based on the percussion-induced sound (audio signal). By integrating deep learning (DL) and shallow learning, the FM-ROCKET model uses the 1D convolutional layer (a DL method) to extract features from the percussion-induced audio signal and adopts the rigid classifier (linear classifier, a shallow learning method) to classify the features. Five different preload levels of the bolted flange are considered. A hammer is utilized to tap the flange surface and the continuous percussion-induced audio signal is collected by a smartphone in an underwater environment. After the audio signal segmentation, single-hit audio signals are fed into the FM-ROCKET model. To verify the effectiveness of the proposed method, three case studies are conducted on two flanges. In case study I, the proposed method slightly outperforms other DL-based methods under different training/test splitting ratios. In case studies II and III, the proposed method is far more effective than other DL-based methods on independent and different test sets. The results demonstrate the superiority of the FM-ROCKET model in the underwater detection of bolted flange looseness. To the best of our knowledge, this article is the first attempt to address the detection of bolted flange looseness in the underwater environment by combining percussion-based method, DL, and shallow learning.

RETRACTED ARTICLE: Audio fingerprint analysis for speech processing using deep learning method

We are generating truly mind-boggling amounts of audio data on a daily basis simply by using the Internet. In different audio-based applications, it increases the complexity of accessing and analyzing audio data. Therefore, the framework or supporting tools needed to retrieve audio data to make intelligent decisions in speech processing. However, non-stationarity and irregularity are insufficient for segmentation and classification of audio signals. Audio classification methods are used in many applications, such as speaker identification, gender recognition, music type classification, natural sound classification, etc. This work proposes a deep learning method based on long-term short-term memory (LSTM) that can be used with preprocessing, segmentation, and retrieval of audio signals from the GTZAN dataset. The simulation results show that the proposed algorithm can effectively improve the audio fingerprint-based data retrieval accuracy and overcome traditional methods' drawbacks. Compared with existing methods, the proposed LSTM method has achieved good results. The precision, recall, accuracy and F-measure of LSTM is 96.54%, 96.15%, 98.56% and 0.96% respectively. In the real world, the recommended audio fingerprint recognition system effectively works through voice applications, especially in heterogeneous portable consumer devices or online audio distributed systems.

Soundscape Audio Signal Classification and Segmentation Using Listeners Perception of Background and Foreground Sound

Soundscape Audio Signal Classification and Segmentation Using Listeners Perception of Background and Foreground Sound

Probabilistic segmentation of time-series audio signals using Support Vector Machines

To allow health tracking, patient monitoring, and provide timely user interventions, sensor signals from body sensor networks need to be processed in real-time. Time subdivisions of the sensor signals are extracted and fed into a supervised learning algorithm, such as Support Vector Machines (SVM), to learn a model capable of distinguishing different class labels. However, selecting a short-duration window from the continuous data stream is a significant challenge, and the window may not be properly centered around the activity of interest. In this work, we address the issue of window selection from a continuous data stream, using an optimized SVM-based probability model. To evaluate the effectiveness of our approach, we apply our algorithm to audio signals acquired from a wearable nutrition-monitoring necklace. Our optimized algorithm is capable of correctly classifying 86.1% of instances, compared to a baseline of 73% which segments the time-series data with fixed-size non-overlapping windows, and an exhaustive-search approach with an accuracy of 92.6%.11This work is funded by the National Science Foundation AIR Option 1: Award Number 1312310.

An Efficient Approach for Segmentation, Feature Extraction and Classification of Audio Signals

Due to the presence of non-stationarities and discontinuities in the audio signal, segmentation and classification of audio signal is a really challenging task. Automatic music classification and annotation is still considered as a challenging task due to the difficulty of extracting and selecting the optimal audio features. Hence, this paper proposes an efficient approach for segmentation, feature extraction and classification of audio signals. Enhanced Mel Frequency Cepstral Coefficient (EMFCC)-Enhanced Power Normalized Cepstral Coefficients (EPNCC) based feature extraction is applied for the extraction of features from the audio signal. Then, multi-level classification is done to classify the audio signal as a musical or non-musical signal. The proposed approach achieves better performance in terms of precision, Normalized Mutual Information (NMI), F-score and entropy. The PNN classifier shows high False Rejection Rate (FRR), False Acceptance Rate (FAR), Genuine Acceptance rate (GAR), sensitivity, specificity and accuracy with respect to the number of classes.

Structural Segmentation of Multitrack Audio

Structural segmentation of musical audio signals is one of many active areas of Music Information Retrieval (MIR) research. One aspect of this important topic which has so far received little attention though is the potential advantage to be gained by utilizing multitrack audio. This paper gives an overview of current segmentation techniques, and demonstrates that by applying a particular segmentation algorithm to multitrack data, rather than the usual case of fully mixed audio, we achieve a significant and quantifiable increase in accuracy when locating segment boundaries. Additionally, we provide details of a structurally annotated multitrack test set available for use by other researchers.

Classification of audio signals using generalized spatial fuzzy clustering.

With the increasing use of multimedia data, the need for automatic classification and retrieval of certain kinds of audio data has become an important issue. In this paper, we propose an efficient method of audio signal segmentation and classification from audiovisual database. While conventional methods apply thresholding to audio features such as energy and zero‐crossing rate to detect the boundaries, causing misclassification for audio signals which contain certain audio effects such as fade‐in, fade‐out, and cross‐fade, the proposed algorithm, called general spatial fuzzy c‐means algorithm (GSFCM), solves the problem by taking into account the local spatial information which is weighted correspondingly to neighbor elements based on their distance attributes. GSFCM detects the boundaries between two different audio signals, classifies segments, and then extracts unique feature vectors. This results in the accurate detection and classification. Experiment results for the audio signal from TV news program at 44.1 kHz with 30‐min long confirm that the proposed method outperforms conventional methods in terms of accuracy of the audio signal classification. These results demonstrate that the proposed method is a suitable candidate for audio‐video indexing which is compressed by MPEG. [Work supported by the MKE, Korea, under the ITRC supervised by IITA (IITA‐2008‐(C1090‐0801‐0039)).]

A Decision-Tree-Based Algorithm for Speech/Music Classification and Segmentation

We present an efficient algorithm for segmentation of audio signals into speech or music. The central motivation to our study is consumer audio applications, where various real-time enhancements are often applied. The algorithm consists of a learning phase and a classification phase. In the learning phase, predefined training data is used for computing various time-domain and frequency-domain features, for speech and music signals separately, and estimating the optimal speech/music thresholds, based on the probability density functions of the features. An automatic procedure is employed to select the best features for separation. In the test phase, initial classification is performed for each segment of the audio signal, using a three-stage sieve-like approach, applying both Bayesian and rule-based methods. To avoid erroneous rapid alternations in the classification, a smoothing technique is applied, averaging the decision on each segment with past segment decisions. Extensive evaluation of the algorithm, on a database of more than 12 hours of speech and more than 22 hours of music showed correct identification rates of 99.4% and 97.8%, respectively, and quick adjustment to alternating speech/music sections. In addition to its accuracy and robustness, the algorithm can be easily adapted to different audio types, and is suitable for real-time operation.

Perceptual Coding of Audio Signals Using Adaptive Time-Frequency Transform

Wide band digital audio signals have a very high data-rate associated with them due to their complex nature and demand for high-quality reproduction. Although recent technological advancements have significantly reduced the cost of bandwidth and miniaturized storage facilities, the rapid increase in the volume of digital audio content constantly compels the need for better compression algorithms. Over the years various perceptually lossless compression techniques have been introduced, and transform-based compression techniques have made a significant impact in recent years. In this paper, we propose one such transform-based compression technique, where the joint time-frequency (TF) properties of the nonstationary nature of the audio signals were exploited in creating a compact energy representation of the signal in fewer coefficients. The decomposition coefficients were processed and perceptually filtered to retain only the relevant coefficients. Perceptual filtering (psychoacoustics) was applied in a novel way by analyzing and performing TF specific psychoacoustics experiments. An added advantage of the proposed technique is that, due to its signal adaptive nature, it does not need predetermined segmentation of audio signals for processing. Eight stereo audio signal samples of different varieties were used in the study. Subjective (mean opinion score--MOS) listening tests were performed and the subjective difference grades (SDG) were used to compare the performance of the proposed coder with MP3, AAC, and HE-AAC encoders. Compression ratios in the range of 8 to 40 were achieved by the proposed technique with subjective difference grades (SDG) ranging from ---0.53 to ---2.27.

Audio signal segmentation and classification using fuzzy c‐means clustering

AbstractThis paper proposes a method of segmentation and classification of audio signals which is coded by MPEG Audio. The proposed method first detects the boundaries between two different audio signals, which are called audio‐cuts, and then classifies segments, which are called audio‐segments, and uses audio‐cuts detected by fuzzy c‐means clustering their boundaries. Since conventional methods detect audio‐cuts by applying thresholding to audio features such as energy and zero‐crossing rate, misdetection often occurs when they are applied to an audio signal which contains several audio effects, such as fade‐in, fade‐out, cross‐fade, and the like. The proposed method represents the possibility that the audio‐cut exists by a real number from 0 to 1, obtained by using fuzzy c‐means clustering; all of the possible candidates for the audio‐cuts are detected. Since audio effects which are difficult to detect by using conventional methods are also detected as the candidates, misdetection can be reduced. Furthermore, the audio‐segments, whose boundaries are the candidates, are subdivided into five audio classes, which are silence, speech, music, speech with music background, and speech with noise background. By using the classification results, unnecessary audio‐cuts can be removed, and thereby both accurate audio‐cut detection and accurate audio‐segment classification can be attained. © 2006 Wiley Periodicals, Inc. Syst Comp Jpn, 37(4): 23–34, 2006; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.20491

Automated Accident Detection in Intersections via Digital Audio Signal Processing

A system for automated traffic accident detection in intersections was designed. The input to the system is a 3-s segment of audio signal. The system can be operated in two modes: the two-class and multiclass modes. The output of the two-class mode is a label of “crash” or “noncrash.” In the multiclass mode of operation, the system identifies crashes as well as several types of noncrash incidents, including normal traffic and construction sounds. The system is composed of three main signal processing stages: feature extraction, feature reduction, and classification. Five methods of feature extraction were investigated and compared; these are based on the discrete wavelet transform, fast Fourier transform, discrete cosine transform, real cepstral transform, and mel frequency cepstral transform. Statistical methods are used for feature optimization and classification. Three types of classifiers are investigated and compared; these are the nearest-mean, maximum-likelihood, and nearest-neighbor methods. The results of the study show that the optimum design uses wavelet-based features in combination with the maximum-likelihood classifier. The system is computationally inexpensive relative to the other methods investigated, and the system consistently results in accident detection accuracies of 95% to 100% when the audio signal has a signal-to-noise-ratio of at least 0 decibels.

A generalized wavelet transform for Fourier analysis: the multiresolution Fourier transform and its application to image and audio signal analysis

A wavelet transform specifically designed for Fourier analysis at multiple scales is described and shown to be capable of providing a local representation which is particularly well suited to segmentation problems. It is shown that, by an appropriate choice of analysis window and sampling intervals, it is possible to obtain a Fourier representation which can be computed efficiently and overcomes the limitations of using a fixed scale of window, yet by virtue of its symmetry properties allows simple estimation of such fundamental signal parameters as instantaneous frequency and onset time/position. The transform is applied to the segmentation of both image and audio signals, demonstrating its power to deal with signal events which are localized in either time/space or frequency. Feature extraction and segmentation are performed through the introduction of a class of multiresolution Markov models, whose parameters represent the signal events underlying the segmentation.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>