Probabilistic Latent Component Analysis Research Articles

Early detection of COPD patients’ symptoms with personal environmental sensors: a remote sensing framework using probabilistic latent component analysis with linear dynamic systems

In this study, we present a cohort study involving 106 COPD patients using portable environmental sensor nodes with attached air pollution sensors and activity-related sensors, as well as daily symptom records and peak flow measurements to monitor patients’ activity and personal exposure to air pollution. This is the first study which attempts to predict COPD symptoms based on personal air pollution exposure. We developed a system that can detect COPD patients’ symptoms one day in advance of symptoms appearing. We proposed using the Probabilistic Latent Component Analysis (PLCA) model based on 3-dimensional and 4-dimensional spectral dictionary tensors for personalised and population monitoring, respectively. The model is combined with Linear Dynamic Systems (LDS) to track the patients’ symptoms. We compared the performance of PLCA and PLCA-LDS models against Random Forest models in the identification of COPD patients’ symptoms, since tree-based classifiers were used for remote monitoring of COPD patients in the literature. We found that there was a significant difference between the classifiers, symptoms and the personalised versus population factors. Our results show that the proposed PLCA-LDS-3D model outperformed the PLCA and the RF models between 4 and 20% on average. When we used only air pollutants as input, the PLCA-LDS-3D forecasting results in personalised and population models were 48.67 and 36.33% accuracy for worsening of lung capacity and 38.67 and 19% accuracy for exacerbation of COPD patients’ symptoms, respectively. We have shown that indicators of the quality of an individual’s environment, specifically air pollutants, are as good predictors of the worsening of respiratory symptoms in COPD patients as a direct measurement.

A Music Cognition–Guided Framework for Multi-pitch Estimation

As one of the most important subtasks of automatic music transcription (AMT), multi-pitch estimation (MPE) has been studied extensively for predicting the fundamental frequencies in the frames of audio recordings during the past decade. However, how to use music perception and cognition for MPE has not yet been thoroughly investigated. Motivated by this, this demonstrates how to effectively detect the fundamental frequency and the harmonic structure of polyphonic music using a cognitive framework. Inspired by cognitive neuroscience, an integration of the constant Q transform and a state-of-the-art matrix factorization method called shift-invariant probabilistic latent component analysis (SI-PLCA) are proposed to resolve the polyphonic short-time magnitude log-spectra for multiple pitch estimation and source-specific feature extraction. The cognitions of rhythm, harmonic periodicity and instrument timbre are used to guide the analysis of characterizing contiguous notes and the relationship between fundamental frequency and harmonic frequencies for detecting the pitches from the outcomes of SI-PLCA. In the experiment, we compare the performance of proposed MPE system to a number of existing state-of-the-art approaches (seven weak learning methods and four deep learning methods) on three widely used datasets (i.e. MAPS, BACH10 and TRIOS) in terms of F-measure ({F}_{1}) values. The experimental results show that the proposed MPE method provides the best overall performance against other existing methods.

Removal of Cochannel Interference Using Probabilistic Latent Component Analysis in Passive Bistatic Radar

This paper examines the problem of cancellation of cochannel interference (CCI) present in the same frequency channel as the signal of interest, which may bring a reduction in the performance of target detection, in passive bistatic radar. We propose a novel approach based on probabilistic latent component analysis for CCI removal. The highlight is that removing CCI is considered as reconstruction, and extraction of Doppler-shifted and time-delayed replicas of the reference signal exploited fully as training data. The results of the simulation show that the developed method is effective.

Dirichlet Latent Variable Model: A Dynamic Model Based on Dirichlet Prior for Audio Processing

We propose a dynamic latent variable model for learning latent bases from time varying, non-negative data. We take a probabilistic approach to modeling the temporal dependence in data by introducing a dynamic Dirichlet prior—a Dirichlet distribution with dynamic parameters. This new distribution allows us to assure non-negativity and avoid intractability when sequential updates are performed otherwise encountered in using Dirichlet prior. We refer to the proposed model as the Dirichlet latent variable model DLVM. We develop an expectation maximization algorithm for the proposed model, and also derive a maximum a posteriori estimate of the parameters. Furthermore, we connect the proposed DLVM to two popular latent basis learning methods—probabilistic latent component analysis PLCA and non-negative matrix factorization NMF. We show that 1 PLCA is a special case of our DLVM, and 2 DLVM can be interpreted as a dynamic version of NMF. The usefulness of DLVM is demonstrated for three audio processing applications—speaker source separation, denoising, and bandwidth expansion. To this end, a new algorithm for source separation is also proposed. Through extensive experiments on benchmark databases, we show that the proposed model outperforms several relevant existing methods in all three applications.

Automatic Transcription of Polyphonic Vocal Music

This paper presents a method for automatic music transcription applied to audio recordings of a cappella performances with multiple singers. We propose a system for multi-pitch detection and voice assignment that integrates an acoustic and a music language model. The acoustic model performs spectrogram decomposition, extending probabilistic latent component analysis (PLCA) using a six-dimensional dictionary with pre-extracted log-spectral templates. The music language model performs voice separation and assignment using hidden Markov models that apply musicological assumptions. By integrating the two models, the system is able to detect multiple concurrent pitches in polyphonic vocal music and assign each detected pitch to a specific voice type such as soprano, alto, tenor or bass (SATB). We compare our system against multiple baselines, achieving state-of-the-art results for both multi-pitch detection and voice assignment on a dataset of Bach chorales and another of barbershop quartets. We also present an additional evaluation of our system using varied pitch tolerance levels to investigate its performance at 20-cent pitch resolution.

On-Bird Sound Recordings: Automatic Acoustic Recognition of Activities and Contexts

We introduce a novel approach to studying animal behavior and the context in which it occurs, through the use of microphone backpacks carried on the backs of individual free-flying birds. These sensors are increasingly used by animal behavior researchers to study individual vocalizations of freely behaving animals, even in the field. However, such devices may record more than an animal's vocal behavior, and have the potential to be used for investigating specific activities movement and context background within which vocalizations occur. To facilitate this approach, we investigate the automatic annotation of such recordings through two different sound scene analysis paradigms: A scene-classification method using feature learning, and an event-detection method using probabilistic latent component analysis. We analyze recordings made with Eurasian jackdaws Corvus monedula in both captive and field settings. Results are comparable with the state of the art in sound scene analysis; we find that the current recognition quality level enables scalable automatic annotation of audio logger data, given partial annotation, but also find that individual differences between animals and/or their backpacks limit the generalization from one individual to another. we consider the interrelation of “scenes” and “events” in this particular task, and issues of temporal resolution.

Polyphonic Sound Event Tracking Using Linear Dynamical Systems

In this paper, a system for polyphonic sound event detection and tracking is proposed, based on spectrogram factorization techniques and state space models. The system extends probabilistic latent component analysis (PLCA) and is modeled around a four-dimensional spectral template dictionary of frequency, sound event class, exemplar index, and sound state. In order to jointly track multiple overlapping sound events over time, the integration of linear dynamical systems (LDS) within the PLCA inference is proposed. The system assumes that the PLCA sound event activation is the (noisy) observation in an LDS, with the latent states corresponding to the true event activations. LDS training is achieved using fully observed data, making use of ground truth-informed event activations produced by the PLCA-based model. Several LDS variants are evaluated, using polyphonic datasets of office sounds generated from an acoustic scene simulator, as well as real and synthesized monophonic datasets for comparative purposes. Results show that the integration of LDS tracking within PLCA leads to an improvement of +8.5-10.5% in terms of frame-based F-measure as compared to the use of the PLCA model alone. In addition, the proposed system outperforms several state-of-the-art methods for the task of polyphonic sound event detection.

Exploiting Continuity/Discontinuity of Basis Vectors in Spectrogram Decomposition for Harmonic-Percussive Sound Separation

In this paper, we present a novel method for harmonic-percussive sound separation (HPSS) by exploiting continuity/discontinuity properties in a matrix decomposition framework. It is widely accepted in the HPSS research that the harmonic and percussive components have anisotropic characteristics: The spectrum of the harmonic components and the time activation of the percussive components are sparse, whereas the spectrum of the percussive components and the time activation of the harmonic components are smooth. However, conventional methods fail to fully utilize the characteristics leading to suboptimal performance. Based on the observations that not the degree of sparseness but the degree of fluctuation is an accurate measure for distinguishing the harmonic and percussive components, we propose a novel HPSS algorithm by incorporating the continuity control in the iterative update formula of the matrix decomposition algorithm. In doing so, we first utilize probabilistic latent component analysis with Dirichlet prior, and later reformulate the algorithm in the nonnegative matrix factorization framework to reduce the computational cost. The comparative evaluation results show that the proposed method outperforms conventional methods in terms of both objective and subjective evaluation.

Multiple-timbre note tracking using linear dynamical systems

This work addresses the problem of multi-pitch detection and note tracking in multiple-instrument polyphonic music recordings. A system is developed that extends probabilistic latent component analysis (PLCA) and supports the use of a five-dimensional dictionary of spectral templates per instrument, pitch, deviation from ideal tuning, and sound state (e.g., attack, sustain, decay). A method based on linear dynamical systems (LDS) is introduced for note tracking, which assumes that the output of the PLCA model is the (noisy) observation in an LDS, with the latent states corresponding to the ideal multi-pitch activation output. The LDS-based process supports the tracking of multiple concurrent pitches and can also be integrated within the PLCA-based model, thus guiding the convergence of the multi-pitch detection process. Experiments performed on several datasets of multiple-instrument polyphonic music demonstrate that the LDS-based method leads to significant improvements in multi-pitch detection as compared to using the frame-based PLCA model alone.

An automatic music transcription system dedicated to the repertoires of the marovany zither

The meeting between Music information retrieval (MIR) and Ethnomusicology has given way to a new scientific discipline called Computational Ethnomusicology, which aims to adapt and develop MIR tools for the multiple corpus of ethnic music. In this paper, we are interested in developing an automatic transcription system dedicated to the marovany musical repertoires. The marovany repertoires are mostly orally transmitted, resulting from a process of memorization/transformation of original base musical motives. These motives represent an important culture patrimony, and are evolving continually under the influences of other musical practices and genres due to globalization. Our transcription system consists mainly of a Probabilistic Latent Component Analysis algorithm post-processed with Hidden Markov Model. It incorporates repertoire-specific music-related prior knowledge about timbre, music language and playing style. The impact of each knowledge component on transcription results has been comparatively evaluated. The most beneficial prior combination brought a transcription gain of 7.4% from the note-based F-measures of state-of-the-art transcription system performance.

A new method for ecoacoustics? Toward the extraction and evaluation of ecologically-meaningful soundscape components using sparse coding methods.

Passive acoustic monitoring is emerging as a promising non-invasive proxy for ecological complexity with potential as a tool for remote assessment and monitoring (Sueur & Farina, 2015). Rather than attempting to recognise species-specific calls, either manually or automatically, there is a growing interest in evaluating the global acoustic environment. Positioned within the conceptual framework of ecoacoustics, a growing number of indices have been proposed which aim to capture community-level dynamics by (e.g., Pieretti, Farina & Morri, 2011; Farina, 2014; Sueur et al., 2008b) by providing statistical summaries of the frequency or time domain signal. Although promising, the ecological relevance and efficacy as a monitoring tool of these indices is still unclear. In this paper we suggest that by virtue of operating in the time or frequency domain, existing indices are limited in their ability to access key structural information in the spectro-temporal domain. Alternative methods in which time-frequency dynamics are preserved are considered. Sparse-coding and source separation algorithms (specifically, shift-invariant probabilistic latent component analysis in 2D) are proposed as a means to access and summarise time-frequency dynamics which may be more ecologically-meaningful.

An investigation of prior knowledge in Automatic Music Transcription systems.

Automatic transcription of music is a long-studied research field with many operational systems available commercially. In this paper, a generic transcription system able to host various prior knowledge parameters has been developed, followed by an in-depth investigation of their impact on music transcription. Explicit links between musical knowledge and algorithmic formalism have been made. Musical knowledge covers classes of timbre, musicology, and playing style of an instrument repertoire. An evaluation sound corpus gathering musical pieces played by human performers from three different instrument repertoires, namely, classical piano, steel-string acoustic guitar, and the marovany zither from Madagascar, has been developed. The different components of musical knowledge have been successively incorporated in a complete transcription system, consisting mainly of a Probabilistic Latent Component Analysis algorithm post-processed with a Hidden Markov Model, and their impact on transcription results have been comparatively evaluated.

Multiple F0 Estimation and Source Clustering of Polyphonic Music Audio Using PLCA and HMRFs

Source transcription of pitched polyphonic music entails providing the pitch (F0) values corresponding to each source in a separate channel. This problem is an important step towards many important problems in music and speech processing. It involves 1) estimating the multiple F0 values in each short time frame, and 2) clustering the F0 values into streams corresponding to different sources. We address the problem in an unsupervised way, with only the total number of sources given beforehand. The framework of probabilistic latent component analysis (PLCA) is used to decompose the polyphonic short-time magnitude spectra for multiple F0 estimation and source-specific feature extraction. It is further embedded into the structure of hidden Markov random fields (HMRF) for clustering the F0s into different sources. This clustering is constrained by the cognitive grouping of continuous F0 contours as well as segregation of simultaneous F0s into different source streams. Such constraints are effectively and elegantly modeled by the HMRF's. Simulated annealing varies the degree of constraints for better clustering. The paper also proposes a novel strategy using the trade-off between precision and recall of multiple F0 estimation for better clustering. Evaluations over a variety of datasets show the efficacy of the proposed algorithm and its robustness to the presence of spurious F0s while clustering. It also outperforms a state-of-the-art unsupervised source streaming algorithm in a set of comparative experiments.

Ground-penetrating radar method used for the characterisation of ornamental stone quarries

In the work non-destructive ground-penetrating radar (GPR) was used to probe the texture and presence of anisotropies in different types of carbonated rocks intended for ornamental use.First, GPR on the different facies of a marble quarry in Macael (Almería, Spain) was applied where alternating layers of marble and mica schists can be found. GPR allows for the differentiation of the marble units from the mica schist units, which makes GPR a good tool for indirectly evaluating the reserves in a deposit. In addition, it allows for the detection of the different anisotropies in the marble units (holes, fractures), and this information could be utilised to formulate a work-plan design. Antennae of different frequencies were employed (100, 250 and 800MHz) for these purposes, and the 250MHz antenna had the most effective probing capacity for obtaining an accurate depth resolution.Second, a GPR study was conducted in three types of rock known commercially as Macael Marble, Crema Marfil and Red Travertine before the block-cutting process. For this configuration, the 800MHz antenna was used to differentiate the textures of each type of rock as well as the location of a variety of anisotropies, and the results showed that GPR is an effective tool for evaluating the block quality, determining whether resins must be injected to consolidate the block and estimating the orientation of the cutting process.In the work, the use of a supervised two-dimensional (i.e., radargram) probabilistic latent component analysis (PLCA) approach is proposed to highlight only the information from target objects (i.e., marble anisotropy) provided by the radargram, which facilitates the data interpretation by the user. This approach can search activations of the georadar pulse across both dimensions. Once the analysis has been performed, a variable-gain compensation process is proposed to outperform the energy losses that result from the reflections of the signal when a different material is found.The results have shown that the GPR method can be utilised as a tool for the diagnosis of stone materials prior to their use in artistic work or monuments; therefore, GPR may be considered a technique for material selection. The presence of discontinuities (sometimes visible to the naked eye) explain many of the phenomena and typologies of the stone after its eventual alteration, and such discontinuities could be avoided by means of this pre-emptive work.

Musical Source Clustering and Identification in Polyphonic Audio

For music transcription or musical source separation, apart from knowing the multi-F0 contours, it is also important to know which F0 has been played by which instrument. This paper focuses on this aspect, i.e. given the polyphonic audio along with its multiple F0 contours, the proposed system clusters them so as to decide ‘which instrument played when.’ For the task of identifying the instrument or singers in the polyphonic audio, there are many supervised methods available. But many times individual source audio is not available for training. To address this problem, this paper proposes novel schemes using semi-supervised as well as unsupervised approach to source clustering. The proposed theoretical framework is based on auditory perception theory and is implemented using various tools like probabilistic latent component analysis and graph clustering, while taking into account various perceptual cues for characterizing a source. Experiments have been carried out over a wide variety of datasets - ranging from vocal to instrumental as well as from synthetic to real world music. The proposed scheme significantly outperforms a state of the art unsupervised scheme, which does not make use of the given F0 contours. The proposed semi-supervised approach also performs better than another semi-supervised scheme, which makes use of the given F0 information, in terms of computations as well as accuracy.

Harmonic Adaptive Latent Component Analysis of Audio and Application to Music Transcription

Recently, new methods for smart decomposition of time-frequency representations of audio have been proposed in order to address the problem of automatic music transcription. However those techniques are not necessarily suitable for notes having variations of both pitch and spectral envelope over time. The HALCA (Harmonic Adaptive Latent Component Analysis) model presented in this article allows considering those two kinds of variations simultaneously. Each note in a constant-Q transform is locally modeled as a weighted sum of fixed narrowband harmonic spectra, spectrally convolved with some impulse that defines the pitch. All parameters are estimated by means of the expectation-maximization (EM) algorithm, in the framework of Probabilistic Latent Component Analysis. Interesting priors over the parameters are also introduced in order to help the EM algorithm converging towards a meaningful solution. We applied this model for automatic music transcription: the onset time, duration and pitch of each note in an audio file are inferred from the estimated parameters. The system has been evaluated on two different databases and obtains very promising results.

Multiple-instrument polyphonic music transcription using a temporally constrained shift-invariant model

A method for automatic transcription of polyphonic music is proposed in this work that models the temporal evolution of musical tones. The model extends the shift-invariant probabilistic latent component analysis method by supporting the use of spectral templates that correspond to sound states such as attack, sustain, and decay. The order of these templates is controlled using hidden Markov model-based temporal constraints. In addition, the model can exploit multiple templates per pitch and instrument source. The shift-invariant aspect of the model makes it suitable for music signals that exhibit frequency modulations or tuning changes. Pitch-wise hidden Markov models are also utilized in a postprocessing step for note tracking. For training, sound state templates were extracted for various orchestral instruments using isolated note samples. The proposed transcription system was tested on multiple-instrument recordings from various datasets. Experimental results show that the proposed model is superior to a non-temporally constrained model and also outperforms various state-of-the-art transcription systems for the same experiment.

Attention-Guided Organized Perception and Learning of Object Categories Based on Probabilistic Latent Variable Models

This paper proposes a probabilistic model of object category learning in conjunction with attention-guided organized perception. This model consists of a model of attention-guided organized perception of object segments on Markov random fields and a model of learning object categories based on a probabilistic latent component analysis. In attention guided organized perception, concurrent figure-ground segmentation is performed on dynamically-formed Markov random fields around salient preattentive points and co-occurring segments are grouped in the neighborhood of selective attended segments. In object category learning, a set of classes of each object category is obtained based on the probabilistic latent component analysis with the variable number of classes from bags of features of segments extracted from images which contain the categorical objects in context and an object category is represented by a composite of object classes. Through experiments using two image data sets, it is shown that the model learns a probabilistic structure of intra-categorical composition and inter-categorical difference of object categories and achieves high performance in object category recognition.

A Shift-Invariant Latent Variable Model for Automatic Music Transcription

In this work, a probabilistic model for multiple-instrument automatic music transcription is proposed. The model extends the shift-invariant probabilistic latent component analysis method, which is used for spectrogram factorization. Proposed extensions support the use of multiple spectral templates per pitch and per instrument source, as well as a time-varying pitch contribution for each source. Thus, this method can effectively be used for multiple-instrument automatic transcription. In addition, the shift-invariant aspect of the method can be exploited for detecting tuning changes and frequency modulations, as well as for visualizing pitch content. For note tracking and smoothing, pitch-wise hidden Markov models are used. For training, pitch templates from eight orchestral instruments were extracted, covering their complete note range. The transcription system was tested on multiple-instrument polyphonic recordings from the RWC database, a Disklavier data set, and the MIREX 2007 multi-F0 data set. Results demonstrate that the proposed method outperforms leading approaches from the transcription literature, using several error metrics.

Multiple-timbre fundamental frequency tracking using an instrument spectrum library

Recently many researchers have attempted automatic pitch estimation of polyphonic music (e.g., Li et al., IEEE Trans ASLP, 2009). Most of these attempts have concerned themselves with the estimation of individual pitches (F0s) while not associating the estimated pitches with the particular instruments that produce them. Estimating pitches for each instrument will lead to full music transcription. Individual instrument F0 tracks can be used in music information retrieval systems to better organize and search music. We propose a method to estimate the F0 tracks for a set of harmonic instruments in a sound mixture, using probabilistic latent component analysis (PLCA) and collections of basis spectra indexed by F0 and instrument learned in advance. The PLCA model is extended hierarchically to explain the observed input mixture spectra as a sum of basis spectra from note(s) of various instruments. The polyphonic pitch tracking problem is posed as inferring the most likely combination of the active note(s) from different instruments. Continuity and sparsity constraints are enforced to better model how the music is produced. The method was trained on a common instrument spectrum library an evaluated using an established polyphonic audio dataset.