Modified Kullback-Leibler Divergence Research Articles

A reduced-order model to estimate first wall particle and heat fluxes for systems codes

Current systems codes (SCs) do not assess the impact of plasma filament parameters and dynamics in the SOL when performing systems-level analyses. Simulation tools based on simplified turbulent transport models could address such need, such as the TOKES code, but these tend to run in timescales that are prohibitive for incorporation in SCs. This work presents a technique to build surrogate models for SCs, obtained with the modes of Reduced-Order Models (ROMs), and is exemplified with results from TOKES. These surrogates were developed with Principal Components Analysis (PCA), multivariate regression against TOKES inputs (filament temperatures, densities, and ejection speeds) and k-fold cross-validation, applied to the results after transformations based on rational powers of the inputs to optimize the prediction capabilities of the surrogates. Selection was performed with a modified Kullback-Leibler Divergence (KLD), and validation, with withheld cases. Main results include validation of the methodology due to the relatively low number of modes needed to represent more than 90% of the data variance, and the data transformation exponents that optimize the regression.

3D multi-object tracking based on informatic divergence-guided data association

In the existing literature, most 3D multi-object tracking algorithms based on the tracking-by-detection framework employed deterministic tracks and detections for similarity calculation in the data association stage. Namely, the inherent uncertainties existing in tracks and detections are overlooked. In this work, we discard the commonly used deterministic tracks and deterministic detections for data association, instead, we propose to model tracks and detections as random vectors in which uncertainties are taken into account. Then, based on a modified Kullback-Leibler divergence, the similarity between two multidimensional distributions, i.e. track and detection, is evaluated for data association purposes. Lastly, the level of track uncertainty is incorporated in our cost function design to guide the data association process. Comparative experiments have been conducted on two typical datasets, KITTI and nuScenes, and the results indicated that our proposed method outperformed the compared state-of-the-art 3D tracking algorithms. For the benefit of the community, our code has been made available at https://github.com/hejiawei2023/UG3DMOT.

A Robust Version of the Empirical Likelihood Estimator

In this paper, we introduce a robust version of the empirical likelihood estimator for semiparametric moment condition models. This estimator is obtained by minimizing the modified Kullback–Leibler divergence, in its dual form, using truncated orthogonality functions. We prove the robustness and the consistency of the new estimator. The performance of the robust empirical likelihood estimator is illustrated through examples based on Monte Carlo simulations.

Dynamical Sampling with Langevin Normalization Flows

In Bayesian machine learning, sampling methods provide the asymptotically unbiased estimation for the inference of the complex probability distributions, where Markov chain Monte Carlo (MCMC) is one of the most popular sampling methods. However, MCMC can lead to high autocorrelation of samples or poor performances in some complex distributions. In this paper, we introduce Langevin diffusions to normalization flows to construct a brand-new dynamical sampling method. We propose the modified Kullback-Leibler divergence as the loss function to train the sampler, which ensures that the samples generated from the proposed method can converge to the target distribution. Since the gradient function of the target distribution is used during the process of calculating the modified Kullback-Leibler, which makes the integral of the modified Kullback-Leibler intractable. We utilize the Monte Carlo estimator to approximate this integral. We also discuss the situation when the target distribution is unnormalized. We illustrate the properties and performances of the proposed method on varieties of complex distributions and real datasets. The experiments indicate that the proposed method not only takes the advantage of the flexibility of neural networks but also utilizes the property of rapid convergence to the target distribution of the dynamics system and demonstrate superior performances competing with dynamics based MCMC samplers.

A New Approach to the Phase-Based Video Motion Magnification for Measuring Microdisplacements

Video motion magnification (VMM) techniques have recently gained attention as a low-cost accurate method for measuring microdisplacements. VMM techniques are known to have incorrect frequency responses when handling dynamic measurements consisting of multiple frequencies. New phase-based VMM method utilizing radial fractional steerable 2-D Hilbert transform is proposed and contrasted, in terms of the frequency response, with two other established VMM methods. Typical VMM workflow includes amplifying phase changes between subsequent video frames on many spatial frequencies separately using image pyramids, which are then collapsed forming new, motion-magnified video. In this paper, Hilbert transform is used to extract local phase changes. The modified Kullback–Leibler divergence was calculated between the spectral estimates of the time series of vibrating plate measured using a high-speed video camera and a reference time series synchronously acquired with high-precision air-coupled ultrasonic transducer. Time series consisted of micrometer range multicomponent vibrations (one, two, and three sine frequencies) recorded with different motion magnification factors. The results show that in the majority of cases, the proposed method, out of three considered, resulted in the most uniform frequency response and generated the lowest amount of unwanted frequencies. To conclude, a novel approach to VMM is proposed, which substantially improves frequency response and makes this technique more suitable for the measurements of phenomena consisting of multiple frequencies.

Reduction of Markov Chains Using a Value-of-Information-Based Approach.

In this paper, we propose an approach to obtain reduced-order models of Markov chains. Our approach is composed of two information-theoretic processes. The first is a means of comparing pairs of stationary chains on different state spaces, which is done via the negative, modified Kullback–Leibler divergence defined on a model joint space. Model reduction is achieved by solving a value-of-information criterion with respect to this divergence. Optimizing the criterion leads to a probabilistic partitioning of the states in the high-order Markov chain. A single free parameter that emerges through the optimization process dictates both the partition uncertainty and the number of state groups. We provide a data-driven means of choosing the ‘optimal’ value of this free parameter, which sidesteps needing to a priori know the number of state groups in an arbitrary chain.

Dysphonic Voice Pattern Analysis of Patients in Parkinson's Disease Using Minimum Interclass Probability Risk Feature Selection and Bagging Ensemble Learning Methods.

Analysis of quantified voice patterns is useful in the detection and assessment of dysphonia and related phonation disorders. In this paper, we first study the linear correlations between 22 voice parameters of fundamental frequency variability, amplitude variations, and nonlinear measures. The highly correlated vocal parameters are combined by using the linear discriminant analysis method. Based on the probability density functions estimated by the Parzen-window technique, we propose an interclass probability risk (ICPR) method to select the vocal parameters with small ICPR values as dominant features and compare with the modified Kullback-Leibler divergence (MKLD) feature selection approach. The experimental results show that the generalized logistic regression analysis (GLRA), support vector machine (SVM), and Bagging ensemble algorithm input with the ICPR features can provide better classification results than the same classifiers with the MKLD selected features. The SVM is much better at distinguishing normal vocal patterns with a specificity of 0.8542. Among the three classification methods, the Bagging ensemble algorithm with ICPR features can identify 90.77% vocal patterns, with the highest sensitivity of 0.9796 and largest area value of 0.9558 under the receiver operating characteristic curve. The classification results demonstrate the effectiveness of our feature selection and pattern analysis methods for dysphonic voice detection and measurement.

New blind source separation method of independent/dependent sources

We introduce a new blind source separation approach, based on modified Kullback–Leibler divergence between copula densities, for both independent and dependent source component signals. In the classical case of independent source components, the proposed method generalizes the mutual information (between probability densities) procedure. Moreover, it has the great advantage to be naturally extensible to separate mixtures of dependent source components. Simulation results are presented showing the convergence and the efficiency of the proposed algorithms.

Analysis of altered gait cycle duration in amyotrophic lateral sclerosis based on nonparametric probability density function estimation

Human locomotion is regulated by the central nervous system (CNS). The neurophysiological changes in the CNS due to amyotrophic lateral sclerosis (ALS) may cause altered gait cycle duration (stride interval) or other gait rhythm. This article used a statistical method to analyze the altered stride interval in patients with ALS. We first estimated the probability density functions (PDFs) of stride interval from the outlier-processed gait rhythm time series, by using the nonparametric Parzen-window approach. Based on the PDFs estimated, the mean of the left-foot stride interval and the modified Kullback–Leibler divergence (MKLD) can be computed to serve as dominant features. In the classification experiments, the least squares support vector machine (LS-SVM) with Gaussian kernels was applied to distinguish the stride patterns in ALS patients. According to the results obtained with the stride interval time series recorded from 16 healthy control subjects and 13 patients with ALS, the key findings of the present study are summarized as follows. (1) It is observed that the mean of stride interval computed based on the PDF for the left foot is correlated with that for the right foot in patients with ALS. (2) The MKLD parameter of the gait in ALS is significantly different from that in healthy controls. (3) The diagnostic performance of the nonlinear LS-SVM, evaluated by the leave-one-out cross-validation method, is superior to that obtained by the linear discriminant analysis. The LS-SVM can effectively separate the stride patterns between the groups of healthy controls and ALS patients with an overall accurate rate of 82.8% and an area of 0.869 under the receiver operating characteristic curve.

Linear and Nonlinear Projective Nonnegative Matrix Factorization

A variant of nonnegative matrix factorization (NMF) which was proposed earlier is analyzed here. It is called projective nonnegative matrix factorization (PNMF). The new method approximately factorizes a projection matrix, minimizing the reconstruction error, into a positive low-rank matrix and its transpose. The dissimilarity between the original data matrix and its approximation can be measured by the Frobenius matrix norm or the modified Kullback-Leibler divergence. Both measures are minimized by multiplicative update rules, whose convergence is proven for the first time. Enforcing orthonormality to the basic objective is shown to lead to an even more efficient update rule, which is also readily extended to nonlinear cases. The formulation of the PNMF objective is shown to be connected to a variety of existing NMF methods and clustering approaches. In addition, the derivation using Lagrangian multipliers reveals the relation between reconstruction and sparseness. For kernel principal component analysis (PCA) with the binary constraint, useful in graph partitioning problems, the nonlinear kernel PNMF provides a good approximation which outperforms an existing discretization approach. Empirical study on three real-world databases shows that PNMF can achieve the best or close to the best in clustering. The proposed algorithm runs more efficiently than the compared NMF methods, especially for high-dimensional data. Moreover, contrary to the basic NMF, the trained projection matrix can be readily used for newly coming samples and demonstrates good generalization.