Leibler Distance Research Articles

Fuzzy stochastic neural network model for structural system identification

This paper presents a dynamic fuzzy stochastic neural network model for nonparametric system identification using ambient vibration data. The model is developed to handle two types of imprecision in the sensed data: fuzzy information and measurement uncertainties. The dimension of the input vector is determined by using the false nearest neighbor approach. A Bayesian information criterion is applied to obtain the optimum number of stochastic neurons in the model. A fuzzy C-means clustering algorithm is employed as a data mining tool to divide the sensed data into clusters with common features. The fuzzy stochastic model is created by combining the fuzzy clusters of input vectors with the radial basis activation functions in the stochastic neural network. A natural gradient method is developed based on the Kullback–Leibler distance criterion for quick convergence of the model training. The model is validated using a power density pseudospectrum approach and a Bayesian hypothesis testing-based metric. The proposed methodology is investigated with numerically simulated data from a Markov Chain model and a two-story planar frame, and experimentally sensed data from ambient vibration data of a benchmark structure.

Linear fitting Kalman filter

Dynamic estimation in signal processing and target tracking often involves non-linear models. These non-linear models are usually linearised through the first-order Taylor approximation in estimation process. However, the error generated by the first-order Taylor approximation is not negligible when the non-linearity of a model is high or the input error is large. This study proposes a new linearisation method through minimising the error between a non-linear function and its linear approximation. A weighted least squares (WLS) algorithm is developed to estimate a linear fitting (LF) function based on the sigma points of the random variable in non-linear transformation. A linear fitting Kalman filter (LKF) is developed based on this principle. The accuracy of the LF transform is analysed using the Kullback–Leibler (KL) distance. The results show that the LF transform has less KL distance to the true distribution compared with the first-order Taylor approximation. To evaluate the estimation performance, simulations are conducted and the results are compared with those of extended Kalman filter (EKF) and unscented Kalman filter (UKF). The results demonstrate that the LKF provides better accuracy than the EKF, and has similar accuracy to the UKF with lower computational cost.

An algorithm for information projection to an affine subspace

We investigate an algorithm to find a point of an affine subspace in the positive orthant such that it is the closest one to the original point with respect to the Kullback---Leibler distance. This problem is solved by means of the classical Darroch---Ratcliff algorithm (see [1]), while we use ideas of the information geometry founded by Chentsov (see [2]) and Csiszar (see [3]). The main theorem of the present work proves the convergence of that algorithm (the method of the proof is different from previous ones). The proposed algorithm can be applied, e.g., to find the maximum likelihood estimates in an exponential family (see the last section of the paper).

Accuracy in Rietveld quantitative phase analysis: a comparative study of strictly monochromatic Mo and Cu radiations.

This study reports 78 Rietveld quantitative phase analyses using Cu Kα1, Mo Kα1 and synchrotron radiations. Synchrotron powder diffraction has been used to validate the most challenging analyses. From the results for three series with increasing contents of an analyte (an inorganic crystalline phase, an organic crystalline phase and a glass), it is inferred that Rietveld analyses from high-energy Mo Kα1 radiation have slightly better accuracies than those obtained from Cu Kα1 radiation. This behaviour has been established from the results of the calibration graphics obtained through the spiking method and also from Kullback-Leibler distance statistic studies. This outcome is explained, in spite of the lower diffraction power for Mo radiation when compared to Cu radiation, as arising because of the larger volume tested with Mo and also because higher energy allows one to record patterns with fewer systematic errors. The limit of detection (LoD) and limit of quantification (LoQ) have also been established for the studied series. For similar recording times, the LoDs in Cu patterns, ∼0.2 wt%, are slightly lower than those derived from Mo patterns, ∼0.3 wt%. The LoQ for a well crystallized inorganic phase using laboratory powder diffraction was established to be close to 0.10 wt% in stable fits with good precision. However, the accuracy of these analyses was poor with relative errors near to 100%. Only contents higher than 1.0 wt% yielded analyses with relative errors lower than 20%.

Influential observation in complex normal data for problems in allometry

The aim of this paper is to investigate how some results related to the complex normal distribution are relevant in size and shape analysis. Our main focus is on the derivation of influential measures. In particular, Cook and Kullback–Leibler distances are combined with their respective asymptotic results as well as to an alternative process of defining cut-off points. Some numerical examples illustrate how these measures are used in practice. We perform an application to simulated and actual data. Results provide evidence that the methodology based on Kullback–Leibler distance outperforms one in terms of the Cook classic distance.

Automatic environmental sound concepts discovery for video retrieval

This paper characterizes a new method for video–soundtrack retrieval based on environmental sounds. Actually, a set of 26 semantic audio concepts is employed. This set is chosen for its helpfulness to the users in terms of video browsing. Additionally, a set of 2000 videos has been annotated with these concepts. To enhance a new signal processing, we start with the separation of the audio sources. In addition, using a fundamental representation of the audio signal as a sequence of Mel Frequency Cepstral Coefficient, we can carry out experiments with three signal representations: the Support Vector machines, the Gaussian Mixture Model and the Hidden Markov Model. Throughout the experiment synthesis, we maintain the Gaussian Mixture Model classifier based on the Kullback–Leibler distance measure. As a matter of fact, we preserve this audio concept classification to integrate a video retrieval system. Hence, the obtained results mirror the effectiveness of our approaches in distinguishing environmental sound and researching video.

Performance of GPS slant total electron content and IRI-Plas-STEC for days with ionospheric disturbance

Total Electron Content (TEC) is an important observable parameter of the ionosphere which forms the main source of error for space based navigation and positioning systems. Since the deployment of Global Navigation Satellite Systems (GNSS), cost-effective estimation of TEC between the earth based receiver and Global Positioning System (GPS) satellites became the major means of investigation of local and regional disturbance for earthquake precursor and augmentation system studies. International Reference Ionosphere (IRI) extended to plasmasphere (IRI-Plas) is the most developed ionospheric and plasmaspheric climatic model that provides hourly, monthly median of electron density distribution globally. Recently, IONOLAB group (www.ionolab.org) has presented a new online space weather service that can compute slant TEC (STEC) on a desired ray path for a given date and time using IRI-Plas model (IRI-Plas-STEC). In this study, the performance of the model based STEC is compared with GPS-STEC computed according to the estimation method developed by the IONOLAB group and includes the receiver bias as IONOLAB-BIAS (IONOLAB-STEC). Using Symmetric Kullback–Leibler Distance (SKLD), Cross Correlation (CC) coefficient and the metric norm (L2N) to compare IRI-Plas-STEC and IONOLAB-STEC for the month of October 2011 over the Turkish National Permanent GPS Network (TNPGN-Active), it has been observed that SKLD provides a good indicator of disturbance for both earthquakes and geomagnetic storms.

A cross-entropy method and probabilistic sensitivity analysis framework for calibrating microscopic traffic models

Car following modeling framework seeks for a more realistic representation of car following behavior in complex driving situations to improve traffic safety and to better understand several puzzling traffic flow phenomena, such as stop-and-go oscillations. Calibration and validation techniques pave the way towards the descriptive power of car-following models and their applicability for analyzing traffic flow. However, calibrating these models is never a trivial task. This is caused by the fact that some parameters, such as reaction time, are generally not directly observable from traffic data. On the other hand, traffic data might be subject to various errors and noises. This contribution puts forward a Cross-Entropy Method (CEM) based approach to identify parameters of deterministic car-following models under noisy data by formulating it as a stochastic optimization problem. This approach allows for statistical analysis of the parameter estimations. Another challenge arising in the calibration of car following models concerns the selection of the most important parameters. This paper introduces a relative entropy based Probabilistic Sensitivity Analysis (PSA) algorithm to identify the important parameters so as to reduce the complexity, data requirement and computational effort of the calibration process. Since the CEM and the PSA are based on the Kullback–Leibler (K–L) distance, they can be simultaneously integrated into a unified framework to further reduce the computational burden. The proposed framework is applied to calibrate the intelligent driving model using vehicle trajectories data from the NGSIM project. Results confirm the great potential of this approach.

Imperceptibility—Robustness tradeoff studies for ECG steganography using Continuous Ant Colony Optimization

ECG Steganography ensures protection of patient data when ECG signals embedded with patient data are transmitted over the internet. Steganography algorithms strive to recover the embedded patient data entirely and to minimize the deterioration in the cover signal caused by the embedding. This paper presents a Continuous Ant Colony Optimization (CACO) based ECG Steganography scheme using Discrete Wavelet Transform and Singular Value Decomposition. Quantization techniques allow embedding the patient data into the ECG signal. The scaling factor in the quantization techniques governs the tradeoff between imperceptibility and robustness. The novelty of the proposed approach is to use CACO in ECG Steganography, to identify Multiple Scaling Factors (MSFs) that will provide a better tradeoff compared to uniform Single Scaling Factor (SSF). The optimal MSFs significantly improve the performance of ECG steganography which is measured by metrics such as Peak Signal to Noise Ratio, Percentage Residual Difference, Kullback–Leibler distance and Bit Error Rate. Performance of the proposed approach is demonstrated on the MIT-BIH database and the results validate that the tradeoff curve obtained through MSFs is better than the tradeoff curve obtained for any SSF. The results also advocate appropriate SSFs for target imperceptibility or robustness.

A non-destructive oil palm ripeness recognition system using relative entropy

This paper introduces a relative entropy based image processing approach for the non-destructive prediction of the maturity of oil palm fresh fruit bunches (FFB) which enables the determination of the correct time for harvesting. The results of an experimental study of applying the Kullback–Leibler distance to the problem of oil palm classification are presented. It is shown that the proposed algorithm has an excellent accuracy and it can be computed very fast. The overall proposed system is simple and useful for oil palm farmers and entrepreneurs.

Assessing the robustness of quantitative fatty acid signature analysis to assumption violations

Summary Knowledge of animal diets can provide important insights into life history and ecology, relationships among species in a community and potential response to ecosystem change or perturbation. Quantitative fatty acid signature analysis (QFASA) is a method of estimating diets from data on the composition, or signature, of fatty acids stored in adipose tissue. Given data on signatures of potential prey, a predator diet is estimated by minimizing the distance between its signature and a mixture of prey signatures. Calibration coefficients, constants derived from feeding trials, are used to account for differential metabolism of individual fatty acids. QFASA has been widely applied since its introduction and several variants of the original estimator have appeared in the literature. However, work to compare the statistical properties of QFASA estimators has been limited. One important characteristic of an estimator is its robustness to violations of model assumptions. The primary assumptions of QFASA are that prey signature data contain representatives of all prey types consumed and the calibration coefficients are known without error. We investigated the robustness of two QFASA estimators to a range of violations of these assumptions using computer simulation and recorded the resulting bias in diet estimates. We found that the Aitchison distance measure was most robust to errors in the calibration coefficients. Conversely, the Kullback–Leibler distance measure was most robust to the consumption of prey without representation in the prey signature data. In most QFASA applications, investigators will generally have some knowledge of the prey available to predators and be able to assess the completeness of prey signature data and sample additional prey as necessary. Conversely, because calibration coefficients are derived from feeding trials with captive animals and their values may be sensitive to consumer physiology and nutritional status, their applicability to free‐ranging animals is difficult to establish. We therefore recommend that investigators first make any improvements to the prey signature data that seem warranted and then base estimation on the Aitchison distance measure, as it appears to minimize risk from violations of the assumption that is most difficult to verify.

Cognitive radio: a study of the role of KL distance function in energy detection scheme

ABSTRACTCognitive radio has been identified as a frontline technology in wireless communication domain where unlicenced spectrum users can utilise unused licensed frequency bands without affecting the licensed users’ transmission process. Spectrum hole detection is the primary requirement to fulfil this objective. Usually energy detector is used for its easy realisation. In this method, a test statistic is formulated by Neyman–Pearson lemma and compared with a threshold to make a decision regarding the vacancy of a frequency band. This article shows an alternative test statistic formulation by Kullback–Leibler distance function and it is established that the proposed approach gives much better result in low signal to noise ratio zone with comparatively lesser number of samples. The performances are adjudicated by false alarm probability and detection probability. Numerical results are provided to validate the proposition.

BEARING COMPOUND FAULT ACOUSTIC DIAGNOSIS BASED ON IMPROVED BLIND DECONVOLUTION ALGORITHM

In the progress of bearing fault acoustic testing, signals picked up by acoustic sensors are usually mixed with fault source signals and other noise signals due to the complexity of mechanical signals and various interference sources. In order to solve the above problems, an improved blind deconvolution algorithm is put forward. The proposed algorithm applies adaptive generalized morphological filtering to the observed signals to retain their characteristic details, and then utilizes an OMP algorithm based on the minimum kurtosis to restore the periodical signals in the mixed signals in order to reduce the impact of the periodic components on blind separation. Finally, the improved Kullback–Leibler (KL) distance algorithm is employed to calculate the distances between independent components, which is used as the clustering index, and then to perform fuzzy C-means clustering. The experiment results of bearing compound fault extraction in real working-environment demonstrate the accuracy and reliability of the proposed algorithm.

Detecting False Data Injection Attacks in AC State Estimation

Estimating power system states accurately is crucial to the reliable operation of power grids. Traditional weighted least square (WLS) state estimation methods face the rising threat of cyber-attacks, such as false data injection attacks, which can pass the bad data detection process in WLS state estimation. In this paper, we propose a new detection method to detect false data injection attacks by tracking the dynamics of measurement variations. The Kullback–Leibler distance (KLD) is used to calculate the distance between two probability distributions derived from measurement variations. When false data are injected into the power systems, the probability distributions of the measurement variations will deviate from the historical data, thus leading to a larger KLD. The proposed method is tested on IEEE 14 bus system using load data from the New York independent system operator with different attack scenarios. We have also tested our method on false data injection attacks that replace current measurement data with historical measurement data. Test results show that the proposed approach can accurately detect most of the attacks.

Region-Based Classification of SAR Images Using Kullback–Leibler Distance Between Generalized Gamma Distributions

For the classification of synthetic aperture radar (SAR) images, traditional pixel-based Bayesian classifiers suffer from an intrinsic flaw that categories with serious overlapped probability density functions cannot be well classified. To solve this problem, in this letter, a region-based classifier for SAR images is proposed, where regions, instead of individual pixels, are treated as elements for classification. In the algorithm, each region is assigned to the class that minimizes a criterion referring to the Kullback–Leibler distance. Besides, the generalized gamma distribution $(\mathrm{G}\Gamma\mathrm{D} ) $ , a flexible empirical model, is employed for the statistical modeling of SAR images. Finally, with a synthetic image and an actual SAR image acquired by the EMISAR system, the effectiveness of the proposed algorithm is validated, compared with the pixel-based maximum-likelihood method and two region-based Bayesian classifiers.

Adaptive square-root cubature–quadrature Kalman particle filter via KLD-sampling for orbit determination

Orbit determination (OD) problem utilizing onboard sensors is a key requirement for many current and future space missions. Though there exists ample research and work on this subject, a novel algorithm is presented in this paper for the nonlinear problem of OD. In this regard, initially a new cubature–quadrature particle filter (CQPF) that uses the square-root cubature–quadrature Kalman filter (SR-CQKF) to generate the importance proposal distribution is developed. The developed CQPF scheme avoids the limitation of the standard particle filter (PF) concerning new measurements. Subsequently, CQPF is enhanced to take advantage of the relative entropy (Kullback–Leibler Distance) criterion to adaptively select the number of particles thus enhancing the efficiency and accuracy of the newly proposed adaptive CQPF (ACQPF). The current study also applies ACQPF for the OD of a low Earth orbit (LEO) satellite equipped with a three-axis magnetometer (TAM) sensor pack that provides noisy geomagnetic field measurements. The results and performance of the proposed filter are compared with a variety of Gaussian approximation filters as well as different versions of PF via a Monte Carlo simulation. It is demonstrated that the proposed ACQPF outperforms the comparative estimators in terms of the root mean square of the estimation error. Moreover, a sensitivity analysis on the orbital elements, plants' and the estimators' parameters is conducted to verify the feasibility and robustness of the ACQPF over a wider acceptable range of operating system and environment.

Saliency detection in MPEG and HEVC video using intra-frame and inter-frame distances

This paper proposes a video saliency detection model for MPEG and HEVC coded videos. The model extracts features from MPEG macroblocks and HEVC coding units. The feature variables are based on syntax elements and statistics of prediction error. The suitability of the selected features is verified through the use of stepwise regression. Three saliency maps are generated based on intra-frame distances, inter-frame distances and global distances. The proposed model is tested using the eye-1 dataset compiled by Laurent Itti Lab in the University of Southern California. The accuracy of the model is quantified by comparing saliency values at human saccade locations against saliency values at random locations. The comparison is performed in terms of Kullback–Leibler distances and receiver operator curves. The proposed solution is compared against existing work using similar experimental setup. Experimental results revealed that a Kullback–Leibler distance of 2.14 and area under the receiver operator curve of 0.936 are achieved.

Towards measuring neuroimage misalignment

To enhance neuro-navigation, high quality pre-operative images must be registered onto intra-operative configuration of the brain. Therefore evaluation of the degree to which structures may remain misaligned after registration is critically important. We consider two Hausdorff Distance (HD)-based evaluation approaches: the edge-based HD (EBHD) metric and the Robust HD (RHD) metric as well as various commonly used intensity-based similarity metrics such as Mutual Information (MI), Normalised Mutual Information (NMI), Entropy Correlation Coefficient (ECC), Kullback–Leibler Distance (KLD) and Correlation Ratio (CR). We conducted the evaluation by applying known deformations to simple sample images and real cases of brain shift. We conclude that the intensity-based similarity metrics such as MI, NMI, ECC, KLD and CR do not correlate well with actual alignment errors, and hence are not useful for assessing misalignment. On the contrary, the EBHD and the RHD metrics correlated well with actual alignment errors; however, they have been found to underestimate the actual misalignment. We also note that it is beneficial to present HD results as a percentile-HD curve rather than a single number such as the 95-percentile HD. Percentile-HD curves present the full range of alignment errors and also facilitate the comparison of results obtained using different approaches. Furthermore, the qualities that should be possessed by an ideal evaluation metric were highlighted. Future studies could focus on developing such an evaluation metric.

Entropy and density approximation from Laplace transforms

How much information does the Laplace transforms on the real line carry about an unknown, absolutely continuous distribution? If we measure that information by the Boltzmann–Gibbs–Shannon entropy, the original question becomes: How to determine the information in a probability density from the given values of its Laplace transform. We prove that a reliable evaluation both of the entropy and density can be done by exploiting some theoretical results about entropy convergence, that involve only finitely many real values of the Laplace transform, without having to invert the Laplace transform.We provide a bound for the approximation error of in terms of the Kullback–Leibler distance and a method for calculating the density to arbitrary accuracy.

Mixing times towards demographic equilibrium in insect populations with temperature variable age structures

In this study, we use entropy related mixing rate modules to measure the effects of temperature on insect population stability and demographic breakdown. The uncertainty in the age of the mother of a randomly chosen newborn, and how it is moved after a finite act of time steps, is modeled using a stochastic transformation of the Leslie matrix. Age classes are represented as a cycle graph and its transitions towards the stable age distribution are brought forth as an exact Markov chain. The dynamics of divergence, from a non equilibrium state towards equilibrium, are evaluated using the Kolmogorov–Sinai entropy. Moreover, Kullback–Leibler distance is applied as information-theoretic measure to estimate exact mixing times of age transitions probabilities towards equilibrium.Using empirically data, we show that on the initial conditions and simulated projection’s trough time, that population entropy can effectively be applied to detect demographic variability towards equilibrium under different temperature conditions. Changes in entropy are correlated with the fluctuations of the insect population decay rates (i.e. demographic stability towards equilibrium). Moreover, shorter mixing times are directly linked to lower entropy rates and vice versa. This may be linked to the properties of the insect model system, which in contrast to warm blooded animals has the ability to greatly change its metabolic and demographic rates. Moreover, population entropy and the related distance measures that are applied, provide a means to measure these rates. The current results and model projections provide clear biological evidence why dynamic population entropy may be useful to measure population stability.