Symmetric Kullback-Leibler Distance Research Articles

2015 Yılında Meydana Gelen Üç Şiddetli Jeomanyetik Fırtına Süresince Deneysel İyonosferik Modellerin Karşılaştırılması

The geomagnetic field acts as both the shield and the electron density regulator for the ionosphere. The effect of the geomagnetic field on the ionosphere can be examined separately for the geomagnetically quiet and disturbed days. In the current study, the performance of the ionospheric models was evaluated for three different severe geomagnetic storms periods during the year of 2015, which was in the beginning of the descending phase of the 24th solar cycle. These three storms occurred during 17-18 March, 22-23 June and 20-21 December of year 2015 in which first one expressed as St. Patrick's Day geomagnetic storm. The relationship between Total Electron Content (TEC) was measured by Global Positioning System (GPS) and evaluated with NeQuick 2, IRI 2016, IRI Plas (without any input- “IRI Plas”) and IRI Plas TEC (with TEC input- “IRI Plas TEC”) global models at three Turkey IGS station namely Ankara (39.57 N, 32.53 E), Istanbul (40.58 N, 29.05 E) and Erzurum (40.39 N, 40.42 E) investigated. The comparison was made separately for pre-storm, during storm and post-storm by using the Mean Absolute Error (MAE), Root Mean Square Error (RMSE) and Mean Absolute Percentage Error (MAPE) metrics and symmetric Kullback-Leibler Distance (KLD) methods. Among the empirical models, IRI Plas TEC is generally present to be better results than other models for all storm processes. It can be stated that IRI 2016 is better in the storm return phase compared to other phases of the storm.

Density Regression with Conditional Support Points

Density regression characterizes the conditional density of the response variable given the covariates, and provides much more information than the commonly used conditional mean or quantile regression. However, it is often computationally prohibitive in applications with massive datasets, especially when there are multiple covariates. In this article, we develop a new data reduction approach for the density regression problem using conditional support points. After obtaining the representative data, we exploit the penalized likelihood method as the downstream estimation strategy. Based on the connections among the continuous ranked probability score, the energy distance, the L 2 discrepancy and the symmetrized Kullback–Leibler distance, we investigate the distributional convergence of the representative points and establish the rate of convergence of the density regression estimator. The usefulness of the methodology is illustrated by modeling the conditional distribution of power output given multivariate environmental factors using a large scale wind turbine dataset.

River Channel Extraction in SAR Images Using Level Sets Driven by Symmetric Kullback–Leibler Distance

This paper proposes a novel level set method (LSM) that improves river channel extraction accuracy in synthetic aperture images (SAR) by developing global median image fitting energies. First, we define a new global median fitting image (GMFI) to approximate the input image and use this GMFI to construct the fitting energy based on the symmetric Kullback-Leibler distance (SKL). Second, to exploit more image grayscale features, a squared global median fitting image (SGMFI) is derived and another fitting energy is similarly constructed using this SGMFI based on SKL. Third, we integrate the above two fitting energies and introduce additional regularized energies. The proposed LSM is verified and compared with several state-of-the-art methods on real SAR images. The river channel extraction results indicate that our proposed LSM has a clear advantage in accuracy and is robust to level set initialization.

Kullback-Leibler distance-based enhanced detection of incipient anomalies

Accurate and effective anomaly detection and diagnosis of modern engineering systems by monitoring processes ensure reliability and safety of a product while maintaining desired quality. In this paper, an innovative method based on Kullback-Leibler divergence for detecting incipient anomalies in highly correlated multivariate data is presented. We use a partial least square (PLS) method as a modeling framework and a symmetrized Kullback-Leibler distance (KLD) as an anomaly indicator, where it is used to quantify the dissimilarity between current PLS-based residual and reference probability distributions obtained using fault-free data. Furthermore, this paper reports the development of two monitoring charts based on the KLD. The first approach is a KLD-Shewhart chart, where the Shewhart monitoring chart with a three sigma rule is used to monitor the KLD of the response variables residuals from the PLS model. The second approach integrates the KLD statistic into the exponentially weighted moving average monitoring chart. The performance of the PLS-based KLD anomaly-detection methods is illustrated and compared to that of conventional PLS-based anomaly detection methods. Using synthetic data and simulated distillation column data, we demonstrate the greater sensitivity and effectiveness of the developed method over the conventional PLS-based methods, especially when data are highly correlated and small anomalies are of interest. Results indicate that the proposed chart is a very promising KLD-based method because KLD-based charts are, in practice, designed to detect small shifts in process parameters.

Performance of GPS slant total electron content and IRI-Plas-STEC for days with ionospheric disturbance

Total Electron Content (TEC) is an important observable parameter of the ionosphere which forms the main source of error for space based navigation and positioning systems. Since the deployment of Global Navigation Satellite Systems (GNSS), cost-effective estimation of TEC between the earth based receiver and Global Positioning System (GPS) satellites became the major means of investigation of local and regional disturbance for earthquake precursor and augmentation system studies. International Reference Ionosphere (IRI) extended to plasmasphere (IRI-Plas) is the most developed ionospheric and plasmaspheric climatic model that provides hourly, monthly median of electron density distribution globally. Recently, IONOLAB group (www.ionolab.org) has presented a new online space weather service that can compute slant TEC (STEC) on a desired ray path for a given date and time using IRI-Plas model (IRI-Plas-STEC). In this study, the performance of the model based STEC is compared with GPS-STEC computed according to the estimation method developed by the IONOLAB group and includes the receiver bias as IONOLAB-BIAS (IONOLAB-STEC). Using Symmetric Kullback–Leibler Distance (SKLD), Cross Correlation (CC) coefficient and the metric norm (L2N) to compare IRI-Plas-STEC and IONOLAB-STEC for the month of October 2011 over the Turkish National Permanent GPS Network (TNPGN-Active), it has been observed that SKLD provides a good indicator of disturbance for both earthquakes and geomagnetic storms.

Mining categorical sequences from data using a hybrid clustering method

The identification of different dynamics in sequential data has become an every day need in scientific fields such as marketing, bioinformatics, finance, or social sciences. Contrary to cross-sectional or static data, this type of observations (also known as stream data, temporal data, longitudinal data or repeated measures) are more challenging as one has to incorporate data dependency in the clustering process. In this research we focus on clustering categorical sequences. The method proposed here combines model-based and heuristic clustering. In the first step, the categorical sequences are transformed by an extension of the hidden Markov model into a probabilistic space, where a symmetric Kullback–Leibler distance can operate. Then, in the second step, using hierarchical clustering on the matrix of distances, the sequences can be clustered. This paper illustrates the enormous potential of this type of hybrid approach using a synthetic data set as well as the well-known Microsoft dataset with website users search patterns and a survey on job career dynamics.

Efficiency of Free Energy Transduction in Autonomous Systems

We consider the thermodynamics of chemical coupling from the viewpoint of free energy transduction efficiency. In contrast to an external parameter-driven stochastic energetics setup, the dynamic change of the equilibrium distribution induced by chemical coupling, adopted, for example, in biological systems, is inevitably an autonomous process. We found that the efficiency is bounded by the ratio between the non-symmetric and the symmetrized Kullback-Leibler distance, which is significantly lower than unity. Consequences of this low efficiency are demonstrated in the simple two-state case, which serves as an important minimal model for studying the energetics of biomolecules.

Speaker Recognition using Excitation Source Parameters

Excitation signal is used in speaker recognition. It corresponds to the frequency of oscillation of vocal cords and is one of the speaker's characteristics. Although this feature gives worse recognition results compared to the vocal tract parameters, but it is more robust to various distortions in the recording channels. As a result, pitch is commonly used in forensic investigations, where different recording channels is one of the main problems. Currently, the pitch distribution generally is modeled using histograms and calculating various distances or similarity measures between two histograms. However, pitch distribution is not Gaussian and view of the histograms and comparison results depend on the number of classes used. We model pitch distribution using Gaussian mixture models (GMM), and calculate similarity and distance measures between the GMM approximations of two comparative records. Best results were achieved using symmetric Kullback-Leibler distance.

Investigation of total electron content variability due to seismic and geomagnetic disturbances in the ionosphere

[1] Variations in solar, geomagnetic, and seismic activity can cause deviations in the ionospheric plasma that can be detected as disturbances in both natural and man-made signals. Total electron content (TEC) is an efficient means for investigating the structure of the ionosphere by making use of GPS receivers. In this study, TEC data obtained for eight GPS stations are compared with each other using the cross-correlation coefficient (CC), symmetric Kullback-Leibler distance (KLD), and L2 norm (L2N) for quiet days of the ionosphere, during severe geomagnetic storms and strong earthquakes. It is observed that only KLD and L2N can differentiate the seismic activity from the geomagnetic disturbance and quiet ionosphere if the stations are in a radius of 340 km. When TEC for each station is compared with an average quiet day TEC for all periods using CC, KLD, and L2N, it is observed that, again, only KLD and L2N can distinguish the approaching seismicity for stations that are within 150 km radius to the epicenter. When the TEC of consecutive days for each station and for all periods are compared, it is observed that CC, KLD, and L2N methods are all successful in distinguishing the geomagnetic disturbances. Using sliding-window statistical analysis, moving averages of daily TEC with estimated variance bounds are also obtained for all stations and for all days of interest. When these bounds are compared with each other for all periods, it is observed that CC, KLD, and L2N are successful tools for detecting ionospheric disturbances.

Statistical Wavelet Subband Characterization Based on Generalized Gamma Density and Its Application in Texture Retrieval

The modeling of image data by a general parametric family of statistical distributions plays an important role in many applications. In this paper, we propose to adopt the three-parameter generalized Gamma density (GGammaD) for modeling wavelet detail subband histograms and for texture image retrieval. The advantage of GGammaD over the existing generalized Gaussian density (GGD) is that it provides more flexibility to control the shape of model which is critical for practical histogram-based applications. To measure the discrepancy between GGammaDs, we use the symmetrized Kullback-Leibler distance (SKLD) and derive a closed form for the SKLD between GGammaDs. Such a distance can be computed directly and effectively via the model parameters, making our proposed scheme particularly suitable for image retrieval systems with large image database. Experimental results on the well-known databases reveal the superior performance of our proposed method compared with the current existing approaches.

Analysis of Emotionally Salient Aspects of Fundamental Frequency for Emotion Detection

During expressive speech, the voice is enriched to convey not only the intended semantic message but also the emotional state of the speaker. The pitch contour is one of the important properties of speech that is affected by this emotional modulation. Although pitch features have been commonly used to recognize emotions, it is not clear what aspects of the pitch contour are the most emotionally salient. This paper presents an analysis of the statistics derived from the pitch contour. First, pitch features derived from emotional speech samples are compared with the ones derived from neutral speech, by using symmetric Kullback-Leibler distance. Then, the emotionally discriminative power of the pitch features is quantified by comparing nested logistic regression models. The results indicate that gross pitch contour statistics such as mean, maximum, minimum, and range are more emotionally prominent than features describing the pitch shape. Also, analyzing the pitch statistics at the utterance level is found to be more accurate and robust than analyzing the pitch statistics for shorter speech regions (e.g., voiced segments). Finally, the best features are selected to build a binary emotion detection system for distinguishing between emotional versus neutral speech. A new two-step approach is proposed. In the first step, reference models for the pitch features are trained with neutral speech, and the input features are contrasted with the neutral model. In the second step, a fitness measure is used to assess whether the input speech is similar to, in the case of neutral speech, or different from, in the case of emotional speech, the reference models. The proposed approach is tested with four acted emotional databases spanning different emotional categories, recording settings, speakers and languages. The results show that the recognition accuracy of the system is over 77% just with the pitch features (baseline 50%). When compared to conventional classification schemes, the proposed approach performs better in terms of both accuracy and robustness.

Density estimation via hybrid splines

The Hybrid Spline method (H-spline) is a method of density estimation which involves regression splines and smoothing splines methods. Using basis functions (B-splines), this method is much faster than Smoothing Spline Density Estimation approach (Gu, 1993). Simulations suggest that with more structured data (e.g., several modes) H-spline method estimates the modes as well as Logspline (Kooperberg and Stone, 1991). The H-spline algorithm is designed to compute a solution to the penalized likelihood problem. The smoothing parameter is updated jointly with the estimate via a cross-validation performance estimate, where the performance is measured by a proxy of the symmetrized Kullback-Leibler. The initial number of knots is determined automatically based on an estimate of the number of modes and the symmetry of the underlying density. The algorithm increases the number of knots by 1 until the symmetrized Kullback-Leibler distance, based on two consecutives estimates satisfies a condition which was determined empirically.

Smoothing Spline Density Estimation: A Dimensionless Automatic Algorithm

Abstract As a sequel to an earlier article by Gu and Qiu, this article describes and illustrates a dimensionless automatic algorithm for nonparametric probability density estimation using smoothing splines. The algorithm is designed to calculate an adaptive finite dimensional solution to the penalized likelihood problem, which was shown by Gu and Qiu to share the same asymptotic convergence rates as the nonadaptive infinite dimensional solution. The smoothing parameter is updated jointly with the estimate in a performance-oriented iteration via a cross-validation performance estimate, where the performance is measured by proxies of the symmetrized Kullback-Leibler distance between the true density and the estimate. Simulations of limited scale are conducted to examine the relative effectiveness of the automatic smoothing parameter selection procedure and to assess the practical statistical performance of the methodology in general. The method is also applied to some real data sets. The algorithm is implemented in a few Ratfor routines, which will be included in later versions of RKPACK for public access.

Smoothing Spline Density Estimation: A Dimensionless Automatic Algorithm

Abstract As a sequel to an earlier article by Gu and Qiu, this article describes and illustrates a dimensionless automatic algorithm for nonparametric probability density estimation using smoothing splines. The algorithm is designed to calculate an adaptive finite dimensional solution to the penalized likelihood problem, which was shown by Gu and Qiu to share the same asymptotic convergence rates as the nonadaptive infinite dimensional solution. The smoothing parameter is updated jointly with the estimate in a performance-oriented iteration via a cross-validation performance estimate, where the performance is measured by proxies of the symmetrized Kullback-Leibler distance between the true density and the estimate. Simulations of limited scale are conducted to examine the relative effectiveness of the automatic smoothing parameter selection procedure and to assess the practical statistical performance of the methodology in general. The method is also applied to some real data sets. The algorithm is implemented in a few Ratfor routines, which will be included in later versions of RKPACK for public access.

Smoothing Spline Density Estimation: Theory

In this article, a class of penalized likelihood probability density estimators is proposed and studied. The true log density is assumed to be a member of a reproducing kernel Hilbert space on a finite domain, not necessarily univariate, and the estimator is defined as the unique unconstrained minimizer of a penalized log likelihood functional in such a space. Under mild conditions, the existence of the estimator and the rate of convergence of the estimator in terms of the symmetrized Kullback-Leibler distance are established. To make the procedure applicable, a semiparametric approximation of the estimator is presented, which sits in an adaptive finite dimensional function space and hence can be computed in principle. The theory is developed in a generic setup and the proofs are largely elementary. Algorithms are yet to follow.