Hellinger Distance Research Articles

Block generation in a two-dimensional space constructed by Hellinger metric and affinity for weather data fusion and learning inputs

The concise and reliable representation of a given source dataset is available for data based learning and decision-making, which can be obtained through a controllable data fusion process within a formatted space. Based on the density matrix representation of a given source weather dataset, this paper first calculates the Hellinger distances and affinities between different density matrices, and then takes them to construct a two-dimensional metric space using the longest Hellinger metric and affinity paths. Within this space, the given source data units are converted into the corresponding rectangle nodes which are determined by the minimum horizontal and vertical distances between different data units. According to a predefined detection size in this space, the basic blocks centred by different rectangle nodes are classified into different subset blocks for fusion. Each subset block is jointed by the basic blocks with overlapped areas. During the detection process, the detection size keeps increasing until a predefined reference variable, such as the relative density for all subset blocks, reaches a predefined turning point. The fusion of the rectangle nodes in a subset block depends on the distances between the included rectangle nodes’ positions and this subset block’s centre position which is calculated according to the included rectangle nodes’ weights and positions. The experimental analysis shows that the proposed weather data fusion method is controllable and stable and can obtain concise and reliable fusion results for learning inputs and decision-making.

An efficient and accurate recommendation strategy using degree classification criteria for item-based collaborative filtering

An efficient and accurate recommender system provides online users with a variety of personalized recommendation services, thus effectively improving the satisfaction and experience of users. However, there is a trade-off between the accuracy and efficiency in recommender systems. Accordingly, this study introduces a recommendation strategy to address this limitation. The extended degree classification criteria are first proposed to assign items to more fine-grained classes. Later, item similarity measure is deployed to quickly evaluate similarities between items within the same class, which greatly reduces the runtime of similarity calculation. To obtain a better recommendation result, a Hellinger distance (HD) based item similarity is presented to calculate item similarity from the perspective of rating probability distribution. Additionally, a sigmoid function is considered in the HD similarity to emphasize the importance of the co-rated items and effectively distinguish differences between a pair of items. The experimental results on two benchmark datasets show that the proposed similarity method using the classification criteria has better performance in both accuracy and efficiency compared to other methods. Also, the results verify the effectiveness of the proposed classification criteria, especially the runtime of item-based CF method is reduced by at least 61% while maintaining a relatively stable or higher accuracy.

Quantifying non-Gaussianity via the Hellinger distance

Non-Gaussianity is an important resource for quantum information processing with continuous variables. We introduce a measure of the non-Gaussianity of bosonic field states based on the Hellinger distance and present its basic features. This measure has some natural properties and is easy to compute. We illustrate this measure with typical examples of bosonic field states and compare it with various measures of non-Gaussianity. In particular, we highlight its similarity to and difference from the measure based on the Bures distance (or, equivalently, fidelity).

Generalized aggregation of misspecified models: With an application to asset pricing

We propose a generalized aggregation approach for model averaging. The entropy-based optimal criterion is a natural choice for aggregating information from many “globally” misspecified models as it adapts better to the underlying model uncertainty and obtains more robust approximations. Unlike almost all other approaches in the existing literature, we do not require a “reference model,” or a true data generation process contained in the set of models — neither implicitly nor in otherwise popular limiting forms. This shift in paradigm prioritizes stochastic optimization and aggregation of information about outcomes over parameter estimation of an optimally selected model. Stochastic optimization is based on a risk function of aggregators across models that satisfies oracle inequalities. Our generalized aggregators relax the common perfect substitutability of the candidate models, implicit in linear averaging and pooling. The aggregation weights are data-driven and obtained from a proper (Hellinger) distance measure. The empirical results illustrate the performance and economic significance of the aggregation approach in the context of stochastic discount factor models and inflation forecasting.

Bayesian Generalized Sparse Symmetric Tensor-on-Vector Regression

Motivated by brain connectome datasets acquired using diffusion weighted magnetic resonance imaging (DWI), this article proposes a novel generalized Bayesian linear modeling framework with a symmetric tensor response and scalar predictors. The symmetric tensor coefficients corresponding to the scalar predictors are embedded with two features: low-rankness and group sparsity within the low-rank structure. Besides offering computational efficiency and parsimony, these two features enable identification of important “tensor nodes” and “tensor cells” significantly associated with the predictors, with characterization of uncertainty. The proposed framework is empirically investigated under various simulation settings and with a real brain connectome dataset. Theoretically, we establish that the posterior predictive density from the proposed model is “close” to the true data generating density, the closeness being measured by the Hellinger distance between these two densities, which scales at a rate very close to the finite dimensional optimal rate of , depending on how the number of tensor nodes grow with the sample size. The theoretical results with proofs are provided in the supplementary materials which are available online.

On the convergence of the Laplace approximation and noise-level-robustness of Laplace-based Monte Carlo methods for Bayesian inverse problems

The Bayesian approach to inverse problems provides a rigorous framework for the incorporation and quantification of uncertainties in measurements, parameters and models. We are interested in designing numerical methods which are robust w.r.t. the size of the observational noise, i.e., methods which behave well in case of concentrated posterior measures. The concentration of the posterior is a highly desirable situation in practice, since it relates to informative or large data. However, it can pose a computational challenge for numerical methods based on the prior measure. We propose to employ the Laplace approximation of the posterior as the base measure for numerical integration in this context. The Laplace approximation is a Gaussian measure centered at the maximum a-posteriori estimate and with covariance matrix depending on the logposterior density. We discuss convergence results of the Laplace approximation in terms of the Hellinger distance and analyze the efficiency of Monte Carlo methods based on it. In particular, we show that Laplace-based importance sampling and Laplace-based quasi-Monte-Carlo methods are robust w.r.t. the concentration of the posterior for large classes of posterior distributions and integrands whereas prior-based importance sampling and plain quasi-Monte Carlo are not. Numerical experiments are presented to illustrate the theoretical findings.

Measuring the drafting alignment of patent documents using text mining.

How would an inventor, entrepreneur, investor, or patent examiner quantify the extent to which the inventive claims listed in a patent document align with patent specification? Since a specification that is poorly aligned with the inventive claims can render an invention unpatentable and can invalidate an already issued patent, an effective measure of alignment is necessary. We define a novel measure of drafting alignment using Latent Dirichlet Allocation (LDA). The measure is defined for each patent document by first identifying the latent topics underlying the claims and the specification, and then using the Hellinger distance to find the proximity between the topical coverages. We demonstrate the use of the novel measure for data processing patent documents related to cybersecurity. The properties of the proposed measure are further investigated using exploratory data analysis, and it is shown that generally alignment is positively associated with the prior patenting efforts as well as the tendency to include figures in a document.

An active fault-tolerant control method based on moving window hidden Markov model

This article proposes a new active fault-tolerant control (AFTC) method based on moving window hidden Markov models (MWHMM) for nonlinear processes. The AFTC approach could be divided into two steps, fault detection and identification (FDI) and fault accommodation. The information obtained from the FDI step is used for fault accommodation, under the framework of model predictive control (MPC). Firstly, MWHMM is utilized for online fault detection and identification. Moreover, to improve the discernibility of the prediction status probability density function (PDF) for identification between the normal and faulty modes, the Hellinger distance, which could measure the similarity between probability distributions, is introduced. As the separability between the normal and faulty modes is increased, the accuracy and reliability of fault identification by MWHMM is greatly improved. At last, not only the occurrence of a single fault but also the occurrence of a hybrid fault, which means that multiple faults occur simultaneously, is considered in this article. The validity of the suggested method is verified by the simulation results on a three-tank water system.

Multidimensional Multilayer Modulation for Broadcast UVLC With Photon Detectors

The design of finite alphabets is an application-oriented work for communication systems. In this paper, we investigate this for photon-counting based broadcast underwater visible light communication (UVLC) systems. For the demand of energy-efficiency and unique identification, our designs are conducted under the constraints of Hellinger distance criterion and uniquely decomposable constellation group (UDCG) theory and generate two kinds of multilayer modulation (MLM) constellation for different overall bit rate cases: Integer and non-integer. Correspondingly, for these two kinds of designs, we provide fast square root (FSR) receivers whose complexity are lower than that of the currently available maximum likelihood (ML) receiver. Finally, we investigate the achievable rates of downlink Poisson non-orthogonal multiple access (NOMA) and our proposed MLMs. The benefits of these designs revealed by demonstrations or simulations include that: 1) For the considered system, the proposed MLMs have a considerable performance advantage over TDMA and admit fast square root receiver at the cost of a moderate performance penalty compared with the ML receiver; 2) Compared with the Poisson NOMA, the proposed MLMs do not require successive interference cancellation (SIC) and their achievable rates exceed the achievable rate region of Poisson NOMA in high power regimes.

Emerging climate signals in the Lena River catchment: a non-parametric statistical approach

Abstract. Climate change has far-reaching implications in permafrost-underlain landscapes with respect to hydrology, ecosystems, and the population's traditional livelihoods. In the Lena River catchment, eastern Siberia, changing climatic conditions and the associated impacts are already observed or expected. However, as climate change progresses the question remains as to how far we are along this track and when these changes will constitute a significant emergence from natural variability. Here we present an approach to investigate temperature and precipitation time series from observational records, reanalysis, and an ensemble of 65 climate model simulations forced by the RCP8.5 emission scenario. We developed a novel non-parametric statistical method to identify the time of emergence (ToE) of climate change signals, i.e. the time when a climate signal permanently exceeds its natural variability. The method is based on the Hellinger distance metric that measures the similarity of probability density functions (PDFs) roughly corresponding to their geometrical overlap. Natural variability is estimated as a PDF for the earliest period common to all datasets used in the study (1901–1921) and is then compared to PDFs of target periods with moving windows of 21 years at annual and seasonal scales. The method yields dissimilarities or emergence levels ranging from 0 % to 100 % and the direction of change as a continuous time series itself. First, we showcase the method's advantage over the Kolmogorov–Smirnov metric using a synthetic dataset that resembles signals observed in the utilized climate models. Then, we focus on the Lena River catchment, where significant environmental changes are already apparent. On average, the emergence of temperature has a strong onset in the 1970s with a monotonic increase thereafter for validated reanalysis data. At the end of the reanalysis dataset (2004), temperature distributions have emerged by 50 %–60 %. Climate model projections suggest the same evolution on average and 90 % emergence by 2040. For precipitation the analysis is less conclusive because of high uncertainties in existing reanalysis datasets that also impede an evaluation of the climate models. Model projections suggest hardly any emergence by 2000 but a strong emergence thereafter, reaching 60 % by the end of the investigated period (2089). The presented ToE method provides more versatility than traditional parametric approaches and allows for a detailed temporal analysis of climate signal evolutions. An original strategy to select the most realistic model simulations based on the available observational data significantly reduces the uncertainties resulting from the spread in the 65 climate models used. The method comes as a toolbox available at https://github.com/pohleric/toe_tools (last access: 19 May 2020).

On the symmetrized s-divergence

Abstract In this study, we work with the relative divergence of type s , s ∈ ℝ s,s\in {\mathbb{R}} , which includes the Kullback-Leibler divergence and the Hellinger and χ 2 distances as particular cases. We study the symmetrized divergences in additive and multiplicative forms. Some basic properties such as symmetry, monotonicity and log-convexity are established. An important result from the convexity theory is also proved.

On the local Lipschitz stability of Bayesian inverse problems

In this note we consider the stability of posterior measures occurring in Bayesian inference w.r.t. perturbations of the prior measure and the log-likelihood function. This extends the well-posedness analysis of Bayesian inverse problems. In particular, we prove a general local Lipschitz continuous dependence of the posterior on the prior and the log-likelihood w.r.t. various common distances of probability measures. These include the total variation, Hellinger, and Wasserstein distance and the Kullback–Leibler divergence. We only assume the boundedness of the likelihoods and measure their perturbations in an Lp-norm w.r.t. the prior. The obtained stability yields under mild assumptions the well-posedness of Bayesian inverse problems, in particular, a well-posedness w.r.t. the Wasserstein distance. Moreover, our results indicate an increasing sensitivity of Bayesian inference as the posterior becomes more concentrated, for example due to more or more accurate data. This confirms and extends previous observations made in the sensitivity analysis of Bayesian inference.

The Hellinger Distance within Posterior Predictive Assessment for Investigating Multidimensionality in IRT Models

Under the Bayesian approach, posterior predictive model checking (PPMC) has become a popular tool for fit assessment of item response theory (IRT) models. In this study, we propose the use of the Hellinger distance within PPMC to quantify the distance between the realized and the predictive distribution of the model-based covariance for item pairs. Specifically, the case of multidimensional data analyzed with a unidimensional approach is taken into account. The results of the simulation study show the effectiveness of the method in detecting model misfit and the sensitivity to the trait correlations. An application to real data on tourism perceptions shows the feasibility of the method in practice and especially the capability of detecting potential misfit attributed to specific items.

Multilook Polarimetric SAR Change Detection Using Stochastic Distances Between Matrix-Variate Gd 0 Distributions

In this article, we propose an efficient heterogeneous change detection algorithm based on stochastic distance measure between two $\mathcal {G}_{d}^{0}$ distributions. Due to its flexibility and simplicity, the matrix-variate $\mathcal {G}_{d}^{0}$ distribution has been successfully used to model the multilook polarimetric synthetic aperture radar (PolSAR) data and has been tested for classification, segmentation, and image analysis. Concretely, closed-form expressions for the Kullback–Leibler, Renyi of order $\beta $ , Bhattacharyya, and Hellinger distances are provided to compute the stochastic distance between $\mathcal {G}_{d}^{0}$ distributions. In this context, we resort to the expectation–maximization (EM) to estimate accurately with low complexity for the parameters of the probability distribution of the two multilook polarimetric covariance matrices to be compared. Finally, the performance of the method is compared firstly to the performance of other known distributions, such as the scaled complex Wishart distribution, and secondly to other known statistical tests using simulated and real multilook PolSAR data.

Hellinger distance-based stable sparse feature selection for high-dimensional class-imbalanced data

BackgroundFeature selection in class-imbalance learning has gained increasing attention in recent years due to the massive growth of high-dimensional class-imbalanced data across many scientific fields. In addition to reducing model complexity and discovering key biomarkers, feature selection is also an effective method of combating overlapping which may arise in such data and become a crucial aspect for determining classification performance. However, ordinary feature selection techniques for classification can not be simply used for addressing class-imbalanced data without any adjustment. Thus, more efficient feature selection technique must be developed for complicated class-imbalanced data, especially in the context of high-dimensionality.ResultsWe proposed an algorithm called sssHD to achieve stable sparse feature selection applied it to complicated class-imbalanced data. sssHD is based on the Hellinger distance (HD) coupled with sparse regularization techniques. We stated that Hellinger distance is not only class-insensitive but also translation-invariant. Simulation result indicates that HD-based selection algorithm is effective in recognizing key features and control false discoveries for class-imbalance learning. Five gene expression datasets are also employed to test the performance of the sssHD algorithm, and a comparison with several existing selection procedures is performed. The result shows that sssHD is highly competitive in terms of five assessment metrics. In addition, sssHD presents limited differences between performing and not performing re-balance preprocessing.ConclusionssssHD is a practical feature selection method for high-dimensional class-imbalanced data, which is simple and can be an alternative for performing feature selection in class-imbalanced data. sssHD can be easily extended by connecting it with different re-balance preprocessing, different sparse regularization structures as well as different classifiers. As such, the algorithm is extremely general and has a wide range of applicability.

Discrete-time survival forests with Hellinger distance decision trees

Random survival forests (RSF) are a powerful nonparametric method for building prediction models with a time-to-event outcome. RSF do not rely on the proportional hazards assumption and can be readily applied to both low- and higher-dimensional data. A remaining limitation of RSF, however, arises from the fact that the method is almost entirely focussed on continuously measured event times. This issue may become problematic in studies where time is measured on a discrete scale t = 1, 2, ..., referring to time intervals [0,a_1), [a_1,a_2), ldots . In this situation, the application of methods designed for continuous time-to-event data may lead to biased estimators and inaccurate predictions if discreteness is ignored. To address this issue, we develop a RSF algorithm that is specifically designed for the analysis of (possibly right-censored) discrete event times. The algorithm is based on an ensemble of discrete-time survival trees that operate on transformed versions of the original time-to-event data using tree methods for binary classification. As the outcome variable in these trees is typically highly imbalanced, our algorithm implements a node splitting strategy based on Hellinger’s distance, which is a skew-insensitive alternative to classical split criteria such as the Gini impurity. The new algorithm thus provides flexible nonparametric predictions of individual-specific discrete hazard and survival functions. Our numerical results suggest that node splitting by Hellinger’s distance improves predictive performance when compared to the Gini impurity. Furthermore, discrete-time RSF improve prediction accuracy when compared to RSF approaches treating discrete event times as continuous in situations where the number of time intervals is small.

Superensemble classifier for improving predictions in imbalanced datasets

Learning from an imbalanced data set presents a tricky problem in which traditional learning algorithms perform poorly. Traditional classifiers usually aim to optimize the overall accuracy without considering the relative distribution of each class. To improve predictions in imbalanced classification problems, this article presents a superensemble classifier that maps Hellinger Distance Decision Trees (HDDT) into Radial Basis Function Networks (RBFN). Regularity conditions for universal consistency and the idea of parameter optimization of the proposed model are given in this article. The proposed distribution-free model can be applied for feature selection cum imbalanced classification problems. We have also provided enough experimental evidence using various real-life data sets to assess the performance of the proposed model. Its effectiveness and competitiveness concerning different state-of-the-art models are shown.

A generalized Hellinger distance for Choquet integral

In this paper, we introduce a generalized Hellinger distance between two monotone measures, including a derivative with respect to monotone measures. Then some properties of this distance are studied. We also obtain a lower bound for the generalized Hellinger distance. Finally, an error bound for the generalized Hellinger distance on conjugates is studied.

Quantum Hellinger distances revisited

This short note aims to study quantum Hellinger distances investigated recently by Bhatia et al. [Lett. Math. Phys. 109 (2019), 1777-1804] with a particular emphasis on barycenters. We introduce the family of generalized quantum Hellinger divergences, that are of the form $\phi(A,B)=\mathrm{Tr} \left((1-c)A + c B - A \sigma B \right),$ where $\sigma$ is an arbitrary Kubo-Ando mean, and $c \in (0,1)$ is the weight of $\sigma.$ We note that these divergences belong to the family of maximal quantum $f$-divergences, and hence are jointly convex and satisfy the data processing inequality (DPI). We derive a characterization of the barycenter of finitely many positive definite operators for these generalized quantum Hellinger divergences. We note that the characterization of the barycenter as the weighted multivariate $1/2$-power mean, that was claimed in the work of Bhatia et al. mentioned above, is true in the case of commuting operators, but it is not correct in the general case.

Proposal of a metric selection index for correspondence analysis: an application in the sensory evaluation of coffee blends

Correspondence analysis is a multivariate dimensionality-reduction technique applied to data structured into contingency tables. The main outcome of this approach is the generation of perceptual maps aimed at the study of similarity between categorical levels. In most cases, interpretations of these similarities present subjectivity when different metrics are considered; e.g., Hellinger distance and Chi-square. Thus, in an attempt to minimize this subjectivity, the present study proposes an index that quantifies the shortest distance between those levels. A simulation study was undertaken in which the generated maps were discussed in relation to real data involving the similarity of blends formed by coffees of different species, with sensory evaluations considering the flavor and acidity attributes. In conclusion, the proposed index—named metric selection index (MSI)—made it possible to include a statistic that justifies the most suitable metric for correspondence analysis, thus preventing subjectivity in interpretations of similarities between blend types and grade classes. In the simulation studies, with the metric proposed by Hellinger distance, MSI showed stabler results regarding total inertia distribution on the first two axes.