Entropy Distance Research Articles

Combination Weighting Method Based on Generalized Mahalanobis Distance and Weighting Relative Entropy

Aimed at combination weighting in multiple attribute decision making, a new approach for combining different weighting vectors is proposed. The proposed approach considers the randomicity of weights themselves and the consistency among weighting vectors, constructs a constrained weighted relative entropy model. Aimed at the disadvantage in the TOPSIS based on Euclidean distance, the TOPSIS based on Mahalanobis distance is adopted to solve the coefficients of optimal weight vector. Finally, an example is conducted and the results show the proposed approach is effective and is more reasonable than three other combination approaches.

Scaling of model approximation errors and expected entropy distances

We compute the expected value of the Kullback-Leibler divergence to various fundamental statistical models with respect to canonical priors on the probability simplex. We obtain closed formulas for the expected model approximation errors, depending on the dimension of the models and the cardinalities of their sample spaces. For the uniform prior, the expected divergence from any model containing the uniform distribution is bounded by a constant $1-\gamma$, and for the models that we consider, this bound is approached if the state space is very large and the models' dimension does not grow too fast. For Dirichlet priors the expected divergence is bounded in a similar way, if the concentration parameters take reasonable values. These results serve as reference values for more complicated statistical models.

Decoding objects of basic categories from electroencephalographic signals using wavelet transform and support vector machines.

Decoding and classification of objects through task-oriented electroencephalographic (EEG) signals are the most crucial goals of recent researches conducted mainly for brain-computer interface applications. In this study we aimed to classify single-trial 12 categories of recorded EEG signals. Ten subjects participated in this study. The task was to select target images among 12 basic object categories including animals, flowers, fruits, transportation devices, body organs, clothing, food, stationery, buildings, electronic devices, dolls and jewelry. In order to decode object categories, we have considered several units namely artifact removing, feature extraction, feature selection, and classification. Data were divided into training, validation, and test sets following the artifact removal process. Features were extracted using three different wavelets namely Daubechies4, Haar, and Symlet2. Features were selected among training data and were reduced afterward via scalar feature selection using three criteria including T test, entropy, and Bhattacharyya distance. Selected features were classified by the one-against-one support vector machine (SVM) multi-class classifier. The parameters of SVM were optimized based on training and validation sets. The classification performance (measured by means of accuracy) was obtained approximately 80% for animal and stationery categories. Moreover, Symlet2 and T test were selected as better wavelet and selection criteria, respectively.

Non-Approximation Addictive Update and Multiplicative Update Online Classifier Algorithm

Since the online learning framework that make a compromise of the correctness and conservativeness is proposed by Kivinen and Warmuth,the framework have been referenced widely,but in gradient descent and exponentiated gradient algorithms proposed by Kivinen and Warmuth,the approximation step in the derivation of loss function of objection function lead to bad results.In this work,by means of duality theory of optimization,the novel non-approximation classifier algorithms based on square distance and relative entropy loss,relative entropy distance and relative entropy loss are proposed.Experimental results show that the proposed classifiers are always more accurate than the gradient descent and exponentiated gradient algorithms proposed by Kivinen and Warmuth in the real datasets.

Juxta-vascular nodule segmentation based on flow entropy and geodesic distance.

Computed aided diagnosis of lung CT data is a new quantitative analysis technique to distinguish malignant nodules from benign ones. Nodule growth rate is a key indicator to discriminate between benign and malignant nodules. Accurate nodule segmentation is the essential for calculating the nodule growth rate. However, it is difficult to segment juxta-vascular nodules, due to the similar gray levels in nodule and attached blood vessels. To distinguish the nodule region from the adjacent vessel region, a flowing direction feature, referred to as the direction of the normal vector for a pixel, is introduced. Since blood is flowing in one single direction through a vessel, the normal vectors of pixels in the vessel region typically point in similar orientations while the directions of those in the nodule region can be viewed as disorganized. The entropy value of the flowing direction features in a neighboring region for a vessel pixel is smaller than that for a nodule pixel. Moreover, vessel pixels typically have a larger geodesic distance to the nodule center than nodule pixels. Based on k -means clustering method, the flow entropy, combined with the geodesic distance, is used to segment vessel attached nodules. The validation of the proposed segmentation algorithm was carried out on juxta-vascular nodules, identified in the Chinalung-CT screening trial and on Lung Image Database Consortium (LIDC) dataset. In fully automated mode, accuracies of 92.9% (26/28), 87.5%(7/8), and 94.9% (149/157) are reached for the outlining of juxta-vascular nodules in the Chinalung-CT, and the first and second datasets of LIDC, respectively. Furthermore, it is demonstrated that the proposed method has low time complexity and high accuracies.

Data Anonymization Approach for Incomplete Microdata

在数据发布过程中，为了防止隐私泄露，需要对数据的准标识符属性进行匿名化，以降低链接攻击风险，实现对数据所有者敏感属性的匿名保护.现有数据匿名方法都建立在数据无缺失的假设基础上，在数据存在缺失的情况下会直接丢弃相关的记录，造成了匿名化前后数据特性不一致.针对缺失数据匿名方法进行研究，基于k-匿名模型提出面向缺失数据的数据匿名方法KAIM(k-anonymity for incomplete mircrodata)，在保留包含缺失记录的前提下，使在同一属性上缺失的记录尽量被分配到同一分组参与泛化.该方法将分组泛化前后的信息熵变化作为距离，基于改进的k-member 算法对数据进行聚类分组，最后通过基于泛化层次的局部泛化算法对组内数据进行泛化.实际数据集的大量实验结果表明，KAIM 造成信息缺损仅为现有算法的43.8%，可以最大程度地保障匿名化前后数据特性不变.;To protect privacy against linking attacks, quasi-identifier attributes of microdata should be anonymized in privacy preserving data publishing. Although lots of algorithms have been proposed in this area, few of them can handle incomplete microdata. Most existing algorithms simply delete records with missing values, causing large information loss. This paper proposes a novel data anonymization approach called KAIM (k-anonymity for incomplete microdata), for incomplete microdata based on k-member algorithm and information entropy distance. Instead of deleting any records, KAIM effectively clusters records with similar characteristics together to minimize information loss, and then generalizes all records with local recording scheme. Results of extensive experiments base on real dataset show that KAIM causes only 43.8% information loss compared with previous algorithms for incomplete microdata, validating that KAIM performs much better than existing algorithms on the utility of anonymized dataset.

Spatial analysis facilitates invasive species risk assessment

Regional scale quantitative invasion risk analyses are needed to allow early detection and rapid response in order to effectively control the spread of exotic invasion. Most of the current invasion risk analyses are qualitative and ad hoc based. In this study, we used a spatial statistics based framework to assess the invasion risks of hemlock woolly adelgid (Adelges tsugae) with the following major steps: (1) invasion probability was first predicted by two widely used spatial statistics tools, maximum entropy (Maxent) and Mahalanobis distance (MD), based on known adelgid infestation locations and a set of environmental and anthropogenic related factors; (2) an ensemble of the above two models and a multi-threshold approach were employed to reduce prediction uncertainties; and (3) a spatial hotspot analysis were applied to enhance invasion prevention and management decision making. Among the factors investigated, variables representing corridors (e.g., trails and railroads) that are inadvertently spreading adelgid were important for the prediction of adelgid invasion. Large portion of the hemlock forests in the study area had a high adelgid invasion probability. The hotspot analysis based on the ensemble model showed three major clustered areas with high adelgid infestation probability. Our study demonstrated the feasibility of regional-scale quantitative invasion risk assessment with the application of a spatial statistics based framework, which can be used for effective and proactive invasion prevention and management.

Models of Relative Nearness Incidence Based on Standard Distance Entropy

The model of grey nearness incidence cannot reflect the nearness degree of sequences correctly. Therefore, the model of relative nearness incidence of sequences, curves, and surfaces is suggested based on standard distance entropy to remove the current bottlenecks and its properties are studied. At last, three cases are exemplified to demonstrate the validity and practicability of relative nearness incidence. The proposed models have enriched the theory of grey nearness incidence, filled the defects of grey nearness incidence.

Convexity of the image of a quadratic map via the relative entropy distance

Let \(\psi : {\mathbb R}^n \longrightarrow {\mathbb R}^k\) be a map defined by \(k\) positive definite quadratic forms on \({\mathbb R}^n\). We prove that the relative entropy (Kullback-Leibler) distance from the convex hull of the image of \(\psi \) to the image of \(\psi \) is bounded above by an absolute constant. More precisely, we prove that for every point \(a=\left( a_1, \ldots , a_k\right) \) in the convex hull of the image of \(\psi \) satisfying \(a_1 + \cdots + a_k=1\) there is a point \(b=\left( b_1, \ldots , b_k\right) \) in the image of \(\psi \) satisfying \(b_1 + \cdots + b_k =1\) and such that \(\sum _{i=1}^k a_i \ln \left( a_i/b_i\right) < 4.8\). Similarly, we prove that for any integer \(m\) one can choose a convex combination \(b\) of at most \(m\) points from the image of \(\psi \) such that \(\sum _{i=1}^k a_i \ln \left( a_i/b_i\right) < 17/\sqrt{m}\).

Quantifying causal influences

Many methods for causal inference generate directed acyclic graphs (DAGs) that formalize causal relations between $n$ variables. Given the joint distribution on all these variables, the DAG contains all information about how intervening on one variable changes the distribution of the other $n-1$ variables. However, quantifying the causal influence of one variable on another one remains a nontrivial question. Here we propose a set of natural, intuitive postulates that a measure of causal strength should satisfy. We then introduce a communication scenario, where edges in a DAG play the role of channels that can be locally corrupted by interventions. Causal strength is then the relative entropy distance between the old and the new distribution. Many other measures of causal strength have been proposed, including average causal effect, transfer entropy, directed information, and information flow. We explain how they fail to satisfy the postulates on simple DAGs of $\leq3$ nodes. Finally, we investigate the behavior of our measure on time-series, supporting our claims with experiments on simulated data.

Comparison and Application of Two Mathematical Models in Predicting and Determining the State of Deposition and Slagging of Coal Burning Boiler

Building a model to predict the state of slag on coal-fired boilers is a good way to optimize the coal combustion and reduce the risk of boiler slag. This paper built new models based on vague sets to predict the state of slag on coal-fired boilers, in which there were six input vectors, which were softening temperature, SiO2-Al2O3 ratio, alkali-acid ratio, percentage of silicon content, the dimensionless average temperature furnace and the dimensionless inscribed circle diameter furnace, and one output vectors, which was slagging degree. Two methods, which were based on the sense of distance and symmetric fuzzy cross entropy, were proposed to calculate the similarity between vague sets. 10 coal burning boilers were selected as known samples and the feasibility of the new methods was proved by the result of predicting the state of slag on the four coal burning boilers from Jilin heat and power plant, Xinli power plant, Jinzhou power plant and Qinhuangdao power plant. Through predicting and determining, it proves that the two pattern recognition models are high in prediction accuracy. Compared with the normal method, it is easier for operators to predict, determine the slagging state and reduce disturbance as far as possible. Besides, a prediction system has been developed by object-oriented high-level language accordingly.

SBHCS: Spike based Histogram Comparison Steganalysis Technique

is the art of secretly transferring of data and steganalysis is the art of detecting that hidden data embedded in cover media. In the past years many powerful and robust methods of steganography and steganalysis have been reported in the literature. In this present work, a Steganalysis technique for Histogram-Shifting Based Data Hiding is designed to detect hidden data by using spike generation and template matching. The proposed work analyzes the characteristics of histogram changes during data hiding procedure, and then uses these features to distinguish between stego and original image.The presented work perform the steganalysis in four steps: First, an input image is filtered by using perwitt operator for edge detection. Second, the spike image is divided into 8x8 blocks and then histogram is generated for each block. Third, histogram of each block of stego-image and original image is compared by using 5 similarity measures (norm distance, cosine distance, Euclidean distance, Chi-squared distance, Entropy distance). Fourth, Neural Network (NN) is trained as a classifier to discriminate stego image from original image. Experimental results indicate that the proposed steganalysis method is better than the method proposed by Der-Chyuan Lou et. al.(1) and can effectively detect stego image at low bit rates.

Influence of Polyproline Region and Macro Domain Genetic Heterogeneity on HEV Persistence in Immunocompromised Patients

Hepatitis E virus (HEV) can chronically infect immunocompromised patients. The polyproline region (PPR) and the macro domain of ORF1 protein may modulate virus production and/or the host immune response. We investigated the association between the genetic heterogeneity of HEV quasispecies in ORF1 and the outcome of infection in solid-organ transplant patients. Both sequence entropy and genetic distances during the hepatitis E acute phase were higher in patients whose infection became chronic than in those who cleared the virus. Hence, great quasispecies heterogeneity in the regions encoding the PPR and the macro domain may facilitate HEV persistence.

Rate of convergence and Edgeworth-type expansion in the entropic central limit theorem

An Edgeworth-type expansion is established for the entropy distance to the class of normal distributions of sums of i.i.d. random variables or vectors, satisfying minimal moment conditions.

Berry–Esseen bounds in the entropic central limit theorem

Berry–Esseen-type bounds for total variation and relative entropy distances to the normal law are established for the sums of non-i.i.d. random variables.

Extension VIKOR for Priority Orders Based on Three Parameters Interval Fuzzy Number

In this study, an improved VIKOR method was presented to deal with multi-attribute decision-making based on three parameters interval fuzzy number. The attribute weights were unknown but alternative priority of object preference was given. A new non-linear rewards and punishment method in positive interval was proposed to make the attributes normal, information covered reliability and relative superiority degree two methods were used to compare and sort the Three Parameters Interval Fuzzy Number (TPIFN) and a quadratic programming based on contribution was constructed to get attribute weights, then defined the information entropy distance between TPIFN and the optimum object orders was obtained by VIKOR. The numerical example was provided to demonstrate the feasibility and validity.

Entropic approach to E. Rio’s central limit theorem for [formula omitted] transport distance

The central limit theorem is considered with respect to the transport distance W2. We discuss an alternative approach to a result of E. Rio, based on a Berry–Esseen-type bound for the entropic distance to the normal distribution.

Game Optimization Theory and Application in Distribution System Expansion Planning, Including Distributed Generation

Based on Game Theory and Multi-objective optimization problems (MOP), Game Optimization Theory (GOT) is discussed in this paper. Optimization Stability Analysis (OSA), Distance Entropy Multi-Objective Particle Swarm Optimization (DEMPSO) and Fuzzy Multi-weights Decision-making Method (FMW) are proposed as well. Game Optimization Theory, which is a comprehensive system, could not only handle multi-objective optimization problems effectively, but also could offset the disadvantages of traditional optimization theories, such as lack of framework and the insufficient consideration of relevant elements. In this paper GOT is used for the first time in solving the distribution systems planning (DSP) issue by implementing distributed generation. The proposed model integrates costs, losses, and voltage index to achieve optimal size and site of distributed generation. The model allows minimizing total system costs, system power losses and maximizing voltage improvement. A detailed DSP example is used for verifying the effectiveness and reasonableness of GOT in this context.

Stability Problems in Cramér-Type Characterization in case of I.I.D. Summands

The stability property in Cramer's characterization of the normal law is considered in the case of identically distributed summands. As opposite results, instability is shown with respect to strong distances including the entropic distance to normality (addressing a question of M. Kac).

(QUANTUMNESS IN THE CONTEXT OF) RESOURCE THEORIES

We review the basic idea behind resource theories, where we quantify quantum resources by specifying a restricted class of operations. This divides the state space into various sets, including states which are free (because they can be created under the class of operations), and those which are a resource (because they cannot be). One can quantify the worth of the resource by the relative entropy distance to the set of free states, and under certain conditions, this is a unique measure which quantifies the rate of state to state transitions. The framework includes entanglement, asymmetry and purity theory. It also includes thermodynamics, which is a hybrid resource theory combining purity theory and asymmetry. Another hybrid resource theory which merges purity theory and entanglement can be used to study quantumness of correlations and discord, and we present quantumness in this more general framework of resource theories.