Entropy Distance Research Articles

Autonomous Fall Detection With Wearable Cameras by Using Relative Entropy Distance Measure

Timely, precise, and reliable detection of fall events is very important for systems monitoring activities of elderly people, especially the ones living independently. In this paper, we propose an autonomous fall detection system by taking a completely different view compared with existing vision-based activity monitoring systems and applying a reverse approach. In our system, in contrast with static sensors installed at fixed locations, the camera is worn by the subject, and thus, monitoring is not limited only to areas where the sensors are located and extends to wherever the subject may travel. Moreover, the camera provides a richer set of data and helps lower the false positive rates compared with accelerometer-only systems. We employ a modified version of the histograms of oriented gradients (HOG) approach together with the gradient local binary patterns (GLBP). It is shown that, with the same training set, the GLBP feature is more descriptive and discriminative than HOG, histograms of template, and semantic local binary patterns. Moreover, we autonomously compute a threshold, for the detection of fall events, from the training data based on relative entropy, which is a member of Ali-Silvey distance measures. Experiments are performed with ten different people and a total of around 300 associated fall events indoors and outdoors. Experimental results show that, with the autonomously computed threshold, the proposed method provides 93.78% and 89.8% accuracy for detecting falls with indoor and outdoor experiments, respectively.

Medical Image Watermarking Technique in the Application of E- diagnosis Using M-Ary Modulation

A robust and lossless ROI medical image watermarking technique using M-Ary modulation is proposed in this paper. The system's effectiveness is experimentally tested for two different medical image modalities of brain using various quality measures as payload, PSNR, MSE and SSIM. The proposed algorithm provides high robustness, higher PSNR and low relative entropy distance as a criterion for protection.

A Distance Model of Intuitionistic Fuzzy Cross Entropy to Solve Preference Problem on Alternatives

In the field of decision-making, for the multiple attribute decision-making problem with the partially unknown attribute weights, the evaluation information in the form of the intuitionistic fuzzy numbers, and the preference on alternatives, this paper proposes a comprehensive decision model based on the intuitionistic fuzzy cross entropy distance and the grey correlation analysis. The creative model can make up the deficiency that the traditional intuitionistic fuzzy distance measure is easy to cause the confusion of information and can improve the accuracy of distance measure; meanwhile, the grey correlation analysis method, suitable for the small sample and the poor information decision-making, is applied in the evaluation. This paper constructs a mathematical optimization model of maximizing the synthesis grey correlation coefficient between decision-making evaluation values and decision-makers’ subjective preference values, calculates the attribute weights with the known partial weight information, and then sorts the alternatives by the grey correlation coefficient values. Taking venture capital firm as an example, through the calculation and the variable disturbance, we can see that the methodology used in this paper has good stability and rationality. This research makes the decision-making process more scientific and further improves the theory of intuitionistic fuzzy multiple attribute decision-making.

Irreversibility and the Arrow of Time in a Quenched Quantum System.

Irreversibility is one of the most intriguing concepts in physics. While microscopic physical laws are perfectly reversible, macroscopic average behavior has a preferred direction of time. According to the second law of thermodynamics, this arrow of time is associated with a positive mean entropy production. Using a nuclear magnetic resonance setup, we measure the nonequilibrium entropy produced in an isolated spin-1/2 system following fast quenches of an external magnetic field. We experimentally demonstrate that it is equal to the entropic distance, expressed by the Kullback-Leibler divergence, between a microscopic process and its time reversal. Our result addresses the concept of irreversibility from a microscopic quantum standpoint.

Production Task Queue Optimization Based on Multi-Attribute Evaluation for Complex Product Assembly Workshop.

The production task queue has a great significance for manufacturing resource allocation and scheduling decision. Man-made qualitative queue optimization method has a poor effect and makes the application difficult. A production task queue optimization method is proposed based on multi-attribute evaluation. According to the task attributes, the hierarchical multi-attribute model is established and the indicator quantization methods are given. To calculate the objective indicator weight, criteria importance through intercriteria correlation (CRITIC) is selected from three usual methods. To calculate the subjective indicator weight, BP neural network is used to determine the judge importance degree, and then the trapezoid fuzzy scale-rough AHP considering the judge importance degree is put forward. The balanced weight, which integrates the objective weight and the subjective weight, is calculated base on multi-weight contribution balance model. The technique for order preference by similarity to an ideal solution (TOPSIS) improved by replacing Euclidean distance with relative entropy distance is used to sequence the tasks and optimize the queue by the weighted indicator value. A case study is given to illustrate its correctness and feasibility.

Athletes trained using stable compared to unstable surfaces exhibit distinct postural control profiles when assessed by traditional and nonlinear measures

Athletes are assumed to exhibit better balance than non-athletes; however, few studies have examined the role of different types of sports on balance measures. Two athlete groups that experience divergent sport-specific balance training are stable- (i.e. basketball) and unstable-surface athletes (i.e. surfers). The purpose of this study was to quantify the effect of stable- compared to unstable-surface sports on postural stability. MethodsEight non-athletes (NON), eight stable-surface athletes (SSA) and eight unstable-surface athletes (USA) performed five 20-s quiet standing trials while ground reaction forces were recorded. Approximate entropy (ApEn), total excursion and root mean square distances (RMS) of the center of pressure position were calculated. Univariate ANOVAs with post hoc tests were conducted for each variable. ResultsApEn values were lower in SSA compared to NON in the ML direction (p=0.012) and USA had lower ApEn values compared to SSA in the AP direction (p=0.036). The USA had smaller AP RMS compared to SSA (p=0.002) while the USA had greater ML RMS (p=0.008) and resultant RMS values compared to SSA (p=0.025). DiscussionThese data suggest that USA and SSA may exhibit direction-specific differences in balance strategy due to feedback paradigm.

Nonequilibrium thermodynamics and a fluctuation theorem for individual reaction steps in a chemical reaction network

We have introduced an approach to nonequilibrium thermodynamics of an open chemical reaction network in terms of the propensities of the individual elementary reactions and the corresponding reverse reactions. The method is a microscopic formulation of the dissipation function in terms of the relative entropy or Kullback-Leibler distance which is based on the analogy of phase space trajectory with the path of elementary reactions in a network of chemical process. We have introduced here a fluctuation theorem valid for each opposite pair of elementary reactions which is useful in determining the contribution of each sub-reaction on the nonequilibrium thermodynamics of overall reaction. The methodology is applied to an oligomeric enzyme kinetics at a chemiostatic condition that leads the reaction to a nonequilibrium steady state for which we have estimated how each step of the reaction is energy driven or entropy driven to contribute to the overall reaction.

Quantum Conditional Mutual Information, Reconstructed States, and State Redistribution.

We give two strengthenings of an inequality for the quantum conditional mutual information of a tripartite quantum state recently proved by Fawzi and Renner, connecting it with the ability to reconstruct the state from its bipartite reductions. Namely, we show that the conditional mutual information is an upper bound on the regularized relative entropy distance between the quantum state and its reconstructed version. It is also an upper bound for the measured relative entropy distance of the state to its reconstructed version. The main ingredient of the proof is the fact that the conditional mutual information is the optimal quantum communication rate in the task of state redistribution.

The Gap between the Conditional Wage Distributions of Incumbents and the Newly Hired Employees: Decomposition and Uniform Ordering

We examine the cardinal gap between wage distributions of the incumbents and newly hired workers based on entropic distances that are well-defined welfare theoretic measures. Decomposition of several effects is achieved by identifying several counterfactual distributions of different groups. These go beyond the usual Oaxaca-Blinder decompositions at the (linear) conditional means. Much like quantiles, these entropic distances are well defined inferential objects and functions whose statistical properties have recently been developed. Going beyond these strong rankings and distances, we consider weak uniform ranking of these wage outcomes based on statistical tests for stochastic dominance. We focus the empirical analysis on employees with at least 35 hours of work in the 1996?2012 monthly Current Population Survey. Among other findings, we find incumbent workers enjoy a better distribution of wages, but the attribution of the gap to wage inequality and human capital characteristics varies between quantiles. For instance, highly paid new workers are mainly due to human capital components and, in some years, even better wage structure.

Similarity Distance Noise Reduction of Entropy Based on Lifting KNN Classification Performance

To overcome the drawback of KNN algorithms based on distance measure which did not measure the contributions for each feature accurately. In this paper, a K-Nearest Neighbor (KNN) de-noise method based on likelihood distance entropy is proposed. The relations of feature parameters are used to measure their contributions for de-noise energy, then according to the contributions for each feature leading de-noise of the feature parameters. In order to compare the performance of these relative methods, the Letter corpora and Pima Indians Diabetes database are employ to carry out the experiments, the experiment results show that comparing with the other de-noise methods mentioned in this paper, this proposed method have a better ability for de-noise.

An information theory based framework for the measurement of population health

This paper proposes a new framework for the measurement of population health and the ranking of the health of different geographies. Since population health is a latent variable, studies which measure and rank the health of different geographies must aggregate observable health attributes into one summary measure. We show that the methods used in nearly all the literature to date implicitly assume that all attributes are infinitely substitutable. Our method, based on the measurement of multidimensional welfare and inequality, minimizes the entropic distance between the summary measure of population health and the distribution of the underlying attributes. This summary function coincides with the constant elasticity of substitution and Cobb–Douglas production functions and naturally allows different assumptions regarding attribute substitutability or complementarity. To compare methodologies, we examine a well-known ranking of the population health of U.S. states, America's Health Rankings. We find that states’ rankings are somewhat sensitive to changes in the weight given to each attribute, but very sensitive to changes in aggregation methodology. Our results have broad implications for well-known health rankings such as the 2000 World Health Report, as well as other measurements of population and individual health levels and the measurement and decomposition of health inequality.

Weights and substitution degree in multidimensional well-being in China

Purpose – The purpose of this paper is to combine multidimensional welfare analysis and entropy metrics to derive not only the best relative weights but also substitution degree among different attributes to construct multidimensional indices of well-being with Chinese Household Income Project Survey 2002 data. Design/methodology/approach – The authors follow Maasoumi’s two-step measures of multivariate inequality to calculate the inequality for three social groups in China, urban residents, migrants, and rural residents. The two-step approach provides an aggregation formula which is numerically identified in this paper based on a metric entropy distance measure between the distribution of the aggregate well-being functions, on the one hand, and the distribution of the self-reported “happiness” indicator. The authors compare the differences in relative weights and substitution degree for the three groups, and link them to some institutional factors. Findings – The authors find that incorporating substitut...

Selection Method of Evaluation Indicators with Three-Parameter Interval Grey Number

The evaluation problem with three-parameter interval grey number (T-PIGN) widely exists in real world. To select effective evaluation indicators of the problem, this paper puts forward evaluation index system selection principle of T-PIGN based on distance entropy model, and gives out evaluation index system selection judgment criterion of T-PIGN. Furthermore, for the redundancy of evaluation index system with T-PIGN, a selection method of evaluation index system with T-PIGN is proposed. Finally, the applicability of the proposed method is verified by concrete examples.

Bell Length in the Entanglement Geometry

A geometric approach to entangled qubit pairs is outlined via Bohm Theory. An entropic quantum correlation distance is here proposed as a mark of the non-local “handshaking” between two systems under the action of Quantum Potential. The Bell-CHSH inequalities and Berry Phase are analyzed in terms of this new correlation measure we called Bell Length, in honour of J.S. Bell (1928-1990).

Information Theory in the mathematical Statistics

In the paper we present an outline of the information theory from the probabilistic and statistical point of view. Such a direction of the information theory has been intensively developed in recent decades and significantly influenced a progress in the statistical methodology. The aim of the article is to introduce the reader into these problems, provide some intuitions and acquaint with a specific information-theoretic approach to the mathematical statistics. The first part of the paper is devoted to brief and easy of approach introduction to the main notions of the information theory like entropy, relative entropy (Kullback-Leibler distance), information projection and Fisher information as well as presentation of their most important properties including de Bruijn's identity, Fisher information inequalities and entropy power inequalities. In the short second part we give applications of the notions and results from the first part to limit theorems of the probability theory such as the asymptotic equipartition property, the convergence of empirical measures in the entropy distance, large deviation principle with emphasis to Sanov theorem, the convergence of distributions of homogeneous Markov chains in the entropy distance and the central limit theorem. The main, last part of the article shows some most significant and important applications of the information theory to the mathematical statistics. We discuss connections of the maximum likelihood estimators with the information projections and the notion of sufficient statistic from the information-theoretic point of view. The problems of source coding, channel capacity and an amount of information provided by statistical experiments are presented in a statistical framework. Some attention is paid to the expansion of Clarke and Barron and its corollaries e.g. in density estimation. Next, applications of the information theory to hypothesis testing is discussed. We give the classical Stein's Lemma and its generalization to testing composite hypothesis obtained by Bahadur and show their connections with the asymptotic efficiency of statistical tests. Finally, we briefly mention the problem of information criteria in a model seletion including the most popular two-stage minimal description length criterion of Rissanen. The enclosed literature is limited only to papers and books which are referred to in the paper.

Neighborhood Evaluation in Recommender Systems Using the Realization Based Entropy Approach

The task of a recommender system evaluation has often been addressed in the literature, however there exists no consensus regarding the best metrics to assess its performance. This research deals with collaborative filtering recommendation systems, and proposes a new approach for evaluating the quality of neighbor selection. It theorizes that good recommendations emerge from good selection of neighbors. Hence, measuring the quality of the neighborhood may be used to predict the recommendation success. Since user neighborhoods in recommender systems are often sparse and differ in their rating range, this paper designs a novel measure to asses a neighborhood quality. First it builds the realization based entropy (RBE), which presents the classical entropy measure from a different angle. Next it modifies the RBE and propose the realization based distance entropy (RBDE), which considers also continuous data. Using the RBDE, it finally develops the consent entropy, which takes into account the absence of rating data. The paper compares the proposed approach with common approaches from the literature, using several recommendation evaluation metrics. It presents offline experiments using the Netflix database. The experimental results confirm that consent entropy performs better than commonly used metrics, particularly with high sparsity neighborhoods. This research is supported by The Israel Science Foundation, Grant #1362/10. This research is supported by NHECD EC, Grant #218639.

Sample-based evaluation of global probabilistic sensitivity measures

The evaluation of probabilistic sensitivity requires scalar measures quantifying the difference between probability density functions. The efficient estimation of two such alternative measures (relative entropy and Hellinger distance) is investigated within the setting of a global sensitivity analysis. This analysis requires evaluation of these measures with respect to different marginal densities, all connected to the same auxiliary joint density. Estimation of these densities is based on implementation of kernel density estimation, using information from samples from their joint distribution. These samples are further explored for improving the computational efficiency whereas theoretical concepts related to these two measures are addressed.

Using information metrics and molecular communication to detect cellular tissue deformation.

Calcium-signaling-based molecular communication has been proposed as one form of communication for short range transmission between nanomachines. This form of communication is naturally found within cellular tissues, where Ca(2+) ions propagate and diffuse between cells. However, the naturally flexible structure of cells usually leads to the cells dynamically changing shape under strain. Since the interconnected cells form the tissue, a change in shape of one cell will change the shape of the neighboring cells and the tissue as a whole. This will in turn dramatically impair the communication channel between the nanomachines. We propose a process for nanomachines utilizing Ca(2+) based molecular communication to infer and detect the state of the tissue, which we term the Molecular Nanonetwork Inference Process. The process employs a threshold based classifier that identifies its threshold boundaries based on a training process. The inference/detection mechanism allows the destination nanomachine to determine: i) the type of tissue deformation; ii) the amount of tissue deformation; iii) the amount of Ca(2+) concentration emitted from the source nanomachine; and iv) its distance from the destination nanomachines. We evaluate the use of three information metrics: mutual information, mutual information with generalized entropy and information distance. Our analysis, which is conducted on two different topologies, finds that mutual information with generalized entropy provides the most accurate inferencing/detection process, enabling the classifier to obtain 80% of accuracy on average.

Information Divergence and Distance Measures for Quantum States

Both information divergence and distance are measures of closeness of two quantum states which are widely used in the theory of information processing and quantum cryptography. For example, the quantum relative entropy and trace distance are well known. Here we introduce a number of new quantum information divergence and distance measures into the literature and discuss their relations and properties. We also propose a method to analyze the properties and relations of various distance and pseudo-distance measures.

An Information Theory Based Framework for the Measurement of Population Health

This paper proposes a new framework for the measurement of population health and the ranking of the health of different geographies. Since population health is a latent variable, studies which measure and rank the health of different geographies must aggregate observable health attributes into one summary measure. We show that the methods used in nearly all the literature to date implicitly assume that all attributes are infinitely substitutable. Our method, based on the measurement of multidimensional welfare and inequality, minimizes the entropic distance between the summary measure of population health and the distribution of the underlying attributes. This summary function coincides with the constant elasticity of substitution and Cobb-Douglas production functions and naturally allows different assumptions regarding attribute substitutability or complementarity. To compare methodologies, we examine a well-known ranking of the population health of U.S. states, America’s Health Rankings. We find that states’ rankings are somewhat sensitive to changes in the weight given to each attribute, but very sensitive to changes in aggregation methodology. Our results have broad implications for well-known health rankings such as the 2000 World Health Report, as well as the measurement of health inequality.