Information Divergence Research Articles

Discrimination of Tomato Maturity Using Hyperspectral Imaging Combined with Graph-Based Semi-supervised Method Considering Class Probability Information

Accurate and nondestructive maturity discrimination of tomatoes is significant for harvest and preservation. Color differences between intermediate adjacent maturity stages are not significant, so it is time-consuming to obtain a large amount of precise maturity labels and difficult to discriminate multiple maturity stages by visual features merely. This study designed a semi-supervised method based on hyperspectral imaging technology for tomato maturity discrimination by using a small amount of labeled samples. Firstly, the hyperspectral data of tomato samples was extracted by the hyperspectral imaging system and pre-processed by the multiple scattering correction algorithm. Then, the class probability information of unlabeled samples was described by the sparse coding of labeled samples. Next, a semi-supervised algorithm based on Laplacian score and spectral information divergence (SIDLS), which used class probability information to construct graphs, was designed to select a representative waveband subset. Finally, the sparse representation model based on class probability information (CSR) was established to construct a connection graph, and label propagation algorithm was used to discriminate tomato maturity. Experimental results demonstrated that SIDLS algorithm had an advantage in the feature selection over semi-supervised algorithm based on Laplacian score (SSLS) and Semi_Fisher Score (SFS) algorithm, and CSR model was also superior to other graph construction methods in constructing a more discriminative graph. The discrimination accuracy of this method could reach 96.78%, which shows a promising prospect in tomato maturity discrimination.

Bounds on the information divergence for hypergeometric distributions

The hypergeometric distributions have many important applications, but they have not had sufficient attention in information theory. Hypergeometric distributions can be approximated by binomial distributions or Poisson distributions. In this paper we present upper and lower bounds on information divergence. These bounds are important for statistical testing and a better understanding of the notion of exchange-ability.

A spatial functional count model for heterogeneity analysis in time

A spatial curve dynamical model framework is adopted for functional prediction of counts in a spatiotemporal log-Gaussian Cox process model. Our spatial functional estimation approach handles both wavelet-based heterogeneity analysis in time, and spectral analysis in space. Specifically, model fitting is achieved by minimising the information divergence or relative entropy between the multiscale model underlying the data, and the corresponding candidates in the spatial spectral domain. A simulation study is carried out within the family of log-Gaussian Spatial Autoregressive \(\ell ^{2}\)-valued processes (SAR\(\ell ^{2}\) processes) to illustrate the asymptotic properties of the proposed spatial functional estimators. We apply our modelling strategy to spatiotemporal prediction of respiratory disease mortality.

Target Detection Within Nonhomogeneous Clutter Via Total Bregman Divergence-Based Matrix Information Geometry Detectors

Information divergences are commonly used to measure the dissimilarity of two elements on a statistical manifold. Differentiable manifolds endowed with different divergences may possess different geometric properties, which can result in totally different performances in many practical applications. In this paper, we propose a total Bregman divergence-based matrix information geometry (TBD-MIG) detector and apply it to detect targets emerged into nonhomogeneous clutter. In particular, each sample data is assumed to be modeled as a Hermitian positive-definite (HPD) matrix and the clutter covariance matrix is estimated by the TBD mean of a set of secondary HPD matrices. We then reformulate the problem of signal detection as discriminating two points on the HPD matrix manifold. Three TBD-MIG detectors, referred to as the total square loss, the total log-determinant and the total von Neumann MIG detectors, are proposed, and they can achieve great performances due to their power of discrimination and robustness to interferences. Simulations show the advantage of the proposed TBD-MIG detectors in comparison with the geometric detector using an affine invariant Riemannian metric as well as the adaptive matched filter in nonhomogeneous clutter.

Learning Log-Determinant Divergences for Positive Definite Matrices.

Representations in the form of Symmetric Positive Definite (SPD) matrices have been popularized in a variety of visual learning applications due to their demonstrated ability to capture rich second-order statistics of visual data. There exist several similarity measures for comparing SPD matrices with documented benefits. However, selecting an appropriate measure for a given problem remains a challenge and in most cases, is the result of a trial-and-error process. In this paper, we propose to learn similarity measures in a data-driven manner. To this end, we capitalize on the αβ-log-det divergence, which is a meta-divergence parametrized by scalars α and β, subsuming a wide family of popular information divergences on SPD matrices for distinct and discrete values of these parameters. Our key idea is to cast these parameters in a continuum and learn them from data. We systematically extend this idea to learn vector-valued parameters, thereby increasing the expressiveness of the underlying non-linear measure. We conjoin the divergence learning problem with several standard tasks in machine learning, including supervised discriminative dictionary learning and unsupervised SPD matrix clustering. We present Riemannian gradient descent schemes for optimizing our formulations efficiently, and show the usefulness of our method on eight standard computer vision tasks.

Applying various satellite image processing methods on Aster and Landsat ETM+ data to identify and separate the alteration zones around gold mine of Akhtarchi, Khomein, Iran

The study area is located 100 km southeast of Arak and in two structural zones of Central Iran in the north and Sanandaj-Sirjan in the southern part. Regarding its geological structures, the area has become the source of important mines including the Akhtarchi gold mine, Aliabad iron mine, Ochestan feldspar mine, and Dali gold and copper mines. Therefore, promising areas for exploration activities are identified using the analysis of satellite images of ASTER and Landsat ETM + in the region to identify alteration areas. For this purpose, the necessary corrections were applied to the satellite images. Then, to identify the alteration parts related to the gold deposits, different satellite image processing methods of ETM + and ASTER were used. These methods include making a false-color composite, band ratio, Selective Principal Components Analysis (SPCA), Spectral Angle Mapper (SAM) method, Spectral Information Divergence Classification (SID), Endmember Collection Dialog Components (ECDC), and innovative methods such as Principal Component Analysis (PCA) and Spectral Angle Mapper, as well as unsupervised classification methods. In the end, the major alterations in the region were observed. In the obtained images, the prophylitic zone and the phyllic and argillic zones in the region were observed. To introduce the optimal method, the results of the various methods mentioned were compared with each other and with the current situation of the mines. The alteration zones were identified through band ratio and SAM methods and the combined methods with more power. Finally, SAM, 2:1 ratio, and the combined methods were identified as successful methods for more accurate separation of the alteration zones.

Maximizing the Bregman divergence from a Bregman family

The problem to maximize the information divergence from an exponential family is generalized to the setting of Bregman divergences and suitably defined Bregman families.

Reducing the quality risk of elderly care services in government procurement from market-oriented private providers through ex ante policy design: lessons from the principal-agent theory analysis

BackgroundGovernment procurement of elderly care services from market-oriented private providers has become an important way to respond to the growing demands of elderly care. However, the government cannot accurately identify the actual quality efforts of these providers, and the government pursues social benefits while the providers pursue economic interests. The existence of asymmetric information and goal divergence increases the quality risk of services. From the perspective of maximizing the government’s net benefits, this study aimed to analyze how to reduce the quality risk through ex ante policy design.MethodsOn the basis of the principal-agent theory, this study defined the asymmetric information of market-oriented private providers’ efforts on quality as a random variable, and constructed the theoretical model in the case of asymmetric information to compare with the one in the reference case of complete information, in both of which the government is the principal and market-oriented private providers are the agents. And the models also introduced several parameters to describe key factors that affect the contract results, including the physical health of the elderly, the spillover benefits to the government and market-oriented private providers, and the market risks.ResultsThe optimal results of the models in the two cases were obtained respectively, and the validity of the theoretical models was verified in a numerical example. Taking the case of complete information as the basic frame of reference, the difference of the optimal results in both cases showed the extent of negative impacts of asymmetric information, and highlighted the role of ex ante policy design in minimizing asymmetric information and reducing its negative impacts. Some ex ante policies that can improve the supervision of market-oriented private providers and their quality efforts, as well as have positive effects on key factors, were also recommended.ConclusionsThe government should attach importance to ex ante policy design to reduce the quality risk of elderly care services supplied by market-oriented private providers in government procurement. Our study provides main framework and critical directions for ex ante policy design, which is conducive to the realization of real and sustained quality improvement.

Random Finite Set Theory and Centralized Control of Large Collaborative Swarms

Controlling large swarms of robotic agents presents many challenges, including, but not limited to, computational complexity due to a large number of agents, uncertainty in the functionality of each agent in the swarm, and uncertainty in the swarm’s configuration. This work generalizes the swarm state using random finite set (RFS) theory and solves a centralized control problem with a quasi-Newton optimization through the use of model predictive control (MPC) to overcome the aforementioned challenges. This work uses the RFS formulation to control the distribution of agents assuming an unknown or unspecified number of agents. Computationally efficient solutions are also obtained via the MPC version of the iterative linear quadratic regulator (ILQR), a variant of differential dynamic programming. Information divergence is used to define the distance between the swarm RFS and the desired swarm configuration through the use of the modified distance. Simulation results using MPC and ILQR show that the swarm intensity converges to the desired intensity. Additionally, the RFS control formulation is shown to be very flexible in terms of the number of agents in the swarm and configuration of the desired Gaussian mixtures. Lastly, the ILQR and the Gaussian mixture probability hypothesis density filter are used in conjunction to solve a spacecraft relative motion problem with imperfect information to show the viability of centralized RFS control for this real-world scenario.

Jensen-information generating function and its connections to some well-known information measures

In this work, we consider the information generating function measure and develop some new results associated with it. We specifically propose two new divergence measures and show that some of the well-known information divergences such as Jensen–Shannon, Jensen-extropy and Jensen–Taneja divergence measures are all special cases of it. Finally, we also discuss the information generating function for residual lifetime variables.

Convex Polygon Maximization-Based Hyperspectral Endmember Extraction Algorithm

Pure endmember extraction from the hyperspectral image is a very essential step in target detection, classification and unmixing applications. Using the concept of convex geometry, a novel algorithm for endmember extraction is proposed in this paper. The algorithm uses convex polygon maximization to determine a convex set that gives maximum convex polygon area according to the surveyor’s formula. The parallel implementation of the proposed algorithm is useful in finding more effective distinctive pixels. The robustness of the proposed algorithm in the presence of noise is demonstrated with synthetic data. The simulation results, conducted using real hyperspectral data, indicate that the proposed algorithm reduces the Spectral Angle Error (SAE) and Spectral Information Divergence (SID) error by 2.4–8.8%. The effectiveness of the proposed algorithm in abundance mapping is also validated using Root Mean Square Error (RMSE). The RMSE of the proposed algorithm is also improved by 1.7–7.6%.

Global Circuit of the Brain through the Thalamus

What is the physiological mechanism of the harmonic dynamics of whole-brain activity? As a clue to answering this question, I have summarized the anatomical findings of global brain circuits, including the basal ganglia and cerebellum. The thalamus always appeared in these circuits as a collection point for all information. The "parallel circuit" seemed to be a basic structure of the global brain circuits. The interaction between parallel circuits was suggested to be achieved by convergence and divergence of information through the parallel circuits, and by switching between them.

Robust Two-Dimensional Linear Discriminant Analysis via Information Divergence

Due to the complexities of collected data that may contain outliers and noises, many variants of LDA and 2DLDA have been proposed to address this problem and have shown that the improved methods produce much better performance than classical LDA and 2DLDA. In this paper we propose a novel two-dimensional linear discriminant analysis method via information divergence. The proposed method applies the weighted L21 norm to learn a robust projection matrix in the image space. In the proposed model, we introduce the weights into the within-class scatter and the total scatter simultaneously, and learn the weights by imposing information divergence on the objective functions. To handle the proposed model, we resort to Dinkelbach’s extended algorithm to solve the proposed ratio minimization problem. Considering the characteristics of the subproblems, we exploit an equivalent representation of subproblems which can be solved by alternating optimization techniques where each block of variables has good optimization properties. The proposed model not only overcomes the small-sample-size problem, but also suppresses outliers by an adaptively weighted scheme with the guidance of information divergences. The experiments on several image data sets demonstrate that the classification performance of the proposed method is superior to that of some existing methods in the presence of outliers.

Optical Remote Sensing Image Registration Using Spatial-Consistency and Average Regional Information Divergence Minimization via Quantum-Behaved Particle Swarm Optimization

Due to invariance to significant intensity differences, similarity metrics have been widely used as criteria for an area-based method for registering optical remote sensing image. However, for images with large scale and rotation difference, the robustness of similarity metrics can greatly determine the registration accuracy. In addition, area-based methods usually require appropriately selected initial values for registration parameters. This paper presents a registration approach using spatial consistency (SC) and average regional information divergence (ARID), called spatial-consistency and average regional information divergence minimization via quantum-behaved particle swarm optimization (SC-ARID-QPSO) for optical remote sensing images registration. Its key idea minimizes ARID with SC to select an ARID-minimized spatial consistent feature point set. Then, the selected consistent feature set is tuned randomly to generate a set of M registration parameters, which provide initial particle warms to implement QPSO to obtain final optimal registration parameters. The proposed ARID is used as a criterion for the selection of consistent feature set, the generation of initial parameter sets, and fitness functions used by QPSO. The iterative process of QPSO is terminated based on a custom-designed automatic stopping rule. To evaluate the performance of SC-ARID-QPSO, both simulated and real images are used for experiments for validation. In addition, two data sets are particularly designed to conduct a comparative study and analysis with existing state-of-the-art methods. The experimental results demonstrate that SC-ARID-QPSO produces better registration accuracy and robustness than compared methods.

Reaching and violating thermodynamic uncertainty bounds in information engines

Thermodynamic uncertainty relations (TURs) set fundamental bounds on the fluctuation and dissipation of stochastic systems. Here, we examine these bounds, in experiment and theory, by exploring the entire phase space of a cyclic information engine operating in a nonequilibrium steady state. Close to its maximal efficiency, we find that the engine violates the original TUR. This experimental demonstration of TUR violation agrees with recently proposed softer bounds: The engine satisfies two generalized TUR bounds derived from the detailed fluctuation theorem with feedback control and another bound linking fluctuation and dissipation to mutual information and Renyi divergence. We examine how the interplay of work fluctuation and dissipation shapes the information conversion efficiency of the engine, and find that dissipation is minimal at a finite noise level, where the original TUR is violated.

A More General Information Theoretic Study of Wireless Location Verification System

In recent years, the issue of location verification system (LVS) has received significant attention in wireless communication networks, especially in Intelligent Transportation Systems (ITSs) and vehicular technology, where the location information is important for users' safety and security. In this article, we consider a vehicular network and analyze a more general Information Theoretic (ITc) study of LVS, specifically, aiming at finding an optimum decision threshold for detection a spoofed location and increasing the system ability in correctly detecting malicious users. For this purpose, we propose different information theoretic measures (Renyi divergence, Renyi mutual information, Kullback-Leibler divergence, Kullback-Leibler mutual information and Jensen-Shannon divergence) for identifying malicious user. Simulation results and comparison between proposed measures show that the optimum decision threshold can be achieved by applying the Jensen-Shannon divergence measure.

Quantitative Analysis of Mixed Pixels in Hyperspectral Image Using Fractal Dimension Technique

In this paper, a novel approach for identification of the components of a mixed pixel along with the quantitative analysis of the determined components of hyperspectral data has been proposed. The overall method is divided into three phases. First, the spectral sensitivity curves (SSCs) of the identified endmembers soil, water and vegetation using N-FINDR algorithm are generated. Next, the percentage of the mixture of the endmembers is computed for calculating the fractal dimension (FD). Afterward, the FD of SSCs is utilized for determining the mathematical model of the obtained curve. In order to verify that the adjacent pixels belong to the class identified using N-FINDR algorithm, two algorithms, i.e., spectral angle mapper and spectral information divergence, are employed for determining the spectral similarity of the mixed pixel with the reference spectrum. Lastly, the quantitative analysis of the SSC of the mixed pixel is performed by using the corresponding calculated FD in the derived mathematical model. The performance of the mathematical model is evaluated on the airborne visible infrared imaging spectrometer image data, and the outcome shows that the proposed framework achieves very promising results with a small number of mixed pixel samples.

Per-pixel analysis of ASTER data for porphyry copper hydrothermal alteration mapping: A case study of NE Isfahan, Iran

This paper evaluates the efficiency and capability of per-pixel methods comprising spectral feature fitting (SFF), multi range spectral feature fitting (MRSFF), and spectral information divergence (SID) algorithms in mapping alteration minerals using the Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) data. The study area is located in the Saveh-Yazd belt, northeastern Isfahan, Iran, which is the central part of the Urumieh–Dokhtar Magmatic Arc (UDMA). Porphyry copper mineralization formed in association with Oligo-Miocene intrusive bodies, are located in this belt. The hydrothermal alteration zones in the study area are manifested by phyllic, argillic, propylitic, and silicic. Mineralogically, these zones are marked by sericite-illite, kaolinite, montmorillonite, chlorite, epidote, and overprint of iron oxide/hydroxides. The pixel purity index (PPI) was used for extraction of image spectra, which were subsequently used as spectral reference. The SFF algorithm could map the Al–OH bearing mineral assemblages as a mixed alteration zone (phyllic-argillic zone), whereas the MRSFF provided an improved discrimination of sericite-illite and kaolinite minerals. The SID algorithm had the best performance for identifying and mapping these alteration minerals, comparing to the SFF and MRSFF. On the other hand, the results showed that SID and MRSFF algorithms were more efficient than SFF in discriminating the propylitic alteration zone. The field surveys, X-ray diffraction, spectral measurements, and microscopic studies confirmed the occurrence of hydrothermally altered minerals containing sericite-illite, kaolinite-dickite, and chlorite– epidote, which corresponded well to the ground criteria. Considering the potential of SID algorithm in discriminating the target minerals, it could be proposed for mineral mapping as part of exploration projects.

Rate-Constrained Shaping Codes for Structured Sources

Shaping codes are used to encode information for use on channels with cost constraints. Applications include data transmission with a power constraint and, more recently, data storage on flash memories with a constraint on memory cell wear. In the latter application, system requirements often impose a rate constraint. In this paper, we study rate-constrained fixed-to-variable length shaping codes for noiseless, memoryless costly channels and general i.i.d. sources. The analysis relies on the theory of word-valued sources. We establish a relationship between the code expansion factor – the ratio of the expected codeword length to the length of the input source word – and the minimum average symbol cost. We then determine the expansion factor that minimizes the average cost per source symbol (total cost), corresponding to a conventional optimal source code with cost. An equivalence is established between codes minimizing average symbol cost and codes minimizing total cost, and a separation theorem is proved, showing that optimal shaping can be achieved by a concatenation of optimal compression and optimal shaping for a uniform i.i.d. source. Shaping codes often incorporate, either explicitly or implicitly, some form of non-equiprobable signaling. We use our results to further explore the connections between shaping codes and codes that map a sequence of i.i.d. source symbols into an output sequence of symbols that are approximately independent and distributed according to a specified target distribution, such as distribution matching (DM) codes. Optimal DM codes are characterized in terms of a new performance measure - generalized expansion factor (GEF) - motivated by the costly channel perspective. The GEF is used to study DM codes that minimize informational divergence and normalized informational divergence.

Performance evaluation and comparative assessment of Sentinel-2B, Landsat-8, and AVIRIS-NG sensor imageries for extraction of road and roof surfaces using different algorithms

Urbanization in India has been rapid over the past few decades, which results in a substantial replacement of the natural surfaces into built-up lands. We present a performance evaluation of Sentinel-2B, Landsat-8 multispectral, and AVIRIS-NG hyperspectral imagery for extraction of road and roof surfaces using proposed spectral index-based and other conventional algorithms. The new road extraction index (NREI) and new building extraction index (NBEI) are developed for extraction of road and roof surfaces, respectively. Moreover, existing spectral angle mapper (SAM), spectral information divergence (SID), matched filter (MF), and support vector machine (SVM) are utilized as angle, information, filtering, and machine learning-based algorithms, respectively, for detection of both the surfaces. The results of our study suggest that the performance of AVIRIS-NG sensor is the best in comparison to aforesaid multispectral sensors, whereas Sentinel-2B performs better in comparison to Landsat-8 for extraction of road and roof surfaces. The comparison of various algorithms suggests that proposed indices, MF, and SVM produce the best results for extraction of road and roof surfaces, while SAM and SID are superior algorithms for extraction of both the surfaces in AVIRIS-NG imagery. Further, NREI and MF performed well for extraction of roads followed by NBEI and SAM for roofs in Landsat-8. Finally, NREI, SAM, and SID are found to be efficient for extraction of roads subsequently NBEI, SAM, and SID for roofs in Sentinel-2B imagery.