Space Of Probability Distributions Research Articles

An Intuitionistic Fuzzy Version of Hellinger Distance Measure and Its Application to Decision-Making Process

Intuitionistic fuzzy sets (IFSs), as a representative variant of fuzzy sets, has substantial advantages in managing and modeling uncertain information, so it has been widely studied and applied. Nevertheless, how to perfectly measure the similarities or differences between IFSs is still an open question. The distance metric offers an elegant and desirable solution to such a question. Hence, in this paper, we propose a new distance measure, named DIFS, inspired by the Hellinger distance in probability distribution space. First, we provide the formal definition of the new distance measure of IFSs, and analyze the outstanding properties and axioms satisfied by DIFS, which means it can measure the difference between IFSs well. Besides, on the basis of DIFS, we further present a normalized distance measure of IFSs, denoted DIFS˜. Moreover, numerical examples verify that DIFS˜ can obtain more reasonable and superior results. Finally, we further develop a new decision-making method on top of DIFS˜ and evaluate its performance in two applications.

Moufang patterns and geometry of information

Technology of data collection and transmission is based on various mathematical models of encoding. The words Geometry of information refer to such models, whereas the words patterns refer to various sophisticated symmetries appearing naturally in such models. In this paper we show that the symmetries of spaces of probability distributions, endowed with their canonical Riemannian metric of geometry, have the structure of a commutative Moufang loop. We also show that the F-manifold structure on the space of probability distribution can be described in terms of differential 3-webs and Malcev algebras. We then present a new construction of (noncommutative) Moufang loops associated to almost-symplectic structures over finite fields, and use then to construct a new class of code loops with associated quantum error-correcting codes and networks of perfect tensors.

Low dimensional measurement of vowels using machine perception.

A method is presented for combining the feature extraction power of neural networks with model based dimensionality reduction to produce linguistically motivated low dimensional measurements of sounds. This method works by first training a convolutional neural network (CNN) to predict linguistically relevant category labels from the spectrograms of sounds. Then, idealized models of these categories are defined as probability distributions in a low dimensional measurement space with locations chosen to reproduce, as far as possible, the perceptual characteristics of the CNN. To measure a sound, the point is found in the measurement space for which the posterior probability distribution over categories in the idealized model most closely matches the category probabilities output by the CNN for that sound. In this way, the feature learning power of the CNN is used to produce low dimensional measurements. This method is demonstrated using monophthongal vowel categories to train this CNN and produce measurements in two dimensions. It is also shown that the perceptual characteristics of this CNN are similar to those of human listeners.

Interpoint distance-based two-sample tests for functional data

This study examines interpoint distance-based tests for random elements from probability distributions in infinite-dimensional space. Some asymptotic properties such as the limiting distributions and the asymptotic power of the Biswas–Ghosh type test (Biswas and Ghosh in J Multivar Anal 123: 160–171, 2014) are presented using the theory of U-statistic. In addition, the p-value approximation based on the jackknife variance estimators and the Welch–Satterthwaite equation is proposed. Simulation studies are conducted to evaluate the performance of the proposed test statistics in functional data analysis. The proposed tests are shown to have better power than the existing method for functional data in some situations.

Driver Identification Through Heterogeneity Modeling in Car-Following Sequences

Intra-driver and inter-driver heterogeneity has been confirmed to exist in human driving behaviors by many studies. This research proposes a driver identification method by modeling such heterogeneities in car following sequences. It is assumed that all drivers share a pool of driver states; under each state, a car-following data sequence obeys a specific probability distribution in feature space; each driver has his/her own probability distribution over the states, called driver profile, which characterize the intra-driver heterogeneity, while the difference between the driver profile of different drivers depicts the inter-driver heterogeneity. Thus, the driver profile can be used to distinguish a driver from others. Based on the assumption, a method of driver identification is proposed to take both intra- and inter-driver heterogeneity into consideration, and a method is developed to jointly learn parameters in behavioral feature extractor, driver states, and driver profiles. Experiments demonstrate the performance of the proposed method in driver identification on naturalistic car-following data: accuracy of 82.3% is achieved in an 8-driver experiment using 10 car-following sequences of duration 15 seconds for online inference. The potential of fast registration of new drivers is demonstrated and discussed.

From data to noise to data for mixing physics across temperatures with generative artificial intelligence

Using simulations or experiments performed at some set of temperatures to learn about the physics or chemistry at some other arbitrary temperature is a problem of immense practical and theoretical relevance. Here we develop a framework based on statistical mechanics and generative artificial intelligence that allows solving this problem. Specifically, we work with denoising diffusion probabilistic models and show how these models in combination with replica exchange molecular dynamics achieve superior sampling of the biomolecular energy landscape at temperatures that were never simulated without assuming any particular slow degrees of freedom. The key idea is to treat the temperature as a fluctuating random variable and not a control parameter as is usually done. This allows us to directly sample from the joint probability distribution in configuration and temperature space. The results here are demonstrated for a chirally symmetric peptide and single-strand RNA undergoing conformational transitions in all-atom water. We demonstrate how we can discover transition states and metastable states that were previously unseen at the temperature of interest and even bypass the need to perform further simulations for a wide range of temperatures. At the same time, any unphysical states are easily identifiable through very low Boltzmann weights. The procedure while shown here for a class of molecular simulations should be more generally applicable to mixing information across simulations and experiments with varying control parameters.

Convergence theorems for random elements in convex combination spaces

A Vitali convergence theorem is proved for subspaces of an abstract convex combination space which admits a complete separable metric. The convergence may be in that metric or, more generally, in a quasimetric satisfying weaker properties. Versions for convergence in probability and in distribution are given. As applications, we show that some dominated convergence theorems in the literature of fuzzy random variables and random compact sets can be recovered or improved, and we derive new convergence theorems in another space of sets and in a space of probability distributions.

Optimal Operation of Battery Energy Storage Under Uncertainty Using Data-Driven Distributionally Robust Optimization

This paper proposes a conditional value-at-risk (CVaR)-based approach to deal with uncertainties in optimizing the participation of a battery storage in both the electricity market and demand-side management (DSM). A data-driven distributionally robust optimization (DDRO) methodology is used to solve the proposed mean-risk portfolio optimization model. The participation of a battery in DSM along with day-ahead and real-time markets (e.g., energy, spinning reserve, regulation up, and regulation down) faces uncertainties in market-clearing prices, energy demand, and available power capacity. Since in this application, the distribution of the uncertain parameters is only observable through a finite training dataset, the proposed DDRO methodology uses the Wasserstein metric to construct a ball in the space of probability distributions, which is centered at the uniform distribution on the training samples. Then, based on the worst-case distribution within this Wasserstein ball, it seeks decisions that perform best. The proposed DDRO problem over Wasserstein balls is reformulated as a finite convex problem, tested, and verified using real market data and supply/demand profiles. The results are compared with the ones obtained from other optimization methods, including deterministic and classical stochastic optimization, which validates the high performance of the proposed DDRO over finite historical sample data.

Information geometry, Pythagorean-theorem extension, and Euclidean distance behind optical sensing via spectral analysis

We present an information-geometric perspective on a generic spectral-analysis task pertaining to a vast class of optical measurements in which a parameter θ needs to be evaluated from θ-dependent spectral features in a measurable optical readout. We show that the spectral shift and line broadening driven by small Δθ variations can be isolated as orthogonal components in a Pythagorean-theorem extension for a Euclidean distance in the space of probability distributions, representing the Δθ-induced information gain, expressible via the relative entropy and the pertinent Fisher information. This result offers important insights into the limits of optical signal analysis, as well as into the ultimate spectral resolution and the limiting sensitivity of a vast class of optical measurements. As one example, we derive a physically transparent closed-form analytical solution for the information-theory bound on the precision of all-optical temperature sensors based on color centers in diamond.

Stochastic telegrapher's approach for solving the random Boltzmann-Lorentz gas

The 1D random Boltzmann-Lorentz equationhas been connected with a set of stochastic hyperbolic equations. Therefore, the study of the Boltzmann-Lorentz gas with disordered scattering centers has been transformed into the analysis of a set of stochastic telegrapher's equations. For global binary disorder (Markovian and non-Markovian) exact analytical results for the second moment, the velocity autocorrelation function, and the self-diffusion coefficient are presented. We have demonstrated that time-fluctuations in the lost of energy in the telegrapher's equation, can delay the entrance to the diffusive regime, this issue has been characterized by a timescale t_{c} which is a function of disorder parameters. Indeed, producing a longer ballistic dynamics in the transport process. In addition, fluctuations of the space probability distribution have been studied, showing that the mean value of a stochastic telegrapher's Fourier mode is a good statistical object to characterize the solution of the random Boltzmann-Lorentz gas. In a different context, the stochastic telegrapher's equationhas also been related to the run-and-tumble model in Biophysics. Then a discussion devoted to the potential applications when swimmers' speed and tumbling rate have time fluctuations has been pointed out.

On-orbit placement optimization of proximity defense shield for space station

Space station is under the increasing threaten caused by large amount of space debris on Earth orbit. Besides traditional active and passive defense measures, it is urgent to study active close-in defense strategy for protecting space station from space debris. In this paper, an active close-in defense mode is proposed for a space station based on a flexible and deployable proximity defense shield, while a hybrid procedure is formulated to optimize the on-orbit placement of the shield. First, an active close-in defense scheme for a space station is proposed. Second, the proximity defense process is modeled including orbit propagation and impact momentum transfer. Third, the orbital spatio-temporal evolution of the space debris and the proximity defense shield during the defense process are transformed into a multi-dimensional space probability distribution using the Poincaré section and the Gaussian mixture model. Then, based on probability optimization, the on-orbit placement of the proximity defense shield is optimized for protecting a space station from tiny space debris. Finally, comparison simulations demonstrate the effectiveness and feasibility of the proposed technique.

NeBula: TEAM CoSTAR’s Robotic Autonomy Solution that Won Phase II of DARPA Subterranean Challenge

This paper presents and discusses algorithms, hardware, and software architecture developed by the TEAM CoSTAR (Collaborative SubTerranean Autonomous Robots), competing in the DARPA Subterranean Challenge. Specifically, it presents the techniques utilized within the Tunnel (2019) and Urban (2020) competitions, where CoSTAR achieved second and first place, respectively. We also discuss CoSTAR’s demonstrations in Martian-analog surface and subsurface (lava tubes) exploration. The paper introduces our autonomy solution, referred to as NeBula (Networked Belief-aware Perceptual Autonomy). NeBula is an uncertainty-aware framework that aims at enabling resilient and modular autonomy solutions by performing reasoning and decision making in the belief space (space of probability distributions over the robot and world states). We discuss various components of the NeBula framework, including (i) geometric and semantic environment mapping, (ii) a multi-modal positioning system, (iii) traversability analysis and local planning, (iv) global motion planning and exploration behavior, (v) risk-aware mission planning, (vi) networking and decentralized reasoning, and (vii) learning-enabled adaptation. We discuss the performance of NeBula on several robot types (e.g., wheeled, legged, flying), in various environments. We discuss the specific results and lessons learned from fielding this solution in the challenging courses of the DARPA Subterranean Challenge competition.

Instance Correlation Graph for Unsupervised Domain Adaptation

In recent years, deep neural networks have emerged as a dominant machine learning tool for a wide variety of application fields. Due to the expensive cost of manual labeling efforts, it is important to transfer knowledge from a label-rich source domain to an unlabeled target domain. The core problem is how to learn a domain-invariant representation to address the domain shift challenge, in which the training and test samples come from different distributions. First, considering the geometry of space probability distributions, we introduce an effective Hellinger Distance to match the source and target distributions on statistical manifold. Second, the data samples are not isolated individuals, and they are interrelated. The correlation information of data samples should not be neglected for domain adaptation. Distinguished from previous works, we pay attention to the correlation distributions over data samples. We design elaborately a Residual Graph Convolutional Network to construct the Instance Correlation Graph (ICG). The correlation information of data samples is exploited to reduce the domain shift. Therefore, a novel Instance Correlation Graph for Unsupervised Domain Adaptation is proposed, which is trained end-to-end by jointly optimizing three types of losses, i.e., Supervised Classification loss for source domain, Centroid Alignment loss to measure the centroid difference between source and target domain, ICG Alignment loss to match Instance Correlation Graph over two related domains. Extensive experiments are conducted on several hard transfer tasks to learn domain-invariant representations on three benchmarks: Office-31, Office-Home, and VisDA2017. Compared with other state-of-the-art techniques, our method achieves superior performance.

Self-Supervised Color-Concept Association via Image Colorization.

The interpretation of colors in visualizations is facilitated when the assignments between colors and concepts in the visualizations match human's expectations, implying that the colors can be interpreted in a semantic manner. However, manually creating a dataset of suitable associations between colors and concepts for use in visualizations is costly, as such associations would have to be collected from humans for a large variety of concepts. To address the challenge of collecting this data, we introduce a method to extract color-concept associations automatically from a set of concept images. While the state-of-the-art method extracts associations from data with supervised learning, we developed a self-supervised method based on colorization that does not require the preparation of ground truth color-concept associations. Our key insight is that a set of images of a concept should be sufficient for learning color-concept associations, since humans also learn to associate colors to concepts mainly from past visual input. Thus, we propose to use an automatic colorization method to extract statistical models of the color-concept associations that appear in concept images. Specifically, we take a colorization model pre-trained on ImageNet and fine-tune it on the set of images associated with a given concept, to predict pixel-wise probability distributions in Lab color space for the images. Then, we convert the predicted probability distributions into color ratings for a given color library and aggregate them for all the images of a concept to obtain the final color-concept associations. We evaluate our method using four different evaluation metrics and via a user study. Experiments show that, although the state-of-the-art method based on supervised learning with user-provided ratings is more effective at capturing relative associations, our self-supervised method obtains overall better results according to metrics like Earth Mover's Distance (EMD) and Entropy Difference (ED), which are closer to human perception of color distributions.

A new divergence measure of interval-valued pythagorean fuzzy sets and its application

As the extension of the Fuzzy sets (FSs) theory, the Interval-valued Pythagorean Fuzzy Sets (IVPFS) was introduced which play an important role in handling the uncertainty. The Pythagorean fuzzy sets (PFSs) proposed by Yager in 2013 can deal with more uncertain situations than intuitionistic fuzzy sets because of its larger range of describing the membership grades. How to measure the distance of Interval-valued Pythagorean fuzzy sets is still an open issue. Jensen–Shannon divergence is a useful distance measure in the probability distribution space. In order to efficiently deal with uncertainty in practical applications, this paper proposes a new divergence measure of Interval-valued Pythagorean fuzzy sets,which is based on the belief function in Dempster–Shafer evidence theory, and is called IVPFSDM distance. It describes the Interval-Valued Pythagorean fuzzy sets in the form of basic probability assignments (BPAs) and calculates the divergence of BPAs to get the divergence of IVPFSs, which is the step in establishing a link between the IVPFSs and BPAs. Since the proposed method combines the characters of belief function and divergence, it has a more powerful resolution than other existing methods.

Taming singularities of the quantum Fisher information

Quantum Fisher information matrices (QFIMs) are fundamental to estimation theory: they encode the ultimate limit for the sensitivity with which a set of parameters can be estimated using a given probe. Since the limit invokes the inverse of a QFIM, an immediate question is what to do with singular QFIMs. Moreover, the QFIM may be discontinuous, forcing one away from the paradigm of regular statistical models. These questions of nonregular quantum statistical models are present in both single- and multiparameter estimation. Geometrically, singular QFIMs occur when the curvature of the metric vanishes in one or more directions in the space of probability distributions, while QFIMs have discontinuities when the density matrix has parameter-dependent rank. We present a nuanced discussion of how to deal with each of these scenarios, stressing the physical implications of singular QFIMs and the ensuing ramifications for quantum metrology.

Semisupervised SAR image change detection based on a siamese variational autoencoder

In synthetic aperture radar (SAR) image change detection, the deep learning has attracted increasingly more attention because the difference images (DIs) of traditional unsupervised technology are vulnerable to speckle noise. However, most of the existing deep networks do not constrain the distributional characteristics of the hidden space, which may affect the feature representation performance. This paper proposes a variational autoencoder (VAE) network with the siamese structure to detect changes in SAR images. The VAE encodes the input as a probability distribution in the hidden space to obtain regular hidden layer features with a good representation ability. Furthermore, subnetworks with the same parameters and structure can extract the spatial consistency features of the original image, which is conducive to the subsequent classification. The proposed method includes three main steps. First, the training samples are selected based on the false labels generated by a clustering algorithm. Then, we train the proposed model with the semisupervised learning strategy, including unsupervised feature learning and supervised network fine-tuning. Finally, input the original data instead of the DIs in the trained network to obtain the change detection results. The experimental results on four real SAR datasets show the effectiveness and robustness of the proposed method.

Analyzing Real Options and Flexibility in Engineering Systems Design Using Decision Rules and Deep Reinforcement Learning

AbstractEngineering systems provide essential services to society, e.g., power generation, transportation. Their performance, however, is directly affected by their ability to cope with uncertainty, especially given the realities of climate change and pandemics. Standard design methods often fail to recognize uncertainty in early conceptual activities, leading to rigid systems that are vulnerable to change. Real options and flexibility in design are important paradigms to improve a system’s ability to adapt and respond to unforeseen conditions. Existing approaches to analyze flexibility, however, do not leverage sufficiently recent developments in machine learning enabling deeper exploration of the computational design space. There is untapped potential for new solutions that are not readily accessible using existing methods. Here, a novel approach to analyze flexibility is proposed based on deep reinforcement learning (DRL). It explores available datasets systematically and considers a wider range of adaptability strategies. The methodology is evaluated on an example waste-to-energy (WTE) system. Low and high flexibility DRL models are compared against stochastically optimal inflexible and flexible solutions using decision rules. The results show highly dynamic solutions, with action space parametrized via artificial neural network (ANN). They show improved expected economic value up to 69% compared with previous solutions. Combining information from action space probability distributions along expert insights and risk tolerance helps make better decisions in real-world design and system operations. Out of sample testing shows that the policies are generalizable, but subject to tradeoffs between flexibility and inherent limitations of the learning process.

Curve Registration of Functional Data for Approximate Bayesian Computation

Approximate Bayesian computation is a likelihood-free inference method which relies on comparing model realisations to observed data with informative distance measures. We obtain functional data that are not only subject to noise along their y axis but also to a random warping along their x axis, which we refer to as the time axis. Conventional distances on functions, such as the L2 distance, are not informative under these conditions. The Fisher–Rao metric, previously generalised from the space of probability distributions to the space of functions, is an ideal objective function for aligning one function to another by warping the time axis. We assess the usefulness of alignment with the Fisher–Rao metric for approximate Bayesian computation with four examples: two simulation examples, an example about passenger flow at an international airport, and an example of hydrological flow modelling. We find that the Fisher–Rao metric works well as the objective function to minimise for alignment; however, once the functions are aligned, it is not necessarily the most informative distance for inference. This means that likelihood-free inference may require two distances: one for alignment and one for parameter inference.

Latent Network Construction for Univariate Time Series Based on Variational Auto-Encode.

Time series analysis has been an important branch of information processing, and the conversion of time series into complex networks provides a new means to understand and analyze time series. In this work, using Variational Auto-Encode (VAE), we explored the construction of latent networks for univariate time series. We first trained the VAE to obtain the space of latent probability distributions of the time series and then decomposed the multivariate Gaussian distribution into multiple univariate Gaussian distributions. By measuring the distance between univariate Gaussian distributions on a statistical manifold, the latent network construction was finally achieved. The experimental results show that the latent network can effectively retain the original information of the time series and provide a new data structure for the downstream tasks.