Fisher Information Matrix Research Articles

Effect of detector placement on joint estimation in X-ray fluorescence emission tomography.

Imaging the spatial distribution of low concentrations of metal is a growing problem of interest with applications in medical and material sciences. X-ray fluorescence emission tomography (XFET) is an emerging metal mapping imaging modality with potential sensitivity improvements and practical advantages over other methods. However, XFET detector placement must first be optimized to ensure accurate metal density quantification and adequate spatial resolution. In this work, we first use singular value decomposition of the imaging model and eigendecomposition of the object-specific Fisher information matrix to study how detector arrangement affects spatial resolution and feature preservation. We then perform joint image reconstructions of a numerical gold phantom. For this phantom, we show that two parallel detectors provide metal quantification with similar accuracy to four detectors, despite the resulting anisotropic spatial resolution in the attenuation map estimate. Two orthogonal detectors provide improved spatial resolution along one axis, but underestimate the metal concentration in distant regions. Therefore, this work demonstrates the minor effect of using fewer, but strategically placed, detectors in the case where detector placement is restricted. This work is a critical investigation into the limitations and capabilities of XFET prior to its translation to preclinical and benchtop uses.

Sloppy model analysis identifies bifurcation parameters without normal form analysis.

Bifurcation phenomena are common in multidimensional multiparameter dynamical systems. Normal form theory suggests that bifurcations are driven by relatively few combinations of parameters. Models of complex systems, however, rarely appear in normal form, and bifurcations are controlled by nonlinear combinations of the bare parameters of differential equations. Discovering reparameterizations to transform complex equationsinto a normal form is often very difficult, and the reparameterization may not even exist in a closed form. Here we show that information geometry and sloppy model analysis using the Fisher information matrix can be used to identify the combination of parameters that control bifurcations. By considering observations on increasingly long timescales, we find those parameters that rapidly characterize the system's topological inhomogeneities, whether the system is in normal form or not. We anticipate that this novel analytical method, which we call time-widening information geometry (TWIG), will be useful in applied network analysis.

Fast and robust quantification of uncertainty in non-linear diffusion MRI models

Diffusion MRI (dMRI) allows for non-invasive investigation of brain tissue microstructure. By fitting a model to the dMRI signal, various quantitative measures can be derived from the data, such as fractional anisotropy, neurite density and axonal radii maps. We investigate the Fisher Information Matrix (FIM) and uncertainty propagation as a generally applicable method for quantifying the parameter uncertainties in linear and non-linear diffusion MRI models. In direct comparison with Markov Chain Monte Carlo (MCMC) sampling, the FIM produces similar uncertainty estimates at much lower computational cost. Using acquired and simulated data, we then list several characteristics that influence the parameter variances, including data complexity and signal-to-noise ratio. For practical purposes we investigate a possible use of uncertainty estimates in decreasing intra-group variance in group statistics by uncertainty-weighted group estimates. This has potential use cases for detection and suppression of imaging artifacts.

Fisher information and shape-morphing modes for solving the Fokker–Planck equation in higher dimensions

The Fokker–Planck equation describes the evolution of the probability density associated with a stochastic differential equation. As the dimension of the system grows, solving this partial differential equation (PDE) using conventional numerical methods becomes computationally prohibitive. Here, we introduce a fast, scalable, and interpretable method for solving the Fokker–Planck equation which is applicable in higher dimensions. This method approximates the solution as a linear combination of shape-morphing Gaussians with time-dependent means and covariances. These parameters evolve according to the method of reduced-order nonlinear solutions (RONS) which ensures that the approximate solution stays close to the true solution of the PDE for all times. As such, the proposed method approximates the transient dynamics as well as the equilibrium density, when the latter exists. Our approximate solutions can be viewed as an evolution on a finite-dimensional statistical manifold embedded in the space of probability densities. We show that the metric tensor in RONS coincides with the Fisher information matrix on this manifold. We also discuss the interpretation of our method as a shallow neural network with Gaussian activation functions and time-varying parameters. In contrast to existing deep learning methods, our method is interpretable, requires no training, and automatically ensures that the approximate solution satisfies all properties of a probability density.

RIS-Aided Localization Under Position and Orientation Offsets in the Near and Far Field

This paper presents a rigorous Bayesian analysis of the information in the signal (consisting of both the line-of-sight (LOS) path and reflections from multiple reconfigurable intelligent surfaces (RISs)) that originate from a single base station (BS) and is received by a user equipment (UE). For a comprehensive Bayesian analysis, both near and far field regimes are considered. The Bayesian analysis views both the location of the RISs and previous information about the UE as <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a priori</i> information for UE localization. With outdated <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a priori</i> information, the position and orientation offsets of the RISs become parameters that need to be estimated and fed back to the BS for correction. We first show that when the RIS elements have a half wavelength spacing, this RIS orientation offset is a factor in the pathloss of the RIS paths. Subsequently, we show through the Bayesian equivalent Fisher information matrix (EFIM) for the channel parameters that the RIS orientation offset cannot be corrected when there is an unknown phase offset in the received signal in the far-field regime. However, the corresponding EFIM for the channel parameters in the received signal observed in the near-field shows that this unknown phase offset does not hinder the estimation of the RIS orientation offset when the UE has more than one receive antenna. Furthermore, we use the EFIM for the UE location parameters to present bounds for UE localization in the presence of RIS uncertainty. We rigorously show that regardless of size and propagation regime, the RISs are only helpful for localization when there is <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a priori</i> information about the location of the RISs. Finally, through numerical analysis of the EFIM and its smallest eigenvalue, we demonstrate the loss in information when the far-field model is <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">incorrectly</i> applied to the signals received at a UE experiencing near-field propagation.

High-dimensional Bernstein–von Mises theorem for the Diaconis–Ylvisaker prior

We study the asymptotic normality of the posterior distribution of canonical parameter in the exponential family under the Diaconis–Ylvisaker prior which is a conjugate prior when the dimension of parameter space increases with the sample size. We prove under mild conditions on the true parameter value θ0 and hyperparameters of priors, the difference between the posterior distribution and a normal distribution centered at the maximum likelihood estimator, and variance equal to the inverse of the Fisher information matrix goes to 0 in the expected total variation distance. The proof assumes dimension of parameter space d grows linearly with sample size n only requiring d=o(n). En route, we derive a concentration inequality of the quadratic form of the maximum likelihood estimator without any specific assumption such as sub-Gaussianity. A specific illustration is provided for the Multinomial-Dirichlet model with an extension to the density estimation and Normal mean estimation problems.

Recursive posterior Cramér–Rao lower bound on Lie groups

The recursive posterior Cramér–Rao Lower Bound (CRLB) introduced by Tichavsky et al. (1998) is an efficient tool to estimate the minimum variance of an estimator. Lately, filters and observers on matrix Lie groups have raised interest in control and nonlinear estimation. However, the recursive CRLB applies to variables belonging to the Euclidean space, and a recursive formulation on matrix Lie groups was not introduced yet. Hence, this paper derives a recursive formula of the Fisher information matrix on Lie groups, which reveals to be a natural extension of the information matrix on the Euclidean space. This generic result is applied to nonlinear Gaussian systems on Lie groups and tested on a navigation problem. The proposed recursive CRLB is consistent with state-of-the-art filters and shows a representative behavior with respect to the estimation errors. This paper provides a simple method to recursively compute the minimal variance of an estimator on matrix Lie groups. It enables to assess the behavior of an estimator, which is fundamental to implement robust algorithms.

Fault detection and isolation in uncertain dynamic systems using composite optimization and inferential sensing

Inferential sensing is a cost-effective and reliable approach to replace expensive and impractical hardware sensing, while yielding robust fault detection and isolation (FDI) during system maintenance or operation. Yet, their high computational footprint necessitates the employment of advanced and efficient estimation techniques, along with symbolic mathematics and automatic differentiation. Herein, we attempt to combine model-driven sensor selection methods with inferential sensing to accurately identify faults in the presence of system noise and uncertainty. During sensor selection, we employ criteria from information theory to maximize the estimability of system faults from available system measurements. The most informative sensor set stems from a comparison of all possible sensor suites via information theoretic criteria expressed as functions of the Fisher Information Matrix. Optimal inferential sensors are calculated through a novel fusion of symbolic regression and information theory. For built-in test deployment, we utilize k-Nearest Neighbors classification to assess the capability of each sensor suite for all plausible instantiations of uncertainty and system noise, and evaluate FDI performance benefits from the inclusion of inferential sensor(s). The proposed computational framework is deployed with steady-state and dynamic models of a cross-flow plate-fin heat exchanger, at varying levels of measurement noise and uncertainty. As shown, the augmented system of composite sensors (i.e., inferential and hardware) provides accurate information on system fault(s) and reduces the evidence of uncertainty. Compared to traditional sensor approaches, the proposed techniques are shown to be more precise while simultaneously granting increased insight into details otherwise hidden by noise. Thus, they form a robust and reliable tool when conducting fault diagnosis and possibly toward system safety.

Fisher information estimation using neural networks

In estimation theory, the Fisher information matrix (FIM) is a fundamental concept from which we can infer the well-known Cramér-Rao bound. A closed-form expression of the FIM is often intractable due to the lack or sophistication of statistical models. In this paper, we propose a Fisher Information Neural Estimator (FINE) based on neural networks and a relation between the f-divergence and the Fisher information. The proposed method produces an estimate of the FIM directly from observed data. It does not require knowledge or an estimate of the probability density function (pdf), and is therefore universally applicable. The proposed FINE is applicable for not only deterministic parameters but also random parameters. We show via numerical results that the proposed FINE can provide a highly-accurate FIM estimate with a low-computational complexity. Furthermore, we also propose an accelerated FINE version which can be used for scenarios with a high parameter dimension. Finally, we develop an algorithm to choose an appropriate size of the employed neural networks.

Construction of Saturated D-optimal Designs for Mixture Experiments with a Non Normal Response using an Algorithmic Search

Background: Mixture experiments belong to the response surface design category, involving the combination of multiple components to create a product. These products are commonly encountered in daily life. In some cases, mixture experiments yield qualitative responses, such as taste in a fruit punch. Qualitative variables often deviate from a normal distribution. Methods: To address non-normal responses, a generalized linear model, specifically the logistic model, is employed. This study utilizes logistic models and develops suitable search algorithms to obtain saturated D-optimal designs for mixture experiments. The validation of D-optimality criteria is based on the General Equivalence Theorem. Result: For generating locally D-optimal designs, the logistic model is utilized considering non-normally distributed errors. While the procedure remains the same for other nonlinear models, the assumptions regarding error distribution impact the Fisher information matrix (FIM).

Performance Analysis of Visual–Inertial–Range Cooperative Localization for Unmanned Autonomous Vehicle Swarm

The swarm of small UAVs is an emerging technology that will enable abundant cooperative tasks. To tackle the positioning problem for the UAV swarm, cooperative localization (CL) has been intensively studied since it uses relative measurement to improve the positioning availability and accuracy for the swarm in GPS-denied environments. Besides relying on inter-UAV range measurement, traditional CL algorithms need to place anchors as location references, which limits their applicability. To implement an infrastructure-less swarm navigation system, a consumer-grade camera together with an inertial device can provide rich environment information, which can be recognized as a kind of local location reference. This paper aims to analyze the fundamental performance of visual–inertial–range CL, which is also a popular metric for UAV planning and sensing optimizing, especially for resource-limited environments. Specifically, a closed-form Fisher information matrix (FIM) of visual–inertial–range CL is constructed in Rn×SO(n) manifold. By introducing an equivalent FIM and utilizing of the sparsity of the FIM, the performance of pose estimation can be efficiently calculated. A series of numerical simulations validate its effectiveness for analyzing the CL performance.

Modelling recall-based competing risks data with k different models

In competing risks model, subjects are exposed to failure due to more than one cause. In this study, we develop a model for recall-based competing risks data when the causes associated with time to event are assumed to follow different distributions respectively. The chances of an individual recalling an event will be high if elapsed time between monitoring time and time to event is less. This information is utilized by taking non-recall probability as a function of this elapsed time. For point estimation, an expectation-maximization based algorithm is developed. For interval estimation, we construct the observed Fisher information matrix by using the missing information principle. The study is further extended to the Bayesian paradigm under suitable choices of prior distributions. The samples from full conditionals are drawn using a Gibbs sampling-based algorithm. To assess the performance of proposed estimators, an extensive simulation study is carried out for different proportions of non-recall and censored data with varying sample sizes under uniform and exponential monitoring patterns. Finally, the median duration of breastfeeding cessation for US women is estimated.

Individual versus simultaneous estimation of two parameters in a correlated noisy environment

Recently, the phase parametric estimation for a single qubit system driven by a phase noisy laser under non-Markovian dynamics has been reported [Annals of Physics 351, 952 (2014)]. We here generalize the single-parameter estimation to the two-parameter scenario where a two-qubit system is influenced by the classical correlated noisy laser, and compare the performances of two-parameter estimation in both individual and simultaneous strategies by using the quantum Fisher information and the quantum Fisher information matrix, respectively. Our results show the phase parametric estimation precision can be significantly improved due to the memory effect arising from successive applications of the noisy laser. With the memory coefficient μ increasing, the precision of parameter estimation becomes more accurate. Besides, the phase parameter estimation precision can be drastically enhanced by engineering the ratios of classical phase diffusion rate and system-environment coupling strength in both Markovian and non-Markovian regions. Remarkably, we find the simultaneous estimation of two-phase parameters is not always advantageous over the individual strategy.

Bayesian Estimation and Prediction for Inverse Power Maxwell Distribution with Applications to Tax Revenue and Health Care Data

This article discusses the problem of estimating the unknown model parameters as well as prediction of future observations from inverse power Maxwell distribution. The maximum likelihood method is applied for estimating the model parameters using Newton-Rapson iterative procedures. The existence and uniqueness of maximum likelihood estimates are established using Cauchy-Schwartz inequality. Approximate confidence intervals are constructed using Fisher information matrix. Using independent gamma informative priors, the Bayes estimates of unknown model parameters are obtained under squared error and Linex loss functions. Two approximation techniques namely: Lindley’s approximation and Metropolis-Hastings within Gibbs sampler algorithm have been employed to derive the Bayes estimators and also to construct the associate highest posterior density credible intervals. Based on the informative (observed) sample, Bayesian prediction, predictive density, and predictive intervals are derived for future observation and decision. The performance of proposed methods are evaluated though a Monte Carlo simulation experiment. Two real-life datasets related to tax revenue and heath are incorporated to show the practical utility of proposed methodology in real phenomenon.

Adaptive Type-II Hybrid Progressive Censoring Samples for Statistical Inference of Comparative Inverse Weibull Distributions

Recently, there has been a lot of interest in comparative life testing for items under jointly censored schemes for products from multiple production lines. The inverse Weibull distribution (IWD) is commonly used in life testing and reliability theory. In this paper, we address the problem of statistical inference from comparative inverse Weibull distributions under joint samples. An adaptive type-II hybrid progressive censoring scheme (HPCS) is used to save the balance between the ideal test time and the number of observed failures. Under the adaptive type-II HPCS, unknown parameters of the inverse Weibull populations are estimated using both maximum likelihood and Bayesian approaches. Asymptotic confidence intervals are established using the observed Fisher information matrix and bootstrap confidence intervals. We suggest using Markov chain Monte Carlo (MCMC) techniques to compute credible intervals under independent gamma priors. Using Monte Carlo simulations, all theoretical conclusions are tested and contrasted. For illustration purposes, an actual sample from comparative populations is analysed.

Adaptive detection of subspace signals from two independent sets of samples drawn from a matrix-variate Student distribution

In this paper we address estimation and detection problems using multiple multichannel observations drawn from a matrix-variate Student distribution, also referred to as uncorrelated t distribution. In a first part we derive the Fisher information matrix when the distribution depends on an unknown parameter vector. Next, we consider adaptive detection of a subspace signal using independent test and training data following a matrix-variate Student distribution. We derive the maximum likelihood estimates of the signal amplitude and of the noise covariance matrix and subsequently the generalized likelihood ratio test. Additionally, the score function and the Fisher information matrix derived in the first part of the paper are used to compute Rao, Wald and gradient tests. The detection performance of these methods is illustrated via numerical simulations.

Optimal Generators for Quantum Sensing.

We propose a computationally efficient method to derive the unitary evolution that a quantum state is most sensitive to. This allows one to determine the optimal use of an entangled state for quantum sensing, even in complex systems where intuition from canonical squeezing examples breaks down. In this paper we show that the maximal obtainable sensitivity using a given quantum state is determined by the largest eigenvalue of the quantum Fisher information matrix (QFIM) and the corresponding evolution is uniquely determined by the coinciding eigenvector. Since we optimize the process of parameter encoding rather than focusing on state preparation protocols, our scheme is relevant for any quantum sensor. This procedure naturally optimizes multiparameter estimation by determining, through the eigenvectors of the QFIM, the maximal set of commuting observables with optimal sensitivity.

Uncertainty laws of experimental modal analysis with known broadband input

‘Uncertainty law’ aims at closed-form asymptotic formulas for the relationship between the identification uncertainties of modal properties (e.g., natural frequency, damping ratio) and test configuration (e.g., noise level, number and location of sensors, data duration). Existing developments focused on the case of unknown-input (ambient), where it has been found that identification uncertainty does not vanish even for noiseless instruments, essentially because the input is unknown. A natural question is then on how the uncertainty depends on test configuration when the input is known, not to mention how the configuration should be quantified. Motivated by these and related questions, this paper develops the uncertainty laws of modal parameters for well-separated modes with known single broadband input, e.g., vibration test with a single shaker as in experimental modal analysis. Asymptotic expressions for the posterior coefficient of variation of modal parameters are derived via the Fisher Information Matrix for long data and small damping scenarios. Assumptions and theory are validated using synthetic and field test data. Governing factors motivated by the theory are investigated, including the equivalent modal signal-to-noise ratio (for known input), the number of measured degrees of freedom, shaker location, and data duration. By virtue of the Cramér-Rao bound in classical statistics, the developed uncertainty laws represent the lower bound of identification uncertainty with known broadband input that can be achieved by any unbiased estimator. They provide a scientific basis for planning and managing identification uncertainties in vibration tests with known input.

Coordinated target localization and tracking with node scheduling for a network of autonomous underwater vehicles

Underwater mobile sensor networks (UMSNs) have the advantages of large area coverage, flexible network structure, and good local resolution. Particularly, spatial diversity can be exploited through node mobility to get more target information. To take those advantages in the context of regional cooperative target localization and tracking, a joint node scheduling and acoustic model fusion method is proposed for an UMSN consisting of multiple autonomous underwater vehicles (AUVs). First, a target tracking algorithm framework based on the random finite set theory is established. To take the acoustic environment into account, the a priori acoustic field and target distribution information are combined to evaluate the detection probability. A path-planning scheme with node scheduling is then developed based on the information gain to optimize the detection performance. To achieve effective node scheduling, three reward functions with different information gains respectively derived from the Rényi divergence and Fisher information matrix are proposed and compared. The rationale and the performance of the developed method are verified via simulations. A simplified version of the approach is also tested in field experiments with three AUVs, verifying the effectiveness of node scheduling in a realistic scenario.

Interval Pareto front-based multi-objective robust optimization for sensor placement in structural modal identification

Considering the multi-performance development of complex systems and the requirement of structural modal identification with typical uncertainties, the nominal single-objective optimization method is not suitable for sensor placement. Therefore, by combining conventional optimal sensor placement with non-probabilistic theory, this study proposes an uncertainty-oriented multi-objective robust optimization method for optimal sensor placement. The Fisher information matrix and ill-posedness comprise one eigenvalue-based optimization objective, and the mean and minimum off-diagonal values in the modal assurance criterion comprise another. Considering the high-cost limitation of the statistical method for handling uncertainties, uncertainty propagations are realized by a dimension-wise analysis with better accuracy and efficiency, thus avoiding the overestimation incurred by the classical Taylor expansion method. The multi-objective robust optimization is established by uncertain eigenvalue- and eigenvector-based indices with interval numbers and solved using the multi-objective optimization algorithm. Considering the solution sets located at the Pareto front, an interval possibility was developed using interval Pareto fronts to determine the optimal number of sensors. A numerical example demonstrated the validity of the proposed method with an optimal number of sensors and corresponding configurations.