Fisher Information Matrix Research Articles

Gill and Massar type bound for estimation of SU(2) channel

Abstract In the estimation for a parametric family of quantum state on a Hilbert space H, the Gill and Massar bound is known as a lower bound of weighted traces of covariances of unbiased estimators. The Gill and Massar bound is derived by considering the convexity of the set of classical Fisher information matrices, and the bound is locally achievable by using randomized strategies when H=C2. In this paper, we show that estimation for a parametric SU(2) unitary channel model has a similar convex structure as qubit state model, and a Gill and Massar type lower bound of weighted traces of covariances of unbiased estimators can be derived for any weight matrix. We show that the Gill and Massar type lower bound is achievable by using randomized strategies when certain conditions are satisfied. To derive a convex strucutre of the set of classical Fisher information matrices, we introduce a Fisher information matrix J(U) for a SU(2) unitary channel model, and we show a upper bound of inverse J(U) weighted trace of classical Fisher information matrix. The optimal randomized strategy we construct in this paper does not require ancilla systems in many cases.

Order-restricted statistical inference and optimal censoring scheme for Gompertz distribution based on joint type-II progressive censored data

This paper considers the order-restricted statistical inference for two populations based on the joint type-II progressive censoring scheme. The lifetime distributions of the two populations are supposed to follow the Gompertz distribution with the same shape parameter but different scale parameters. The maximum likelihood estimates of the unknown parameters are derived by employing the Newton-Raphson algorithm and the expectation-maximization algorithm, respectively. The Fisher information matrix is then employed to construct asymptotic confidence intervals. For Bayes estimation, we assume an ordered Beta-Gamma prior for the scale parameters and a Gamma prior for the common shape parameter. Bayes estimations and the highest posterior density credible intervals for unknown parameters under two different loss functions are obtained with the importance sampling technique. To evaluate the performance of order-restricted inference, extensive Monte Carlo simulations are performed and two air-conditioning systems datasets are used to illustrate the proposed inference methods. In addition, the results are compared with the case when there is no order restriction on the parameters. Finally, the optimal censoring scheme is obtained by four optimality criteria.

Random Natural Gradient

Hybrid quantum-classical algorithms appear to be the most promising approach for near-term quantum applications. An important bottleneck is the classical optimization loop, where the multiple local minima and the emergence of barren plateaux make these approaches less appealing. To improve the optimization the Quantum Natural Gradient (QNG) method \cite{stokes2020quantum} was introduced – a method that uses information about the local geometry of the quantum state-space. While the QNG-based optimization is promising, in each step it requires more quantum resources, since to compute the QNG one requires O(m2) quantum state preparations, where m is the number of parameters in the parameterized circuit. In this work we propose two methods that reduce the resources/state preparations required for QNG, while keeping the advantages and performance of the QNG-based optimization. Specifically, we first introduce the Random Natural Gradient (RNG) that uses random measurements and the classical Fisher information matrix (as opposed to the quantum Fisher information used in QNG). The essential quantum resources reduce to linear O(m) and thus offer a quadratic "speed-up", while in our numerical simulations it matches QNG in terms of accuracy. We give some theoretical arguments for RNG and then benchmark the method with the QNG on both classical and quantum problems. Secondly, inspired by stochastic-coordinate methods, we propose a novel approximation to the QNG which we call Stochastic-Coordinate Quantum Natural Gradient that optimizes only a small (randomly sampled) fraction of the total parameters at each iteration. This method also performs equally well in our benchmarks, while it uses fewer resources than the QNG.

Prospect of Detecting Magnetic Fields from Strong-magnetized Binary Neutron Stars

Binary neutron star mergers are unique sources of gravitational waves in multi-messenger astronomy. The inspiral phase of binary neutron stars can emit gravitational waves as chirp signals. The present waveform models of gravitational waves only considered the gravitational interaction. In this paper, we derive the waveform of the gravitational wave signal taking into account the presence of magnetic fields. We found that the electromagnetic interaction and radiation can introduce different frequency-dependent power laws for both the amplitude and frequency of the gravitational wave. We show from the results of the Fisher information matrix that the third-generation observation may detect magnetic dipole moments if the magnetic field is ∼1017 G.

Detecting the tidal heating with the generic extreme mass-ratio inspirals

The horizon of a classical black hole (BH), functioning as a one-way membrane, plays a vital role in the dynamic evolution of binary BHs, capable of absorbing fluxes entirely.Tidal heating, stemming from this phenomenon, exerts a notable influence on the production of gravitational waves (GWs). If at least one member of a binary is an exotic compact object (ECO) instead of a BH, the absorption of fluxes is expected to be incomplete and the tidal heating would be different. Thus, tidal heating can be utilized for model-independent investigations into the nature of compact object. In this paper, assuming that the extreme mass-ratio inspiral (EMRI) contains a stellar-mass compact object orbiting around a massive ECO with a reflective surface, we compute the GWs from the generic EMRI orbits. Using the accurate and analytic flux formulas in the black hole spacetime, we adapted these formulas in the vicinity of the ECO surface by incorporating a reflectivity parameter.Under the adiabatic approximation, we can evolve the orbital parameters and compute the EMRI waveforms. The effect of tidal heating for the spinning and non-spinning objects can be used to constrain the reflectivity of the surface at the level of 𝒪(10-6) by computing the mismatch and fisher information matrix.

Natural Gradient Variational Bayes Without Fisher Matrix Analytic Calculation and Its Inversion

This article introduces a method for efficiently approximating the inverse of the Fisher information matrix, a crucial step in achieving effective variational Bayes inference. A notable aspect of our approach is the avoidance of analytically computing the Fisher information matrix and its explicit inversion. Instead, we introduce an iterative procedure for generating a sequence of matrices that converge to the inverse of Fisher information. The natural gradient variational Bayes algorithm without analytic expression of the Fisher matrix and its inversion is provably convergent and achieves a convergence rate of order O ( log s / s ) , with s the number of iterations. We also obtain a central limit theorem for the iterates. Implementation of our method does not require storage of large matrices, and achieves a linear complexity in the number of variational parameters. Our algorithm exhibits versatility, making it applicable across a diverse array of variational Bayes domains, including Gaussian approximation and normalizing flow Variational Bayes. We offer a range of numerical examples to demonstrate the efficiency and reliability of the proposed variational Bayes method. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

Natural gradient hybrid variational inference with application to deep mixed models

Stochastic models with global parameters and latent variables are common, and for which variational inference (VI) is popular. However, existing methods are often either slow or inaccurate in high dimensions. We suggest a fast and accurate VI method for this case that employs a well-defined natural gradient variational optimization that targets the joint posterior of the global parameters and latent variables. It is a hybrid method, where at each step the global parameters are updated using the natural gradient and the latent variables are generated from their conditional posterior. A fast to compute expression for the Tikhonov damped Fisher information matrix is used, along with the re-parameterization trick, to provide a stable natural gradient. We apply the approach to deep mixed models, which are an emerging class of Bayesian neural networks with random output layer coefficients to allow for heterogeneity. A range of simulations show that using the natural gradient is substantially more efficient than using the ordinary gradient, and that the approach is faster and more accurate than two cutting-edge natural gradient VI methods. In a financial application we show that accounting for industry level heterogeneity using the deep mixed model improves the accuracy of asset pricing models. MATLAB code to implement the method and replicate the results can be found at https://github.com/WeibenZhang07/NG-HVI

Inference for depending competing risks from Marshall–Olikin bivariate Kies distribution under generalized progressive hybrid censoring

This paper explores inferences for a competing risk model with dependent causes of failure. When the lifetimes of competing risks are modelled by a Marshall–Olikin bivariate Kies distribution, classical and Bayesian estimations are studied under generalized progressive hybrid censoring. The existence and uniqueness results for maximum likelihood estimators of unknown parameters are established, whereas approximate confidence intervals are constructed using the observed Fisher information matrix. In addition, Bayes estimates are explored based on a flexible Gamma-Dirichlet prior information. Furthermore, when there is a priori order information on competing risk parameters being available, traditional classical likelihood and Bayesian estimates are also established under restricted parameter case. The behavior of the proposed estimators is evaluated through extensive simulation studies, and a real data study is presented for illustrative purposes.

Comparison of estimation limits for quantum two-parameter estimation

Measurement estimation bounds for local quantum multiparameter estimation, which provide lower bounds on possible measurement uncertainties, have so far been formulated in two ways: by extending the classical Cramér-Rao bound (e.g., the quantum Cramér-Rao bound and the Nagaoka Cramér-Rao bound) and by incorporating the parameter estimation framework with the uncertainty principle, as in the Lu-Wang uncertainty relation. In this work, we present a general framework that allows a direct comparison between these different types of estimation limits. Specifically, we compare the attainability of the Nagaoka Cramér-Rao bound and the Lu-Wang uncertainty relation, using analytical and numerical techniques. We show that these two limits can provide different information about the physically attainable precision. We present an example where both limits provide the same attainable precision and an example where the Lu-Wang uncertainty relation is not attainable even for pure states. We further demonstrate that the unattainability in the latter case arises because the figure of merit underpinning the Lu-Wang uncertainty relation (the difference between the quantum and classical Fisher information matrices) does not necessarily agree with the conventionally used figure of merit (mean-squared error). The results offer insights into the general attainability and applicability of the Lu-Wang uncertainty relation. Furthermore, our proposed framework for comparing bounds of different types may prove useful in other settings. Published by the American Physical Society 2024

Learning under singularity: an information criterion improving WBIC and sBIC

AbstractWe introduce a novel information criterion, termed learning under singularity (LS), designed to enhance the functionality of the widely applicable Bayes information criterion (WBIC) and the singular Bayesian information criterion (sBIC). LS is effective without regularity constraints and demonstrates stability. Watanabe defined a statistical model or a learning machine as regular if the mapping from a parameter to a probability distribution is one-to-one and its Fisher information matrix is positive definite. In contrast, models not meeting these conditions are termed singular. Over the past decade, several information criteria for singular cases have been proposed, including WBIC and sBIC. WBIC is applicable in non-regular scenarios but faces challenges with large sample sizes and redundant estimation of known learning coefficients. Conversely, sBIC is limited in its broader application due to its dependence on maximum likelihood estimates. LS addresses these limitations by enhancing the utility of both WBIC and sBIC. It incorporates the empirical loss from the widely applicable information criterion (WAIC) to represent the goodness of fit to the statistical model, along with a penalty term similar to that of sBIC. This approach offers a flexible and robust method for model selection, free from regularity constraints.

Constraining the EdGB theory with extreme mass-ratio inspirals

The Einstein-dilaton-Gauss–Bonnet (EdGB) theory is a modified theory of gravity which include a scalar field to couple with the higher order curvature terms. It has already been constrained with various observations include the gravitational wave (GW) with LIGO, Virgo and KAGRA (LVK) Collaboration. In this work, we study the capability for space-borne GW detectors to constrain the EdGB theory using the signal of Extreme Mass-Ratio Inspiral (EMRIs). We use the “numerical kludge (NK)” method to construct the waveform of EMRI in the EdGB theory, focusing on the case when the central black hole is spinless. We then study how a future space-borne gravitational wave detector, TianQin, for example, can place constraints on the EdGB theory through the detection of EMRIs. With the analysis using mismatch and Fisher Information Matrix (FIM), we find that the EdGB parameter α\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\sqrt{\\alpha }$$\\end{document} is expected to be constrained to the level of ∼O(0.1)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\sim \\mathcal {O}(0.1)$$\\end{document} km.

Statistical inference for marshall-olkin bivariate Kumaraswamy distribution under adaptive progressive hybrid censored dependent competing risks data

Abstract The statistical inference under competing risks model is of great significance in reliability analysis and it is more practical to assume that they have dependent competing causes of failure in actual situations. In this article, we make inference for unknown parameters of a Marshall-Olkin bivariate Kumaraswamy distribution under adaptive progressive hybrid censoring mechanism. The maximum likelihood estimations of the unknown parameters are derived, and the Fisher information matrix is then employed to construct asymptotic confidence intervals. Bayes estimates are evaluated against squared error and linex loss functions assuming ordered Gamma-Dirichlet and Gamma-Dirichlet prior distributions for order restriction and without order restriction cases respectively. The Metropolis-Hasting and Lindley techniques are applied to acquire the estimates of all unknown parameters. A thorough simulation analysis is demonstrated to assess the performance of the supplied approaches across various sample sizes. The usefulness of the techniques is illustrated using real engineering data to prove their versatility in practical applications.

Analysis and comparison of two-level KFAC methods for training deep neural networks

As a second-order method, the Natural Gradient Descent (NGD) has the ability to accelerate training of neural networks. However, due to the prohibitive computational and memory costs of computing and inverting the Fisher Information Matrix (FIM), efficient approximations are necessary to make NGD scalable to Deep Neural Networks (DNNs). Many such approximations have been attempted. The most sophisticated of these is KFAC, which approximates the FIM as a block-diagonal matrix, where each block corresponds to a layer of the neural network. By doing so, KFAC ignores the interactions between different layers. In this work, we investigate the interest of restoring some low-frequency interactions between the layers by means of two-level methods. Inspired from domain decomposition, several two-level corrections to KFAC using different coarse spaces are proposed and assessed. The obtained results show that incorporating the layer interactions in this fashion does not really improve the performance of KFAC. This suggests that it is safe to discard the off-diagonal blocks of the FIM, since the block-diagonal approach is sufficiently robust, accurate and economical in computation time.

Calculating the power of a planned individual participant data meta-analysis to examine prognostic factor effects for a binary outcome.

Collecting data for an individual participant data meta-analysis (IPDMA) project can be time consuming and resource intensive and could still have insufficient power to answer the question of interest. Therefore, researchers should consider the power of their planned IPDMA before collecting IPD. Here we propose a method to estimate the power of a planned IPDMA project aiming to synthesise multiple cohort studies to investigate the (unadjusted or adjusted) effects of potential prognostic factors for a binary outcome. We consider both binary and continuous factors and provide a three-step approach to estimating the power in advance of collecting IPD, under an assumption of the true prognostic effect of each factor of interest. The first step uses routinely available (published) aggregate data for each study to approximate Fisher's information matrix and thereby estimate the anticipated variance of the unadjusted prognostic factor effect in each study. These variances are then used in step 2 to estimate the anticipated variance of the summary prognostic effect from the IPDMA. Finally, step 3 uses this variance to estimate the corresponding IPDMA power, based on a two-sided Wald test and the assumed true effect. Extensions are provided to adjust the power calculation for the presence of additional covariates correlated with the prognostic factor of interest (by using a variance inflation factor) and to allow for between-study heterogeneity in prognostic effects. An example is provided for illustration, and Stata code is supplied to enable researchers to implement the method.

Observability-Aware Differential Dynamic Programming with Impulsive Maneuvers

In autonomous space systems, the reliability of navigation systems is essential. Observability in autonomous orbit determination techniques depends on the spacecraft’s orbital motion, making the design of autonomous navigation systems and orbital maneuvers a coupled process. This study develops a stable and efficient algorithm based on differential dynamic programming to design maneuver sequences that improve navigation performance. Our approach incorporates the Fisher information matrix into a cost function to quantify state observability and facilitates its convergence using a semi-analytic gradient and Hessian derived under impulsive maneuvers. Two numerical examples show the validity and effectiveness of our algorithm. The results indicate the stability and efficiency in determining maneuver sequences and the improvement of state estimation accuracy along an optimized trajectory. It is also applicable to other observability-aware optimal control problems because the algorithm is independent of specific systems.

Optimization of Sensor Placement for a Measurement System for the Determination of Local Magnetic Material Properties

The aim of this work is to optimize the sensor positions of a sensor–actuator measurement system for identifying local variations in the magnetic permeability of cut steel sheets. Before solving the actual identification problem, i.e., finding the material distribution, the sensor placement of the measurement setup should be improved in order to reduce the uncertainty of the identification of the material distribution. The Fisher information matrix (FIM), which allows one to quantify the amount of information that the measurements carry about the unknown parameters, is used as the main metric for the objective function of this design optimization. The forward problem is solved by the finite element method. The results show that the proposed method is able to find optimal sensor positions as well as the minimum number of sensors to achieve a desired maximum parameter uncertainty.

Reinforcement-Learning-Based Robust Guidance for Asteroid Approaching

This paper presents a reinforcement-learning (RL)-based robust low-thrust guidance method for asteroid approaching under process uncertainties. Markov decision processes with stochastic dynamics are formulated for RL. To overcome the problem of low terminal accuracy in RL-based transfer trajectory design, robust zero-effort-miss/zero-effort-velocity (R-ZEM/ZEV) guidance is proposed. Originally, an eigenvalue-related term is defined according to the stability conditions of the ZEM/ZEV feedback system and chosen as the learning parameter, which can significantly improve the robustness of the agent to process uncertainties under the low-thrust magnitude constraint. Moreover, the navigation performance of the asteroid approaching is modeled via the Fisher information matrix and incorporated in the reward function design, which enables optimizing the optical observation performance together with the propellant cost. Thereafter, the proximal policy optimization is adopted to train an RL agent that can efficiently deal with the uncertainties. The effectiveness and efficiency of the proposed method are validated through simulations of a low-thrust spacecraft approaching the asteroid Bennu. The promising results indicate that the proposed method can not only deal with various uncertainties efficiently and autonomously but can also guarantee high terminal accuracy.

Versatile Descent Algorithms for Group Regularization and Variable Selection in Generalized Linear Models

This article proposes an adaptively bounded gradient descent (ABGD) algorithm for group elastic net penalized regression. Unlike previously proposed algorithms, the proposed algorithm adaptively bounds the Fisher information matrix, which results in a flexible and stable computational framework. In particular, the proposed algorithm (i) does not require orthogonalization of the predictors, and (ii) can be easily applied to any combination of exponential family response distribution and link function. The proposed algorithm is implemented in the grpnet R package (available from CRAN), which implements the approach for common response distributions (Gaussian, binomial, and Poisson), as well as several response distributions not previously considered in the group penalization literature (i.e., multinomial, negative binomial, gamma, and inverse Gaussian). Simulated and real data examples demonstrate that the proposed algorithm is as or more efficient than existing methods for Gaussian, binomial, and Poisson distributions. Furthermore, using two genomic examples, I demonstrate how the proposed algorithm can be applied to high-dimensional multinomial regression problems with grouped predictors. R code to reproduce the results is included as supplementary materials.

Lifelong reinforcement learning tracking control of nonlinear strict-feedback systems using multilayer neural networks with constraints

This paper presents a novel safe integral reinforcement learning (IRL)-based optimal trajectory tracking scheme for nonlinear systems with uncertain dynamics that is subject to constraints. We leverage multilayer neural networks (MNNs) for actor-critic MNNs along with an NN identifier in the backstepping process for minimizing a discounted value function. A time-varying barrier Lyapunov function (TVBLF) is utilized for handling constraints and to provide safety assurances. Online weight update laws for the actor and critic MNNs are derived that are driven by Bellman error and control input error. We introduce an online lifelong learning (LL) method in the critic NN, utilizing the Bellman error in MNNs to address catastrophic forgetting. The method’s effectiveness is demonstrated through simulations on mobile robot multitask tracking. The paper concludes with a stability analysis of the closed-loop system.

Different Statistical Inference Algorithms for the New Pareto Distribution Based on Type-II Progressively Censored Competing Risk Data with Applications

In this research, the statistical inference of unknown lifetime parameters is proposed in the presence of independent competing risks using a progressive Type-II censored dataset. The lifetime distribution associated with a failure mode is assumed to follow the new Pareto distribution, with consideration given to two distinct competing failure reasons. Maximum likelihood estimators (MLEs) for the unknown model parameters, as well as reliability and hazard functions, are derived, noting that they are not expressible in closed form. The Newton–Raphson, expectation maximization (EM), and stochastic expectation maximization (SEM) methods are employed to generate maximum likelihood (ML) estimations. Approximate confidence intervals for the unknown parameters, reliability, and hazard rate functions are constructed using the normal approximation of the MLEs and the normal approximation of the log-transformed MLEs. Additionally, the missing information principle is utilized to derive the closed form of the Fisher information matrix, which, in turn, is used with the delta approach to calculate confidence intervals for reliability and hazards. Bayes estimators are derived under both symmetric and asymmetric loss functions, with informative and non-informative priors considered, including independent gamma distributions for informative priors. The Monte Carlo Markov Chain sampling approach is employed to obtain the highest posterior density credible intervals and Bayesian point estimates for unknown parameters and reliability characteristics. A Monte Carlo simulation is conducted to assess the effectiveness of the proposed techniques, with the performances of the Bayes and maximum likelihood estimations examined using average values and mean squared errors as benchmarks. Interval estimations are compared in terms of average lengths and coverage probabilities. Real datasets are considered and examined for each topic to provide illustrative examples.