Fisher Information Matrix Research Articles

Deep learning forecasts of cosmic acceleration parameters from DECi-hertz Interferometer Gravitational-wave Observatory

Context. Validating the accelerating expansion of the universe is an important aspect in improving our understanding of the evolution of the universe. By constraining the cosmic acceleration parameter XH, we can discriminate between the cosmological constant plus cold dark matter (ΛCDM) model and the Lemaître–Tolman–Bondi (LTB) model. Aims. In this paper, we explore the possibility of constraining the cosmic acceleration parameter with the inspiral gravitational waveform of neutron star binaries (NSBs) in the frequency range of 0.1 Hz–10 Hz, which can be detected by the second-generation space-based gravitational wave detector DECIGO. Methods. We used a convolutional neural network (CNN) and a long short-term memory (LSTM) network combined with a gated recurrent unit (GRU), along with a Fisher information matrix to derive constraints on the cosmic acceleration parameter, XH. Results. We assumed that our networks estimate the cosmic acceleration parameter without biases (the expected value of the estimation is equal to the true value). Under this assumption, based on the simulated gravitational wave data with a time duration of one month, we conclude that CNN can limit the relative error to 15.71%, while LSTM network combined with GRU can limit the relative error to 14.14%. Additionally, using a Fisher information matrix for gravitational wave data with a five-year observation can limit the relative error to 32.94%. Conclusions. Under the assumption of an unbiased estimation, the neural networks can offer a high-precision estimation of the cosmic acceleration parameter at different redshifts. Therefore, DECIGO is expected to provide direct measurements of the acceleration of the universe by observing the chirp signals of coalescing binary neutron stars.

The Best Fit Bayesian Hierarchical Generalized Linear Model Selection Using Information Complexity Criteria in the MCMC Approach

Both frequentist and Bayesian statistics schools have improved statistical tools and model choices for the collected data or measurements. Model selection approaches have advanced due to the difficulty of comparing complicated hierarchical models in which linear predictors vary by grouping variables, and the number of model parameters is not distinct. Many regression model selection criteria are considered, including the maximum likelihood (ML) point estimation of the parameter and the logarithm of the likelihood of the dataset. This paper demonstrates the information complexity (ICOMP), Bayesian deviance information, or the widely applicable information criterion (WAIC) of the BRMS to hierarchical linear models fitted with repeated measures with a simulation and two real data examples. The Fisher information matrix for the Bayesian hierarchical model considering fixed and random parameters under maximizing a posterior estimation is derived. Using Gibbs sampling and Hybrid Hamiltonian Monte Carlo approaches, six different models were fitted for three distinct application datasets. The best-fitted candidate models were identified under each application dataset with the two MCMC approaches. In this case, the Bayesian hierarchical (mixed effect) linear model with random intercepts and random slopes estimated using the Hamiltonian Monte Carlo method best fits the two application datasets. Information complexity (ICOMP) is a better indicator of the best-fitted models than DIC and WAIC. In addition, the information complexity criterion showed that hierarchical models with gradient-based Hamiltonian Monte Carlo estimation are the best fit and have supper convergence relative to the gradient-free Gibbs sampling methods.

Data Augmentation for Regression Machine Learning Problems in High Dimensions

Machine learning approaches are currently used to understand or model complex physical systems. In general, a substantial number of samples must be collected to create a model with reliable results. However, collecting numerous data is often relatively time-consuming or expensive. Moreover, the problems of industrial interest tend to be more and more complex, and depend on a high number of parameters. High-dimensional problems intrinsically involve the need for large amounts of data through the curse of dimensionality. That is why new approaches based on smart sampling techniques have been investigated to minimize the number of samples to be given to train the model, such as active learning methods. Here, we propose a technique based on a combination of the Fisher information matrix and sparse proper generalized decomposition that enables the definition of a new active learning informativeness criterion in high dimensions. We provide examples proving the performances of this technique on a theoretical 5D polynomial function and on an industrial crash simulation application. The results prove that the proposed strategy outperforms the usual ones.

Joint energy beamforming and bandwidth allocation for cooperative localization in passive sensor networks

Passive sensor networks (PSNs) harvest continuous radio-frequency (RF) energy to work. However, the location information is difficult to maintain with unstable power supply for each passive sensor node. Cooperative localization is a promising solution utilizes the cooperation between the nodes with unknown positions to maintain the location information. And the localization performance relies on the attained energy from the energy access point (E-AP) and also the allocated bandwidth for each node. In this paper, we analyze the impacts of energy beamforming and bandwidth allocation on the cooperative localization performance of PSNs. We formulate the Fisher information matrix (FIM) and deduce the squared position error bound (SPEB) for the proposed network. Then, we propose an alternative optimization method for jointly energy beamforming and bandwidth allocation. Firstly, we employ the first-order Taylor expansions based on the trust region method to allocate the bandwidth of each node. Secondly, the Lagrangian multiplier based energy beamforming strategy is proposed. Then, the bandwidth allocation and energy beamforming calculations are executed alternatively and iteratively until the global optimal solution is achieved. The simulation results indicate that our proposed two-step method can achieve high precision positioning in different scenarios, and the SPEBs of proposed method reach 10−3 m2.

Observed Fisher information for radar clutter modeled with sub-Gaussian α-stable distribution

It is well known that the clutter in such applications as synthetic aperture radar (SAR) follows heavy-tailed distributions. In such applications, knowledge of the exact statistical properties and/or errors in estimation of the clutter parameters plays an important role in successful implementation of the related applications like target tracking using SAR. In this study, the observed Fisher information (OFI) matrix will be derived for the received clutter assuming that the statistical behavior of clutter follows a sub-Gaussian α-stable (SGS) distribution along with a density function for an SGS distribution based on a newly proposed expansion. The new expansion is characterized with high speed computation in one hand, and avoids issues related to numerical computation of SGS density function, on the other. The minimum standard error of the clutter parameters, then, will be derived as the OFI matrix is inversely related to the Cramer-Rao lower bound (CRLB) of the clutter's estimated parameters. In order to compute the density function and OFI of the SGS distribution, the performance of the proposed methodology has been tested through numerical simulation where the implemented computer code of the proposed numerical method has been demonstrated to be fast and accurate. The R code is available at https://github.com/mahditeimouri/Sub-Gaussian/blob/main/OFI-sub-Gaussian.r.

Log‐density gradient covariance and automatic metric tensors for Riemann manifold Monte Carlo methods

AbstractA metric tensor for Riemann manifold Monte Carlo particularly suited for nonlinear Bayesian hierarchical models is proposed. The metric tensor is built from symmetric positive semidefinite log‐density gradient covariance (LGC) matrices, which are also proposed and further explored here. The LGCs generalize the Fisher information matrix by measuring the joint information content and dependence structure of both a random variable and the parameters of said variable. Consequently, positive definite Fisher/LGC‐based metric tensors may be constructed not only from the observation likelihoods as is current practice, but also from arbitrarily complicated nonlinear prior/latent variable structures, provided the LGC may be derived for each conditional distribution used to construct said structures. The proposed methodology is highly automatic and allows for exploitation of any sparsity associated with the model in question. When implemented in conjunction with a Riemann manifold variant of the recently proposed numerical generalized randomized Hamiltonian Monte Carlo processes, the proposed methodology is highly competitive, in particular for the more challenging target distributions associated with Bayesian hierarchical models.

Deep learning for parameter estimation of supermassive binary black holes with simulated LISA data

Previous studies have demonstrated that deep learning is an effective approach for processing gravitational wave (GW) data obtained from ground-based detectors, as it can enhance the efficiency of data processing and offer great potential for real-time parameter estimation. In this paper, we explore three different deep learning architectures (MCD, TFP, and VI) for inferring the chirp mass (Mc) and the luminosity distance (DL) of supermassive binary black holes (SMBBHs) using simulated data from the Laser Interferometer Space Antenna (LISA). We train the neural networks with the simulated data and evaluate their performance on predicting the parameters (Mc,DL). The results show that more than 97.5% of the true values fall within the 3σ confidence interval of the predicted values with the optimal network. To verify the accuracy of the network on parameter estimation, we also calculate the estimation error of the parameters using the Fisher Information Matrix (FIM) with the same simulated data. By comparing the root-mean-square error (RMSE) between deep learning and FIM, we find that the two methods are comparable, which implies that deep learning can be reliable for GW parameter estimation.

Closed-Loop Estimation Method of Neurostimulation Strength-Duration Curve Using Fisher Information Optimization.

Strength-duration (SD) curve, rheobase and chronaxie parameters provide insights about neural activation dynamics and interdependence between pulse amplitude and duration, for diagnostics and therapeutic applications. The existing SD curve estimation methods are based on open-loop uniform and/or random pulse durations, which are chosen without feedback from neuronal data. To develop a method for closed-loop estimation of the SD curve, where the pulse durations are adjusted iteratively using the neuronal data. In the proposed method, after the selection of each pulse duration through Fisher information matrix (FIM) optimization, the corresponding motor threshold (MT) is computed, the SD curve estimation is updated, and the process continues until satisfaction of a stopping rule based on the successive convergence of the SD curve parameters. The results are compared with various iterative uniform and random sampling techniques, where the SD curve estimation is updated after each sample. 250 simulation cases were run. The FIM method satisfied the stopping rule in 225 (90%) runs, and estimated the rheobase (chronaxie in parenthesis) with an average absolute relative error (ARE) of 1.57% (2.15%), with an average of 85 samples. At the FIM termination sample, methods with two and all random pulse durations, and uniform methods with descending, ascending and random orders led to 5.69% (20.09%), 2.22% (3.93%), 7.34% (40.90%), 3.10% (4.44%), and 2.05% (3.45%) AREs. In all 250 runs, the FIM method has chosen the minimum and maximum pulse durations as the optimal pulse durations for the SD curve estimation. As proposed by the FIM method, the SD curve is identifiable by fitting to the data of the minimum and maximum pulse durations. However, the range of pulse duration should cover the vertical and horizontal parts of the SD curve. Also, iterative random or uniform samples from only the vertical or horizontal areas of the curve might not result in satisfactory estimation. This paper provides insights about pulse durations selection for closed-loop and open-loop SD curve estimation.

Оптимальное планирование эксперимента на основе винеровской деградационной модели с ковариатами с использованием условной информационной матрицы Фишера

The Wiener degradation models with covariates are widely used for reliability analysis by degradation index paths. In this paper, we suggest an algorithm for constructing the optimal design based on the Wiener degradation model, which involves determining optimal stress levels, the number of objects under each stress level and the time moments for measuring degradation index, taking into account the limitation on the duration of the experiment and the critical value of the degradation index. The proposed algorithm is based on optimizing the sum of functionals from the Fisher information matrix (FIM) and the conditional Fisher information matrix (CFIM) under given restrictions on the stress levels, the duration of the experiment and the minimal interval between the time points of measuring the degradation index. As an example, the application of the developed algorithm is considered for the LED reliability analysis.

Sparsified federated learning with differential privacy for intrusion detection in VANETs based on Fisher Information Matrix.

With the continuous development of vehicular ad hoc networks (VANET) security, using federated learning (FL) to deploy intrusion detection models in VANET has attracted considerable attention. Compared to conventional centralized learning, FL retains local training private data, thus protecting privacy. However, sensitive information about the training data can still be inferred from the shared model parameters in FL. Differential privacy (DP) is sophisticated technique to mitigate such attacks. A key challenge of implementing DP in FL is that non-selectively adding DP noise can adversely affect model accuracy, while having many perturbed parameters also increases privacy budget consumption and communication costs for detection models. To address this challenge, we propose FFIDS, a FL algorithm integrating model parameter pruning with differential privacy. It employs a parameter pruning technique based on the Fisher Information Matrix to reduce the privacy budget consumption per iteration while ensuring no accuracy loss. Specifically, FFIDS evaluates parameter importance and prunes unimportant parameters to generate compact sub-models, while recording the positions of parameters in each sub-model. This not only reduces model size to lower communication costs, but also maintains accuracy stability. DP noise is then added to the sub-models. By not perturbing unimportant parameters, more budget can be reserved to retain important parameters for more iterations. Finally, the server can promptly recover the sub-models using the parameter position information and complete aggregation. Extensive experiments on two public datasets and two F2MD simulation datasets have validated the utility and superior performance of the FFIDS algorithm.

Inference and other aspects for $ q- $Weibull distribution via generalized order statistics with applications to medical datasets

<abstract><p>This work utilizes generalized order statistics (GOSs) to study the $ q $-Weibull distribution from several statistical perspectives. First, we explain how to obtain the maximum likelihood estimates (MLEs) and utilize Bayesian techniques to estimate the parameters of the model. The Fisher information matrix (FIM) required for asymptotic confidence intervals (CIs) is generated by obtaining explicit expressions. A Monte Carlo simulation study is conducted to compare the performances of these estimates based on type Ⅱ censored samples. Two well-established measures of information are presented, namely extropy and weighted extropy. In this context, the order statistics (OSs) and sequential OSs (SOSs) for these two measures are studied based on this distribution. A bivariate $ q $-Weibull distribution based on the Farlie-Gumbel-Morgenstern (FGM) family and its relevant concomitants are studied. Finally, two concrete instances of medical real data are ultimately provided.</p></abstract>

Reliability analysis for two populations Nadarajah-Haghighi distribution under Joint progressive type-II censoring

<abstract><p>In order to evaluate the competitive advantages and dependability of two products in a competitive environment, comparative lifespan testing becomes essential. We examine the inference problems that occur when two product lines follow the Nadarajah-Haghighighi distribution in the setting of joint type-II censoring. In the present study, we derived the maximum likelihood estimates for the Nadarajah-Haghighi population parameters. Additionally, a Fisher information matrix was constructed based on these maximum likelihood estimations. Furthermore, Bayesian estimators and their corresponding posterior risks were calculated, considering both gamma and non-informative priors under symmetric and asymmetric loss functions. To assess the performance of the overall parameter estimators, we conducted a Monte Carlo simulation using numerical methods. Lastly, a real data analysis was carried out to validate the accuracy of the models and methods discussed.</p></abstract>

Joint Optimization Method of Trajectory of UAV Swarm and MIMO Radar Radiation Power based on Posterior Fisher Information Matrix

Joint Optimization Method of Trajectory of UAV Swarm and MIMO Radar Radiation Power based on Posterior Fisher Information Matrix

An Inflated Modelling of Zero Truncated Poisson Ailamujia Distribution and Its Application to Child Mortality and Genetics Count Data

In this paper, an attempt has been made to develop a model as Inflated form of zero truncated Poisson Ailamujia distribution. The proposed model involved two parameters, estimated by the maximum likelihood estimation method and derived as an expression of the Fisher Information Matrix. To evaluate the adequacy and applicability of the proposed model, it is applied to real data sets on child mortality and genetic count data.

Effect of detector placement on joint estimation in X-ray fluorescence emission tomography.

Imaging the spatial distribution of low concentrations of metal is a growing problem of interest with applications in medical and material sciences. X-ray fluorescence emission tomography (XFET) is an emerging metal mapping imaging modality with potential sensitivity improvements and practical advantages over other methods. However, XFET detector placement must first be optimized to ensure accurate metal density quantification and adequate spatial resolution. In this work, we first use singular value decomposition of the imaging model and eigendecomposition of the object-specific Fisher information matrix to study how detector arrangement affects spatial resolution and feature preservation. We then perform joint image reconstructions of a numerical gold phantom. For this phantom, we show that two parallel detectors provide metal quantification with similar accuracy to four detectors, despite the resulting anisotropic spatial resolution in the attenuation map estimate. Two orthogonal detectors provide improved spatial resolution along one axis, but underestimate the metal concentration in distant regions. Therefore, this work demonstrates the minor effect of using fewer, but strategically placed, detectors in the case where detector placement is restricted. This work is a critical investigation into the limitations and capabilities of XFET prior to its translation to preclinical and benchtop uses.

Sloppy model analysis identifies bifurcation parameters without normal form analysis.

Bifurcation phenomena are common in multidimensional multiparameter dynamical systems. Normal form theory suggests that bifurcations are driven by relatively few combinations of parameters. Models of complex systems, however, rarely appear in normal form, and bifurcations are controlled by nonlinear combinations of the bare parameters of differential equations. Discovering reparameterizations to transform complex equationsinto a normal form is often very difficult, and the reparameterization may not even exist in a closed form. Here we show that information geometry and sloppy model analysis using the Fisher information matrix can be used to identify the combination of parameters that control bifurcations. By considering observations on increasingly long timescales, we find those parameters that rapidly characterize the system's topological inhomogeneities, whether the system is in normal form or not. We anticipate that this novel analytical method, which we call time-widening information geometry (TWIG), will be useful in applied network analysis.

Fast and robust quantification of uncertainty in non-linear diffusion MRI models

Diffusion MRI (dMRI) allows for non-invasive investigation of brain tissue microstructure. By fitting a model to the dMRI signal, various quantitative measures can be derived from the data, such as fractional anisotropy, neurite density and axonal radii maps. We investigate the Fisher Information Matrix (FIM) and uncertainty propagation as a generally applicable method for quantifying the parameter uncertainties in linear and non-linear diffusion MRI models. In direct comparison with Markov Chain Monte Carlo (MCMC) sampling, the FIM produces similar uncertainty estimates at much lower computational cost. Using acquired and simulated data, we then list several characteristics that influence the parameter variances, including data complexity and signal-to-noise ratio. For practical purposes we investigate a possible use of uncertainty estimates in decreasing intra-group variance in group statistics by uncertainty-weighted group estimates. This has potential use cases for detection and suppression of imaging artifacts.

Fisher information and shape-morphing modes for solving the Fokker–Planck equation in higher dimensions

The Fokker–Planck equation describes the evolution of the probability density associated with a stochastic differential equation. As the dimension of the system grows, solving this partial differential equation (PDE) using conventional numerical methods becomes computationally prohibitive. Here, we introduce a fast, scalable, and interpretable method for solving the Fokker–Planck equation which is applicable in higher dimensions. This method approximates the solution as a linear combination of shape-morphing Gaussians with time-dependent means and covariances. These parameters evolve according to the method of reduced-order nonlinear solutions (RONS) which ensures that the approximate solution stays close to the true solution of the PDE for all times. As such, the proposed method approximates the transient dynamics as well as the equilibrium density, when the latter exists. Our approximate solutions can be viewed as an evolution on a finite-dimensional statistical manifold embedded in the space of probability densities. We show that the metric tensor in RONS coincides with the Fisher information matrix on this manifold. We also discuss the interpretation of our method as a shallow neural network with Gaussian activation functions and time-varying parameters. In contrast to existing deep learning methods, our method is interpretable, requires no training, and automatically ensures that the approximate solution satisfies all properties of a probability density.

RIS-Aided Localization Under Position and Orientation Offsets in the Near and Far Field

This paper presents a rigorous Bayesian analysis of the information in the signal (consisting of both the line-of-sight (LOS) path and reflections from multiple reconfigurable intelligent surfaces (RISs)) that originate from a single base station (BS) and is received by a user equipment (UE). For a comprehensive Bayesian analysis, both near and far field regimes are considered. The Bayesian analysis views both the location of the RISs and previous information about the UE as <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a priori</i> information for UE localization. With outdated <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a priori</i> information, the position and orientation offsets of the RISs become parameters that need to be estimated and fed back to the BS for correction. We first show that when the RIS elements have a half wavelength spacing, this RIS orientation offset is a factor in the pathloss of the RIS paths. Subsequently, we show through the Bayesian equivalent Fisher information matrix (EFIM) for the channel parameters that the RIS orientation offset cannot be corrected when there is an unknown phase offset in the received signal in the far-field regime. However, the corresponding EFIM for the channel parameters in the received signal observed in the near-field shows that this unknown phase offset does not hinder the estimation of the RIS orientation offset when the UE has more than one receive antenna. Furthermore, we use the EFIM for the UE location parameters to present bounds for UE localization in the presence of RIS uncertainty. We rigorously show that regardless of size and propagation regime, the RISs are only helpful for localization when there is <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a priori</i> information about the location of the RISs. Finally, through numerical analysis of the EFIM and its smallest eigenvalue, we demonstrate the loss in information when the far-field model is <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">incorrectly</i> applied to the signals received at a UE experiencing near-field propagation.

High-dimensional Bernstein–von Mises theorem for the Diaconis–Ylvisaker prior

We study the asymptotic normality of the posterior distribution of canonical parameter in the exponential family under the Diaconis–Ylvisaker prior which is a conjugate prior when the dimension of parameter space increases with the sample size. We prove under mild conditions on the true parameter value θ0 and hyperparameters of priors, the difference between the posterior distribution and a normal distribution centered at the maximum likelihood estimator, and variance equal to the inverse of the Fisher information matrix goes to 0 in the expected total variation distance. The proof assumes dimension of parameter space d grows linearly with sample size n only requiring d=o(n). En route, we derive a concentration inequality of the quadratic form of the maximum likelihood estimator without any specific assumption such as sub-Gaussianity. A specific illustration is provided for the Multinomial-Dirichlet model with an extension to the density estimation and Normal mean estimation problems.