Conditional Probability Density Function Research Articles

BO-GRNN: Machine Learning for Underwater Acoustic Source Localization

This article proposes a Bayesian optimization-based generalized regression neural network (BO-GRNN) framework for underwater source localization in shallow water environment. Specially, by leveraging kernel regression analysis, BO-GRNN fits a conditional probability density function using training and test samples to infer the source location of unseen test samples. Furthermore, BO-GRNN models the objective function as a Gaussian process (GP) based on observed samples, utilizing an acquisition function to sample around the best objective function values and high variance regions. By evaluating these samples and updating the GP iteratively, the optimal hyperparameters are identified, ultimately improving the training efficiency and localization performance of the GRNN. Simulation experiments and error analysis validate this framework’s effectiveness. Further validation using SWellEx-96 experimental data demonstrates the superior localization performance of this framework, even with limited data snapshots. This framework estimates the range and depth of underwater sources within a 10 km range, achieving mean absolute errors of less than 0.15 km for range and 0.2 m for depth. Compared to GRNN, this framework demonstrates a 5 times improvement in training efficiency. These advantages make BO-GRNN an efficient, accurate, and robust solution for underwater source localization.

분포동학을 통한 중국 위안화 환율의 안정성 및 결정요인 분석

This study is focused on the stability and determinants of the Chinese Yuan (CNY) against the U.S. dollar exchange rate movement. To this end, a distribution dynamics model is introduced. We establish the non-linearity of the Generalized normal (GEN) distribution and estimate the impact of macroeconomic variables, such as interest rates, on exchange rate dynamic conditional probability density functions (PDFs). As the distribution allows for fluctuations in the exchange rate, the conditional probability density function (PDF) reveals a time-varying multimodality (i.e., unimodality and bimodality), based on the change in the shape parameters of the generalized normal (GEN) distribution. A unimodal distribution corresponds to a stable equilibrium for exchange rate movement. In contrast, a bimodal distribution contains an unstable equilibrium. Moreover, the bimodality may offer insights into the exchange rate fluctuations resulting from foreign exchange crises or global financial crises. In contrast with previous research on the linearity between macroeconomic variables and exchange rate movements, this study presents novel evidence on the dynamic non-linearity distribution of CNY exchange rate movements and their determinants. Our empirical analysis results will serve as a reference point for macroeconomic policymakers.

Efficient path integral approach via analytical asymptotic expansion for nonlinear systems under Gaussian white noise

In this paper an efficient formulation of the Path integral (PI) approach is developed for determining the response probability density functions (PDFs) and first-passage statistics of nonlinear oscillators subject to stationary and time-modulated external Gaussian white noise excitations. Specifically, the evolution of the response PDF is obtained in short time steps, by using a discrete version of the Chapman-Kolmogorov equation and assuming a Gaussian form for the conditional response PDF. Next, the technique involves proceeding to treating the problem via an analytical asymptotic expansion procedure, namely the Laplace’s method of integration. In this manner, the repetitive double integrals involved in the standard implementation of the PI approach are evaluated in a closed form, while the response and first-passage PDFs are obtained by mundane step-by-step application of the derived approximate analytical expression. It is shown that the herein proposed formulation can drastically decrease the associated computational cost by several orders of magnitude, as compared to both the standard PI technique and Monte Carlo solution (MCS) approach. A number of nonlinear oscillators are considered in the numerical examples. Notably, for these systems both response PDFs and first-passage probabilities are presented, whereas comparisons with pertinent MCS data demonstrate the efficiency and accuracy of the technique.

A probabilistic identification of the buckling resistance of imperfect cylindrical shells in axial compression

The buckling resistance of imperfect circular cylindrical shells of elastic and isotropic material subject to axial compression is studied focusing on initial geometric imperfections. Considering the imperfections as a random field, the suggested probabilistic representation of the buckling resistance parallels the deterministic dependence of buckling loads on the size and shape of imperfections. It derives from the probability density function of an imperfection size measure and the conditional probability density functions of the resistance confined to the realizations of imperfections given their size levels. The separated representation of the buckling resistance allows: (1) supplementary checking the accuracy of the random field representation, (2) classifying measured and simulated imperfections by their size and shape effects on buckling loads, (3) prospective delimiting the range of working imperfections for fabrication tolerance classes, and (4) defining the characteristic imperfection for the semi-probabilistic design. Introductory calculations illustrate the dependence of the normalized buckling loads of a shell on the shape and size of imperfections.

Sequential Detection under Correlated Observations using Recursive Method

Sequential analysis has been used in many cases when the decision is supposed to be taken quickly such as for signal detection in statistical signal processing, namely sequential detector. For identical error probabilities, a sequential detector needs a smaller average sample number (ASN) than its counterpart of a fixed sample number quadrature detector based on Neyman-Pearson criteria. The optimum sequential detector was derived based on the assumption that the observations are uncorrelated (independent). However, in realistic scenario, such as in radar, the assumption is commonly violated. Using a sequential detector under correlated observations is sub-optimal and it poses a problem. It demands a high computational complexity, since it needs to recalculate the inverse and the determinant of the signal covariance matrix for each new sample taken. This paper presents a technique for reducing the computational complexity, which involves using recursive matrix inverse to subsequently calculate conditional probability density functions (pdf). This eliminates the need to recalculate the inverse and determinant, leading to a more reasonable solution in real-world scenario. We evaluate the performance of the proposed (recursive) sequential detector by using Monte-Carlo simulations and we use the conventional and non-recursive sequential detectors for comparisons. The results show that the recursive sequential detector has equal probabilities of false alarm and miss-detection with the conventional sequential detector and performs better than the non-recursive sequential detector. In terms of ASN, it maintains comparable results to the two conventional detectors. The recursive approach has reduced the computational complexity for matrix multiplication to from and it also has rendered the calculation of matrix determinant to be unnecessary. Therefore, by having better probabilities of error and reduced computational complexities under correlated observations, the proposed recursive sequential detector may become a viable alternative to obtain a more agile detection system as required in future applications, such as in radar and cognitive radio.

TEILP: Time Prediction over Knowledge Graphs via Logical Reasoning

Conventional embedding-based models approach event time prediction in temporal knowledge graphs (TKGs) as a ranking problem. However, they often fall short in capturing essential temporal relationships such as order and distance. In this paper, we propose TEILP, a logical reasoning framework that naturaly integrates such temporal elements into knowledge graph predictions. We first convert TKGs into a temporal event knowledge graph (TEKG) which has a more explicit representation of time in term of nodes of the graph. The TEKG equips us to develop a differentiable random walk approach to time prediction. Finally, we introduce conditional probability density functions, associated with the logical rules involving the query interval, using which we arrive at the time prediction. We compare TEILP with state-of-the-art methods on five benchmark datasets. We show that our model achieves a significant improvement over baselines while providing interpretable explanations. In particular, we consider several scenarios where training samples are limited, event types are imbalanced, and forecasting the time of future events based on only past events is desired. In all these cases, TEILP outperforms state-of-the-art methods in terms of robustness.

A full‐probabilistic cloud analysis for structural seismic fragility via decoupled M‐PDEM

AbstractPerformance‐based earthquake engineering (PBEE) is essential for ensuring engineering safety. Conducting seismic fragility analysis within this framework is imperative. Existing methods for seismic fragility analysis often rely heavily on double loop reanalysis and empirical data fitting, leading to challenges in obtaining high‐precision results with a limited number of representative structural analysis instances. In this context, a new methodology for seismic fragility based on a full‐probabilistic cloud analysis is proposed via the decoupled multi‐probability density evolution method (M‐PDEM). In the proposed method, the assumption of a log‐normal distribution is not required. According to the random event description of the principle of preservation of probability, the transient probability density functions (PDFs) of intensity measure (IM) and engineering demand parameter (EDP), as key response quantities of the seismic‐structural system, are governed by one‐dimensional Li‐Chen equations, where the physics‐driven forces are determined by representative analysis data of the stochastic dynamic system. By generating ground motions based on representative points of basic random variables and performing structural dynamic analysis, the decoupled M‐PDEM is employed to solve the one‐dimensional Li‐Chen equations. This yields the joint PDF of IM and EDP, as well as the conditional PDF of EDP given IM, resulting in seismic fragility analysis outcomes. The numerical implementation procedure is elaborated in detail, and validation is performed using a six‐story nonlinear reinforced concrete (RC) frame subjected to non‐stationary stochastic ground motions. Comparative analysis against Monte Carlo simulation (MCS) and traditional cloud analysis based on least squares regression (LSR) reveals that the proposed method achieves higher computational precision at comparable structural analysis costs. By directly solving the physics‐driven Li‐Chen equations, the method provides the full‐probabilistic joint information of IM and EDP required for cloud analysis, surpassing the accuracy achieved by traditional methods based on statistical moment fitting and empirical distribution assumptions.

Probability distributions for dynamic and extreme responses of linear elastic structures under quasi-stationary harmonizable loads

The non-stationary load models based on the evolutionary power spectral density (EPSD) may lead to ambiguous structural responses. Quasi-stationary harmonizable processes with non-negative Wigner-Ville spectra are suitable for modeling non-stationary loads and analyzing their induced structural responses. In this study, random environmental loads are modeled as quasi-stationary harmonizable processes. The Loève spectrum of a harmonizable load process contains several random physical parameters. An explicit approach to calculate the probability distributions for the dynamic and extreme responses of a linear elastic structure subjected to a quasi-stationary harmonizable load is proposed. Conditioned on the specific values of the load spectral parameters, the harmonizable load process is assumed to be Gaussian. The conditional joint probability density function (PDF) of structural dynamic responses at any finite time instants and the conditional cumulative distribution function (CDF) of the structural extreme response are provided. By multiplying these two conditional probability distributions with the joint PDF of the load spectral parameters, and then integrating these two products over the parameter sample space, the joint PDF of structural dynamic responses at any finite time instants and the CDF of the structural extreme response can be calculated. The efficacy of the proposed approach is numerically validated using two linear elastic systems, which are subjected to non-stationary and non-Gaussian wind and seismic loads, respectively. The merit of the harmonizable load process model is highlighted through a comparative analysis with the EPSD load model.

Filtering of hidden Markov renewal processes by continuous and counting observations

<p>This paper introduces a subclass of Markov renewal processes (MRPs) and presents a solution to the optimal filtering problem in a stochastic observation system, where the state is modeled by an MRP and observed indirectly through noisy measurements. The MRPs considered here can be interpreted as continuous-time Markov chains (CTMCs) with a finite set of abstract states representing distributions of random vectors. The paper outlines the probabilistic properties of MRPs, emphasizing the ability to express any arbitrary function of the MRP as the solution to a linear stochastic differential system (SDS) with a martingale on the right-hand side (RHS). Using these properties, an optimal filtering problem is formulated in stochastic observation systems, where the hidden state belongs to the class of MRPs, and the observations consist of both diffusion and counting components. The drift terms in all observations depend on the system state. An optimal filtering estimate for a scalar function of the MRP is provided through the solution of an SDS with innovation processes on the RHS. Additionally, the paper presents a version of the Kushner-Stratonovich equation, describing the evolution of the conditional probability density function (PDF). To demonstrate the practical application of the estimation method, the paper presents a communications-related example, focusing on monitoring the qualitative state and numerical characteristics of a network channel using noisy observations of round-trip time (RTT) and packet loss flow. The paper also highlights the robustness of the filtering algorithm in scenarios where the MRP distribution is uncertain.</p>

Vine Copula Model: Application to Chemical Elements in Water Samples

Copula can link the bivariate distribution function with marginal distribution functions without requiring specific information about the interdependence among random variables. There are several types of copulas, such as elliptical copulas, Archimedean copulas, and extreme value copulas. However, in multivariate modeling, each type of copula has limitations in modeling complex dependence structures in terms of symmetry and tail dependence properties. The class of vine copulas overcomes these limitations by constructing multivariate models using bivariate copulas in a tree-like structure. The bivariate copulas used in this study include the Clayton, Gumbel, Frank, Gaussian, and Student's t copula families. This study discusses the construction of vine copula models, parameter estimation, and their applications. The construction of vine copulas is done through the decomposition of conditional probability density functions and substituting bivariate copula density functions into the decomposition results. The data used in the study is the logarithm of the concentration of chemical elements in water samples in Colorado. The parameter estimation method used is pseudo-maximum likelihood with sequential estimation. Model selection is then performed using the Akaike information criterion (AIC) to determine the most suitable model. The results indicate that Caesium and Titanium have a dependency relationship with Scandium. Moreover, Scandium and Titanium exhibit the strongest dependence compared to other variable pairs.

Test of conditional independence in factor models via Hilbert–Schmidt independence criterion

This work is concerned with testing conditional independence under a factor model setting. We propose a novel multivariate test for non-Gaussian data based on the Hilbert–Schmidt independence criterion (HSIC). Theoretically, we investigate the convergence of our test statistic under both the null and the alternative hypotheses, and devise a bootstrap scheme to approximate its null distribution, showing that its consistency is justified. Methodologically, we generalize the HSIC-based independence test approach to a situation where data follow a factor model structure. Our test requires no nonparametric smoothing estimation of functional forms including conditional probability density functions, conditional cumulative distribution functions and conditional characteristic functions under the null or alternative, is computationally efficient and is dimension-free in the sense that the dimension of the conditioning variable is allowed to be large but finite. Further extension to nonlinear, non-Gaussian structure equation models is also described in detail and asymptotic properties are rigorously justified. Numerical studies demonstrate the effectiveness of our proposed test relative to that of several existing tests.

T-shape data and probabilistic remaining useful life prediction for Li-ion batteries using multiple non-crossing quantile long short-term memory

This paper introduces and formalizes the concept of T-shape data, which arises in several engineering and natural contexts, where the initial data are richer and cover a wider range of operations than the data acquired in the following time. The specific context considered is the Li-ion battery experimental testing before the actual operation, where the T-shape data is cast as right-censored data in the application of battery remaining useful life (RUL) prediction. Existing RUL prediction techniques focused on RUL point estimation and provided rough calculations concerning the RUL distribution, thus are insufficient for battery degradation information. Based on the T-shape structure, this paper investigates a time-varying construction of the RUL probability density function. The proposed computationally efficient method, the multiple non-crossing quantiles Long Short-Term Memory (MNQ-LSTM), learns long-term dependencies among battery RUL and operation process quantities, including operating conditions. With several predicted quantile levels, the construction of the RUL conditional probability density function can capture the underlying survival distribution and statistical inferences of battery RUL with richer and more accurate information. Numerical results verify the performance of the proposed MNQ-LSTM with T-shape data. Compared to the conventional LSTM, Gaussian process regression, and a hybrid deep learning model of convolutional neural network and the bi-directional gated recurrent unit, the proposed model outperforms and can achieve the coefficient of determination up to 95.74% regarding point predictions. 100% testing results of battery RUL are not outside the range of 90% prediction intervals even in the worst case of 80% T-shape data.

Modeling orientation perception adaptation to altered gravity environments with memory of past sensorimotor states.

Transitioning between gravitational environments results in a central reinterpretation of sensory information, producing an adapted sensorimotor state suitable for motor actions and perceptions in the new environment. Critically, this central adaptation is not instantaneous, and complete adaptation may require weeks of prolonged exposure to novel environments. To mitigate risks associated with the lagging time course of adaptation (e.g., spatial orientation misperceptions, alterations in locomotor and postural control, and motion sickness), it is critical that we better understand sensorimotor states during adaptation. Recently, efforts have emerged to model human perception of orientation and self-motion during sensorimotor adaptation to new gravity stimuli. While these nascent computational frameworks are well suited for modeling exposure to novel gravitational stimuli, they have yet to distinguish how the central nervous system (CNS) reinterprets sensory information from familiar environmental stimuli (i.e., readaptation). Here, we present a theoretical framework and resulting computational model of vestibular adaptation to gravity transitions which captures the role of implicit memory. This advancement enables faster readaptation to familiar gravitational stimuli, which has been observed in repeat flyers, by considering vestibular signals dependent on the new gravity environment, through Bayesian inference. The evolution and weighting of hypotheses considered by the CNS is modeled via a Rao-Blackwellized particle filter algorithm. Sensorimotor adaptation learning is facilitated by retaining a memory of past harmonious states, represented by a conditional state transition probability density function, which allows the model to consider previously experienced gravity levels (while also dynamically learning new states) when formulating new alternative hypotheses of gravity. In order to demonstrate our theoretical framework and motivate future experiments, we perform a variety of simulations. These simulations demonstrate the effectiveness of this model and its potential to advance our understanding of transitory states during which central reinterpretation occurs, ultimately mitigating the risks associated with the lagging time course of adaptation to gravitational environments.

Non parametric estimation of transition density for second-order diffusion processes

The transition density of the diffusion process plays an important role in calculating the dynamic characteristics of the underlying variables as well as the model estimation. In this article, we combine the idea of conditional probability density function with non parametric kernel regression, and introduce the kernel estimation of joint density function and marginal density function, then construct the non parametric kernel estimator of the transition density of second-order diffusion process based on discrete observational samples. In order to obtain the asymptotic properties of the new kernel estimator, we analyze the asymptotic expectation and asymptotic variance of the proposed estimator under some mild conditions. Finally, the consistency and asymptotic normality of the new proposed non parametric estimator of the transition density function are proved.

Оптимальная нелинейная фильтрация оценок информационного воздействия в стохастической модели информационного противоборства

A computationally efficient algorithmic solution to the problem of optimal nonlinear filtering of information impact estimates in a generalized stochastic model of information warfare is developed in the article. The formed solution is applicable in the presence of heterogeneous rules for measuring the parameters of the information warfare model, on the basis of which a pair of systems of stochastic differential equations is formed. According to the criterion of maximum likelihood according to the determined evolution of the a posteriori conditional probability density function at a given observation interval, the evaluation of the information impact in the optimal nonlinear filtering model is performed. Taking into account the probability addition theorem, as the probability of the sum of two joint events, the density functions of which are established from the numerical solution of the corresponding robust Duncan-Mortensen-Zakai equations, finding a posteriori conditional probability density function at a given time is performed. For the first event, it is assumed that the first system of stochastic differential equations is the equation of state, and the second – is the equation of observation. For the second event, their definition is set in reverse order. The solution of the robust Duncan-Mortensen-Zakai equation is carried out in the formulation of the Galerkin spectral method when sampling the observation interval into subintervals and reducing the initial solution to a numerical recurrent study of the sequence of subtasks using the so-called Yau-Yau's algorithm, which assumes an estimate of the probability measure from the solution of the direct Kolmogorov equation with its subsequent correction by observation. To highlight the features of the algorithmic implementation of the compiled solution, an algorithm for optimal nonlinear filtering of information impact estimates in a generalized stochastic model of information confrontation when specifying the listing of the function implementing it, which is represented by a pseudocode, has been formed. To identify the preference of the compiled algorithmic solution for optimal nonlinear filtering of information impact assessments, a series of computational experiments on large-volume test samples was carried out. The result of the information impact assessment obtained by the proposed algorithm is compared with the determined solution: 1) by the average sample values from the observation models; 2) by an ensemble extended Kalman filter; 3) by a filtering algorithm involving a numerical study of the Duncan-Mortensen-Zakai equation. According to the conducted a posteriori study, quantitative indicators that establish the gain of the compiled algorithm and the limits of its applicability are highlighted.

The Mori–Zwanzig formulation of deep learning

We develop a new formulation of deep learning based on the Mori–Zwanzig (MZ) formalism of irreversible statistical mechanics. The new formulation is built upon the well-known duality between deep neural networks and discrete dynamical systems, and it allows us to directly propagate quantities of interest (conditional expectations and probability density functions) forward and backward through the network by means of exact linear operator equations. Such new equations can be used as a starting point to develop new effective parameterizations of deep neural networks and provide a new framework to study deep learning via operator-theoretic methods. The proposed MZ formulation of deep learning naturally introduces a new concept, i.e., the memory of the neural network, which plays a fundamental role in low-dimensional modeling and parameterization. By using the theory of contraction mappings, we develop sufficient conditions for the memory of the neural network to decay with the number of layers. This allows us to rigorously transform deep networks into shallow ones, e.g., by reducing the number of neurons per layer (using projection operators), or by reducing the total number of layers (using the decay property of the memory operator).

A Statistical Dependence Framework Based on a Multivariate Normal Copula Function and Stochastic Differential Equations for Multivariate Data in Forestry

Stochastic differential equations and Copula theories are important topics that have many advantages for applications in almost every discipline. Many studies in forestry collect longitudinal, multi-dimensional, and discrete data for which the amount of measurement of individual variables does not match. For example, during sampling experiments, the diameters of all trees, the heights of approximately 10% of the trees, and the tree crown base height and crown width for a significantly smaller number of trees are measured. In this study, for estimating five-dimensional dependencies, we used a normal copula approach, where the dynamics of individual tree variables (diameter, potentially available area, height, crown base height, and crown width) are described by a stochastic differential equation with mixed-effect parameters. The approximate maximum likelihood method was used to obtain parameter estimates of the presented stochastic differential equations, and the normal copula dependence parameters were estimated using the pseudo-maximum likelihood method. This study introduced the normalized multi-dimensional interaction information index based on differential entropy to capture dependencies between state variables. Using conditional copula-type probability density functions, the exact form equations defining the links among the diameter, potentially available area, height, crown base height, and crown width were derived. All results were implemented in the symbolic algebra system MAPLE.

Asynchronous Spatiotemporal Spike Metric for Event Cameras.

Event cameras as bioinspired vision sensors have shown great advantages in high dynamic range and high temporal resolution in vision tasks. Asynchronous spikes from event cameras can be depicted using the marked spatiotemporal point processes (MSTPPs). However, how to measure the distance between asynchronous spikes in the MSTPPs still remains an open issue. To address this problem, we propose a general asynchronous spatiotemporal spike metric considering both spatiotemporal structural properties and polarity attributes for event cameras. Technically, the conditional probability density function is first introduced to describe the spatiotemporal distribution and polarity prior in the MSTPPs. Besides, a spatiotemporal Gaussian kernel is defined to capture the spatiotemporal structure, which transforms discrete spikes into the continuous function in a reproducing kernel Hilbert space (RKHS). Finally, the distance between asynchronous spikes can be quantified by the inner product in the RKHS. The experimental results demonstrate that the proposed approach outperforms the state-of-the-art methods and achieves significant improvement in computational efficiency. Especially, it is able to better depict the changes involving spatiotemporal structural properties and polarity attributes.

Data Assimilation Networks

AbstractData Assimilation aims at estimating the posterior conditional probability density functions based on error statistics of the noisy observations and the dynamical system. State of the art methods are sub‐optimal due to the common use of Gaussian error statistics and the linearization of the non‐linear dynamics. To achieve a good performance, these methods often require case‐by‐case fine‐tuning by using explicit regularization techniques such as inflation and localization. In this paper, we propose a fully data driven deep learning framework generalizing recurrent Elman networks and data assimilation algorithms. Our approach approximates a sequence of prior and posterior densities conditioned on noisy observations using a log‐likelihood cost function. By construction our approach can then be used for general nonlinear dynamics and non‐Gaussian densities. As a first step, we evaluate the performance of the proposed approach by using fully and partially observed Lorenz‐95 system in which the outputs of the recurrent network are fitted to Gaussian densities. We numerically show that our approach, without using any explicit regularization technique, achieves comparable performance to the state‐of‐the‐art methods, IEnKF‐Q and LETKF, across various ensemble size.

The mechanical behaviors of random curved fiber networks by numerical simulations

Random curved fiber networks (RCFNs) are ubiquitous in biological materials and artificial composites, attracting much attention because of their unique mechanical properties and functionalities. Based on numerical simulations, the effects of fiber morphology, fiber aspect ratio and relative density on mechanical properties of two dimensional (2D) RCFNs are investigated in terms of the elastic modulus, yield strength as well as energy ratio of stretching to bending deformation. Based on the conditional probability density function (CPDF), a general theoretical model is extended for calculating the crosslink density of fiber networks, which elucidates the independence of crosslink density on fiber shape in RCFNs. According to numerical results, the elastic modulus and yield strength decrease with rising fiber curvature and are enhanced by increasing of aspect ratio and relative density. The influence of fiber aspect ratio in RCFNs exhibits thresholds, beyond which the mechanical properties will reach upper limits for constant curvature. The scaled yield strength to modulus of RCFNs versus curvature is found to collapse into a nonlinear curve regardless of aspect ratios. The energy ratio is influenced by fiber curvature more vigorously which is explained by deformations of curved segments and load paths deviations. This paper is expected to contribute to deep understanding of relationships between overall mechanical properties and structural parameters of RCFNs.