Multivariate Probability Distribution Research Articles

Intrinsic Information-Theoretic Models.

With this follow-up paper, we continue developing a mathematical framework based on information geometry for representing physical objects. The long-term goal is to lay down informational foundations for physics, especially quantum physics. We assume that we can now model information sources as univariate normal probability distributions N (μ, σ0), as before, but with a constant σ0 not necessarily equal to 1. Then, we also relaxed the independence condition when modeling m sources of information. Now, we model m sources with a multivariate normal probability distribution Nm(μ,Σ0) with a constant variance-covariance matrix Σ0 not necessarily diagonal, i.e., with covariance values different to 0, which leads to the concept of modes rather than sources. Invoking Schrödinger's equation, we can still break the information into m quantum harmonic oscillators, one for each mode, and with energy levels independent of the values of σ0, altogether leading to the concept of "intrinsic". Similarly, as in our previous work with the estimator's variance, we found that the expectation of the quadratic Mahalanobis distance to the sample mean equals the energy levels of the quantum harmonic oscillator, being the minimum quadratic Mahalanobis distance at the minimum energy level of the oscillator and reaching the "intrinsic" Cramér-Rao lower bound at the lowest energy level. Also, we demonstrate that the global probability density function of the collective mode of a set of m quantum harmonic oscillators at the lowest energy level still equals the posterior probability distribution calculated using Bayes' theorem from the sources of information for all data values, taking as a prior the Riemannian volume of the informative metric. While these new assumptions certainly add complexity to the mathematical framework, the results proven are invariant under transformations, leading to the concept of "intrinsic" information-theoretic models, which are essential for developing physics.

R‐Vine Copulas for Data‐Driven Quantification of Descriptor Relationships in Porous Materials

AbstractLocal variations in the 3D microstructure can control the macroscopic behavior of heterogeneous porous materials. For example, the permittivity through porous sheets or membranes is governed by local high‐volume pathways or bottlenecks. Due to local variations, unfeasibly large amounts of microstructure data may be needed to reliably predict such material properties directly from image data. Here it is demonstrated that a vine copula approach provides parametric models for local microstructure descriptors that compactly capture the 3D microstructure including its local variations and efficiently probe it with respect to selected, measurable properties. In contrast to common methods of complexity reduction, the proposed approach creates parametric models for the multivariate probability distribution of high‐dimensional descriptor vectors that inherently contain the complex, nonlinear dependencies between these descriptors. Therein, material properties are offered in physically motivated distributions of microstructure descriptors rather than as normally distributed data. Applied to porous fiber networks (paper) before and after unidirectional compression, it is shown that the copula‐based models reveal material‐characteristic relationships between two or more microstructure descriptors. In this way, the presented modeling approach can provide deeper insight into the microscopic origin of effective macroscopic properties of heterogeneous porous materials.

Discrete search stability for a hidden target on M-intervals with a known probabilistic distributed effort

We consider a discrete search problem to detect a hidden target on one of the [Formula: see text]-intersected real lines at the origin. The target position is a vector of independent random variables with a known multivariate probability distribution that is symmetric around the origin. From the available information about the probability of the target, we can consider one bounded interval as a search space on each line. To maximize the detection probability, we divide each interval into a number of small subintervals. Some subintervals have low target probabilities; thus, we delete them and distribute their probabilities among the searched subintervals. The problem has a discrete version where each line contains a set of independent search subintervals. The search effort is bounded by a normal distribution. More than obtaining the maximum detection probability, we also need to get the minimum search effort to detect the target. After studying the stability of the minimum search effort, we present an illustrative example to show the effectiveness of our model.

Using Copula functions to predict climatic change impacts on floods in river source regions

Flood frequency in river source regions is significantly affected by rainfall and snowmelt as part of climatic changes. A traditional univariate flood frequency analysis cannot reflect the complexity of floods, and when used in isolation, it can only underestimate flood risk. For effective flood prevention and mitigation, it is essential to consider the combined effects of precipitation and snowmelt. Copula functions can effectively quantify the joint distribution relationship between floods and their associated variables without restrictions on their distribution characteristics. This study uses copula functions to consider a multivariate probability distribution model of flood peak flow (Q) with cumulative snowmelt (CSm) and cumulative precipitation (CPr) for the Hutubi River basin located in northern Xinjiang, China. The joint frequencies of rainfall and snowmelt floods are predicted using copula models based on the Coupled Model Intercomparison Project Phase 6 data. The results show that Q has a significant positive correlation with 24-d CSm (r = 0.559, p = 0.002) and 23-d CPr (r = 0.965, p < 0.05). Flood frequency will increase in the future, and mid- (2050–2074) and long-term (2075–2099) floods will be more severe than those in the near-term (2025–2049). The probability of flood occurrence is higher under the SSP2-4.5 and SSP1-2.6 scenarios than under SSP5-8.5. Precipitation during the historical period (1990–2014) led to extreme floods, and increasing future precipitation trends are found to be insignificant. Snowmelt increases with rising temperatures and occurs earlier than estimated, leading to an earlier flood period in the basin and more frequent snowmelt floods. The Q under the joint return period is larger than that during the same univariate return period. This difference indicates that neglecting the interaction between precipitation and snowmelt for floods leads to an underestimation of the flood risk (with underestimations ranging from 0.3% to 22%). The underestimations decrease with an increase in the return period. The joint risks of rainfall or snowmelt according to various flood periods should be considered for rivers with multi-source runoff recharge in flood control design. This study reveals the joint impact of precipitation and snowmelt on extreme floods under climate change in river source regions. This study also provides a scientific basis for regional flood prevention and mitigation strategies, as well as for the rational allocation of water resources.

Prediction for underground seismic intensity measures using conditional generative adversarial networks

With the escalating development and utilization of subterranean spaces, the seismic hazards faced by underground structures are progressively increasing. However, owing to the challenges associated with acquiring underground seismic data and historical seismic design norms, pertinent regulations and research in this domain are scarce. This study focused on three crucial intensity measures in the seismic design process of underground structures peak ground acceleration (PGA), peak ground velocity (PGV), and peak ground displacement (PGD). The research leverages seismic data obtained from the California Strong Motion Instrumentation Program (CSMIP) to train and evaluate a conditional generative adversarial network (CGAN) model. This model was employed to establish a multivariate joint conditional probability distribution among the intensity measures at varying depths, facilitating the stochastic prediction of shallow intensity measures. In contrast to empirical formulas, the CGAN model eliminates the need for a predefined equation structure and enables the simultaneous prediction of multiple intensity measures. The performance of the model was evaluated by comparing the predictive accuracy of the CGAN model and empirical fitting formulas across diverse site conditions and depth intervals using metrics such as relative error coefficients. It can be concluded that the proposed CGAN model can accurately predict shallow seismic intensity measures, and the predictions conform to a specific conditional distribution while retaining the stochastic nature of seismic motion. Compared with empirical formula models, the CGAN model exhibited an enhanced predictive capability.

Measuring multidimensional heterogeneity in emergent social phenomena

Abstract Measuring inequalities in a multidimensional framework is a challenging problem, which is common to most field of science and engineering. Nevertheless, despite the enormous amount of researches illustrating the fields of application of inequality indices, and of the Gini index in particular, very few consider the case of a multidimensional variable. In this paper, we consider in some details a new inequality index, based on the Fourier transform, that can be fruitfully applied to measure the degree of inhomogeneity of multivariate probability distributions. This index exhibits a number of interesting properties that make it very promising in quantifying the degree of inequality in datasets of complex and multifaceted social phenomena.

Neural Network Approximators for Marginal MAP in Probabilistic Circuits

Probabilistic circuits (PCs) such as sum-product networks efficiently represent large multi-variate probability distributions. They are preferred in practice over other probabilistic representations, such as Bayesian and Markov networks, because PCs can solve marginal inference (MAR) tasks in time that scales linearly in the size of the network. Unfortunately, the most probable explanation (MPE) task and its generalization, the marginal maximum-a-posteriori (MMAP) inference task remain NP-hard in these models. Inspired by the recent work on using neural networks for generating near-optimal solutions to optimization problems such as integer linear programming, we propose an approach that uses neural networks to approximate MMAP inference in PCs. The key idea in our approach is to approximate the cost of an assignment to the query variables using a continuous multilinear function and then use the latter as a loss function. The two main benefits of our new method are that it is self-supervised, and after the neural network is learned, it requires only linear time to output a solution. We evaluate our new approach on several benchmark datasets and show that it outperforms three competing linear time approximations: max-product inference, max-marginal inference, and sequential estimation, which are used in practice to solve MMAP tasks in PCs.

Autonomous Behavior Selection For Self-driving Cars Using Probabilistic Logic Factored Markov Decision Processes

ABSTRACT We propose probabilistic logic factored Markov decision processes (PL-fMDPs) as a behavior selection scheme for self-driving cars. Probabilistic logic combines logic programming with probability theory to achieve clear, rule-based knowledge descriptions of multivariate probability distributions, and a flexible mixture of deductive and probabilistic inferences. Factored Markov decision processes (fMDPs) are widely used to generate reward-optimal action policies for stochastic sequential decision problems. For evaluation, we developed a simulated self-driving car with reliable modules for behavior selection, perception, and control. The behavior selection module is composed of a two-level structure of four action policies obtained from PL-fMDPs. Three main tests were conducted focused on the selection of the appropriate actions in specific driving scenarios, and the overtaking of static obstacle vehicles and dynamic obstacle vehicles. We performed 520 repetitions of these tests. The self-driving car completed its task without collisions in 99.2% of the repetitions. Results show the suitability of the overall self-driving strategy and PL-fMDPs to construct safe action policies for self-driving cars.

Characterization of metal elements in deep-seabed polymetallic nodules: A multivariate statistical approach

Deep-seabed polymetallic nodules have been recognized as a potential solution to the depletion of many metals that are produced by terrestrial minerals. Mineral resources obtained by deep-seabed mining vehicles significantly affect the economic viability of underwater mining activities. Therefore, an accurate prediction of the harvested mineral resources is significantly important. Probabilistic approach-based prediction, which enhances the accuracy of the economic evaluation, requires a statistical model of the variability of each metal element in the harvested polymetallic nodules. However, the probability distribution of the metal elements in the polymetallic nodules has rarely been studied thus far. A multivariate joint probability distribution must be adopted because the variabilities of these metal elements is correlated with each other. However, multivariate statistical approaches have not been actively studied owing to their highly sophisticated theories. The objective of this study was to establish a systematic framework for modeling a multivariate joint probability distribution of correlated random variables. A case study was performed to characterize the metal elements of the polymetallic nodules using the proposed approach.

Modeling forecast errors for microgrid operation using Gaussian process regression

Microgrids, denoting small-scale and self-sustaining grids, constitute a pivotal component in future power systems with a high penetration of renewable generators. The inherent uncertainty tied to renewable power generation, typified by photovoltaic and wind turbine systems, necessitates counterbalancing mechanisms. These mechanisms encompass Energy storage systems or conventional thermal fossil-fuel generators imbued with heightened flexibility. Addressing the uncertainty stemming from renewable generators mandates a cost-effective assessment and operational strategy for said compensatory devices. To this end, myriad uncertainty factors warrant scrutiny, conceivably concretized into a unified probability distribution function (PDF) that takes into account their temporal inter-dependencies. Diverse uncertainty factors, characterized by varying marginal distributions and scales, can be assimilated into a multivariate probability distribution through a conversion to normal distributions via rank correlation. However, with the escalation in the number of uncertainty factors embraced within a microgrid context, the endeavour becomes notably intricate when aiming to define conditional probability distributions originating from joint PDFs. This paper presents a method proposing the modelling of net-load forecast error distribution, considering the interplay among uncertainty factors. The approach introduces a data-driven Gaussian process regression technique for training and validating conditional PDFs among these uncertainty factors. Notably, this approach facilitates the transformation of said factors into normal distributions while preserving their inherent marginal characteristics. The resultant conditional density function, as per the proposed methodology, exhibits enhanced suitability for estimating net-load error distribution. Consequently, the conditional density function stemming from this proposed approach demonstrates superior aptitude in approximating the distribution of net load error.

Charging management of electric vehicles with the presence of renewable resources

Considering the increasing use of electric vehicles, the establishment of charging stations to exchange power between the grid and electric devices, and the integration of charging stations with solar power generation sources, the optimal use of electric vehicle charging stations in the power system. The purpose of cost reduction in the presence of the intelligent environment is a challenge that must be investigated so that this platform is suitable for predicting the behaviour of vehicles and, as a result, optimizing their presence in the power network. This research presents a relatively complete radial distribution network development planning model in two scenarios. In the first scenario, the effects of electric vehicles are not considered, and only the effects of distributed production (renewable and dispatchable) are considered. Studies have been done on a sample 54-bus network, a common system in most Distribution expansion planning (DEP) articles for distribution networks. In addition, the real data of American highways have been used to create raw input data. Also, due to the distance limit, the information on vehicles under 100 miles has been received as electric vehicle information. The clustering method and Capiola multivariate probability distribution functions have created suitable vehicle scenarios during different planning years. Capiola’s method increases the accuracy of vehicle load forecasting according to a predetermined growth rate. The DEP problem in this research is modeled as an optimization problem based on scenario, dynamic, and in 5 one-year time frames (5-year time horizon and one-year accuracy). The results indicate that, in the presence of electric vehicles and distributed production sources, the technical characteristics of the network are improved.Similarly, the use of DGs, in addition to reducing the cost of equipment, has reduced undistributed energy in the system. But 10,000 vehicles, which have been applied to the network as an uncontrolled load, have caused an increase in undistributed energy. The cost of equipment required for the network development is almost as much as 5%.

Studying the Influence of Antimicrobial Resistance on the Probability Distribution of Densities for Synchronization Growing of Different Kinds of Bacteria

Multiple types of microorganisms impact people with immunological illnesses simultaneously. In the presence of antibiotic resistance and random fluctuations such as rapid changes in the environment, the synchronization growth density varies stochastically from one species to another. Therefore, in this work, we study the multivariate probability distribution of the density of all species at the same time. Using the principle of independence of growth density for each species, we provide the statistical properties of this distribution. Furthermore, we estimate the parameters of the intrinsic rates of antibiotic action and bacterial cell growth for each variety, along with the strength of the intra-competitive pressures influencing this distribution.

Data-driven predictions of shield attitudes using Bayesian machine learning

Shield attitude prediction is essential in shield tunnelling to prevent excessive attitude deviations and is highly correlated with shield operational parameters. Furthermore, geological and geotechnical uncertainty are the primary sources of attitude deviation risks; however, these uncertainties were neglected in previous studies. In this paper, a novel data-driven Bayesian machine learning method for site-specific shield attitude predictions is proposed by simultaneously considering the spatial variability of geotechnical parameters and the generic cross-correlations between shield attitude parameters and operational parameters. The Gaussian process model is first used to effectively capture the spatial variability of geotechnical parameters using field borehole data. The multivariate probability distribution model is then constructed to characterise the generic cross-correlation between four attitude parameters, four geotechnical parameters and eight shield operational parameters. In the Bayesian theory framework, a hybrid strategy is proposed to fuse the generic data from other shield tunnelling projects with the ongoing site-specific data, ensuring that the shield attitude prediction is dominated by site-specific data when those data are abundant or by generic data when the site-specific data are sparse. Finally, the Hamiltonian Monte Carlo sampling method is employed to generate the posterior distributions of shield attitudes conditioned on geotechnical parameters and shield operational parameters. The proposed method has the benefit of rigorously quantifying the uncertainties of shield attitude predictions. The proposed approach is examined on a real-world tunnelling project in Nanjing City, indicating that the proposed method outperforms existing techniques in terms of both accuracy and robustness. Parametric analyses are conducted to explore the impacts of key factors on the performance of the proposed approach.

Multidimensional characterization of particle morphology and mineralogical composition using CT data and R-vine copulas

Computed tomography (CT) can capture volumes large enough to measure a statistically meaningful number of micron-sized particles with a sufficiently good resolution to allow for the analysis of individual particles. However, the development of methods to efficiently investigate such image data and interpretably model the observed particle features is still an active field of research. When image data of particles exhibiting a wide range of shapes and sizes is considered, traditional image segmentation methods, such as the classic watershed algorithm, struggle to recognize particles with satisfying accuracy. Thus, more advanced methods of machine learning must be utilized for image segmentation to improve the validity of subsequent analyses. Moreover, CT data does not include information about the mineralogical composition of particles and, therefore, additional SEM-EDS image data has to be acquired. In this paper, micro-CT image data of a particle system mostly consisting of zinnwaldite–quartz composites is considered. First, an image segmentation method is applied which uses deep convolutional neural networks, in particular an adaptation of the U-net architecture. This has the advantage of requiring less hand-labeling than other machine learning methods, while also being more flexible with the possibility of transfer learning. Then, fully parameterized models based on vine copulas are designed to determine multivariate probability distributions of descriptor vectors for the size, shape, texture and composition of particles—allowing for the estimation and interpretable characterization of interdependencies between particle descriptors. For model fitting, the segmented three-dimensional CT data and co-registered two-dimensional SEM-EDS data are used. The models are applied to predict the mineralogical composition of particles, solely on the basis of particle descriptors observed in CT data.

Estimation of site-specific multivariate probability distribution of soil properties using a mixed sampling technique

Estimation of multivariate probability density function (PDF) of soil properties based on limited site investigation data plays an important role in data driven-based geotechnical engineering analysis and design. However, this estimation is often intractable because the site-specific investigation data generally are multivariate, uncertain and unique, sparse, incomplete, sometimes corrupt and spatially variable. The estimation is more challenging when the uncertainties in model parameters of the distribution (i.e., mean, standard deviations and scale of fluctuation of soil properties) are taken into consideration. To address these difficulties, this study proposed a mixed sampling technique by combing the Gibbs sampler and transitional Markov Chain Monte Carlo sampler to effectively estimate the multivariate probability density function of soil properties. The proposed method is demonstrated through a virtual site example while the effect of statistical uncertainty in the scale of fluctuation on reliability analyses is illustrated in an infinite slope example. It is shown that the proposed method can not only handle the complex features of site investigation data but also effectively quantify the statistical uncertainties of the mean values, standard deviations and scale of fluctuation of the investigated soil properties. Ignoring the statistical uncertainties of scale of fluctuation can affect the estimation of mean and standard deviations of the soil parameters. When considering the statistical uncertainty of scale of fluctuation, the 95% confidence intervals for the profiles of the soil properties can rationally cover the actual soil data at the unobserved points. Ignoring the statistical uncertainty of scale of fluctuation can lead to the underestimation of the probability of failure of the infinite slope.

Multivariate probability distributions for index and mechanical clay parameters in Shenzhen

Probabilistic site characterization is the cornerstone of data-centric geotechnics, which underpins the digitalization in geotechnical engineering. The essential aspect of probabilistic site characterization is to reasonably infer the probability density distributions of geotechnical parameters, based on field and laboratory data. In this paper, a clay property database, SZ-CLAY/11/5130, was established by collecting geotechnical investigation reports from 52 engineering projects in Shenzhen city. This SZ-CLAY/11/5130 database was first compared with the global clay database CLAY/10/7490 and the Shanghai clay database SH-CLAY/11/4051, in terms of their sample ranges, correlations, and scatter plots. Then, the multivariate probability distribution of eight clay parameters was established using the training records selected from SZ-CLAY/11/5130, to characterize the generic correlations between multiple clay parameters. Based on Bayesian theory, the posterior distributions of clay mechanical parameters from a construction site in Shenzhen city were updated using both the site-specific correlations and the generic correlations given by SZ-CLAY/11/5130. The constructed multivariate probability density distribution is necessary for this updating step.

Dynamic prediction of over-excavation gap due to posture adjustment of shield machine in soft soil

The probability analysis of ground deformation is becoming a trend to estimate and control the risk brought by shield tunnelling. The gap parameter is regarded as an effective tool to estimate the ground loss of tunnelling in soft soil. More specifically, ω, which is a gap parameter component defined as the over (or insufficient) excavation due to the change in the posture of the shield machine, may contribute more to the uncertainty of the ground loss. However, the existing uncertainty characterization methods for ω have several limitations and cannot explain the uncertain correlations between the relevant parameters. Along these lines, to better characterize the uncertainty of ω, the multivariate probability distribution was developed in this work and a dynamic prediction was proposed for it. To attain this goal, 1 523 rings of the field data coming from the shield tunnel between Longqing Road and Baiyun Road in Kunming Metro Line 5 were utilized and 44 parameters including the construction, stratigraphic, and posture parameters were collected to form the database. According to the variance filter method, the mutual information method, and the value of the correlation coefficients, the original 44 parameters were reduced to 10 main parameters, which were unit weight, the stoke of the jacks (A, B, C, and D groups), the pressure of the pushing jacks (A, C groups), the chamber pressure, the rotation speed, and the total force. The multivariate probability distribution was constructed based on the Johnson system of distributions. Moreover, the distribution was satisfactorily verified in explaining the pairwise correlation between ω and other parameters through 2 million simulation cases. At last, the distribution was used as a prior distribution to update the marginal distribution of ω with any group of the relevant parameters known. The performance of the dynamic prediction was further validated by the field data of 3 shield tunnel cases.

Exploring complex multivariate probability distributions with simple and robust bayesian network topology for classification

Exploring complex multivariate probability distributions with simple and robust bayesian network topology for classification

Learning high-dependence Bayesian network classifier with robust topology

The increase in data variability and quantity makes it urgent for learning complex multivariate probability distributions. The state-of-the-art Tree Augmented Naive Bayes (TAN) classifier uses maximum weighted spanning tree (MWST) to graphically model data with excellent time and space complexity. In this paper, we theoretically prove the feasibility of scaling up one-dependence MWST to model high-dependence relationships, and then propose to apply a heuristic search strategy to improve the fitness of extended topology to data. The newly added edges to each attribute node may provide a local optimal solution. Then ensemble learning is introduced to improve the generalization performance and reduce the sensitivity to variation in training data. The experimental results on 32 benchmark datasets reveal that this highly scalable algorithm inherits the expressive power of TAN and achieves an excellent bias–variance tradeoff, and it also demonstrates competitive classification performance when compared to a range of high-dependence or ensemble learning algorithms.

A Framework for Analyzing Individual-Tree and Whole-Stand Growth by Fusing Multilevel Data: Stochastic Differential Equation and Copula Network

In forestry, growth functions form the basis of research and are widely used for the mathematical modeling of stand variables, e.g., tree or stand basal area, stand height, stand volume, site index, and many more. In this study, to estimate five-dimensional dependencies between tree diameter at breast height, potentially available area, height, crown area and crown base height, we used a normal copula approach whereby the growths of individual variables are described using a stochastic differential equation with mixed-effect parameters. The normal copula combines the marginal distributions of tree diameter at breast height, potentially available area, height, crown area, and crown base height into a joint multivariate probability distribution. Copula models have the advantage of being able to use collected longitudinal, multivariate, and discrete data for which the number of measurements of individual variables does not match. This study introduced a normalized multivariate interaction information measure based on differential entropy to assess the causality between tree size variables. In order to accurately and quantitatively assess the stochastic processes of the tree size variables’ growth and to provide a scientific basis for the formalization of models, an analysis method of the synergetic theory of information entropy has been proposed. Theoretical findings are illustrated using an uneven-aged, mixed-species empirical dataset of permanent experimental plots in Lithuania.