Deterministic Measure Research Articles

Deciphering Clusters With a Deterministic Measure of Clustering Tendency

Deciphering Clusters With a Deterministic Measure of Clustering Tendency

Random distortion risk measures

This paper presents one type of random risk measures, named as the random distortion risk measure. The random distortion risk measure is a generalization of the traditional deterministic distortion risk measure by randomizing the deterministic distortion function and the risk distribution respectively, where a stochastic distortion is introduced to randomize the distortion function, and a sub-σ-algebra is introduced to illustrate the influence of the known information on the risk distribution. Some theoretical properties of the random distortion risk measure are provided, such as normalization, conditional positive homogeneity, conditional comonotonic additivity, monotonicity in stochastic dominance order, and continuity from below, and a method for specifying the stochastic distortion and the sub-σ-algebra is provided. Based on some stochastic axioms, a representation theorem of the random distortion risk measure is proved. For considering the randomization of a given deterministic distortion risk measure, some families of random distortion risk measures are introduced with the stochastic distortions constructed from a Poisson process, a Brownian motion, and a Dirichlet process, respectively. A numerical analysis is carried out for showing the influence of the stochastic distortion and the sub-σ-algebra by focusing on the sample statistics, empirical distributions, and tail behavior of the random distortion risk measures.

Orthogonal Procrustes and Machine Learning: Predicting Bill of Materials errors on time

In an industrial product development process, the Bill of Materials (BOM) is a hierarchical, multi-level representation of all components, parts and quantities of a product. With increasing complexity of industrial products, also BOMs become more complex and thus prone to errors, for example when the individual parts of a product are changed during the product development process. Frequently, these Bill of Materials errors have to be identified manually or by using simple, rule-based schemes. In this paper, we provide a technical background of BOMs, showing the intricacy of temporal BOMs errors in an industrial product development process. The work of other authors, which focused on association mining and tree reconciliation to detect Bill of Materials errors, is analysed. We found that there is currently no system being able to prescribe where in a Bill of Materials and when in the product development process, errors are probable to occur. Also, Machine Learning (ML) methods have not been applied yet. Based on these findings, we formalize the notions Bill of Materials and Bill of Materials errors. Furthermore, we present a deterministic distance measure for BOMS. We provide an answer to the main question of how to represent a Bill of Materials for Machine Learning tasks by solving the orthogonal Procrustes problem for dynamic, hierarchical datasets. Then, we describe an isolation forest based approach to temporal anomaly detection, which points at potential errors in a Bill of Materials at a specific timestamp. Furthermore, we apply Machine Learning and present a multi-output Multi Layer Perceptron for the prediction of temporal Bill of Materials errors. The model predicts where and at which point of time Bill of Materials errors are probable to occur, which renders it a prescriptive system. Eventually, we optimize the performance of our model using contextualization via k-means clustering. Finally, we apply our prescriptive pipeline to a real world dataset and show its superiority to existing methods using a qualitative comparison.

Convergence of persistence diagram in the sparse regime

The objective of this paper is to examine the asymptotic behavior of persistence diagrams associated with Čech filtration. A persistence diagram is a graphical descriptor of a topological and algebraic structure of geometric objects. We consider Čech filtration over a scaled random sample rn−1Xn={rn−1X1,…,rn−1Xn}, such that rn→0 as n→∞. We treat persistence diagrams as a point process and establish their limit theorems in the sparse regime: nrnd→0, n→∞. In this setting, we show that the asymptotics of the kth persistence diagram depends on the limit value of the sequence nk+2rnd(k+1). If nk+2rnd(k+1)→∞, the scaled persistence diagram converges to a deterministic Radon measure almost surely in the vague metric. If rn decays faster so that nk+2rnd(k+1)→c∈(0,∞), the persistence diagram weakly converges to a limiting point process without normalization. Finally, if nk+2rnd(k+1)→0, the sequence of probability distributions of a persistence diagram should be normalized, and the resulting convergence will be treated in terms of the M0-topology.

System resilience distribution identification and analysis based on performance processes after disruptions.

Resilience is a system's ability to withstand a disruption and return to a normal state quickly. It is a random variable due to the randomness of both the disruption and resilience behavior of a system. The distribution characteristics of resilience are the basis for resilience design and analysis, such as test sample size determination and assessment model selection. In this paper, we propose a systematic resilience distribution identification and analysis (RDIA) method based on a system's performance processes after disruptions. Typical performance degradation/recovery processes have linear, exponential, and trigonometric functions, and they have three key parameters: the maximum performance degradation, the degradation duration, and the recovery duration. Using the Monte Carlo method, these three key parameters are first sampled according to their corresponding probability density functions. Combining the sample results with the given performance function type, the system performance curves after disruptions can be obtained. Then the sample resilience is computed using a deterministic resilience measure and the resilience distribution can be determined through candidate distribution identification, parameter estimation, and a goodness-of-fit test. Finally, we apply our RDIA method to systems with typical performance processes, and both the orthogonal experiment method and the control variable method are used to investigate the resilience distribution laws. The results show that the resilience of these systems follows the Weibull distribution. An end-to-end communication system is also used to explain how to apply this method with simulation or test data in practice.

Redefine carbon footprint performance with the total factor productivity: A case from a regional food production system

By observing the disturbance of indicators, models, and samples variability to the robustness of low carbon performance evaluation, the importance of random errors and heterogeneity of sample data is clarified. The carbon footprint joint total factor productivity conceptual framework is proposed in this study, then we apply it to a food production system in a region of China to reasonably judge the carbon footprint performance level of peasant households. With regard to the single-output production frontier, the internal method changes did not cause significant changes in the level of crop production technical efficiency in the deterministic and stochastic estimation strategies respectively. Although the results of partial low-carbon economic technical efficiency are consistent in the two estimation strategies, a significant difference is obvious for the low-carbon technical efficiency; The total low-carbon economic technical efficiency is significantly different between deterministic non-parametric slacks-based measure and parameterized random distance function estimation strategies under the framework of the multi-output production frontier; Ultimately, the total factor productivity including the carbon footprint of the food system in our research area close to the technical level of the industry, as well as the technical efficiency of the sample is at an efficient level.

TCMI: a non-parametric mutual-dependence estimator for multivariate continuous distributions

The identification of relevant features, i.e., the driving variables that determine a process or the properties of a system, is an essential part of the analysis of data sets with a large number of variables. A mathematical rigorous approach to quantifying the relevance of these features is mutual information. Mutual information determines the relevance of features in terms of their joint mutual dependence to the property of interest. However, mutual information requires as input probability distributions, which cannot be reliably estimated from continuous distributions such as physical quantities like lengths or energies. Here, we introduce total cumulative mutual information (TCMI), a measure of the relevance of mutual dependences that extends mutual information to random variables of continuous distribution based on cumulative probability distributions. TCMI is a non-parametric, robust, and deterministic measure that facilitates comparisons and rankings between feature sets with different cardinality. The ranking induced by TCMI allows for feature selection, i.e., the identification of variable sets that are nonlinear statistically related to a property of interest, taking into account the number of data samples as well as the cardinality of the set of variables. We evaluate the performance of our measure with simulated data, compare its performance with similar multivariate-dependence measures, and demonstrate the effectiveness of our feature-selection method on a set of standard data sets and a typical scenario in materials science.

Hydrodynamics of the Generalized N-urn Ehrenfest Model

In this paper we are concerned with a generalized N-urn Ehrenfest model, where balls perform independent random walks between N boxes uniformly laid on [0,1]. After a proper scaling of the transition rates function of the aforesaid random walk, we derive the hydrodynamic limit of the model, i.e., the law of large numbers which the empirical measure of the model follows, under an assumption where the initial number of balls in each box independently follows a Poisson distribution. We show that the empirical measure of the model converges weakly to a deterministic measure with density driven by an integral equation. Furthermore, we derive non-equilibrium fluctuation of the model, i.e, the central limit theorem from the above hydrodynamic limit. We show that the non-equilibrium fluctuation of the model is driven by a time-inhomogeneous generalized O-U process on the dual of C[0,1]. At last, we prove a large deviation principle from the hydrodynamic limit under an assumption where the transition rates function from [0,1] × [0,1] to $[0, +\infty )$ of the aforesaid random walk is a product of two marginal functions from [0,1] to $[0, +\infty )$ .

On the operator norm of non-commutative polynomials in deterministic matrices and iid Haar unitary matrices

Let \(U^N = (U_1^N,\dots ,U^N_p)\) be a p-tuple of \(N\times N\) independent Haar unitary matrices and \(Z^{NM}\) be any family of deterministic matrices in \({\mathbb {M}}_N({\mathbb {C}})\otimes {\mathbb {M}}_M({\mathbb {C}})\). Let P be a self-adjoint non-commutative polynomial. In Voiculescu (Int Math Res Notices 1:41–63, 1998), Voiculescu showed that the empirical measure of the eigenvalues of this polynomial evaluated in Haar unitary matrices and deterministic matrices converges towards a deterministic measure defined thanks to free probability theory. Now, let f be a smooth function. The main technical result of this paper is a precise bound of the difference between the expectation of $$\begin{aligned} \frac{1}{MN} {{\,\mathrm{Tr}\,}}_{{\mathbb {M}}_N({\mathbb {C}})}\otimes {{\,\mathrm{Tr}\,}}_{{\mathbb {M}}_M({\mathbb {C}})}\left( f(P(U^N\otimes I_M,Z^{NM})) \right) , \end{aligned}$$and its limit when N goes to infinity. If f is seven times differentiable, we show that it is bounded by \(M^2 \left\| f\right\| _{{\mathcal {C}}^6} \ln ^2(N)\times N^{-2}\). As a corollary we obtain a new proof with quantitative bounds of a result of Collins and Male which gives sufficient conditions for the operator norm of a polynomial evaluated in Haar unitary matrices and deterministic matrices to converge almost surely towards its free limit. Our result also holds in much greater generality. For instance, it allows to prove that if \(U^N\) and \(Y^{M_N}\) are independent and \(M_N=o(N^{1/3}\ln ^{-2/3}(N))\), then the norm of any polynomial in \((U^N\otimes I_{M_N}, I_N\otimes Y^{M_N})\) converges almost surely towards its free limit. Previous results required that \(M=M_N\) is constant.

The Spectra of Principal Submatrices in Rotationally Invariant Hermitian Random Matrices and the Markov– Krein Correspondence

We prove a concentration phenomenon on the empirical eigenvalue distribution (EED) of the principal submatrix in a random hermitian matrix whose distribution is invariant under unitary conjugacy; for example, this class includes GUE (Gaussian Unitary Ensemble) and Wishart matrices. More precisely, if the EED of the whole matrix converges to some deterministic probability measure $\mathfrak{m}$, then its fluctuation from the EED of the principal submatrix, after a rescaling, concentrates at the Rayleigh measure (in general, a Schwartz distribution) associated with $\mathfrak{m}$ by the Markov--Krein correspondence. For the proof, we use the moment method with Weingarten calculus and free probability. At some stage of calculations, the proof requires a relation between the moments of the Rayleigh measure and free cumulants of $\mathfrak{m}$. This formula is more or less known, but we provide a different proof by observing a combinatorial structure of non-crossing partitions.

On the operator norm of non-commutative polynomials in deterministic matrices and iid GUE matrices

Let $X^N = (X_1^N,\dots, X^N_d)$ be a d-tuple of $N\times N$ independent GUE random matrices and $Z^{NM}$ be any family of deterministic matrices in $\mathbb{M}_N(\mathbb{C})\otimes \mathbb{M}_M(\mathbb{C})$. Let $P$ be a self-adjoint non-commutative polynomial. A seminal work of Voiculescu shows that the empirical measure of the eigenvalues of $P(X^N)$ converges towards a deterministic measure defined thanks to free probability theory. Let now $f$ be a smooth function, the main technical result of this paper is a precise bound of the difference between the expectation of $$\frac{1}{MN}\text{Tr}\left( f(P(X^N\otimes I_M,Z^{NM})) \right)$$ and its limit when $N$ goes to infinity. If $f$ is six times differentiable, we show that it is bounded by $M^2\left\Vert f\right\Vert_{\mathcal{C}^6}N^{-2}$. As a corollary we obtain a new proof of a result of Haagerup and Thorbj\o rnsen, later developed by Male, which gives sufficient conditions for the operator norm of a polynomial evaluated in $(X^N,Z^{NM},{Z^{NM}}^*)$ to converge almost surely towards its free limit. Restricting ourselves to polynomials in independent GUE matrices, we give concentration estimates on the largest eingenvalue of these polynomials around their free limit. A direct consequence of these inequalities is that there exists some $\beta>0$ such that for any $\varepsilon_1<3+\beta)^{-1}$ and $\varepsilon_2<1/4$, almost surely for $N$ large enough, $$-\frac{1}{N^{\varepsilon_1}} \| P(X^N)\| - \left\Vert P(x)\right\Vert \leq \frac{1}{N^{\varepsilon_2}}.$$ Finally if $X^N$ and $Y^{M_N}$ are independent and $M_N = o(N^{1/3})$, then almost surely, the norm of any polynomial in $(X^N\otimes I_{M_N},I_N\otimes Y^{M_N})$ converges almost surely towards its free limit. This result is an improvement of a Theorem of Pisier, who was himself using estimates from Haagerup and Thorbj\o rnsen, where $M_N$ had size $o(N^{1/4})$.

Homotopy types of random cubical complexes

We study the topology of a random cubical complex associated to Bernoulli site percolation on a cubical grid. We begin by establishing a limit law for homotopy types. More precisely, looking within an expanding window, we define a sequence of normalized counting measures (counting connected components according to homotopy type), and we show that this sequence of random probability measures converges in probability to a deterministic probability measure. We then investigate the dependence of the limiting homotopy measure on the coloring probability p, and our results show a qualitative change in the homotopy measure as p crosses the percolation threshold $$p=p_c$$ . Specializing to the case of $$d=2$$ dimensions, we also present empirical results that raise further questions on the p-dependence of the limiting homotopy measure.

Use of the p-values as a size-dependent function to address practical differences when analyzing large datasets

Biomedical research has come to rely on p-values as a deterministic measure for data-driven decision-making. In the largely extended null hypothesis significance testing for identifying statistically significant differences among groups of observations, a single p-value is computed from sample data. Then, it is routinely compared with a threshold, commonly set to 0.05, to assess the evidence against the hypothesis of having non-significant differences among groups, or the null hypothesis. Because the estimated p-value tends to decrease when the sample size is increased, applying this methodology to datasets with large sample sizes results in the rejection of the null hypothesis, making it not meaningful in this specific situation. We propose a new approach to detect differences based on the dependence of the p-value on the sample size. We introduce new descriptive parameters that overcome the effect of the size in the p-value interpretation in the framework of datasets with large sample sizes, reducing the uncertainty in the decision about the existence of biological differences between the compared experiments. The methodology enables the graphical and quantitative characterization of the differences between the compared experiments guiding the researchers in the decision process. An in-depth study of the methodology is carried out on simulated and experimental data. Code availability at https://github.com/BIIG-UC3M/pMoSS.

Random Geometric Complexes and Graphs on Riemannian Manifolds in the Thermodynamic Limit

We investigate some topological properties of random geometric complexes and random geometric graphs on Riemannian manifolds in the thermodynamic limit. In particular, for random geometric complexes we prove that the normalized counting measure of connected components, counted according to isotopy type, converges in probability to a deterministic measure. More generally, we also prove similar convergence results for the counting measure of types of components of each k-skeleton of a random geometric complex. As a consequence, in the case of the 1-skeleton (i.e., for random geometric graphs) we show that the empirical spectral measure associated to the normalized Laplace operator converges to a deterministic measure.

On the behavior of large empirical autocovariance matrices between the past and the future

The asymptotic behavior of the distribution of the squared singular values of the sample autocovariance matrix between the past and the future of a high-dimensional complex Gaussian uncorrelated sequence is studied. Using Gaussian tools, it is established that the distribution behaves as a deterministic probability measure whose support [Formula: see text] is characterized. It is also established that the squared singular values are almost surely located in a neighborhood of [Formula: see text].

Hydrodynamics of the weakly asymmetric normalized binary contact path process

We are concerned with the weakly asymmetric normalized binary contact path process. The symmetric version was introduced in Griffeath (1983). The model describes the spread of an infectious disease on the lattice Zd. The configuration at each site x∈Zd takes value in [0,∞). Site x is recovered at rate 1, and is infected by its neighbor x±ei at rate λ∓λ1∕N, where {ei}1≤i≤d is the canonical basis of the d-dimensional lattice, λ,λ1 are non-negative constants and N is the scaling parameter. When the infection occurs, the seriousness of the disease at site x is added with that of y. When there is neither recovery nor infection occurring during some time interval, the value at site x evolves according to some ODE. We prove that for d≥3 and large value λ, the empirical measure of the process, under diffusive scaling, converges in probability to a deterministic measure whose density is the unique weak solution to a linear parabolic equation, while for small λ and in all dimensions, the limit is zero. We also conjecture that the fluctuations field in the former case is driven by a generalized Ornstein–Uhlenbeck process, while a rigorous proof is absent. The main difficulty in proving the hydrodynamics is to prove the absolute continuity of the limiting path, where the theory of linear systems introduced in Liggett (1985) is utilized.

Topologies of Random Geometric Complexes on Riemannian Manifolds in the Thermodynamic Limit

Abstract We investigate the topologies of random geometric complexes built over random points sampled on Riemannian manifolds in the so-called “thermodynamic” regime. We prove the existence of universal limit laws for the topologies; namely, the random normalized counting measure of connected components (counted according to homotopy type) is shown to converge in probability to a deterministic probability measure. Moreover, we show that the support of the deterministic limiting measure equals the set of all homotopy types for Euclidean connected geometric complexes of the same dimension as the manifold.

Sequential Recommendation with Relation-Aware Kernelized Self-Attention

Recent studies identified that sequential Recommendation is improved by the attention mechanism. By following this development, we propose Relation-Aware Kernelized Self-Attention (RKSA) adopting a self-attention mechanism of the Transformer with augmentation of a probabilistic model. The original self-attention of Transformer is a deterministic measure without relation-awareness. Therefore, we introduce a latent space to the self-attention, and the latent space models the recommendation context from relation as a multivariate skew-normal distribution with a kernelized covariance matrix from co-occurrences, item characteristics, and user information. This work merges the self-attention of the Transformer and the sequential recommendation by adding a probabilistic model of the recommendation task specifics. We experimented RKSA over the benchmark datasets, and RKSA shows significant improvements compared to the recent baseline models. Also, RKSA were able to produce a latent space model that answers the reasons for recommendation.

Eigenvalue rigidity for truncations of random unitary matrices

We consider the empirical eigenvalue distribution of an [Formula: see text] principal submatrix of an [Formula: see text] random unitary matrix distributed according to Haar measure. For [Formula: see text] and [Formula: see text] large with [Formula: see text], the empirical spectral measure is well approximated by a deterministic measure [Formula: see text] supported on the unit disc. In earlier work, we showed that for fixed [Formula: see text] and [Formula: see text], the bounded-Lipschitz distance [Formula: see text] between the empirical spectral measure and the corresponding [Formula: see text] is typically of order [Formula: see text] or smaller. In this paper, we consider eigenvalues on a microscopic scale, proving concentration inequalities for the eigenvalue counting function and for individual bulk eigenvalues.

Assessment practice of geomechanical risks for slopes and retaining walls

The methods of geomechanical analysis are considered, which together allow to correct and propose new proactive approaches to ensure the geomechanical safety of slopes and retaining walls of mining structures. A very promising solution to this problem is the addition of a traditional deterministic measure for assessing the condition of slopes and retaining walls, i.e. safety margin factors, with simulation quantitative assessments of risk levels that objectively reflect the instability of the environment. In such a complex, the deterministic block performs insurance functions and determines that the results of the analysis do not contradict the requirements of regulatory documents; simulation (according to the Monte Carlo method) provides an unambiguous solution on the factors of economic and social security, eliminates or significantly reduces the likelihood of making unnecessary costs.