Underlying Probability Distribution Research Articles

Reformulation and Enhancement of Distributed Robust Optimization Framework Incorporating Decision-Adaptive Uncertainty Sets

Distributionally robust optimization (DRO) is an advanced framework within the realm of optimization theory that addresses scenarios where the underlying probability distribution governing the data is uncertain or ambiguous. In this paper, we introduce a novel class of DRO challenges where the probability distribution of random variables is contingent upon the decision variables, and the ambiguity set is defined through parameterization involving the mean and a covariance matrix, which also depend on the decision variables. This dependency makes DRO difficult to solve directly; therefore, first, we demonstrate that under the condition of a full-space support set, the original problem can be reduced to a second-order cone programming (SOCP) problem. Subsequently, we solve this second-order cone programming problem using a projection differential equation approach. Compared with the traditional methods, the differential equation method offers advantages in providing continuous and smooth solutions, offering inherent stability analysis, and possessing a rich mathematical toolbox, which make the differential equation a powerful and versatile tool for addressing complex optimization challenges.

Robust concave utility maximization over chance constraints

This paper first studies an expected utility problem with chance constraints and incomplete information on a decision maker’s utility function. The model maximizes the worst-case expected utility of random outcome over a set of concave functions within a novel ambiguity set, while the underlying probability distribution is known with the assumption of a discretization of possible realizations for the uncertainty parameter. To obtain computationally tractable formulations, we employ a discretization approach to derive a max–min chance-constrained approximation of this problem. This approximation is further reformulated as a mixed-integer program. We show that the discrete approximation converges to the true counterpart under mild assumptions. We also present a row generation algorithm for optimizing the max–min program. A computational study for a bin-packing problem and a multi-item newsvendor problem is conducted to demonstrate the benefit of the proposed framework and the computational efficiency of our algorithm. We find that the row generation algorithm can significantly reduce the computational time, and our robust policy could achieve a better out-of-sample performance when compared with the non-robust policy and the one without the chance constraints.

Back analysis of geomechanical parameters based on a data augmentation algorithm and machine learning technique

Accurate geomechanical parameters are key factors for stability evaluation, disaster forecasting, structural design, and supporting optimization. The intelligent back analysis method based on the monitored information is widely recognized as the most efficient and cost-effective technique for inverting parameters. To address the low accuracy of measured data, and the scarcity of comprehensive datasets, this study proposes an innovative back analysis framework tailored for small sample sizes. We introduce a multi-faceted back analysis approach that combines data augmentation with advanced optimization and machine learning techniques. The auxiliary classifier generative adversarial network (ACGAN)-based data augmentation algorithm is first employed to generate synthetic yet realistic samples that adhere to the underlying probability distribution of the original data, thereby expanding the dataset and mitigating the impact of small sample sizes. Subsequently, we harness the power of optimized particle swarm optimization (OPSO) integrated with support vector machine (SVM) to mine the intricate nonlinear relationships between input and output variables. Then, relying on a case study, the validity of the augmented data and the performance of the developed OPSO-SVM algorithms based on two different sample sizes are studied. Results show that the new datasets generated by ACGAN almost coincide with the actual monitored convergences, exhibiting a correlation coefficient exceeding 0.86. Furthermore, the superiority of the OPSO-SVM algorithm is also demonstrated by comparing the displacement prediction capability of various algorithms through four indices. It is also indicated that the relative error of the predicted displacement values reduces from almost 20% to 5% for the OPSO-SVM model trained with 25 samples and that trained with 625 samples. Finally, the inversed parameters and corresponding convergences predicted by the two OPSO-SVM models trained with different samples are discussed, indicating the feasibility of the combination application of ACGAN and OPSO-SVM in back analysis of geomechanical parameters. This endeavor not only facilitates the progression of underground engineering analysis in scenarios with limited data, but also serves as a pivotal reference for both researchers and practitioners alike.

Discriminative Deep Canonical Correlation Analysis for Multi-View Data.

Over the past few years, multimodal data analysis has emerged as an inevitable method for identifying sample categories. In the multi-view data classification problem, it is expected that the joint representation should include the supervised information of sample categories so that the similarity in the latent space implies the similarity in the corresponding concepts. Since each view has different statistical properties, the joint representation should be able to encapsulate the underlying nonlinear data distribution of the given observations. Another important aspect is the coherent knowledge of the multiple views. It is required that the learning objective of the multi-view model efficiently captures the nonlinear correlated structures across different modalities. In this context, this article introduces a novel architecture, termed discriminative deep canonical correlation analysis (D2CCA), for classifying given observations into multiple categories. The learning objective of the proposed architecture includes the merits of generative models to identify the underlying probability distribution of the given observations. In order to improve the discriminative ability of the proposed architecture, the supervised information is incorporated into the learning objective of the proposed model. It also enables the architecture to serve as both a feature extractor as well as a classifier. The theory of CCA is integrated with the objective function so that the joint representation of the multi-view data is learned from maximally correlated subspaces. The proposed framework is consolidated with corresponding convergence analysis. The efficacy of the proposed architecture is studied on different domains of applications, namely, object recognition, document classification, multilingual categorization, face recognition, and cancer subtype identification with reference to several state-of-the-art methods.

Histopathologic Differential Diagnosis and Estrogen Receptor/Progesterone Receptor Immunohistochemical Evaluation of Breast Carcinoma Using a Deep Learning–Based Artificial Intelligence Architecture

In breast carcinoma, invasive ductal carcinoma (IDC) is the most common histopathological subtype, and ductal carcinoma in situ (DCIS) is a precursor of IDC. They are often concomitant. The immunohistochemical staining of estrogen receptor (ER)/progesterone receptor (PR) in IDC/DCIS on whole-slide histopathological images (WSIs) can predict the prognosis of patients. However, the inter-observer variability among pathologists in reading WSIs is inevitable. Thus, artificial intelligence (AI) technology is crucial. Herein, IDC/DCIS detection was conducted by deep learning approach, including Faster R-CNN, RetinaNet, SSD300, YOLOv3, YOLOv5, YOLOv7, YOLOv8, and Swin transformer. Their performance was estimated by mean average precision (mAP) values. Cell recognition and counting were performed using AI technology to evaluate the intensity and proportion of ER/PR-immunostained cancer cells in IDC/DCIS. A three-round ring study (RS) was conducted to assess WSIs. A database for modelling the underlying probability distribution of a dataset with labels was established. YOLOv8 exhibits the highest detection performance with an mAP@0.5 of 0.944 and an mAP@0.5-0.95 of 0.790. With the assistance of YOLOv8, the scoring concordance across all pathologists was boosted to excellent in RS3 (0.970) from moderate in RS1 (0.724) and good in RS2 (0.812). Deep learning detection can be applied in clinicopathological field. To facilitate the histopathological diagnosis of IDC/DCIS and immunostaining scoring of ER/PR, a novel AI architecture and well-organized dataset were developed.

On the stochastic significance of peaks in the least-squares wavelet spectrogram and an application in GNSS time series analysis

In this paper, the mathematical derivation of the underlying probability distribution function for the normalized least-squares wavelet spectrogram is presented. The impact of empirical and statistical weights on the estimation of the spectral peaks and their significance are demonstrated from the statistical point of view both theoretically and practically. The simulation results show an improvement of approximately 0.02mm (RMSE) for annual signal estimation when statistical weights are considered in the least-squares wavelet analysis (LSWA). The weighted LSWA estimates the signals more accurately than the ordinary LSWA for different percentage amount of missing data. As a real-world application, Global Navigation Satellite Systems (GNSS) time series for a station in Rome, Italy are analyzed. The analyses of the GNSS time series provided by different agencies for the same station reveal statistically significant annual peaks, more significant in 2010 but less significant between 2018 and 2020, while the higher frequency components show different spectral patterns over time. A declining trend of approximately −0.42 mm/year since 2004 is estimated for the GNSS height time series, likely due to gradual land subsidence. The results not only highlight the advantages of LSWA but can also help to better understand the uncertainties involved in signal estimation.

Maximum likelihood estimation of probabilistically described loads in beam structures

In recent years, focus has been shifted towards predictive maintenance in an effort to improve the reliability of operating structures. Processing structural response data obtained from in-situ sensors during operation can provide added value towards this direction. Structural Health Monitoring (SHM) methods are uniquely suited for this task; however, accounting for the effect of stochastic structural loads is critical for their robustness. In this work, a framework based on Maximum Likelihood Estimation (MLE) is presented, whose goal is to obtain inferences on typically unobservable quantities that describe stochastic structural loading. A structural beam is employed as a demonstrative case study, that is subjected to point loads with stochastic magnitude and application points. The hyperparameters that govern their underlying probability distribution functions (pdf) are the quantities of inferential interest. The inverse (load) identification process is performed using a marginalized MLE objective, where stochastic Monte Carlo (MC) integration is employed to perform the marginalization and Genetic Algorithms (GAs) are used as the optimizer. The Cramer–Rao (CR) lower bound is used to produce 95 % Confidence Intervals (CIs) to quantify estimation uncertainty.

Elective Surgery Sequencing and Scheduling Under Uncertainty

Problem definition: We consider a surgery sequencing and scheduling problem with uncertain durations of surgeries in the context of an operating theater. From real data collected from a hospital, we observe the common practice, namely, “to follow,” in which surgeries are conducted sequentially and immediately one after another, according to a specific schedule. Methodology/results: Based on this practice, we propose a mathematical framework to balance the risk of delay and idling using the punctuality index, which takes into consideration both the probability and intensity of delay and idle time. We develop a computationally efficient procedure based on Benders decomposition to derive exact solutions for the problem. The scheduling problem is solvable in polynomial time when the sequence is given. The framework can also accommodate a robust setting when the underlying probability distribution is not fully available. For practical use, we propose two effective heuristics for sequencing decisions by approximating the model. Using real data, we demonstrate that our framework is significantly better than the risk-neutral and probability-maximizing approaches in both performance and computational efficiency. Moreover, the robust setting can effectively lessen the risk of extremely long delay and idle time, and the heuristics are efficient with only a small sacrifice of performance. Managerial implications: With the to-follow policy, our sequencing and scheduling model describes the actual practice better. The two heuristics can be applied easily and directly to help managers of operating theaters to make decisions. Funding: J. Qi is supported by the Hong Kong Research Grants Council [Grants 26202016, 16212419, and 16204521]. Supplemental Material: The online appendix is available at https://doi.org/10.1287/msom.2022.0029 .

Fair Sampling in Diffusion Models through Switching Mechanism

Diffusion models have shown their effectiveness in generation tasks by well-approximating the underlying probability distribution. However, diffusion models are known to suffer from an amplified inherent bias from the training data in terms of fairness. While the sampling process of diffusion models can be controlled by conditional guidance, previous works have attempted to find empirical guidance to achieve quantitative fairness. To address this limitation, we propose a fairness-aware sampling method called \textit{attribute switching} mechanism for diffusion models. Without additional training, the proposed sampling can obfuscate sensitive attributes in generated data without relying on classifiers. We mathematically prove and experimentally demonstrate the effectiveness of the proposed method on two key aspects: (i) the generation of fair data and (ii) the preservation of the utility of the generated data.

Tropical Modeling of Battery Swapping and Charging Station

We propose and investigate a queueing model of a battery swapping and charging station (BSCS) for electric vehicles (EVs). A new approach to the analysis of the queueing model is developed, which combines the representation of the model as a stochastic dynamic system with the use of the methods and results of tropical algebra, which deals with the theory and applications of algebraic systems with idempotent operations. We describe the dynamics of the queueing model by a system of recurrence equations that involve random variables (RVs) to represent the interarrival time of incoming EVs. A performance measure for the model is defined as the mean operation cycle time of the station. Furthermore, the system of equations is represented in terms of the tropical algebra in vector form as an implicit linear state dynamic equation. The performance measure takes on the meaning of the mean growth rate of the state vector (the Lyapunov exponent) of the dynamic system. By applying a solution technique of vector equations in tropical algebra, the implicit equation is transformed into an explicit one with a state transition matrix with random entries. The evaluation of the Lyapunov exponent reduces to finding the limit of the expected value of norms of tropical matrix products. This limit is then obtained using results from the tropical spectral theory of deterministic and random matrices. With this approach, we derive a new exact formula for the mean cycle time of the BSCS, which is given in terms of the expected value of the RVs involved. We present the results of the Monte Carlo simulation of the BSCS’s operation, which show a good agreement with the exact solution. The application of the obtained solution to evaluate the performance of one BSCS and to find the optimal distribution of battery packs between stations in a network of BSCSs is discussed. The solution may be of interest in the case when the details of the underlying probability distributions are difficult to determine and, thus, serves to complement and supplement other modeling techniques with the need to fix a distribution.

Fog, Friction, and Failure in Organized Conflict: A Formal Study

Organized conflict, while confined by the laws of physics—and, under profound strategic incompetence, by the Lanchester equations—is not a physical process but rather an extended exchange between cognitive entities that have been shaped by path-dependent historical trajectories and cultural traditions. Cognition itself is confined by the necessity of duality, with an underlying information source constrained by the asymptotic limit theorems of information and control theories. We introduce the concept of a ‘basic underlying probability distribution’ characteristic of the particular cognitive process studied. The dynamic behavior of such systems is profoundly different for ‘thin-tailed’ and ‘fat-tailed’ distributions. The perspective permits the construction of new probability models that may provide useful statistical tools for the analysis of observational and experimental data associated with organized conflict, and, in some measure, for its management.

A parsimonious, computationally efficient machine learning method for spatial regression

AbstractWe introduce the modified planar rotator method (MPRS), a physically inspired machine learning method for spatial/temporal regression. MPRS is a non-parametric model which incorporates spatial or temporal correlations via short-range, distance-dependent “interactions” without assuming a specific form for the underlying probability distribution. Predictions are obtained by means of a fully autonomous learning algorithm which employs equilibrium conditional Monte Carlo simulations. MPRS is able to handle scattered data and arbitrary spatial dimensions. We report tests on various synthetic and real-word data in one, two and three dimensions which demonstrate that the MPRS prediction performance (without hyperparameter tuning) is competitive with standard interpolation methods such as ordinary kriging and inverse distance weighting. MPRS is a particularly effective gap-filling method for rough and non-Gaussian data (e.g., daily precipitation time series). MPRS shows superior computational efficiency and scalability for large samples. Massive datasets involving millions of nodes can be processed in a few seconds on a standard personal computer. We also present evidence that MPRS, by avoiding the Gaussian assumption, provides more reliable prediction intervals than kriging for highly skewed distributions.

Maximum entropy-based method for extracting the underlying probability distributions of z-number

Maximum entropy-based method for extracting the underlying probability distributions of z-number

Bayesian inference for parameter identification in mechanistic models, exemplified using a cow lifetime performance model

Mechanistic models are valuable tools for studying the underlying mechanisms of complex biological phenomena. For example, cow lifespan models can be used to identify differences in resource acquisition and allocation strategies between individuals, which is relevant for decision-making in breeding programs. In such models, differences in simulated traits between individuals are consequences of the parameter set that represents the genetic potential of each animal and its interaction with the environment. This indicates that the identification of these differences is essentially a search for individual parameters. In mechanistic models, this search is generally a non-convex problem that has different local minima because the parameters interact within these models. Due to this and to the simulation time length (e.g. years), there is uncertainty associated with the inference of the parameter values for each individual. This uncertainty can be quantified using Bayesian inference since this approach treats the model parameters as random variables with an underlying probability distribution that describes them. The objective of this work was to employ the Delayed Rejection Adaptive Metropolis (DRAM) algorithm to identify the parameters of a cows’ lifespan model using two datasets of Holstein cows. The datasets contain periodic measurements of Milk Yield (MY), BW, and Body Condition Score (BCS). Additionally, one of the two datasets has information of BW from birth to first calving. The average Mean Absolute Percentage Error (MAPE) minimisation between the simulated and experimental data (MY, BW and BCS) was used as the objective function for parameter search. The Bayesian inference performance was compared with four optimisation metaheuristic approaches: Differential Evolution, Genetic Algorithm, Particle Swarm Optimisation, and Simulated Annealing. Although the results show that all methods are efficient in finding parameter values that reduce the distance between the simulated and experimental data (MAPE < 10%), the DRAM method is more efficient in terms of computational cost, and the parameter distributions obtained with this method offer more information about the statistical properties of each parameter (e.g. median).

The pseudo-information entropy of Z-number and its applications in multi-attribute decision-making

In order to find a reasonable and effective approach to describe the information contained in a specific Z-number, we introduce information entropy into Z-number environment in this paper, and investigate its applications in multi-attribute decision-making (MADM) issues. Moreover, aiming at the problem with the weighting indicated by Z-number values, we propose two novel weighting methodologies based on conditional entropy and sigmoid function, respectively. Firstly, on the basis of the maximum entropy principle, the optimization model to calculate the underlying probability distribution of Z-number is introduced. And then, we define Z-number pseudo-information entropy, and a novel Z-VIKOR method is proposed to solve a selecting regional circular economy development plan issue from the perspective of information entropy. Furthermore, we propose Z-number pseudo-conditional entropy, and the relationship between Z-number pseudo-information entropy and Z-number pseudo-conditional entropy is investigated. Subsequently, a weighting method based on information entropy of Z-number is proposed. In addition, we also introduce the weighting approach based on the sigmoid function decision-making method. Finally, we introduce a government new energy investment problem to verify and compare the effectiveness of the new weighting approaches. The new method gives a solution to the problem related to Z-number from the perspective of information entropy.

A generating-function approach to modelling complex contagion on clustered networks with multi-type branching processes

Abstract Understanding cascading processes on complex network topologies is paramount for modelling how diseases, information, fake news and other media spread. In this article, we extend the multi-type branching process method developed in Keating et al., (2022), which relies on networks having homogenous node properties, to a more general class of clustered networks. Using a model of socially inspired complex contagion we obtain results, not just for the average behaviour of the cascades but for full distributions of the cascade properties. We introduce a new method for the inversion of probability generating functions to recover their underlying probability distributions; this derivation naturally extends to higher dimensions. This inversion technique is used along with the multi-type branching process to obtain univariate and bivariate distributions of cascade properties. Finally, using clique-cover methods, we apply the methodology to synthetic and real-world networks and compare the theoretical distribution of cascade sizes with the results of extensive numerical simulations.

Diversity Subsampling: Custom Subsamples from Large Data Sets

Subsampling from a large unlabeled (i.e., no response values are available yet) data set is useful in many supervised learning contexts to provide a global view of the data based on only a fraction of the observations. In this paper, we borrow concepts from the well-known sampling/importance resampling technique, which samples from a specified probability distribution, to develop a diversity subsampling approach that selects a subsample from the original data with no prior knowledge of its underlying probability distribution. The goal is to produce a subsample that is independently and uniformly distributed over the support of distribution from which the data are drawn, to the maximum extent possible. We give an asymptotic performance guarantee of the proposed method and provide experimental results to show that the proposed method performs well for typical finite-size data. We also compare the proposed method with competing diversity subsampling algorithms and demonstrate numerically that subsamples selected by the proposed method are closer to a uniform sample than subsamples selected by other methods. The proposed diversity subsampling (DS) algorithm is more efficient than known methods. It takes only a few minutes to select tens of thousands of subsample points from a data set of size one million. Our DS algorithm easily generalizes to select subsamples following distributions other than uniform. We provide a Python package (FADS) that implements the proposed method. History: Kwok-Leung Tsui served as the senior editor for this article. Funding: This work was supported by the National Science Foundation [Grant CMMI-1436574], Northwestern University, the Advanced Research Projects Agency-Energy, and the U.S. Department of Energy [Award DE-AR0001209]. Data Ethics &amp; Reproducibility Note: No data ethics considerations are foreseen related to this article. The code capsule is available on Code Ocean at https://doi.org/10.24433/CO.8309237.v3 and in the e-Companion to this article (available at https://doi.org/10.1287/ijds.2022.00017 ).

Data-driven distributionally robust optimization for long-term contract vs. spot allocation decisions: Application to electricity markets

There are numerous industrial settings in which a decision maker must decide whether to enter into long-term contracts to guarantee price (and hence cash flow) stability or to participate in more volatile spot markets. In this paper, we investigate a data-driven distributionally robust optimization (DRO) approach aimed at balancing this tradeoff. Unlike traditional risk-neutral stochastic optimization models that assume the underlying probability distribution generating the data is known, DRO models assume the distribution belongs to a family of possible distributions, thus providing a degree of immunization against unseen and potential worst-case outcomes. We compare and contrast the performance of a risk-neutral model, conditional value-at-risk formulation, and a Wasserstein distributionally robust model to demonstrate the potential benefits of a DRO approach for an “elasticity-aware” price-taking decision maker.

Quantum Quantile Mechanics: Solving Stochastic Differential Equations for Generating Time‐Series

AbstractA quantum algorithm is proposed for sampling from a solution of stochastic differential equations (SDEs). Using differentiable quantum circuits (DQCs) with a feature map encoding of latent variables, the quantile function is represented for an underlying probability distribution and samples extracted as DQC expectation values. Using quantile mechanics the system is propagated in time, thereby allowing for time‐series generation. The method is tested by simulating the Ornstein‐Uhlenbeck process and sampling at times different from the initial point, as required in financial analysis and dataset augmentation. Additionally, continuous quantum generative adversarial networks (qGANs) are analyzed, and the authors show that they represent quantile functions with a modified (reordered) shape that impedes their efficient time‐propagation. The results shed light on the connection between quantum quantile mechanics (QQM) and qGANs for SDE‐based distributions, and point the importance of differential constraints for model training, analogously with the recent success of physics informed neural networks.

Simplifying Complexity: Scenario Reduction Techniques in Stochastic Programming

Stochastic programming problems arise as mathematical models for optimizing problems under stochastic uncertainty. Computational approaches for solving these models often involve approximating the underlying probability distribution with a probability measure that has finite support. To mitigate the computational complexity associated with increasing the number of scenarios, it may be necessary to reduce their quantity. The scenario is selected as the first element of supp , and the separable structure is used to determine the second element of supp while keeping the first element fixed. The process is repeated to establish the remaining indices, and each subsequent scenario is reduced accordingly. This iterative process continues until scenario is reduced