Input Probability Distribution Research Articles

Adaptive temporal transformer method for short-term wind power forecasting considering shift in time series distribution

In wind power prediction, the input probability distributions in the different sub-periods are shifted owing to the strong randomness of the input features, such as wind speed and direction. This may violate the assumption for machine learning that the training and test data meet the condition of being independent and identically distributed, resulting in an insufficient generalization ability of the prediction model that is trained with the training data and applied to unknown test data. To address this problem, this study proposes an adaptive temporal transformer method for short-term wind power forecasting. First, a temporal transformer model with a gate recurrent unit and multi-head attention layers was used to extract the short- and long-term temporal information of the multiple input variables. Then, an adaptive learning strategy consisting of two stages—temporal distribution characterization and temporal distribution matching—was developed to explore the common knowledge hidden in each sub-period. The case results for an actual wind farm in northwest China showed that the proposed method could effectively weaken the adverse effects of the shifts in time series distribution on forecasting and improve the accuracy and generalization of short-term wind power prediction.

On the External Validity of Average-case Analyses of Graph Algorithms

The number one criticism of average-case analysis is that we do not actually know the probability distribution of real-world inputs. Thus, analyzing an algorithm on some random model has no implications for practical performance. At its core, this criticism doubts the existence of external validity ; i.e., it assumes that algorithmic behavior on the somewhat simple and clean models does not translate beyond the models to practical performance real-world input. With this article, we provide a first step toward studying the question of external validity systematically. To this end, we evaluate the performance of six graph algorithms on a collection of 2,740 sparse real-world networks depending on two properties: heterogeneity (variance in the degree distribution) and locality (tendency of edges to connect vertices that are already close). We compare this with the performance on generated networks with varying locality and heterogeneity. We find that the performance in the idealized setting of network models translates surprisingly well to real-world networks. Moreover, heterogeneity and locality appear to be the core properties impacting the performance of many graph algorithms.

Practical use of data lake for better fatigue life estimation

The qualification of components in terms of Durability and Reliability is based on the analysis on a lot of sensors, CAN, IIOT data, which needs a data management infrastructure, to understand the customer usage and variability. Such big data infrastructure is often called « data lake », and may lead to storing huge amount of data. This infrastructure must be generic yet test data-oriented to understand the data structure and its analysis required, and to be optimized for such application. All those data may come from connected equipment, instrumented fleets, test lab or proving ground measurements, digital twins and multi-body dynamics simulations. Further, the data must be managed in terms of quality and traceability. It must be indexed to be able to be retrieved through searches (customer, vehicle, measurement site, engine specification, road condition, usage conditions).Once this step is achieved, the product development team is able to have a better understanding of customer usage and inputs variabilities in different environments and conditions, which have to be taken into account in the product's mission profile or duty cycle. This ad-hoc mission profile enables the creation of realistic, meaningful design and test specifications.Understanding the customer usage and input variabilities enables a probabilistic approach to fatigue life prediction. The uncertainties on inputs (geometry, material and loading) may be propagated through the life process, knowing each input's probability distribution function, using a Monte Carlo analysis. The infrastructure enables the life analysis to be done through multiple runs on cloud-oriented server, which enables automation and streamlining the whole process. A use case will be presented to illustrate the approach and its benefits.

Implementation of process optimization to maximize line productivity in pharmaceutical industries

Macro Group is the largest player in Egypt’s fast-growing cosmeceutical industry founded in 2002, and Macro has about 149 stock-keeping units (“SKUs”) with a 31.4% market share in 2022. The macro group has three-ampoule products for hair, namely, Atrakta, Verdex, and Solodex with an expected forecast of 134,655 FG in 2023 and it’s expected to reach 261,000 FG in 2027. 188 working days per year in 2023 are required to manufacture the forecasted units and 365 working days to manufacture 2027 forecasted units. Through observing the production line of Atrakta, it has been noticed that the production line could be enhanced to increase the production rate, and hence, meet market demands efficiently. Therefore, this paper aims to study and investigate the Atrakta hair ampoules production line to reach the optimum line productivity and to commit to market needs. Studying the Atrakta hair ampules production line is conducted through interviews, production line observation, data collection, and modeling and simulation of the production line. Two scenarios recommending improvements to the production lines are proposed and simulated. The two scenarios differ mainly in the probability distribution of probabilistic inputs. Simulating the system with the recommended improvements for the two scenarios shows similar results of increased productivity. Simulation results of improved systems show an increase of 300% in daily production from 720 FG per day to 2176 FG per day. According to Atrakta's results, the Macro group decided to implement the same recommended improvements on the similar process of Verdex and Solodex ampoules to reach 2176 FG. The required working days to reach forecasted units in 2023 decreased from 188 to 67 days and also decreased from 365 to 132 days in 2027.ArticleHighlightsThe primary pack items are decreased, and secondary pack of ampoules SKUs has been changed to avoid bottlenecks.The change of primary and secondary pack plays a significant role in increasing the productivity 3 times for the non-sterile ampoules production unit.The increasing of production capacity decreased the required working days in 2023 till 2027.

REIN: Reliability Estimation via Importance sampling with Normalizing flows

We introduce a novel framework called REIN: Reliability Estimation by learning an Importance sampling (IS) distribution with Normalizing flows (NFs). The NFs learn probability space maps that transform the probability distribution of the input random variables into a quasi-optimal IS distribution. NFs stack together invertible neural networks to construct differentiable bijections with efficiently computed Jacobian determinants. The NF ‘pushes forward’ a realization from the input probability distribution into a realization from the IS distribution, with importance weights calculated using the change of variables formula. We also propose a loss function to learn a NF map that minimizes the reverse Kullback–Leibler divergence between the ‘pushforward’ distribution and a sequentially updated target distribution obtained by modifying the optimal IS distribution. We demonstrate REIN’s efficacy on a set of benchmark problems that feature very low failure rates, multiple failure modes and high dimensionality, while comparing against other variance reduction methods. We also consider two simple applications, the reliability analyses of a thirty-four story building and a cantilever tube, to demonstrate the applicability of REIN to practical problems of interest. As compared to other methods, REIN is shown to be useful for high-dimensional reliability estimation problems with very small failure probabilities.

Verified propagation of imprecise probabilities in non-linear ODEs

We combine reachability analysis and probability bounds analysis, which allow for imprecisely known random variables (multivariate intervals or p-boxes) to be specified as the initial states of a dynamical system. In combination, the methods allow for the temporal evolution of p-boxes to be rigorously computed, and they give interval probabilities for formal verification problems, also called failure probability calculations in reliability analysis. The methodology places no constraints on the input probability distribution or p-box and can handle dependencies generally in the form of copulas. We also provide a consonant approximation method for multivariate p-boxes, which allows for the prediction sets of dynamical systems to be efficiently computed. The presented methodology is rigorous and automatically verified, as both the dynamics and uncertainties are represented and solved with guaranteed enclosures.

Central Limit Theorem-based Stochastic Economic Evaluation (CLT-SEE) Model for Evaluating Oil Wells: Case Study from Niger Delta, Nigeria

Oil and natural gas production has been highly contributing to world economy and is some countries’ economy root. After the discovery of a new oil/gas field, the operator has to decide whether or not to develop that field. Such decisions rely on the economic evaluation of potential oil/gas fields development when they will be discovered and of the proven oil/gas reserves. The economic indicators used for that purpose are actually computed with deterministic and/or stochastic methods. Deterministic models show limitations while stochastic ones reduce the risks and doubts in the decision making. Stochastic models require the knowledge of the probability distribution of the model inputs, what is costeous in terms of software, data and conditions to be satisfied. Our study proposes a technique, called “Central Limit Theorem-based Stochastic Economic Evaluation (CLT-SEE) Model’’ that eases projects NPV probability distribution determination and the computation of P10, P50 and P90 of projects NPV, IRR and PI. A case study is carried out on a Nigerian’s Niger Delta onshore oil well. The results show the well NPV, IRR and PI are respectively MM$ 84.112, 24.5%, 1.169. The well project P10(NPV), P50(NPV) and P90(NPV) are respectively MM$ 96.4, MM$ 84.16 and MM$ 71.89; P10(IRR), P50(IRR) and P90(IRR) are respectively 27%, 24.75% and 22%; P10(PI), P50(PI) and P90(PI) are respectively 1.34, 1.17 and 1. These stochastic outputs show that the company has 90% of chance to earn at least MM$ 71.89 which is its investment and the likelihood that the project IRR be more than 22% is 0.9. As a result, the use of CLT-SEE model for oil wells economic evaluation offers much more chance and confidence to oil companies to decide righteously in field and well development projects.

Assessing techno-economic uncertainties in nuclear power-to-X processes: The case of nuclear hydrogen production via water electrolysis

Nuclear assisted low carbon hydrogen production by water electrolysis represents a potential application of nuclear cogeneration towards deep decarbonization of several fossil fuel-dependent industrial sectors. This work builds a probabilistic techno-commercial model of a water electrolysis plant coupled to an existing nuclear reactor for base load operations. The objective is to perform discounted cash flow (DCF) calculations for levelized nuclear hydrogen production cost under input parameter uncertainty. The probability distributions of inputs are used with the Monte Carlo-Latin Hypercube (MC-LH) sampling technique to generate 105 input scenarios and corresponding distribution of the levelized or life cycle hydrogen production cost instead of deterministic point values. Based on current techno-economic conditions, the levelized production costs of electrolytic hydrogen using electricity from large water-cooled nuclear reactors are determined to be US $ 12.205 ± 1.342, 8.384 ± 1.148 and 6.385 ± 1.051/kg H2 respectively at rated alkaline water electrolyser capacities of 1.25 MW(e), 2.5 MW(e) and 5 MW(e). The corresponding values for PEM water electrolysers are US $ 13.162 ± 1.356, 8.891 ± 1.141 and 6.663 ± 1.057/kg H2. The potential for flexible nuclear reactor operation and management of power demand uncertainties through nuclear hydrogen cogeneration is also examined through a case study.

Managing Geotechnical Uncertainty With Simulation Models: An Introduction

In a standard deterministic analysis discrete scenarios are considered, and a moderately conservative “characteristic” value is used as a design basis. However, fixed or exact values in a real-world geotechnical site seldom occurs. Deterministic approaches may not explicitly consider the ground uncertainty. Simulations using various probabilities provides for this uncertainty as each parameter input is treated as a random variable within certain measured ranges or ability to evaluate. Monte Carlo (MC) sampling is a traditional technique for generating random numbers to sample from a probability distribution. When low probability events occur, a small number of MC iterations might not sample sufficient quantities of these outcomes for inclusion in the simulation model. Latin Hypercube (LH) sampling uses stratification of the input probability distributions, to overcome the limitations of Monte Carlo sampling. The simulation results show low probability outcomes are included in the sampling for the simulation model. At a high number of simulation iterations both provide similar outputs, but at low simulation iterations the LH is more reliable. However, both the MC and LH sampling suffer from impractical values at low or high probability events when the normal probability density function (PDF) is adopted. The normal PDF is commonly used in statistical modelling. Non-normal PDFs often represent the best fit PDF when a goodness of fit test is carried out. The errors associated with using the common normal PDF are shown with the above-mentioned simulation models. This best fit PDF applies whether simulation models as described above is used or even with simple “what if” sensitivity models in traditional analysis.

Digitally predicting protein localization and manipulating protein activity in fluorescence images using 4D reslicing GAN.

While multi-channel fluorescence microscopy is a vital imaging method in biological studies, the number of channels that can be imaged simultaneously is limited by technical and hardware limitations such as emission spectra cross-talk. One solution is using deep neural networks to model the localization relationship between two proteins so that the localization of one protein can be digitally predicted. Furthermore, the input and predicted localization implicitly reflect the modeled relationship. Accordingly, observing the response of the prediction via manipulating input localization could provide an informative way to analyze the modeled relationships between the input and the predicted proteins. We propose a protein localization prediction (PLP) method using a cGAN named 4D Reslicing Generative Adversarial Network (4DR-GAN) to digitally generate additional channels. 4DR-GAN models the joint probability distribution of input and output proteins by simultaneously incorporating the protein localization signals in four dimensions including space and time. Because protein localization often correlates with protein activation state, based on accurate PLP, we further propose two novel tools: digital activation (DA) and digital inactivation (DI) to digitally activate and inactivate a protein, in order to observing the response of the predicted protein localization. Compared with genetic approaches, these tools allow precise spatial and temporal control. A comprehensive experiment on six pairs of proteins shows that 4DR-GAN achieves higher-quality PLP than Pix2Pix, and the DA and DI responses are consistent with the known protein functions. The proposed PLP method helps simultaneously visualize additional proteins, and the developed DA and DI tools provide guidance to study localization-based protein functions. The open-source code is available at https://github.com/YangJiaoUSA/4DR-GAN. Supplementary data are available at Bioinformatics online.

Generalized polynomial chaos based surrogate models for acoustics and vibrations

This work explores generalized polynomial chaos (GPC) surrogate models and their effectiveness for general acoustics and vibration applications. GPC is primarily known as an uncertainty quantification (UQ) technique and in that context the underlying polynomial-based model has been shown to be effective in mapping input probability distributions to the corresponding output probability distributions. In this study, GPC surrogate models – including those generated by both quadrature and regression methods – are evaluated for their effectiveness in non-UQ focused analyses. As points of comparison, Krylov subspace and other more traditional reduced order modeling techniques are demonstrated and compared to GPC models so that the differences may be better understood. Example problems with several different levels of complexity are used to show how the computational burden as well as the overall effectiveness of the method changes as the number of input variables that must be considered grows.

TCMI: a non-parametric mutual-dependence estimator for multivariate continuous distributions

The identification of relevant features, i.e., the driving variables that determine a process or the properties of a system, is an essential part of the analysis of data sets with a large number of variables. A mathematical rigorous approach to quantifying the relevance of these features is mutual information. Mutual information determines the relevance of features in terms of their joint mutual dependence to the property of interest. However, mutual information requires as input probability distributions, which cannot be reliably estimated from continuous distributions such as physical quantities like lengths or energies. Here, we introduce total cumulative mutual information (TCMI), a measure of the relevance of mutual dependences that extends mutual information to random variables of continuous distribution based on cumulative probability distributions. TCMI is a non-parametric, robust, and deterministic measure that facilitates comparisons and rankings between feature sets with different cardinality. The ranking induced by TCMI allows for feature selection, i.e., the identification of variable sets that are nonlinear statistically related to a property of interest, taking into account the number of data samples as well as the cardinality of the set of variables. We evaluate the performance of our measure with simulated data, compare its performance with similar multivariate-dependence measures, and demonstrate the effectiveness of our feature-selection method on a set of standard data sets and a typical scenario in materials science.

Second‐level global sensitivity analysis of numerical simulators with application to an accident scenario in a sodium‐cooled fast reactor

AbstractNumerical simulators are widely used to model physical phenomena and global sensitivity analysis (GSA) aims at studying the global impact of the input uncertainties on the simulator output. To perform GSA, statistical tools based on inputs/output dependence measures are commonly used. We focus here on the Hilbert–Schmidt independence criterion (HSIC). Sometimes, the probability distributions modeling the uncertainty of inputs may be themselves uncertain and it is important to quantify their impact on GSA results. We call it here the second‐level global sensitivity analysis (GSA2). However, GSA2, when performed with a Monte Carlo double‐loop, requires a large number of model evaluations, which is intractable with CPU time expensive simulators. To cope with this limitation, we propose a new statistical methodology based on a Monte Carlo single‐loop with a limited calculation budget. First, we build a unique sample of inputs and simulator outputs, from a well‐chosen probability distribution of inputs. From this sample, we perform GSA for various assumed probability distributions of inputs by using weighted HSIC measures estimators. Statistical properties of these weighted estimators are demonstrated. Subsequently, we define 2nd‐level HSIC‐based measures between the distributions of inputs and GSA results, which constitute GSA2 indices. The efficiency of our GSA2 methodology is illustrated on an analytical example, thereby comparing several technical options. Finally, an application to a test case simulating a severe accidental scenario on nuclear reactor is provided.

Rayleigh Fading Channel Capacity for Coherent Signaling With Asymmetric Constellations

This paper investigates the channel capacity of coherent reception with antenna diversity in Rayleigh fading using asymmetric constellations, such as nonbinary two-sided amplitude-shift keying (ASK) and quadrature amplitude modulation (QAM) with ≥ 8 levels. Due to the asymmetries, the capacity achieving input or a priori probability distributions deviate from uniformity. To perform the investigation, an analytical expression for the mutual information, in terms of an integral over a single variable, between the input (transmitted symbol) and the output (received signal vector comprising signals at receive antennas), is derived. For two-sided ASK, 8-QAM with cross constellation, and 16-QAM, analytical closed form expressions for the high and low signal-to-noise (SNR) approximations, respectively, of the capacity achieving input probability distribution, the capacity, and the relative gain in mutual information (between using the uniform and the capacity achieving input probability distributions) are obtained. Numerical results show that at low SNRs, the use of the capacity achieving nonuniform input probability distribution offers significant advantage over the use of the uniform input probability distribution, while at high SNRs, this advantage is significant only for configurations with one receive antenna. Furthermore, the high and low SNR approximations are shown to be reasonably accurate.

Empirical approach for reliability evaluation of tunnel excavation stability using the Q rock mass classification system

The critical strain concept has been widely used in analytical or numerical approaches to evaluate the stability of underground excavations. Analytical, empirical, and numerical procedures are usually used to determine the critical strain values. This paper presents a reliability assessment procedure for evaluating excavation stability using the empirical approach based on the rock mass classification Q and the first order reliability method (FORM). In contrast to deterministic critical strain values, a probabilistic critical strain, which considers uncertainties in rock mass parameters, was incorporated in a limit state function for reliability analysis. Using the rock mass classification Q, the empirically estimated tunnel stain was included in the limit state function. The critical strain and estimated tunnel strain were probabilistically characterized based on the rock mass classification Q-derived rock mass properties. Monte Carlo simulations were also conducted for comparing the reliability analysis results with those derived from the FORM algorithm. A highway tunnel case study was used to demonstrate the reliability assessment procedure. The effects of the input ground parameter correlations, probability distributions, and coefficients of variation on tunnel reliability were investigated. Results show that uncorrelated and normally distributed input parameters (intact rock strength and elastic modulus) have generated more conservative reliability. The reliability analysis results also show that the tunnel had relatively high reliability (reliability index of 2.78 and probability of failure of 0.27%), indicating the tunnel is not expected to experience instability after excavation. The tunnel excavation stability was assessed using analytical and numerical approaches for comparison. The results were consistent with the reliability analysis using the FORM algorithm's Q-based empirical method.

A Data-Driven Uncertainty Quantification Method for Stochastic Economic Dispatch

This letter proposes a data-driven sparse polynomial chaos expansion-based surrogate model for the stochastic economic dispatch problem considering uncertainty from wind power. The proposed method can provide accurate estimations for the statistical information (e.g., mean, variance, probability density function, and cumulative distribution function) for the stochastic economic dispatch solution efficiently without requiring the probability distributions of random inputs. Simulation studies on an integrated electricity and gas system (IEEE 118-bus system integrated with a 20-node gas system) are presented, demonstrating the efficiency and accuracy of the proposed method compared to the Monte Carlo simulations.

Design Centering of Compact Microwave Components Using Response Features and Trust Regions

Fabrication tolerances, as well as uncertainties of other kinds, e.g., concerning material parameters or operating conditions, are detrimental to the performance of microwave circuits. Mitigating their impact requires accounting for possible parameter deviations already at the design stage. This involves optimization of appropriately defined statistical figures of merit such as yield. Although important, robust (or tolerance-aware) design is an intricate endeavor because manufacturing inaccuracies are normally described using probability distributions, and their quantification has to be based on statistical analysis. The major bottleneck here is high computational cost: for reliability reasons, miniaturized microwave components are evaluated using full-wave electromagnetic (EM) models, whereas conventionally utilized analysis methods (e.g., Monte Carlo simulation) are associated with massive circuit evaluations. A practical approach that allows for circumventing the aforementioned obstacles offers surrogate modeling techniques, which have been a dominant trend over the recent years. Notwithstanding, a construction of accurate metamodels may require considerable computational investments, especially for higher-dimensional cases. This paper brings in a novel design-centering approach, which assembles forward surrogates founded at the level of response features and trust-region framework for direct optimization of the system yield. Formulating the problem with the use of characteristic points of the system response alleviates the issues related to response nonlinearities. At the same time, as the surrogate is a linear regression model, a rapid yield estimation is possible through numerical integration of the input probability distributions. As a result, expenditures related to design centering equal merely few dozens of EM analyses. The introduced technique is demonstrated using three microstrip couplers. It is compared to recently reported techniques, and its reliability is corroborated using EM-based Monte Carlo analysis.

Channel Capacity of an Asymmetric Constellation in Rayleigh Fading With Noncoherent Energy Detection

The channel capacity of noncoherent reception of multi-level one-sided amplitude-shift keying (ASK), which is an asymmetric constellation, in Rayleigh fading with receive diversity and energy detection is considered. The asymmetries result in capacity achieving input probability distributions, that is, a priori probability distributions, that deviate from uniformity. An analytical expression for the mutual information in terms of a single integral is derived, and from it the set of equations, which can be solved to obtain the optimum or capacity achieving input probabilities, is obtained. High and low signal-to-noise ratio (SNR) approximations of the optimum input probabilities and the capacity are derived next. Furthermore, a logarithmic upper bound on the mutual information is obtained. Numerical results confirm that the uniform distribution of input probabilities is not capacity achieving. For example, with average SNR per symbol per branch of 6 dB, the relative deviation of the mutual information (with uniform input distribution) from the capacity is nearly 20% for 4-level ASK with one transmit diversity branch and two receive diversity branches. Furthermore, the derived high and low SNR approximations to the capacity are shown to be reasonably accurate.

Capacity-Achieving Input Distribution in Per-Sample Zero-Dispersion Model of Optical Fiber

The per-sample zero-dispersion channel model of the optical fiber is considered. It is shown that capacity is uniquely achieved by an input probability distribution that has continuous uniform phase and discrete amplitude that takes on finitely many values. This result holds when the channel is subject to general input cost constraints, that include a peak amplitude constraint and a joint average and peak amplitude constraint.

Improving neural network’s performance using Bayesian inference

Neural networks can be used as a data-driven model for system identification. But the probability properties of the training data are not included. The Bayesian approach can model the input and output probability distribution, but it cannot give points estimation. In this paper, we propose a special neural model, which combines the neural model and Bayesian inference. The probability distributions of input and output are obtained by the Bayesian approach. This statistical model changes the neural network’s structure and improve the accuracy of the neural modeling. We also propose a neural network training method using Bayesian inference. The approach capabilities are analyzed. We use three examples to compare our method with the other black box methods. The results show that this new model is much better, when there are large noises, and the dynamics of the system is complex.