Expected Performance Measure Research Articles

Stochastic Approximation for Multi-period Simulation Optimization with Streaming Input Data

We consider a continuous-valued simulation optimization (SO) problem, where a simulator is built to optimize an expected performance measure of a real-world system while parameters of the simulator are estimated from streaming data collected periodically from the system. At each period, a new batch of data is combined with the cumulative data and the parameters are re-estimated with higher precision. The system requires the decision variable to be selected in all periods. Therefore, it is sensible for the decision-maker to update the decision variable at each period by solving a more precise SO problem with the updated parameter estimate to reduce the performance loss with respect to the target system. We define this decision-making process as the multi-period SO problem and introduce a multi-period stochastic approximation (SA) framework that generates a sequence of solutions. Two algorithms are proposed: Re-start SA ( ReSA ) reinitializes the stepsize sequence in each period, whereas Warm-start SA ( WaSA ) carefully tunes the stepsizes, taking both fewer and shorter gradient-descent steps in later periods as parameter estimates become increasingly more precise. We show that under suitable strong convexity and regularity conditions, ReSA and WaSA achieve the best possible convergence rate in expected sub-optimality either when an unbiased or a simultaneous perturbation gradient estimator is employed, while WaSA accrues significantly lower computational cost as the number of periods increases. In addition, we present the regularized ReSA which obviates the need to know the strong convexity constant and achieves the same convergence rate at the expense of additional computation.

Queueing networks for supporting container storage and retrieval

PurposeAn integrated queueing network focused on container storage/retrieval operations occurring on the yard of a transshipment hub is proposed. The purpose of the network is to support decisions related to the organization of the yard area, while also accounting for operations policies and times on the quay.Design/methodology/approachA discrete-event simulation model is used to reproduce container handling on both the quay and yard areas, along with the transfer operations between the two. The resulting times, properly estimated by the simulation output, are fed to a simpler queueing network amenable to solution via algorithms based on mean value analysis (MVA) for product-form networks.FindingsNumerical results justify the proposed approach for getting a fast, yet accurate analytical solution that allows carrying out performance evaluation with respect to both organizational policies and operations management on the yard area.Practical implicationsPractically, the expected performance measures on the yard subsystem can be obtained avoiding additional time-expensive simulation experiments on the entire detailed model.Originality/valueAs a major takeaway, deepening the MVA for generally distributed service times has proven to produce reliable estimations on expected values for both user- and system-oriented performance metrics.

Modeling and Control of Probabilistic Fuzzy Discrete Event Systems

We investigate modeling and control of probabilistic fuzzy discrete event systems (PFDES). PFDES is a new type of fuzzy discrete event systems. It allows the use of probabilities to describe the chances of occurrences of different events. Our new model for PFDES consists of a fuzzy automaton and a crisp automaton that specifies what sequences of events can occur and their probabilities of occurrences. Based on the new model, optimal control is designed using an on-line and limited lookahead method. Control is calculated one step at a time, after an occurrence of an event. At each step, a lookahead window of $N$ events is constructed. The performance measures for all states in the window are determined, which is a function of fuzzy states. Control is calculated to maximize the expected performance measure after the occurrences of $N$ events. To reduce computational complexity, a “dynamic-programming” approach is proposed. We prove that the control obtained is optimal. Examples are given in the paper to illustrate the results.

A Neyman–Pearson problem with ambiguity and nonlinear pricing

We consider a problem of the Neyman–Pearson type arising in the theory of portfolio choice in the presence of probability weighting, such as in markets with Choquet pricing (as in Araujo et al. in Econ Theory 49(1):1–35, 2011; Cerreia-Vioglio et al. in J Econ Theory 157(1):730–762, 2015; Chateauneuf and Cornet in Submodular financial markets with frictions. Working Paper, 2015; Chateauneuf et al. in Math Finance 6(3):323–330, 1996) and ambiguous beliefs about the payoffs of contingent claims (see Gilboa and Marinacci, in: Acemoglu, Arellano, Dekel (eds) Advances in economics and econometrics: theory and applications, tenth world congress of the econometric society, Cambridge University Press, Cambridge, 2013). Specifically, we consider a problem of optimal choice of a contingent claim so as to minimize a non-linear pricing functional (or a distortion risk measure), subject to a minimum expected performance measure (or a minimum expected return or utility), where expectations with respect to distorted probabilities are taken in the sense of Choquet. Such contingent claims are called cost-efficient. We give an analytical characterization of cost-efficient contingent claims under very mild assumptions on the probability weighting functions, thereby extending some of the results of Ghossoub (Math Financ Econ 10(1):87–111, 2016), and we provide examples of some special cases of interest. In particular, we show how a cost-efficient contingent claim exhibits a desirable monotonicity property: It is anti-comonotonic with the random mark-to-market value (or return, etc.) of the underlying financial position, and it is hence a hedge against such variability.

Performance expectations and the tenure of head coaches: Evidence from NCAA football

Previous research on coaching turnover indicates that a number of factors, including age, firm performance, and deviation from expected performance affect turnover. We examine the relationship between performance expectations and head coach turnover using data from NCAA Division I-A college football using a market-determined measure of expected performance, winning percentage against point spreads set in betting markets; this expected performance measure does not suffer from many of the problems that plague analysts’ earnings forecasts. We find that performance expectations, actual performance, and tenure affect coaching tenure in NCAA Division I-A college football, based on performance data from 102 Division I-A football programs over the period 1980–2004.

Mathematical modeling and a memetic algorithm for the integration of process planning and scheduling considering uncertain processing times

The integration of process planning and scheduling is important for an efficient utilization of manufacturing resources. However, the focus of existing works is mainly on deterministic constraints of jobs. This article proposes a novel memetic algorithm for the integrated process planning and scheduling problem with processing time uncertainty based on processing time scenarios. First, a mathematical model for the stochastic integrated process planning and scheduling problem based on the network graph is established. Due to the nonlinearity in the model and the complexity of the problem, a memetic algorithm is then suggested for this problem. A novel local search (variable neighborhood search) algorithm is incorporated into the memetic algorithm. Two effective neighborhood structures are employed in the variable neighborhood search algorithm to improve the overall performance of the population. Furthermore, for the uncertainty in processing times, a set of scenarios have been generated to evaluate each individual. Finally, two performance measures—the expected performance measure and the worst-case deviation measure—are introduced and compared. In the experimental studies, the proposed memetic algorithm is tested on typical benchmark instances. Computational results show that the expected makespan measure performs better than the worst-case deviation measure and the proposed method exhibits high performance especially for large-scale instances. In addition, the results obtained by the proposed memetic algorithm are more satisfactory than those obtained by the algorithm that considers deterministic processing times only.

Model Risk Analysis via Investment Structuring

"What are the origins of risks?" and "How material are they?" -- these are the two most fundamental questions of any risk analysis. Quantitative Structuring -- a technology for building financial products -- provides economically meaningful answers for both of these questions. It does so by considering risk as an investment opportunity. The structure of the investment reveals the precise sources of risk and its expected performance measures materiality. We demonstrate these capabilities of Quantitative Structuring using a concrete practical example -- model risk in options on vol-targeted indices.

Critical success factors of green supply chain management for achieving sustainability in Indian automobile industry

The aim of this study was to identify and analyse the key success factors behind successful achievement of environment sustainability in Indian automobile industry supply chains. Here, critical success factors (CSFs) and performance measures of green supply chain management (GSCM) have been identified through extensive literature review and discussions with experts from Indian automobile industry. Based on the literature review, a questionnaire was designed and 123 final responses were considered. Six CSFs to implement GSCM for achieving sustainability and four expected performance measures of GSCM practices implementation were extracted using factor analysis. interpretive ranking process (IRP) modelling approach is employed to examine the contextual relationships among CSFs and to rank them with respect to performance measures. The developed IRP model shows that the CSF ‘Competitiveness’ is the most important CSF for achieving sustainability in Indian automobile industry through GSCM practices. This study is one of the few that have considered the environmental sustainability practices in the automobile industry in India and their implications on sectoral economy. The results of this study may help the mangers/SC practitioners/Governments/Customers in making strategic and tactical decisions regarding successful implementation of GSCM practices in Indian automobile industry with a sustainability focus. The developed framework provides a comprehensive perspective for assessing the synergistic impact of CSFs on GSCM performances and can act as ready reckoner for the practitioners. As there is very limited work presented in literature using IRP, this piece of work would provide a better understanding of this relatively new ranking methodology.

Laboratory Performance Based Cost Assessment of Warm-Mix Asphalt Concrete Technologies

With growing environmental concerns and ever-increasing budgetary constraints, the use of warm-mix asphalt (WMA) is on the rise, since WMA has the potential to reduce cost and provide environmental benefits. One hurdle in the adoption of WMA is the uncertainty of its long-term performance. Currently a literature gap exists on the performance of WMA, as most studies tend to focus on the economics and environmental benefits of the emerging technology. Laboratory experimentation, in the form of rutting, fatigue life, and moisture susceptibility as measured by tensile-strength-ratio (TSR), was conducted on four different WMA technologies in order to compare the change in performance between WMA and hot-mix asphalt. Linear regression models were then fit to the laboratory results to identify variables which were statistically significant in affecting performance. Life cycle cost analysis principles were then applied to compare the warm mix technologies. The resulting dollar amount from the cost analysis was compared with the regression models, which allow for direct comparison between cost savings and performance. The result of this study shows that addition of low and moderate amounts of WMA additive was found to increase fatigue life; however, high dosages of additive negatively affected the fatigue life. It was also shown that reducing the mixing temperature of any asphalt mixture tested in this study increased the likelihood of rutting. Lastly, TSR and fatigue life were found to be dependent on the type of additive chosen. In general, every WMA technology tested has the potential to reduce costs and improve at least one measure of the expected performance of asphalt mixtures.

Modeling and Analyzing Budget-Constrained Pavement Preservation Strategies

The Washington State pavement management system (WSPMS) has a long and well-known history of describing network condition, predicting necessary rehabilitation, and guiding the Washington State Department of Transportation (DOT) to the best possible value for pavements meeting an acceptable level of service. By definition, lowest life-cycle cost (LLCC) is the most economical way to manage roadways to a required level of service. What does an agency do when it is currently, and projected to be, funded at less than half of what LLCC calculations recommend? This deceptively difficult question reflects the situation that the Washington State DOT currently faces with its preservation budget. To assist the agency in answering this question, the WSPMS developed a tool called the Forecaster, allowing definition of a forecast scenario that enables the user to define a yearly budget with optional allocations, preservation techniques, prioritization of roadways and activities, and more, and analyzes the results of the forecast by using several different performance measures. Presented are an overview of the new WSPMS tool to create forecasts, a general overview of the forecast process, and the performance measures that the WSPMS gives to users to analyze the results of a forecast. Finally, several funding scenarios are compared by the expected performance measure outcomes, and a specific prioritization methodology is recommended for the severely constrained preservation budget of the Washington State DOT.

Screening and selection procedures with control variates and correlation induction techniques

AbstractWe consider the problem of identifying the simulated system with the best expected performance measure when the number of alternatives is finite and small (often < 500). Recently, more research efforts in the simulation community have been directed to develop ranking and selection (R&S) procedures capable of exploiting variance reduction techniques (especially the control variates). In this article, we propose new R&S procedures that can jointly use control variates and correlation induction techniques (including antithetic variates and Latin hypercube sampling). Empirical results and a realistic illustration show that the proposed procedures outperform the conventional procedures using sample means or control variates alone. © 2012 Wiley Periodicals, Inc. Naval Research Logistics, 2012

Quantification of margins and uncertainties of complex systems in the presence of aleatoric and epistemic uncertainty

Performance assessment of complex systems is ideally done through full system-level testing which is seldom available for high consequence systems. Further, a reality of engineering practice is that some features of system behavior are not known from experimental data, but from expert assessment, only. On the other hand, individual component data, which are part of the full system are more readily available. The lack of system level data and the complexity of the system lead to a need to build computational models of a system in a hierarchical or building block approach (from simple components to the full system). The models are then used for performance prediction in lieu of experiments, to estimate the confidence in the performance of these systems. Central to this are the need to quantify the uncertainties present in the system and to compare the system response to an expected performance measure. This is the basic idea behind Quantification of Margins and Uncertainties (QMU). QMU is applied in decision making—there are many uncertainties caused by inherent variability (aleatoric) in materials, configurations, environments, etc., and lack of information (epistemic) in models for deterministic and random variables that influence system behavior and performance. This paper proposes a methodology to quantify margins and uncertainty in the presence of both aleatoric and epistemic uncertainty. It presents a framework based on Bayes networks to use available data at multiple levels of complexity (i.e. components, subsystem, etc.) and demonstrates a method to incorporate epistemic uncertainty given in terms of intervals on a model parameter.

Multinomial Selection Problem: A Study of BEM and AVC Algorithms

The two well-known and widely used multinomial selection procedures Bechhofor, Elmaghraby, and Morse (BEM) and all vector comparison (AVC) are critically compared in applications related to simulation optimization problems. Two configurations of population probability distributions in which the best system has the greatest probability p i of yielding the largest value of the performance measure and has or does not have the largest expected performance measure were studied. The numbers achieved by our simulations clearly show that none of the studied procedures outperform the other in all situations. The user must take into consideration the complexity of the simulations and the performance measure probability distribution properties when deciding which procedure to employ. An important discovery was that the AVC does not work in populations in which the best system has the greatest probability p i of yielding the largest value of the performance measure but does not have the largest expected performance measure.

Selecting designs with the smallest variance of normal populations

Indifference-zone selection procedures have been used to select a design with the minimum or maximum expected performance measure among a finite number of simulated designs. While there have been significant advancements in selection methodologies, the majority of the selection procedures are developed to process the selection when the performance measures are the mean of some output. In this paper, we state and justify two sampling procedures for selecting a subset of size m that contains at least c of the v best of k independent normal populations, when the ranking parameters are the population variances. An experimental performance evaluation demonstrates the validity of these selection procedures.

Some Procedures of Selecting the Best Designs with Respect to Quantile

Indifference-zone selection procedures have been used to select a design with the minimum or maximum expected performance measure among a finite number of simulated designs. While there have been significant advancements in selection methodologies, the majority of the selection procedures are developed to process the selection when the performance measures are the mean of some output. In some situations, quantiles provide more suitable information. Quantiles are also more robust to outliers than the mean and standard deviation. Moreover, selection procedures are often derived based on the assumption that the input data are independent and identically distributed (i.i.d.) normal. In this paper we state and justify selection procedures when the ranking parameter is quantile. It is our intention for the quantile estimates to play the role of the i.i.d. normal observations that the original versions of selection procedures process. That is, we assume that our quantile estimates are approximately i.i.d. normal. We perform an empirical study of several stochastic processes to evaluate the performance of the procedure.

The Valuation of Athletes as Risky Investments: A Theoretical Model

The desire to hire the best athletes and coaches in order to maximize team performance necessitates generous compensation contracts, which in turn increase the risk of financial distress or even bankruptcy for team owners. Indeed, one of the largest expense items in the budget of professional sport teams is the remuneration of players and coaches. Yet an investment made today in a given team yields an uncertain income in the future because team profitability depends on the uncertain performance of each player and the synchronization among players—both influenced by the coach. We present a formal theoretical model that assesses athletes’ valuation and accounts for the aforementioned factors. The optimal compensation schedule is determined empirically by regressing expected performance measures of each player with the aggregate team performance. Once the optimal schedule has been determined, the expected rate of return for the owner is earned at the lowest possible risk.

Comparison with a standard via fully sequential procedures

We develop fully sequential procedures for comparison with a standard. The goal is to find systems whose expected performance measures are larger or smaller than a single system referred to as a standard and, if there is any, to find the one with the largest or smallest performance. The general formulation of comparison with a standard gives the standard a special status and tries to protect it when its performance is better than or even equal to performance measures of all the other alternatives. Therefore, the problem cannot be formulated as the selection of the best and a specialized procedure is required. Our procedures allow for unequal variances across systems, the use of common random numbers, and known or unknown expected performance of the standard. Experimental results are provided to compare the efficiency of the procedure with other existing procedures.

Selecting the best stochastic system for large scale problems in DEDS

We consider the problem of selecting the stochastic system with the best expected performance measure, when the number of alternative systems is large. We consider the case of discrete event dynamic systems (DEDS) where the standard clock simulation technique can be used for simulating multiple systems using only one sample path. In this paper, we use a two-phase procedure that uses the standard clock simulation technique. In the first phase, we screen out non-competent alternatives and construct a confidence set that contains the best alternative with a pre-specified large probability. In the second phase, we use the indifference-zone ranking and selection procedure to select the best expected alternative among the survivals of the first phase. We implement this algorithm for solving a practical simulation optimization problem.

A Bayesian View on Detecting Drifts by Nonparametric Methods

We study a nonparametric sequential detection procedure, which aims at detecting the first time point where a drift term appears in a stationary process, from a Bayesian perspective. The approach is based on a nonparametric model for the drift, a nonparametric kernel smoother which is used to define the stopping rule, and a performance measure which determines for each smoothing kernel and each given drift the asymptotic accuracy of the method. We look at this approach by parameterizing the drift and putting a prior distribution on the parameter vector. We are able to identify the optimal prior distribution which minimizes the expected performance measure. Consequently, we can judge whether a certain prior distribution yields good or even optimal asymptotic detection. We consider several important special cases where the optimal prior can be calculated explicitly.

Capacity reliability of a road network: an assessment methodology and numerical results

Existing reliability studies of road networks are mainly limited to connectivity and travel time reliability and may not be sufficient for a comprehensive network performance measure. Recently Chen et al. (J. Adv. Transp. 33 (2) (1999) 183–200) introduced capacity reliability as a new network performance index. It is defined as the probability that the network can accommodate a certain traffic demand at a required service level, while accounting for drivers' route choice behavior. The proposed capacity reliability index includes connectivity reliability as a special case and also provides travel time reliability as a side product. This paper extends the capacity reliability analysis by providing a comprehensive methodology, which combines reliability and uncertainty analysis, network equilibrium models, sensitivity analysis of equilibrium network flow and expected performance measure, as well as Monte Carlo methods, to assess the performance of a degradable road network. Numerical results are also provided to demonstrate the feasibility of the proposed framework.