Decision Analysis Practice Research Articles

From the Editor: Belated Recognition for the 2021 Clemen–Kleinmuntz Decision Analysis Best Paper Award Winner and Finalist

Free AccessAboutSectionsView PDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareShare onFacebookTwitterLinked InEmail Go to SectionFree Access HomeDecision AnalysisAhead of Print From the Editor: Belated Recognition for the 2021 Clemen–Kleinmuntz Decision Analysis Best Paper Award Winner and FinalistVicki M. BierVicki M. BierPublished Online:9 May 2023https://doi.org/10.1287/deca.2023.0477The winning paper for 2021 was “Friction and Decision Rules in Portfolio Decision Analysis,” by Gary J. Summers (Summers 2021). To put this paper in context, the practice of decision analysis typically involves identifying performance objectives and managerial preferences, quantifying uncertainties, and then using them as inputs to assess alternatives. An exhaustive theory-driven literature in decision analysis provides foundations for this practice, assuming that objectives, preferences, and uncertainties are well understood and available to quantify. The award-winning paper by Summers highlights that this quantification may not be accurate in practice, leading to “friction” in decision analysis models. This friction can systematically lead to a loss in decision quality. The article provides examples from multiple domains, and new analysis to illustrate this friction. As such, the article underscores the importance of calibrating the inputs to decision analysis models. It promises to be a springboard for future research on understanding the causes of friction in decision analysis models, its implications, and mitigation methods.I congratulate Gary J. Summers on this excellent paper and am especially glad that Decision Analysis was able to help a full-time practitioner bring forth important and insightful ideas in a manner that is compelling to both academics and practitioners.The finalist for 2021 was “Preference-Approval Structures in Group Decision Making: Axiomatic Distance and Aggregation,” by Yucheng Dong, Yao Li, Ying He, and Xia Chen (Dong et al. 2021). Although decision analysis commonly focuses on decision making by an individual or a group acting as an individual, this paper focuses on group decision making. It combines two popular approaches for aggregating individual preferences, ranked voting and approval voting, which have compensating strengths and weaknesses. Ranked voting leverages the preference ranking central to the decision analysis approach, but is subject to strategic manipulation (where individuals misrepresent their preferences); approval voting is immune to strategic manipulation, but does not provide the complete ranking required by decision analysis. The article shows that combining them leads to superior performance. Future empirical research and practice is likely to significantly benefit from the rigorous foundational treatment provided by this article.Congratulations to Dong et al. for this excellent and pragmatic contribution to the challenging topic of group decision making.Thanks also to committee members Saurabh Bansal (Pennsylvania State University) and Robert Bordley (University of Michigan) for cochairing the award committee for 2021, and for providing extensive input to the paper descriptions given above. My sincere apologies for this belated recognition to all.ReferencesDong Y, Li Y, He Y, Chen X (2021) Preference–approval structures in group decision making: Axiomatic distance and aggregation. Decision Anal. 18(4):273–295. https://doi.org/10.1287/deca.2021.0430.Link, Google ScholarSummers GJ (2021) Friction and decision rules in portfolio decision analysis. Decision Anal. 18(2):101–120. https://doi.org/10.1287/deca.2020.0421.Link, Google Scholar Previous Back to Top Next FiguresReferencesRelatedInformation Articles In Advance Article Information Metrics Information Published Online:May 09, 2023 Copyright © 2023, INFORMSCite asVicki M. Bier (2023) From the Editor: Belated Recognition for the 2021 Clemen–Kleinmuntz Decision Analysis Best Paper Award Winner and Finalist. Decision Analysis 0(0). https://doi.org/10.1287/deca.2023.0477 PDF download

Innovations in the Science and Practice of Decision Analysis: The Role of Management Science

In this paper, we reflect on research in decision analysis that has appeared in Management Science and its impact on decision analysis practice and applications. We consider professional applications of decision analysis in business and government settings as well as everyday conversational applications. This paper was accepted by David Simchi-Levi, Special Section of Management Science: 65th Anniversary.

Improving stochastic analysis for tradeoffs in multi‐criteria value models

AbstractIn multi‐criteria value modelling (MCVM), uncertainty clouds decision making by complicating choices that inevitably must be made among an offering of competing alternatives. In stochastic MCVM settings involving value versus cost (risk) tradeoffs, exposing and communicating ramifications that choice instigates becomes increasingly difficult directly in response to the increased sophistication of mathematical methods needed to distinguish between alternatives, leaving the decision maker potentially unprepared for eventual opportunity or disaster. This article introduces an approach called realization analysis that leverages stochastic simulation, probability theory and tradespace geometry to estimate full spectrum outcome probabilities in an effort to enrich decision making and reduce the potential for surprise or regret, demonstrating its effectiveness when competing alternatives are tightly co‐mingled in their potential realizations causing issues for traditional stochastic dominance methods. The approach is surprisingly straightforward to implement, with results easily communicated to technical and non‐technical decision makers. A brief acquisition case study involving competing alternatives is presented to illustrate how this approach offers advantages for multi‐criteria decision analysis practice.

IJAHP team at INFORMS 2017 - Valentina Ferretti wins award

On October 2017 at the INFORMS Annual meeting which took place in Houston, our IJAHP News and Events Section Editor, Dr Valentina Ferretti, won the 2017 INFORMS Decision Analysis Practice Award.https://doi.org/10.13033/ijahp.v9i3.531

Modeling project preferences in multiattribute portfolio decision analysis

When choosing a portfolio of projects with a multi-attribute weighting model, it is necessary to elicit trade-off statements about how important these attributes are relative to each other. Such statements correspond to weight constraints, and thus impact on which project portfolios are potentially optimal or non-dominated in view of the resulting set of feasible attribute weights. In this paper, we extend earlier preference elicitation approaches by allowing the decision maker to make direct statements about the selection and rejection of individual projects. We convert such project preference statements to weight information by determining the weights for which (i) the selected project is included in all potentially optimal or non-dominated portfolios, or (ii) the rejected project is not included in any potentially optimal or non-dominated portfolio. We prove that the two complementary selection rules will exclude exactly the same set of weights. However, analyses that apply the dominance structure often lead to multiple, mutually exclusive feasible weight sets, and therefore the approach based on potential optimality is more relevant for practical decision analysis. We also propose ex ante value of information measures to guide the elicitation of project preference statements, and illustrate our results by analyzing a real case on the selection of infrastructure maintenance projects.

Johnson Quantile-Parameterized Distributions

It is common decision analysis practice to elicit quantiles of continuous uncertainties and then fit a continuous probability distribution to the corresponding probability-quantile pairs. This process often requires curve fitting and the best-fit distribution will often not honor the assessed points. By strategically extending the Johnson Distribution System, we develop a new distribution system that honors any symmetric percentile triplet of quantile assessments (e.g., the 10th-50th-90th) in conjunction with specified support bounds. Further, our new system is directly parameterized by the assessed quantiles and support bounds, eliminating the need to apply a fitting procedure. Our new system is practical, flexible, and, as we demonstrate, able to match the shapes of numerous commonly named distributions.

Measurement issues in the evaluation of projects in a project portfolio

A common problem arising in many domains is how value the benefits of projects in a project portfolio. Recently, there has been some attention given to a decision analysis practice whereby analysts define a value function on some criterion by setting 0 as the value of the worst project. In particular, Clemen and Smith have argued this practice is not sound as it gives different results from the case where projects are “priced out”, and makes a strong implicit assumption about the value of not doing a project. In this paper we underscore the criticism of this way of using value functions by showing it can lead to a rank reversal. We provide a measurement theoretic account of the phenomenon, showing that the problem arises from using evaluating projects on an interval scale (such as a value scale) whereas to guard against such rank reversals, benefits must be measured on at least a ratio scale. Seen from this perspective, we discuss how the solution proposed by Clemen and Smith, of reformulating the underlying optimisation problem to allow for explicitly non-zero values, addresses the issue, and explore in what sense it may be open to similar problems. In closing we discuss what lessons from practice may be drawn from this analysis, focussing on settings where the Clemen and Smith proposal may not be the most natural way of modelling.

Integrated decision model for knowledge offshoring and outsourcing

Knowledge management (KM) has played a critical role in business management fields ever since the 1990s. In today's global marketplace, firms not only offshore and/or outsource tangible resources, but they also acquire tacit knowledge. The study herein attempts to construct a knowledge offshoring and outsourcing (KO&O) model from the basis of decision analysis and decision making to help organizations consider whether, what, and how to offshore and/or outsource knowledge. After reviewing KM-related articles (in particular Arikan’s model, 2009) and conducting a case study of Yulon (Taiwan’s largest automobile manufacturer), this study presents a four-mode KO&O model by considering four managerial questions and eight criteria. The model consists of four modes: knowledge allocation, development, deal, and growth. Accordingly, this study classifies KO&O as having 16 activities and details them. This study also forms an integrated KO&O framework and a checklist for practical decision analysis and managerial application. Finally, this study provides applications, suggestions, and directions toward KO&O practices and research for the future.

About the Authors

About the Authors

From the Editors—Copulas, Group Preferences, Multilevel Defenders, Sharing Rewards, and Communicating Analytics

We review the past year and this Editor-in-Chief's editorship for the last six annual volumes in this issue's “From the Editors” column, which is coauthored with Managing Editor Kelly M. Kophazi, and then we preview this issue's research articles. Our first article, by Dorota Kurowicka is on “Conditionalization of Copula-Based Models.” The next three papers address groups or teams. First is “A Note on a Group Preference Axiomatization with Cardinal Utility,” by Luis C. Dias and Paula Sarabando. Next, Andrew Samuel and Seth D. Guikema address “Resource Allocation for Homeland Defense: Dealing with the Team Effect.” Then, Arthur Carvalho and Kate Larson present a way for “Sharing Rewards Among Strangers Based on Peer Evaluations.” Finally, Jeffrey M. Keisler and Patrick S. Noonan provide guidance for decision analysis practice in “Communicating Analytic Results: A Tutorial for Decision Consultants.”

About the Authors

About the Authors

About the Authors

About the Authors

Quantile-Parameterized Distributions

This paper introduces a new class of continuous probability distributions that are flexible enough to represent a wide range of uncertainties such as those that commonly arise in business, technology, and science. In many such cases, the nature of the uncertainty is more naturally characterized by quantiles than by parameters of familiar continuous probability distributions. In the practice of decision analysis, it is common to fit a hand-drawn curve to quantile outputs from probability elicitations on a continuous uncertain quantity and to then discretize the curve. The resulting discrete probability distribution is an approximation that cuts off the distribution's tails and eliminates intermediate values. Quantile-parameterized distributions address this problem by using quantiles themselves to parameterize a continuous probability distribution. We define quantile-parameterized distributions, illustrate their flexibility and range of applicability, and conclude with practical considerations when parameterizing distributions using inconsistent quantile assessments.

About the Authors

About the Authors

From the Editor…

Our first two articles in this issue focus on decision analysis in practice, from charitable decisions to government policy making. Kevin F. McCardle, Kumar Rajaram, and Christopher S. Tang present “A Decision Analysis Tool for Evaluating Fundraising Tiers.” Next, Rex V. Brown draws upon his years as a consultant in “Working with Policy Makers on Their Choices: A Decision Analyst Reminisces.” The next two articles are on aggregating probability judgments from multiple experts. Ali E. Abbas presents “A Kullback-Leibler View of Linear and Log-Linear Pools.” The final article, on “Combining the Opinions of Experts Who Partition Events Differently,” is by Robert F. Bordley. The call for papers on auctions for a special issue in memory of Michael Rothkopf is also included in this issue.

Aggregating Probabilistic Forecasts from Incoherent and Abstaining Experts

Decision makers often rely on expert opinion when making forecasts under uncertainty. In doing so, they confront two methodological challenges: the elicitation problem, which requires them to extract meaningful information from experts; and the aggregation problem, which requires them to combine expert opinion by resolving disagreements. Linear averaging is a justifiably popular method for addressing aggregation, but its robust simplicity makes two requirements on elicitation. First, each expert must offer probabilistically coherent forecasts; second, each expert must respond to all our queries. In practice, human judges (even experts) may be incoherent, and may prefer to assess only the subset of events about which they are comfortable offering an opinion. In this paper, a new methodology is developed for combining expert assessment of chance. The method retains the conceptual and computational simplicity of linear averaging, but generalizes the standard approach by relaxing the requirements on expert elicitation. The method also enjoys provable performance guarantees, and in experiments with real-world forecasting data is shown to offer both computational efficiency and competitive forecasting gains as compared to rival aggregation methods. This paper is relevant to the practice of decision analysis, for it enables an elicitation methodology in which judges have freedom to choose the events they assess.

General Methodology Combining Engineering Optimization of Primary HVAC&R Plants with Decision Analysis Methods—Part I: Deterministic Analysis

This paper is the first of a two-part sequence that proposes a general methodology for dynamic scheduling and optimal control of complex primary HVAC&R plants, which combines engineering analyses within a practical decision analysis framework by modeling risk attitudes of the operator. The methodology involves a computationally efficient, deterministic engineering optimization phase for scheduling and controlling primary systems over the planning horizon, followed by a systematic and comprehensive stochastic sensitivity and decision analysis phase, where various sources of uncertainties are evaluated along with alternative non-optimal but risk-averse operating strategies. This paper describes the deterministic component of the analysis methodology, which essentially involves the development of response surface models for different combinations of system configurations to be used for static optimization and then using them in conjunction with the modified Dijkstra's algorithm for dynamic scheduling and optimal control under different operating conditions and pricing signals. The proposed methodology is illustrated for a semi-real hybrid cooling plant operated under two different pricing schemes: real-time pricing and time-of-use with electricity demand. We feel that the general methodology framework proposed sacrifices very little in accuracy while being much more efficient computationally than the more complicated optimization methods proposed in thegeneral literature. Moreover, this approach is suitable for online implementation, and it is also general enough to be relevant to other energy systems.

Combining scenario planning and multi‐criteria decision analysis in practice

AbstractThe integrated use of scenario planning and multi‐criteria decision analysis (MCDA) has been advocated as a powerful combination for providing decision support in strategic decisions. Scenario planning helps decision makers in devising strategies and thinking about possible future scenarios; while MCDA can support an in‐depth performance evaluation of each strategy, as well as in the design of more robust and better options. One of the frameworks proposed recently, by Goodwin & Wright, suggests the use of scenario planning with multi‐attribute value theory, a mathematically simple, yet extensively researched and widely employed multi‐criteria method. However, so far, such framework has been presented only using hypothetical problems. In this paper, we describe two case studies where this approach was used to support real‐world strategic decisions. We discuss the challenges and limitations we encountered in applying it and suggest some possible improvements that could be made to such framework. Copyright © 2007 John Wiley & Sons, Ltd.

Attribute Dominance Utility

We present an analogy between joint cumulative probability distributions and a class of multiattribute utility functions, which we call attribute dominance utility functions. Attribute dominance utility functions permit assessing multiattribute utility functions using common techniques of joint probability assessment such as marginal-conditional assessments and the method of copulas. By itself, this class of utility functions appears in many cases of decision analysis practice. Furthermore, we show that many functional forms of multiattribute utility function can be decomposed into attribute dominance utility functions that are easier to elicit. We introduce the notion of utility inference analogous to Bayes’ rule for probability inference and provide a graphic representation of attribute dominance utility functions, which we call utility diagrams.

Subjective Probability Assessment in Decision Analysis: Partition Dependence and Bias Toward the Ignorance Prior

Decision and risk analysts have considerable discretion in designing procedures for eliciting subjective probabilities. One of the most popular approaches is to specify a particular set of exclusive and exhaustive events for which the assessor provides such judgments. We show that assessed probabilities are systematically biased toward a uniform distribution over all events into which the relevant state space happens to be partitioned, so that probabilities are “partition dependent.” We surmise that a typical assessor begins with an “ignorance prior” distribution that assigns equal probabilities to all specified events, then adjusts those probabilities insufficiently to reflect his or her beliefs concerning how the likelihoods of the events differ. In five studies, we demonstrate partition dependence for both discrete events and continuous variables (Studies 1 and 2), show that the bias decreases with increased domain knowledge (Studies 3 and 4), and that top experts in decision analysis are susceptible to this bias (Study 5). We relate our work to previous research on the “pruning bias” in fault-tree assessment (e.g., Fischhoff et al. 1978) and show that previous explanations of pruning bias (enhanced availability of events that are explicitly specified, ambiguity in interpreting event categories, and demand effects) cannot fully account for partition dependence. We conclude by discussing implications for decision analysis practice.