Multistage Decision Research Articles

Decision programming for mixed-integer multi-stage optimization under uncertainty

Influence diagrams are widely employed to represent multi-stage decision problems in which each decision is a choice from a discrete set of alternative courses of action, uncertain chance events have discrete outcomes, and prior decisions may influence the probability distributions of uncertain chance events endogenously. In this paper, we develop the Decision Programming framework which extends the applicability of influence diagrams by developing mixed-integer linear programming formulations for such problems. In particular, Decision Programming makes it possible to (i) solve problems in which earlier decisions cannot necessarily be recalled later, for instance, when decisions are taken by agents who cannot communicate with each other; (ii) accommodate a broad range of deterministic and chance constraints, including those based on resource consumption, logical dependencies or risk measures such as Conditional Value-at-Risk; and (iii) determine all non-dominated decision strategies in problems which multiple value objectives. In project portfolio selection problems, Decision Programming allows scenario probabilities to depend endogenously on project decisions and can thus be viewed as a generalization of Contingent Portfolio Programming (Gustafsson & Salo, 2005). We present several illustrative examples, evidence on the computational performance of Decision Programming formulations, and directions for further development.

Mixed data-driven sequential three-way decision via subjective–objective dynamic fusion

In the context of granular computing, sequential three-way decision is a useful tool to triadic thinking, triadic computing and triadic processing from coarser to finer under multilevel and multiview granularity space. In this paper, we mainly explore a novel framework of sequential three-way decision for the fusion of mixed data from the subjective and objective dynamic perspectives. The former focuses on the decision maker’s dynamic behavior without considering the time-evolving data, and the latter emphasizes on dealing with dynamic mixed data over time by multi-stage decision-making. We firstly utilize four T-norm operators and kernel-based similarity relations to integrate different types of dynamic data. Then the subjective and objective models of sequential three-way decision are investigated based on decision thresholds, attribute importance and cost reduction. Finally, the comparative experiments are reported to verify that our proposed models can achieve the lower decision cost and the acceptable accuracy.

Artificial Intelligence-Based Methods for Decision Support to Avoid Collisions at Sea

Ship collisions cause major losses in terms of property, equipment, and human lives. Therefore, more investigations should be focused on this problem, which mainly results from human error during ship control. Indeed, to reduce human error and considerably improve the safe traffic of ships, an intelligent tool based on fuzzy set theory is proposed in this paper that helps navigators make fast and competent decisions in eventual collision situations. Moreover, as a result of selecting the shortest collision avoidance trajectory, our tool minimizes energy consumption. The main aim of this paper was the development of a decision-support system based on an artificial intelligence technique for safe ship trajectory determination in collision situations. The ship’s trajectory optimization is ensured by multistage decision making in collision situations in a fuzzy environment. Furthermore, the navigator’s subjective evaluation in decision making is taken into account in the process model and is included in the modified membership function of constraints. A comparative analysis of two methods, i.e., a method based on neural networks and a method based on the evolutionary algorithm, is presented. The proposed technique is a promising solution for use in real time in onboard decision-support systems. It demonstrated a high accuracy in finding the optimal collision avoidance trajectory, thus ensuring the safety of the crew, property, and equipment, while minimizing energy consumption.

Comparing Assessment Methods of Attribute Importance in Teachers’ Decisions: The Importance of Different Criteria for Tracking Recommendations after Primary School

The importance of different criteria for tracking recommendations is usually inferred using regression weights as a cross-student measure. The few studies that have applied alternative approaches or differentiated between student groups sometimes reach different conclusions. According to research on judgment and decision making (JDM), different methods operationalize different facets of importance. Given this, we investigate whether the importance of criteria for tracking recommendations depends on a direct vs. indirect operationalization (regression weights vs. ratings). A total of 181 teachers selected four students from their most recent fourth-grade class using a 2 × 2 design (certain vs. uncertain qualification for the Realschule (vocational track) vs. the Gymnasium (academic track)). Then, they reported on the level and the importance of predetermined criteria for each student. Contrary to JDM research, we found few method-related differences, but striking differences between cases with a certain vs. an uncertain qualification. For the latter, the importance of the criteria is more homogeneous, the regression prediction is less successful and the importance varies with the dependent variable in the regression (actual recommendation vs. perceived qualification). We conclude that further research should focus on uncertain cases rather than method-related differences and suspect that, in uncertain cases, the formation of the recommendation is a multistage decision process.

Restricted sub-tree learning to estimate an optimal dynamic treatment regime using observational data.

Dynamic treatment regimes (DTRs), consisting of a sequence of tailored treatment decision rules that span multiple stages of care, present a unique opportunity in our drive toward personalized medicine. Given that estimation of optimal DTRs is often exploratory and communication with clinicians is vital, robust and flexible methods that yield interpretable results are needed. Tree-based methods utilizing a purity measure defined on the full set of covariates have enjoyed much success in meeting this goal. Often, however, it is necessary for clinical, practical, or ethical reasons to restrict certain covariates that should be used when making treatment decisions. Herein we present restricted sub-tree learning (ReST-L), a flexible and robust, sub-tree-based method to estimate an optimal multi-stage multi-treatment DTR that enables restrictions to the set of prespecified candidate tailoring variables. ReST-L employs a purity measure derived from an augmented inverse probability weighted estimator for the counterfactual mean outcome, using observational data to build multi-stage decision trees that are restricted in sub-tree spaces defined by the corresponding prescriptive covariates. We show that ReST-L is able to correctly estimate the optimal DTR searching over a large number of variables with relatively small sample sizes and improves upon competing estimation methods. We demonstrate the utility of ReST-L to estimate a two-stage fluid resuscitation strategy for patients admitted to an intensive care unit with acute emergent sepsis.

Optimal manpower recruitment and promotion policies for the finitely graded systems using dynamic programming

BackgroundAt the heart in the development of any organization or nation is human resource. Over the years the world over, a sharp increase in too many numbers of qualified persons is being experienced yearly. This has impacted on recruitment and promotion costs to increase immensely, thereby affecting negatively manpower system costs. Dynamic Programming (DP) approach to optimal manpower recruitment and promotion policies for the two grade system has been proposed. MethodsConsidering the fact that contemporary manpower systems are not limited to just two grades - a kind of “switch-approach” to manpower systems, we first establish the link between a manpower planning problem and a dynamic decision-making process. This linkage resulted to a multistage real-life decision-making problem whose solution demands that decisions be made sequentially at different levels and at different points in time and space. Dynamic Programming is a mathematical technique well appropriate for the optimization of multistage decision problems. This allows us to give a generalization to manpower systems by modifying the model to finite grades which came out to be more robust and actionable, a constrained deterministic Dynamic Programming (CDDP) found to function computationally as the very well-known Wagner-Whitin Model in inventory/production management. Five cost variables associated with manpower planning were identified and used as inputs to the modified deterministic DP model. Results/RelevanceThe data shows yearly recruitments and promotions totaling 507 and 266 staff respectively for a ten-period (year) planning horizon. Total manpower system cost (in ooo's of Nigerian Naira) occasioned by yearly recruitments and promotions exercises for the period is 11334 (7092 for recruiting, 4100 for promoting, and 142 for overstaffing). Our DP model minimizes the manpower system cost to 9462 making a significant reduction of 1872. The optimal policy for the planning period calls for recruiting and promoting respectively 79 and 41 in period 1 only, 86 and 24 in period 2 for periods 2 and 3, 86 and 46 in period 4 for periods 4 and 5, 89 and 29 in period 6 only, 85 and 70 in period 7 for periods 7 and 8, and 82 and 56 in period 9 for periods 9 and 10.The study will contribute to the growing literature on applications of OR models to problems in manpower planning. The model outcomes would provide the basis for evaluating decision policies aimed at conducting recruitment promptly and to eliminate over-stagnated promotions. Conclusion/Further researchWe formulate decisions making in a finitely-graded manpower system as a multistage decision-making optimization problem which are best handled by dynamic programming. Five cost variables associated with manpower planning were identified and used as inputs to the modified deterministic DP model. Our model is resolute for minimizing manpower system costs occasioned by recruitments and promotions exercises in a wide range of multi-graded manpower systems instead of just two grades. The study's limitations and scope for future research work are presented in the end.

A Stochastic Dual Dynamic Integer Programming for the Uncapacitated Lot-Sizing Problem with Uncertain Demand and Costs

We study the uncapacitated lot-sizing problem with uncertain demand and costs. We consider a multi-stage decision process and rely on a scenario tree to represent the uncertainty. We propose to solve this stochastic combinatorial optimization problem thanks to a new extension of the stochastic dual dynamic integer programming algorithm. Our results show that this approach can provide good quality solutions in a reasonable time for large-size instances.

Decide Now or Wait for the Next Forecast? Testing a Decision Framework Using Real Forecasts and Observations

AbstractUsers of meteorological forecasts are often faced with the question of whether to make a decision now, on the basis of the current forecast, or to wait for the next and, it is hoped, more accurate forecast before making the decision. Following previous authors, we analyze this question as an extension of the well-known cost–loss model. Within this extended cost–loss model, the question of whether to decide now or to wait depends on two specific aspects of the forecast, both of which involve probabilities of probabilities. For the special case of weather and climate forecasts in the form of normal distributions, we derive a simple simulation algorithm, and equivalent analytical expressions, for calculating these two probabilities. We apply the algorithm to forecasts of temperature and find that the algorithm leads to better decisions in most cases relative to three simpler alternative decision-making schemes, in both a simulated context and when we use reforecasts, surface observations, and rigorous out-of-sample validation of the decisions. To the best of our knowledge, this is the first time that a dynamic multistage decision algorithm has been demonstrated to work using real weather observations. Our results have implications for the additional kinds of information that forecasters of weather and climate could produce to facilitate good decision-making on the basis of their forecasts.

Dynamic aggregation operators and Einstein operations based on interval-valued picture hesitant fuzzy information and their applications in multi-period decision making

The traditional picture hesitant fuzzy aggregation operators are generally suitable for aggregating information acquired in the form of picture hesitant fuzzy numbers, but they will fail in dealing with interval-valued picture hesitant fuzzy information. In this paper, we describe the notion of interval-valued picture hesitant fuzzy set and the operational laws of interval-valued picture hesitant fuzzy variables. Moreover, we derive some dynamic interval-valued picture hesitant fuzzy aggregation operators (based on Einstein operators) to aggregate the interval-valued picture hesitant fuzzy information collected at different periods. Some desirable properties of these aggregation operators are discussed in detail. In addition, we develop the approaches to tackle the multi-period decision-making problems, where all decision information takes the form of interval-valued picture hesitant fuzzy information collected at different periods. In an attempt to illustrate the applications of the proposed approaches, two numerical examples are given to measure the impact of Coronavirus Disease 2019 (COVID-19) in daily life and to identify the optimal investment opportunity. Finally, a comparative analysis of the proposed and existing studies are conducted to demonstrate the effectiveness of the proposed approaches. The presented interval-valued picture hesitant fuzzy operations, aggregation operators, and decision-making approaches can widely apply to dynamic decision analysis and multi-stage decision analysis in real life.

An integrated regional water quality assessment method considering interrelationships among monitoring indicators.

To monitor and manage water environments, China developed a centralized multi-level administrative system where governments and agencies at each level are responsible for water quality within their regions. In this case, regional water quality assessment has become a critical issue. However, as a complex multi-criteria decision making (MCDM) problem, it faces many challenges such as diverse implement indicator framework, complicated indicator interrelations, and lack of reliable assessment methods. Therefore, this paper constructs a novel multistage decision support framework for regional water quality assessment. In phase I, we determine indicator framework strictly according to the national standards, involving PH, dissolved oxygen (DO), chemical oxygen demand (COD), etc., totally 21 water quality indicators where the temperature indicator is excluded due to its lack of assessment standard. In addition, considering the matching between the characteristic of water quality data and the probabilistic linguistic term set (PLTS) technique, we employ PLTS theory to process massive monitoring data. In phase II, relative weight considering indicators' interrelationship is produced by the proposed regression-based decision-making trial and evaluation laboratory (DEMATEL) method, and further forms combined weight by balancing single-factor weight. In phase III, we present a new PLTS measure and extend the fuzzy technique for order performance by similarity to ideal solution (FTOPSIS) method to generate assessment results. Then, we investigate water quality status of 16 administrative districts in Shanghai, China, with the proposed method. The collected data are derived from 26 water quality monitoring sites and covers the period during September 2018 to February 2019. The results confirm a hypothesis that the statistically significant interrelationship does exist among indicators, and point out that Huang Pu District remains the best water quality with highest values of [Formula: see text] in the range of (0.79-0.85) over the 6 months. Moreover, the parameter analysis and comparative analysis are further given that verifies the robustness and reliability of the model in details.

Decision-Making Under Uncertainty for a Digital Thread-Enabled Design Process

AbstractDigital thread is a data-driven architecture that links together information from all stages of the product lifecycle. Despite increasing application in manufacturing, maintenance/operations, and design related tasks, a principled formulation of analyzing the decision-making problem under uncertainty for the digital thread remains absent. The contribution of this article is to present a formulation using Bayesian statistics and decision theory. First, we address how uncertainty propagates in the product lifecycle and how the digital thread evolves based on the decisions we make and the data we collect. By using these mechanics, we explore designing over multiple product generations or iterations and provide an algorithm to solve the underlying multistage decision problem. We illustrate our method on an example structural design problem where our method can quantify and optimize different types and sequences of decisions, ranging from experimentation, manufacturing, and sensor placement/selection, to minimize total accrued costs.

Drought grade assessment method based on grey cloud incidence clustering model

PurposeThe purpose of this paper is to establish a grey cloud incidence clustering model to assess the drought disaster degree under 15 indexes in 18 cities of Henan province.Design/methodology/approachThe grey incidence degree between each index and ideal index is used to determine the index weight and combined with the subjective weight, the comprehensive weight is given; the traditional possibility function is transformed into grey cloud possibility function by using the principle of maximum deviation and maximum entropy, which fully reflects the coexistence of multiple decision-making conclusions and constructs the grey cloud incidence clustering model.FindingsThe drought disaster degree of Henan province is divided into four grades under the selected 15 indexes. The drought grades show obvious regional differences. The risk levels of the east and southwest are higher, and the risk levels of the north and southeast are lower. This result is consistent with the study of drought disaster grades in Henan province, which shows the practicability and usefulness of the model.Practical implicationsIt provides an effective method for the assessment of drought disaster grade and the basis for formulating disaster prevention and mitigation plan.Originality/valueBy studying the method of multiattribute and multistage decision-making with interval grey number information. The objective weight model of index value is designed, and the subjective weight is given by experts. On the basis of the two, the comprehensive weight of subjective and objective combination is proposed, which effectively weakens the randomness of subjective weight and reasonably reflects the practicality of index decision-making. The time weight reflects the dynamic of the index. The traditional possibility function is transformed into the grey cloud possibility function, which effectively takes advantage of the grey cloud model in dealing with the coexistence of fuzzy information, grey information and random information. Thus, the conflict between the decision-making results and the objective reality is effectively solved. The interval grey number can make full use of the effective information and improve the accuracy of the actual information.

Stochastic single machine scheduling problem as a multi-stage dynamic random decision process

In this work, we study a stochastic single machine scheduling problem in which the features of learning effect on processing times, sequence-dependent setup times, and machine configuration selection are considered simultaneously. More precisely, the machine works under a set of configurations and requires stochastic sequence-dependent setup times to switch from one configuration to another. Also, the stochastic processing time of a job is a function of its position and the machine configuration. The objective is to find the sequence of jobs and choose a configuration to process each job to minimize the makespan. We first show that the proposed problem can be formulated through two-stage and multi-stage Stochastic Programming models, which are challenging from the computational point of view. Then, by looking at the problem as a multi-stage dynamic random decision process, a new deterministic approximation-based formulation is developed. The method first derives a mixed-integer non-linear model based on the concept of accessibility to all possible and available alternatives at each stage of the decision-making process. Then, to efficiently solve the problem, a new accessibility measure is defined to convert the model into the search of a shortest path throughout the stages. Extensive computational experiments are carried out on various sets of instances. We discuss and compare the results found by the resolution of plain stochastic models with those obtained by the deterministic approximation approach. Our approximation shows excellent performances both in terms of solution accuracy and computational time.

Multiagent Reinforcement Learning: Rollout and Policy Iteration

We discuss the solution of complex multistage decision problems using methods that are based on the idea of policy iteration (PI), i.e., start from some base policy and generate an improved policy. Rollout is the simplest method of this type, where just one improved policy is generated. We can view PI as repeated application of rollout, where the rollout policy at each iteration serves as the base policy for the next iteration. In contrast with PI, rollout has a robustness property: it can be applied on-line and is suitable for on-line replanning. Moreover, rollout can use as base policy one of the policies produced by PI, thereby improving on that policy. This is the type of scheme underlying the prominently successful AlphaZero chess program. In this paper we focus on rollout and PI-like methods for problems where the control consists of multiple components each selected (conceptually) by a separate agent. This is the class of multiagent problems where the agents have a shared objective function, and a shared and perfect state information. Based on a problem reformulation that trades off control space complexity with state space complexity, we develop an approach, whereby at every stage, the agents sequentially (one-at-a-time) execute a local rollout algorithm that uses a base policy, together with some coordinating information from the other agents. The amount of total computation required at every stage grows linearly with the number of agents. By contrast, in the standard rollout algorithm, the amount of total computation grows exponentially with the number of agents. Despite the dramatic reduction in required computation, we show that our multiagent rollout algorithm has the fundamental cost improvement property of standard rollout: it guarantees an improved performance relative to the base policy. We also discuss autonomous multiagent rollout schemes that allow the agents to make decisions autonomously through the use of precomputed signaling information, which is sufficient to maintain the cost improvement property, without any on-line coordination of control selection between the agents. For discounted and other infinite horizon problems, we also consider exact and approximate PI algorithms involving a new type of one-agent-at-a-time policy improvement operation. For one of our PI algorithms, we prove convergence to an agent-by-agent optimal policy, thus establishing a connection with the theory of teams. For another PI algorithm, which is executed over a more complex state space, we prove convergence to an optimal policy. Approximate forms of these algorithms are also given, based on the use of policy and value neural networks. These PI algorithms, in both their exact and their approximate form are strictly off-line methods, but they can be used to provide a base policy for use in an on-line multiagent rollout scheme.

Multiobjective dynamic programming in bipolar multistage method

The multicriteria Bipolar method can be extended and used to control multicriteria, multistage decision processes. In this extension, at each stage of the given multistage process two sets of reference points are determined, constituting a reference system for the evaluation of stage alternatives. Multistage alternatives, which are compositions of stage alternatives, are assigned to one of six predefined hierarchical classes and then ranked. The aim of this paper is to show the possibility of finding the best multistage alternative, using Bellman’s optimality principle and optimality equations. Of particular importance is a theorem on the non-dominance of the best multistage alternative, proven here. The methodology proposed allows to avoid reviewing each multistage alternative, which is important in large-size problems. The method is illustrated by a numerical example and a brief description of the sustainable regional development problem. The problem can be solved by means of the proposed procedure.

Multi-period investment pathways - Modeling approaches to design distributed energy systems under uncertainty

Multi-modal distributed energy system planning is applied in the context of smart grids, industrial energy supply, and in the building energy sector. In real-world applications, these systems are commonly characterized by existing system structures of different age where monitoring and investment are conducted in a closed-loop, with the iterative possibility to invest. The literature contains two main approaches to approximate this computationally intensive multi-period investment problem. The first approach simplifies the temporal decision-making process collapsing the multi-stage decision to a two-stage decision, considering uncertainty in the second stage decision variables. The second approach considers multi-period investments under the assumption of perfect foresight. In this work, we propose a multi-stage stochastic optimization model that captures multi-period investment decisions under uncertainty and solves the problem to global optimality, serving as a first-best benchmark to the problem. To evaluate the performance of conventional approaches applied in a multi-year setup, we propose a rolling horizon heuristic that on the one hand reveals the performance of conventional approaches applied in a multi-period set-up and on the other hand enables planners to identify approximate solutions to the original multi-stage stochastic problem. We conduct a real-world case study and investigate solution quality as well as the computational performance of the proposed approaches. Our findings indicate that the approximation of multi-period investments by two-stage stochastic approaches yield the best results regarding constraint satisfaction, while deterministic multi-period approximations yield better economic and computational performance.

Optimal Action Strategy approach for Desert Crossing Based on Dynamic Programming

It is a novel but challenging task that simulated desert crossing strategy for the game players under the specified conditions. This study is aimed to develop an optimal action strategy approach for desert crossing based on dynamic programming. According to the game rules, a multi-stage decision optimization model was first established to maximize the remaining funds of players within given game time. Then, a simulation-based dynamic programming algorithm was designed to solve the optimization model, and the optimal action strategy was obtained by backtracking crossing routes. Finally, two experimental case studies with different initial conditions were used to validate the model. In comparison with the benchmark exhaustive search algorithm, the proposed approach has higher performance and efficiency. The case studies confirm that the method can obtain the optimal action strategy under different game maps and initial resources.

Procedurally Justifiable Strategies: Integrating Context Effects into Multistage Decision Making

Procedurally Justifiable Strategies: Integrating Context Effects into Multistage Decision Making

Deduplication-Oriented Mutual-Assisted Cooperative Video Upload for Mobile Crowd Sensing

Deduplication (redundancy elimination) and cooperative video delivery are two effective ways to save the bandwidth and energy consumption and ensure video collection in damaged networks. However, deduplication in mobile crowd sensing (MSC) is primarily performed on texts and images. Furthermore, most of deduplication technologies require global information and are separated from video routing. To solve such problems, this paper propose a cooperative upload method for sensing videos, which performs the local video deduplication without excessive comparison and feature exchange. Also, we combine the content-aware deduplication with the dynamic relay selection to avoid the propagation of redundant items caused by the content-free video routing. Besides, we integrate a novel mutual-assisted mechanism into our method to motivate relay cooperation and load balance. We formulate the deduplication-supported cooperative video upload as a multi-stage decision problem. To solve the uncertainty of destinations in the decision problem, we develop a stepwise Mutual-Assisted Video Upload Algorithm (MAVU) to schedule video chunks and remove duplicates. Extensive experiments are conducted to compare MAVU with the existing algorithms. The numerical results validate that our MAVU has advantages over the other algorithms in collected video size and upload latency

Supply Chain Scheduling Using Double Deep Time-Series Differential Neural Network

The purpose of supply chain scheduling is to be able to find an optimized plan and strategy so as to optimize the benefits of the entire supply chain. This paper proposes a method for processing tightly coordinated supply chain task scheduling problems based on an improved Double Deep Timing Differential Neural Network (DDTDN) algorithm. The Semi-Markov Decision Process (SMDP) modeling of the state characteristics and action characteristics of the supply chain scheduling problem is realized, so as to transform the task scheduling problem of the tightly coordinated supply chain into a multi-stage decision problem. The deep neural network model can help fit the state value function, and the unique reinforcement learning online evaluation mechanism can realize the selection of the best action strategy combination, and optimize it under the condition of only the stator processing time. Finally, the optimal action strategy group is obtained.