Finite Action Sets Research Articles

On the Evaluation of Government Programs to Reduce Environmental Risk

This paper attempts to identify the correct benefit measure for a government project which might reduce the subjective probability of an environmental risk to health. The evaluation of government projects whose costs or benefits are uncertain occupies a considerable literature including Arrow and Lind, Fisher, and Graham. The value of reducing the probability of an adverse state (outcome) has been examined by Cook and Graham and more recently by Gallagher and Smith. In the latter two papers it was assumed that the individual at risk had access to perfect insurance or contingent claims markets. In such a perfect world the value of reducing the probability of an adverse state could be bounded by measures which Cook and Graham referred to as ransom and compensation and which Gallagher and Smith show to correspond to the more traditional Hicksian notions of compensating and equivalent variation. If the assumption of perfect contingent claims markets does not hold, it is not possible to set upper and lower bounds on the value of reducing the probability of an adverse state. While general forms of insurance are available to individuals at risk from an environmental hazard, specific policies do not allow for the continuous adjustment (reallocation of wealth) assumed to occur when the individual has access to perfect markets. In the next section a more plausible model is constructed. It assumes that the individual has a finite set of self-protective actions which he or she may take in response to a risk to future health. In addition, the government may undertake a project to reduce the risk to health. It is assumed that both the selfprotective action and the government project have the ability to influence the subjective (personal) probability of the future states, continued good health or ill-health. There are three major conclusions which emerge from this model. (a) The value of the government project is an option price (Graham) which may be zero or positive. If positive, the benefits from the project may take the form of cost-savings, risk (probability) reduction, or both. (b) The value of the government project is dependent on the set of self-protective actions and may change as new actions become available or the effectiveness of existing actions is reassessed by the individual. (c) The method of contingent valuation, in theory, would appear capable of eliciting the proper option price estimates for the project's value to the individual. It is even possible to distinguish between the value attributable to cost-savings and risk reduction. In the next section I present the model and demonstrate the above conclusions. In the final section an assessment is made of the in-

An Algorithm for Learning Without External Supervision and Its Application to Learning Control Systems

An algorithm is proposed for the design of ``on-line'' learning controllers to control a discrete stochastic plant. The subjective probabilities of applying control actions from a finite set of allowable actions using random strategy, after any plant-environment situation (called an ``event'') is observed, are modified through the algorithm. The subjective probability for the optimal action is proved to approach one with probability one for any observed event. The optimized performance index is the conditional expectation of the instantaneous performance evaluations with respect to the observed events and the allowable actions. The algorithm is described through two transformations, T1, and T2. After the ``ordering transformation'' T1 is applied on the estimates of the performance indexes of the allowable actions, the ``learning transformation'' T2 modifies the subjective probabilities. The cases of discrete and continuous features are considered. In the latter, the Potential Function Method is employed. The algorithm is compared with a linear reinforcement schenme and computer simulation results are presented.

How to select and how to rank projects: The Promethee method

In this paper, we present the Promethee methods, a new class of outranking methods in multicriteria analysis. Their main features are simplicity, clearness and stability. The notion of generalized criterion is used to construct a valued outranking relation. All the parameters to be defined have an economic signification, so that the decision maker can easily fix them. Two ways of treatment are proposed: It is possible to obtain either a partial preorder ( Promethee I) or a complete one ( Promethee II), both on a finite set of feasible actions. A comparison is made with the Electre III method. The stability of the results given by the two methods is analysed. Numerical applications are given in order to illustrate the properties of the new methods and some further problems are discussed.

Finite state dynamic programming with the total reward criterion

Dynamic programming problems with the expected total reward criterion and finite state space are considered. Two different value functions, the supremum over all policies and the supremum over all stationary policies, are examined. It is shown that both value functions satisfy the optimality equation and upper and lower bounds as well as conditions for equality for these functions are presented. Using these results characterizations for those policies which achieve the maximum within all policies and for those which achieve the maximum within all stationary policies are derived. Finally a modified version of the Howard policy improvement is proposed which yields an optimal policy for problems with finite action sets.

Preference relations on actions and criteria in multicriteria decision making

Let A be a finite set of feasible actions which are judged following several criteria. An outranking relation is defined on A by considering preference of the decision maker as a weak order on each criterion and the relation among criteria as a semi-order on the given set of criteria. Several ways of constructing outranking relations have been proposed. One of the most popular, introduced by B. Roy, for instance ELECTRE(s), is based on the use of weights related to criteria. In our approach, the knowledge of weights is replaced by the existence of a semi-order. A case study is developed. It deals with a computer selection problem.

Asynchronous Interactive Control Systems

A class of interactive control systems is derived by generalizing interactive manipulator control systems. The general structural properties of such systems are discussed and an appropriate general software implementation is proposed. This is based on the fact that tasks of interactive control systems can be represented as a network of a finite set of actions which have specific operational characteristics and specific resource requirements, and which are of limited duration. This has enabled the decomposition of the overall control algorithm into a set of subalgorithms, called subcontrollers, which can operate simultaneously and asynchronously. Coordinate transformations of sensor feedback data and actuator set-points have enabled the further simplification of the subcontrollers and have reduced their conflicting resource requirements. The modules of the decomposed control system are implemented as parallel processes with disjoint memory space communicating only by I/O. The synchronization mechanisms for dynamic resource allocation among subcontrollers and other synchronizaton mechanisms are also discussed in this paper. Such a software organization is suitable for the general form of multiprocessing using computer networks with distributed storage.

Multiple‐attribute decision making with partial information: The expected‐value criterion

AbstractWe consider the multiple‐attribute decision problem with finite action set and additive utility function. We suppose that the decision maker cannot specify nonnegative weights for the various attributes which would resolve the problem, but that he/she supplies ordinal information about these weights which can be translated into a set of linear constraints restricting their values. A bounded polytope W of feasible weight vectors is thus determined. Supposing that each element of W has the same chance of being the “appropriate one,” we compute the expected utility value of each action. The computation method uses a combination of numerical integration and Monte Carlo simulation and is equivalent to finding the center of mass of the bounded polytope W. Comparisons are made with another criterion already presented, the comparative hyper‐volume criterion, and two small examples are presented.

Solution of continuous‐time markovian decision models using infinite linear programming

AbstractInfinite‐horizon, countable‐state, continuous‐time Markovian decision models are solved by formulating as a pair of infinite linear‐programming problems. Expected discounted and average returns are considered as criterion functions. For both criterion functions, the existence of deterministic optimal stationary policies is established by solving the associated infinite linear‐programming problems. Computational procedures for finite state and action sets are discussed by considering associated finite linear‐programming problems.

An algorithm for learning without external supervision and its application to learning control systems

An algorithm is proposed for the design of on-line controllers to control a discrete stochastic plant. The subjective probabilities of applying control actions from a finite set of allowable actions using random strategy, after any plant-environment situation (called an event) is observed, are modified through the algorithm. The subjective probability for the optimal action is proved to approach one with probability one for any observed event. The optimized performance index is the conditional expectation of the instantaneous performance evaluations with respect to the observed events and the allowable actions. The algorithm is described through two transformations, T 1 and T 2 . After the ordering transformation T 1 is applied on the estimates of the performance indexes of the allowable actions, the learning transformation T 2 modifies the subjective probabilities. The cases of discrete and continuous features are considered. In the latter, the Potential Function Method is employed. The algorithm is compared with a linear reinforcement scheme and computer simulation results are presented.