Regret Function Minimization Algorithms

Introduction. When making decisions under uncertainty, Savage's criterion is sometimes used, or the criterion of minimizing regrets [1]. Usually, in the literature, this decision situation is described in matrix language. In other words, both the number of decision alternatives and the number of states of nature are finite. Of particular interest are situations where the admissible domain of decision variants is a convex set, and the regrets with respect to each state of nature are expressed by convex functions. In this paper, we propose numerical methods for minimizing Savage's regret function, constructed based on the subgradient projection method with automatic step size adjustment [2, 3]. The convergence of these methods is demonstrated. Goal. In the article, the Savage function is defined as a function that expresses the maximum regret value, assumed to be a convex function with respect to the decision factors. This function measures the effectiveness of each decision relative to the set of states of nature. It is important to note that computing the values of these functions is complex because of the need to know the optimal solution for each state of nature. This difficulty is successfully overcome in the process of solving the problem of minimizing functions on convex sets, thanks to parallel solutions of m "internal" algorithms based on the number of states of nature, and one external algorithm, aimed at minimizing the Savage function. Each of the m+1 algorithms represent modifications of the subgradient projection method with a programmable way of adjusting the step size. Depending on the complexity of the constraints and the required precision, three theorems have been proven, confirming the convergence of the investigated methods. Results obtained. Constructive numerical algorithms have been developed for determining optimal decision alternatives under uncertainty, when the number of states of nature is finite, the admissible domain of control factors is convex and compact, and the Savage regret function serves as a decision criterion. The convergence of the corresponding algorithms to the set of optimal solutions has been proven, without knowing the exact values of the Savage function. Instead, estimates obtained from parallel runs of algorithms were used, aimed at determining optimal solutions for each state of nature. Conclusions. Uncertainty poses significant difficulties in designing and making decisions. Any decision made under uncertainty represents a certain risk or a certain regret. In cases where the number of states of nature is finite, the decision domain is convex, the target function with respect to each state of nature is convex, and the Savage regret function is adopted as the decision criterion, the decision-making problem can be successfully solved using numerical algorithms based on the generalized gradient method. The implementation of the algorithm is relatively simple, and the fields of application can be very diverse. Keywords: uncertainty, decisions, Savage function, optimization.

Algorithms have been available for exact performance evaluation of multi-state k-out-of-n systems. However, especially for complex systems with a large number of components, and a large number of possible states, obtaining would be an interesting, significant issue. Reliability bounds will give us a range of the system reliability in a much shorter computation time, which allow us to make decisions more efficiently. The systems under consideration are multi-state k-out-of-n systems with i.i.d. components. We will focus on the probability of the system in states below a certain state d, denoted by Qsd. Based on the recursive algorithm proposed by Zuo & Tian [14] for performance evaluation of multi-state k-out-of-n systems with i.i.d. components, a reliability bounding approach is developed in this paper. The upper, and lower bounds of Qsd are calculated by reducing the length of the k vector when using the recursive algorithm. Using the bounding approach, we can obtain a good estimate of the exact Qsd value while significantly reducing the computation time. This approach is attractive, especially to complex systems with a large number of components, and a large number of possible states. A numerical example is used to illustrate the significance of the proposed bounding approach.

Dynamic facility location when the total number of facilities is uncertain: A decision analysis approach

The convex feasibility problem (CFP) is at the core of the modeling of many problems in various areas of science. Subgradient projection methods are important tools for solving the CFP because they enable the use of subgradient calculations instead of orthogonal projections onto the individual sets of the problem. Working in a real Hilbert space, we show that the sequential subgradient projection method is perturbation resilient. By this we mean that under appropriate conditions the sequence generated by the method converges weakly, and sometimes also strongly, to a point in the intersection of the given subsets of the feasibility problem, despite certain perturbations which are allowed in each iterative step. Unlike previous works on solving the convex feasibility problem, the involved functions, which induce the feasibility problem's subsets, need not be convex. Instead, we allow them to belong to a wider and richer class of functions satisfying a weaker condition, that we call "zero-convexity". This class, which is introduced and discussed here, holds a promise to solve optimization problems in various areas, especially in non-smooth and non-convex optimization. The relevance of this study to approximate minimization and to the recent superiorization methodology for constrained optimization is explained.

Conditional subgradient optimization — Theory and applications

Contents volume 71

Focuses on the development of a learning-based heuristic for scheduling heterogeneous machines. Although list scheduling methods have been widely used for a large class of scheduling problems, including the heterogeneous machine scheduling problem, they involve designing priority rules, which usually require a fair amount of insights on the characteristics of the problem to be solved. Instead of elaborate design of priority rules in a single step, we propose an iterative list scheduling process, which refines priority rules while generating a number of schedules. The proposed iterative list scheduling is formulated as a reinforcement learning problem, with states and actions defined in list scheduling. Due to the large number of possible states, reinforcement learning algorithms which use value functions in constructing an optimal policy may not be suitable for scheduling problems. Thus, to directly work with policies rather than the values of states, we propose genetic reinforcement learning (GRL), in which the policies of reinforcement learning are encoded into the chromosomes of genetic algorithms and a near-optimal policy is searched for by genetic algorithms. A GRL-based scheduler, called evolutionary intracell scheduler (EVIS), has been developed and applied to various scheduling problems such as the heterogeneous machine scheduling, the processor scheduling, the job-shop scheduling, the flow-shop scheduling, and the open-shop scheduling problems. The proposed model of EVIS, which has a linear order of population-fitness convergence, is verified by computer experiments. Even without fine tuning EVIS, the quality of solutions achieved by EVIS is comparable to that of problem-tailored heuristics for most of the problem instances.

Advanced diagnostic reasoners are being developed to make use of Bayesian fault probability distributions, experiential learning algorithms and actual net centric historical failure data to change dynamically or optimize the testing sequence. What is not covered is how also to reconfigure dynamically the system or unit under test and the test equipment from its present state to the state required for the desired test. This issue is especially acute in organizational level at-system testing where there can be a large number of possible states for the system under test and test hook ups when compared to Intermediate or Depot level Line Replaceable Unit (LRU) and/or board testing. This paper describe the work performed by the U.S. Army Automated Test Systems Division (ATSD) to design and implement a model based State Configuration Controller. How this solution can be used with any or even multiple reasoners, provided the proper interface is implemented, will also be discussed.

Phase transitions in models of human cooperation

In order to perform recognition and forecasting of an object state basing on its operation indicators monitoring results methods of machine learning can be applied: binary classification when separating objects into operable and faulty, multidimensional classification at larger number of possible states of a technical object and time series systems for forecasting. The present paper proposes a number of approaches providing enhancement of recognition performance and complex technical objects state forecasting accuracy; examples of calculations for real systems are given. Properties of an object can change in time so the parameters and structure of the models to be applied should also be changed. To achieve this, approaches to model updating are proposed: pseudogradient method of model parameters updating and then (if necessary) an algorithm of model structure updating.

C. Alvani and J. Naegele Commission of the European Communities, Joint Research Centre, European Institute for Transuranium Elements, Postfach 2266, D-7500 Karlsruhe 1, F.R.G. Resume. — L'etude systematique de la reflectivite optique a ete entreprise sur des echantillons de thorium dont la surface a ete preparee par polissage mecanique et electrolytique ou qui ont ete utilises tels quels apres obtention par le procede van Arkel. Les resultats des mesures de reflectivite optique sont compares avec les resultats publies. Une augmentation de la rugosite de la surface provoque une diminution de la reflectivite et induit probablement des plasmons de surface mis en evidence par le minimum de reflectivite mesure a 4 eV. Les meilleurs resultats n'ont ete obtenus qu'avec des surfaces electropolies, cependant, dans des conditions bien determinees, le procede van Arkel conduit a de bons resultats. Abstract. — Thorium samples, the surfaces of which were mechanically polished, electropolished and as grown from the van Arkel process, were systematically studied by optical reflectivity measurements. These optical data were compared with published measurements showing significant discrepancies. Increasing surface roughness was found to lower the reflectivity considerably and probably to induce surface plasmons indicated by a deep minimum in reflectivity at 4 eV. Best reflectivity data were obtained only for electropolished surfaces. Well chosen conditions for the van Arkel process give also good results. 1. Introduction. — To determine the character of the 5f-electrons in actinide metals, several optical investigations have been performed on thorium [1, 2] and americium [3]. For thorium, the surfaces were prepared in different ways (Ref. [1] : electro­polished single crystal ; Ref. [2] : bulk material elec­tropolished and sputter etched in a glow discharge). The results show significant discrepancies. Conse­quently these data must be interpreted differently. It must be pointed out as has been stated earlier [1] that serious difficulties arise from attri­buting features in the spectroscopic data to specific transitions in the calculated band scheme since a large number of possible states might be involved. Therefore, a better, not only qualitative [1,2] but quantitative comparison needs detailed matrix ele­ment calculations which are not available at present. On the other hand a comparison of the reflectivity of thorium with that of protactinium, the first metal in the actinide series which is assumed to have occu­pied 5f-levels, could help greatly to identify transi­tions involving 5f-states. The aim of this study was to find a preparation procedure that reliably avoids surface induced re­flectivity structures. Thorium was taken for compa­rison with and as stand-in for protactinium the metal to be measured next. For this purpose the reflectivity of differently prepared thorium samples has been measured from 2 eV to 5 eV and compared with published data. 2. Sample preparation. — The surfaces were pre­pared using three different methods : — Mechanical polishing down to 0.05 urn grain size. — Van Arkel process. In a sealed evacuated quartz tube, thorium is transported as iodide in the vapour phase to a hot substrate (electropolished tungsten) where the iodide dissociates and a poly-crystalline Th-layer of high purity is deposited. In order to get the Th-layer as compact and flat as possible the temperature of the substrate and the partial pressure of the iodide have been varied systematically. — Electrochemical polishing. The advantage of this procedure over other methods, specifically me­chanical abrasion, is principally in obtaining an unworked, strain free and microscopically flat surface [4]. A solution of perchloric acid (1.5 %) in methanol [5, 6, 7] has been used in a stainless steel beaker as cathode. A temperature of — 80 °C was maintained by a flow of cooled nitrogen gas. This procedure has been applied to single crystal and to polycrystalline van Arkel deposited Th samples. After preparation the samples were enclosed in a window-sealed sample holder under an argon atmo­sphere. 3. Results and discussion. — Figure 1 shows the reflectivity obtained for mechanically (curve a) and electrochemically (curve b) polished Th single crys­tals together with the published data. Curve a shows a remarkable reflectivity minimum at 4 eV as report­ed by Veal et al. [2], whereas curve b shows much higher reflectivity without the minimum as published

Management studies can be regarded as applied cybernetics because both disciplines are concerned with designing and steering complex and dynamic systems. All managers are cyberneticists, intentionally or otherwise, because they are trying to bring under control a system of incredibly high variety. According to a pioneer of management cybernetics, Stafford Beer (1966, 239), management is the mastery of “high variety systems”. As discussed above, even a medium-sized company has an incredibly large number of possible states. A manager will undoubtedly find indications in cybernetics as to how they can perform complex tasks.

A main research direction in the field of evolutionary machine learning is to develop a scalable classifier system to solve high-dimensional problems. Recently work has begun on autonomously reusing learned building blocks of knowledge to scale from low-dimensional problems to high-dimensional ones. An XCS-based classifier system, known as XCSCFC, has been shown to be scalable, through the addition of expression tree-like code fragments, to a limit beyond standard learning classifier systems. XCSCFC is especially beneficial if the target problem can be divided into a hierarchy of subproblems and each of them is solvable in a bottom-up fashion. However, if the hierarchy of subproblems is too deep, then XCSCFC becomes impractical because of the needed computational time and thus eventually hits a limit in problem size. A limitation in this technique is the lack of a cyclic representation, which is inherent in finite state machines (FSMs). However, the evolution of FSMs is a hard task owing to the combinatorially large number of possible states, connections, and interaction. Usually this requires supervised learning to minimize inappropriate FSMs, which for high-dimensional problems necessitates subsampling or incremental testing. To avoid these constraints, this work introduces a state-machine-based encoding scheme into XCS for the first time, termed XCSSMA. The proposed system has been tested on six complex Boolean problem domains: multiplexer, majority-on, carry, even-parity, count ones, and digital design verification problems. The proposed approach outperforms XCSCFA (an XCS that computes actions) and XCSF (an XCS that computes predictions) in three of the six problem domains, while the performance in others is similar. In addition, XCSSMA evolved, for the first time, compact and human readable general classifiers (i.e., solving any n-bit problems) for the even-parity and carry problem domains, demonstrating its ability to produce scalable solutions using a cyclic representation.

The fact that relatively simple entities, such as particles or neurons, or even ants or bees or humans, give rise to fascinatingly complex behaviour when interacting in large numbers is the hallmark of complex systems science. Agent-based models are frequently employed for modelling and obtaining a predictive understanding of complex systems. Since the sheer number of equations that describe the behaviour of an entire agent-based model often makes it impossible to solve such models exactly, Monte Carlo simulation methods must be used for the analysis. However, unlike pairwise interactions among particles that typically govern solid-state physics systems, interactions among agents that describe systems in biology, sociology or the humanities often involve group interactions, and they also involve a larger number of possible states even for the most simplified description of reality. This begets the question: when can we be certain that an observed simulation outcome of an agent-based model is actually stable and valid in the large system-size limit? The latter is key for the correct determination of phase transitions between different stable solutions, and for the understanding of the underlying microscopic processes that led to these phase transitions. We show that a satisfactory answer can only be obtained by means of a complete stability analysis of subsystem solutions. A subsystem solution can be formed by any subset of all possible agent states. The winner between two subsystem solutions can be determined by the average moving direction of the invasion front that separates them, yet it is crucial that the competing subsystem solutions are characterised by a proper composition and spatiotemporal structure before the competition starts. We use the spatial public goods game with diverse tolerance as an example, but the approach has relevance for a wide variety of agent-based models.

The interaction between ligands and proteins usually induces changes in protein thermal stability with modifications in the midpoint denaturation temperature, enthalpy of unfolding, and heat capacity. These modifications are due to the coupling of unfolding with binding equilibrium. Furthermore, they can be attained by changes in protein structure and conformational flexibility induced by ligand interaction. To study these effects we have used bovine serum albumin (BSA) interacting with three different anilinonaphthalene sulfonate derivatives (ANS). These ligands have different effects on protein stability, conformation, and dynamics. Protein stability was studied by differential scanning calorimetry and fluorescence spectroscopy, whereas conformational changes were detected by circular dichroism and infrared spectroscopy including kinetics of hydrogen/deuterium exchange. The order of calorimetric midpoint of denaturation was: 1,8-ANS-BSA > 2,6-ANS-BSA > free BSA >> (nondetected) bis-ANS-BSA. Both 1,8-ANS and 2,6-ANS did not substantially modify the secondary structure of BSA, whereas bis-ANS induced a distorted alpha-helix conformation with an increase of disordered structure. Protein flexibility followed the order: 1,8-ANS-BSA < 2,6-ANS-BSA < free BSA << bis-ANS-BSA, indicating a clear correlation between stability and conformational flexibility. The structure induced by an excess of bis-ANS to BSA is compatible with a molten globule-like state. Within the context of the binding landscape model, we have distinguished five conformers (identified by subscript): BSA(1,8-ANS), BSA(2,6-ANS), BSA(free), BSA(bis-ANS), and BSA(unfolded) among the large number of possible states of the conformational dynamic ensemble. The relative population of each distinguishable conformer depends on the type and concentration of ligand and the temperature of the system.