Binary Actions Research Articles

Is Reinforcement Learning Good at American Option Valuation?

This paper investigates algorithms for identifying the optimal policy for pricing American Options. The American Option pricing is reformulated as a Sequential Decision-Making problem with two binary actions (Exercise or Continue), transforming it into an optimal stopping time problem. Both the least square Monte Carlo simulation method (LSM) and Reinforcement Learning (RL)-based methods were utilized to find the optimal policy and, hence, the fair value of the American Put Option. Both Classical Geometric Brownian Motion (GBM) and calibrated Stochastic Volatility models served as the underlying uncertain assets. The novelty of this work lies in two aspects: (1) Applying LSM- and RL-based methods to determine option prices, with a specific focus on analyzing the dynamics of “Decisions” made by each method and comparing final decisions chosen by the LSM and RL methods. (2) Assess how the RL method updates “Decisions” at each batch, revealing the evolution of the decisions during the learning process to achieve optimal policy.

The binary actions of simple groups with a single conjugacy class of involutions

Abstract We continue our investigation of binary actions of simple groups. In this paper, we demonstrate a connection between the graph Γ ⁢ ( C ) \Gamma(\mathcal{C}) based on the conjugacy class 𝒞 of the group 𝐺, which was introduced in our previous work, and the notion of a strongly embedded subgroup of 𝐺. We exploit this connection to prove a result concerning the binary actions of finite simple groups that contain a single conjugacy class of involutions.

The Set of Equilibria in Max-Min Two Groups Contests with Binary Actions and a Private Good Prize

The Set of Equilibria in Max-Min Two Groups Contests with Binary Actions and a Private Good Prize

The Set of Equilibria in Max-Min Two Groups Contests with Binary Actions and a Private Good Prize

The Set of Equilibria in Max-Min Two Groups Contests with Binary Actions and a Private Good Prize

Selling information in competitive environments

Data buyers compete in a game of incomplete information about which a single data seller owns some payoff-relevant information. The seller faces a joint information- and mechanism-design problem: deciding which information to sell, while eliciting the buyers' types and imposing payments. We derive the welfare- and revenue-optimal mechanisms for a class of games with binary actions and states. Our results highlight the critical properties of selling information in competitive environments: (i) the negative externalities arising from buyer competition increase the profitability of recommending the correct action to one buyer exclusively; (ii) for the buyers to follow the seller's recommendations, the degree of exclusivity must be limited; (iii) the buyers' obedience constraints also reduce the distortions in the allocation of information introduced by a monopolist; (iv) as competition becomes fiercer, these limitations become more severe, weakening the impact of market power on the optimal allocation of information.

Cooperative Dilemmas with Binary Actions and Multiple Players

The prisoner’s dilemma, the snowdrift game, and the stag hunt are two-player symmetric games that are often considered as prototypical examples of cooperative dilemmas across disciplines. However, surprisingly little consensus exists about the precise mathematical meaning of the words “cooperation” and “cooperative dilemma” for these and other binary-action symmetric games, in particular when considering interactions among more than two players. Here, we propose definitions of these terms and explore their evolutionary consequences on the equilibrium structure of cooperative dilemmas in relation to social optimality. We show that our definition of cooperative dilemma encompasses a large class of collective action games often discussed in the literature, including congestion games, games with participation synergies, and public goods games. One of our main results is that regardless of the number of players, all cooperative dilemmas—including multi-player generalizations of the prisoner’s dilemma, the snowdrift game, and the stag hunt—feature inefficient equilibria where cooperation is underprovided, but cannot have equilibria in which cooperation is overprovided. We also find simple conditions for full cooperation to be socially optimal in a cooperative dilemma. Our framework and results unify, simplify, and extend previous work on the structure and properties of cooperative dilemmas with binary actions and two or more players.

Optimal Targeting in Super-Modular Games

We study an optimal targeting problem for super-modular games with binary actions and finitely many players. The considered problem consists in the selection of a subset of players of minimum size, such that when the actions of these players are forced to a controlled value while the others are left to repeatedly play a best response action, the system will converge to the greatest Nash equilibrium of the game. Our main contributions consist in showing that the problem is NP-complete and in proposing an efficient iterative algorithm for its solution with provable probabilistic convergence properties. We discuss in detail the special case of network coordination games and its relation with the graph-theoretic notion of cohesiveness. Finally, through numerical simulations we compare the efficacy of our approach with respect to naive heuristics based on classical network centrality measures.

Extending the concept of pro-environmental action and behaviour: a binary perspective

Many studies have explored pro-environmental actions. This paper aims to extend the concept of pro-environmental action in the context of binary themes or perspectives. The extended conceptualisation is drawn from a study that investigates the influence of residential outdoor environmental education courses on Malaysian participants’ environmental attitudes and behaviours using a life history approach. Data analysis revealed six binary pro-environmental actions: conforming/transforming action, compulsory/optional action, intentional/unintentional action, public/private sphere action, activist/non-activist action, and biospheric/anthropocentric-oriented action. The binary actions presented in this paper pull together all of the binary themes, both pre-existing ones and those emerging from the current study data. The paper provides recommendations about how one can contribute to environmental quality and positively impact the sustainable development agenda.

Computing Equilibria in Binary Networked Public Goods Games

Public goods games study the incentives of individuals to contribute to a public good and their behaviors in equilibria. In this paper, we examine a specific type of public goods game where players are networked and each has binary actions, and focus on the algorithmic aspects of such games. First, we show that checking the existence of a pure-strategy Nash equilibrium is NP-complete. We then identify tractable instances based on restrictions of either utility functions or of the underlying graphical structure. In certain cases, we also show that we can efficiently compute a socially optimal Nash equilibrium. Finally, we propose a heuristic approach for computing approximate equilibria in general binary networked public goods games, and experimentally demonstrate its effectiveness. Due to space limitation, some proofs are deferred to the extended version1.

On Information Design with Spillovers

An information designer has access to a set of experiments and decides which of these to assign to each of the agents in a directed network. The network encodes informational spillovers: an agent has access to the experiments assigned to her, as well as to those assigned to any other agent who has a directed path to her. We establish that the designer's problem in any network can be reduced to an equivalent problem in a directed acyclic network. We show that when in the latter network each agent follows at most one other agent (i.e., each node has in-degree at most one), the optimal information structure can be obtained in a tractable way. The problem becomes intractable if some agents follow multiple other agents. Thus, qualitatively, following multiple information sources is what makes information design problems intractable in the presence of spillovers. We also study a voting game with binary actions in the presence of spillovers. We show that when the followers are more pessimistic (i.e., have higher posterior mean requirements to take action $1$), the network effects do not play a role, and a certain monotone information structure is optimal. When the followers are more optimistic a monotone information structure is still optimal if each agent follows at most one other agent, but not in general. That said, in the latter case an optimal monotone information structure can be obtained in a tractable way by using an algorithm we provide, provided that the underlying network has bounded treewidth.

Trade clustering and power laws in financial markets

This study provides an explanation for the emergence of power laws in asset trading volume and returns. We consider a two‐state model with binary actions, where traders infer other traders' private signals regarding the value of an asset from their actions and adjust their own behavior accordingly. We prove that this leads to power laws for equilibrium volume and returns whenever the number of traders is large and the signals for asset value are sufficiently noisy. We also provide numerical results showing that the model reproduces observed distributions of daily stock volume and returns.

Sequential Naive Learning

We analyze boundedly rational updating from aggregate statistics in a model with binary actions and binary states. Agents each take an irreversible action in sequence after observing the unordered set of previous actions. Each agent first forms her prior based on the aggregate statistic, then incorporates her signal with the prior based on Bayes rule, and finally applies a decision rule that assigns a (mixed) action to each belief. If priors are formed according to a discretized DeGroot rule, then actions converge to the state (in probability), i.e., \emph{asymptotic learning}, in any informative information structure if and only if the decision rule satisfies probability matching. This result generalizes to unspecified information settings where information structures differ across agents and agents know only the information structure generating their own signal. Also, the main result extends to the case of $n$ states and $n$ actions.

Partial Policy-Based Reinforcement Learning for Anatomical Landmark Localization in 3D Medical Images.

Utilizing the idea of long-term cumulative return, reinforcement learning (RL) has shown remarkable performance in various fields. We follow the formulation of landmark localization in 3D medical images as an RL problem. Whereas value-based methods have been widely used to solve RL-based localization problems, we adopt an actor-critic based direct policy search method framed in a temporal difference learning approach. In RL problems with large state and/or action spaces, learning the optimal behavior is challenging and requires many trials. To improve the learning, we introduce a partial policy-based reinforcement learning to enable solving the large problem of localization by learning the optimal policy on smaller partial domains. Independent actors efficiently learn the corresponding partial policies, each utilizing their own independent critic. The proposed policy reconstruction from the partial policies ensures a robust and efficient localization, where the sub-agents uniformly contribute to the state-transitions based on their simple partial policies mapping to binary actions. Experiments with three different localization problems in 3D CT and MR images showed that the proposed reinforcement learning requires a significantly smaller number of trials to learn the optimal behavior compared to the original behavior learning scheme in RL. It also ensures a satisfactory performance when trained on fewer images.

Optimality of Double Intervals in Persuasion: A Convex Programming Framework

We study information design problems, where the payoff of the information designer is an increasing step function of the posterior mean of the state her signals induce. Settings where the designer employs public signaling mechanisms to persuade multiple receivers (who have (i) finitely many actions that exhibit strategic complementarities, and (ii) best responses that are increasing in the state), and where her payoff is increasing in the receivers’ actions, are special cases. We provide a novel and tractable convex optimization framework for obtaining an optimal signaling mechanism. This mechanism is deterministic and admits a double-interval structure: it pools some “good” and “bad” states, and with each signal realization it associates at most two subintervals of the set of states. Our framework is broadly applicable, and we illustrate this through two very different applications. In the first application, we focus on receivers in a network who take binary actions (that can represent, e.g., adoption of a product of unknown quality), and characterize the optimal mechanism in terms of the network’s cores. Interestingly, as a consequence of the double-interval structure, the set of states for which a receiver takes action 1 is in general nonconvex. We show that in large random networks the designer can obtain asymptotically optimal mechanisms even in the absence of detailed network information, by using only the degree distribution. In our second application, we focus on persuading a single receiver whose type is private. Using a slight modification of our framework, we establish that the optimal mechanism is based on a laminar interval structure – a generalization of the double-interval structure – where the number of subintervals associated with each signal realization increases with the number of types.

Naive Learning Through Probability Matching

We analyze boundedly rational updating in a repeated interaction network model with binary actions and binary states. Agents form beliefs according to discretized DeGroot updating and apply a decision rule that assigns a (mixed) action to each belief. We first show that under weak assumptions random decision rules are sufficient to achieve agreement in finite time in any strongly connected network. Our main result establishes that naive learning can be achieved in any large strongly connected network. That is, if beliefs satisfy a high level of inertia, then there exist corresponding decision rules coinciding with probability over-matching such that the eventual agreement action matches the true state, with a probability converging to one as the network size goes to infinity.

Persuasion in Networks: Public Signals and k-Cores

We consider a setting where agents in a social network take binary actions, which exhibit local strategic complementarities. The agents are a priori uninformed about an underlying payoff-relevant state. An information designer wants to maximize the expected number of agents who take action 1, and she can commit to a signaling mechanism which upon the realization of the state sends an informative signal to all the agents. We study the structure and design of the optimal public signaling mechanisms. We establish that given a signal realization, the set of agents who take action 1 correspond to a k-core of the underlying network, for some k. Using this we show that the designer’s payoff is an increasing step function of the posterior mean her signals induce. We provide a convex optimization formulation and an algorithm that obtain the optimal information structure whenever the designer’s payoff exhibits this structure. The latter structural property is prevalent, thereby making our approach useful well beyond network persuasion. The optimal mechanism is based on a double-interval partition of the set of states. In particular, it associates up to two subintervals of the set of possible state realizations with each k-core. When the state is in these intervals the mechanism publicly reveals this, which induces the agents in k-core to take action 1. We also study a class of random networks with known limiting degree distributions, and provide a framework for obtaining asymptotically optimal public signaling mechanisms for these networks. Our approach relies on characterizing the sizes of the cores of such networks (asymptotically) and uses only the degree distribution of the random networks, thereby making it useful even when the network structure is not fully known. Finally, we explore which networks are more amenable to persuasion, and show that more assortative connection structures lead to larger payoffs for the designer. On the other hand, the dependence of the designer’s payoff to the agents’ degrees can be quite counterintuitive. In particular, we focus on networks sampled uniformly at random from the set of all networks consistent with a degree sequence. We show that when the degrees of some nodes increase, this can reduce the designer’s payoff, despite the increase in the extent of the network externalities.

Enhancement of Crude Oil Biodegradation by Immobilized Bacterial Consortium in Small Batch and Continuous Bioreactor Modes

Oil spills besides the discarding of oil refineries byproducts directly into the environment is a major source of nature pollution which contributes to environmental contamination and health problems. Firstly for solving this problem oil degrading bacterial consortium (Pseudomonas sp. sp48 and Bacillus cereus M12 which were isolated from soil samples from Bahary area, SidiKerir branch, Alexandria, Egypt, respectively) was immobilized with different carriers, where rice straw was selected as the best one for oil degradation. Conditional optimization for immobilized bacterial consortium has been explored by testing various amounts of preculture (2, 3, 4, 5.5 and 7%), different incubation time with rice straw (1-5 h), enumeration of viable cells immobilized and different oil concentrations (1, 2, 3, and 4%). The degradation of crude oil (1% concentration) reached to 61.6% due to biological action alone while it raised to 97.14% by binary actions of biological and physical after 6 days incubation time and applying the optimum conditions (4% bacterial consortium preculture and 3 h incubation time with the carrier). Finally application study has been carried out using the ideal model conditions in small batch bioreactor and continuous to treat oil contaminated water in an open system. The results showed that the crude oil degradation biologically only and biologically/ physically reached to 56.2% and 87.2%, respectively. Whereas the continuous addition of 1% oil concentration for three times the removal reached to 81.27%. This technique of immobilization produces a promising result for solving the oil pollution problem due to the collaborating effects for both physical and biological actions.

МЕТОДИКА ОРГАНІЗАЦІЇ САМОСТІЙНОЇ РОБОТИ З ДИСЦИПЛІНИ “АПАРАТНЕ ЗАБЕЗПЕЧЕННЯ ПЕРСОНАЛЬНОГО КОМП’ЮТЕРА”

The urgency of the research is conditioned by the need to find the scientific developments regarding the organization of independent work of future specialists in computer technology and the development of a methodology for its implementation on the basis of modern educational trends. The purpose of the study is to develop a methodology of organizing the independent work on the discipline “Hardware of the personal computer” of future specialists in network administration. In the article the didactic characteristics of the method are presented: the tasks, principles, methods, forms and means.The key is to determine the tasks of the methodology: to increase the motivation of students; to ensure the formation of fundamental cognitive processes and the formation of self-regulation skills, self-planning, self-control of the organization; to implement the “immersion procedures” of students in the situation of future professional activities, the search for ways to perform the tasks that are the components of the project design discipline. The basis of the methodology is the task for independent work. The content of the tasks is based on the use of basic cognitive laws: modeling, experiment, forecast, evaluation, diagnosis, planning, causality, judgment, negotiation or negotiation skills, influence, teamwork, description. That is, the conditions of the problem are formulated for the implementation of these processes. The article gives an example of a task. The article presents the stages, actions of the teacher and student on the tasks. The given binary actions allow to provide self-control loading tasks (self-regulation), self-control of the plan of implementation of the organization (self-planning), self-control of the organization (individual self-organization) activities related to the task.The methods of research are: theoretical, and empirical. The criteria of the assessment of effectiveness of the developed methodology are defined: the criterion of the influence of the methodology on the formation of the motivation of professional activity K1; criteria of the formation of knowledge and skills K2; the criterion of the influence of the methodology on the formation of professionally important qualities of K3 and their indicators. The results of experimental research are presented. The practical significance: the materials of the research can be used in the process of preparation of specialists in computer technology.

Bayesian Group Decisions: Algorithms and Complexity

We study the computations that Bayesian agents undertake when exchanging opinions over a network. The agents act repeatedly on their private information and take myopic actions that maximize their expected utility according to a fully rational posterior belief. We show that such computations are NP-hard for two natural utility functions: one with binary actions, and another where agents reveal their posterior beliefs. In fact, we show that distinguishing between posteriors that are concentrated on different states of the world is NP-hard. Therefore, even approximating the Bayesian posterior beliefs is hard. We also describe a natural search algorithm to compute agents' actions, which we call elimination of impossible signals, and show that if the network is transitive, the algorithm can be modified to run in polynomial time.

Cost‐effective conservation of an endangered frog under uncertainty

How should managers choose among conservation options when resources are scarce and there is uncertainty regarding the effectiveness of actions? Well-developed tools exist for prioritizing areas for one-time and binary actions (e.g., protect vs. not protect), but methods for prioritizing incremental or ongoing actions (such as habitat creation and maintenance) remain uncommon. We devised an approach that combines metapopulation viability and cost-effectiveness analyses to select among alternative conservation actions while accounting for uncertainty. In our study, cost-effectiveness is the ratio between the benefit of an action and its economic cost, where benefit is the change in metapopulation viability. We applied the approach to the case of the endangered growling grass frog (Litoria raniformis), which is threatened by urban development. We extended a Bayesian model to predict metapopulation viability under 9 urbanization and management scenarios and incorporated the full probability distribution of possible outcomes for each scenario into the cost-effectiveness analysis. This allowed us to discern between cost-effective alternatives that were robust to uncertainty and those with a relatively high risk of failure. We found a relatively high risk of extinction following urbanization if the only action was reservation of core habitat; habitat creation actions performed better than enhancement actions; and cost-effectiveness ranking changed depending on the consideration of uncertainty. Our results suggest that creation and maintenance of wetlands dedicated to L.raniformis is the only cost-effective action likely to result in a sufficiently low risk of extinction. To our knowledge we are the first study to use Bayesian metapopulation viability analysis to explicitly incorporate parametric and demographic uncertainty into a cost-effective evaluation of conservation actions. The approach offers guidance to decision makers aiming to achieve cost-effective conservation under uncertainty.