State-dependent Strategy Research Articles

Nonlinear state-dependent pulse control for an SIRS epidemic model with varying size and its application to the transmission of brucellosis

As the disease spreads, it will inevitably cause important damage to the life and health of the population, resulting in changes in the population quantity. In addition, in some economically underdeveloped areas, limited medical resources will also have an important impact on the prevention and control of diseases. Based on these, a susceptible-infected-recovered-susceptible (SIRS) epidemic model is established, where state-dependent pulse control strategy, varying total population and limited medical resources are introduced. By using the qualitative theory of ordinary differential equation, differential inequality techniques, Poincaré map, and other methods, some sufficient conditions of the existence and orbital asymptotical stability of positive order-1 or order-2 periodic solution are obtained in various situations. Theoretical results imply that the proportion of infected class can be controlled at a desired low level for a long time and disease will not break out among population. Finally, based on realistic parameters of brucellosis in ruminants, numerical simulations have been performed to expalin/extend our analytical results and the feasibility of the state-dependent feedback control strategy.

Finite-horizon semi-Markov game for time-sensitive attack response and probabilistic risk assessment in nuclear power plants

Cybersecurity has drawn increasing attention in the nuclear industry. To improve the cyber-security posture, it is important to develop effective methods for cyber-attack response and cyber-security risk assessment. In this research, we develop a finite-horizon semi-Markov general-sum game between the defender (i.e., plant operator) and the attacker to obtain the time-sensitive attack response strategy and the real-time risk assessment in nuclear power plants. We propose methods for identifying system states of concern to reduce the state space and for determining state transition probabilities by integrating probabilistic risk assessment techniques. After a proper discretization of the developed continuous-time model, we use dynamic programming to derive the time-varying and state-dependent strategy of the defender based on the solution concept of the mixed-strategy Nash equilibrium. For risk assessment, three risk metrics are considered, and an exact analytical algorithm and a Monte Carlo simulation-based algorithm for obtaining the metrics are developed. Both players’ strategies and the risk metrics are illustrated using a digital feedwater control system used in pressurized water reactors. The results show that the proposed method can support plant operators in timely cyber-attack response and effective risk assessment, reduce the risk, and improve the resilience of nuclear power plants to malicious cyber-attacks.

Deficit of state-dependent risk attitude modulation in gambling disorder

Gambling disorder (GD) is often considered as a problem of trait-like risk preference. However, the symptoms of GD cannot be fully understood by this trait view. In the present study, we hypothesized that GD patients also had problem with a flexible control of risk attitude (state-dependent strategy optimization), and aimed to investigate the mechanisms underlying abnormal risk-taking of GD. To address this issue, we tested GD patients without comorbidity (GD group: n=21) and age-matched healthy control participants (HC group: n=29) in a multi-step gambling task, in which participants needed to clear ‘block quota’ (required units to clear a block, 1000–7000 units) in 20 choices, and conducted a task-functional magnetic resonance imaging (fMRI) experiment. Behavioral analysis indeed revealed a less flexible risk-attitude change in the GD group; the GD group failed to avoid risky choice in a specific quota range (low-quota condition), in which risky strategy was not optimal to solve the quota. Accordingly, fMRI analysis highlighted diminished functioning of the dorsolateral prefrontal cortex (dlPFC), which has been heavily implicated in cognitive flexibility. To our knowledge, the present study provided the first empirical evidence of a deficit of state-dependent strategy optimization in GD. Focusing on flexible control of risk attitude under quota may contribute to a better understanding of the psychopathology of GDs.

Flexible modulation of risk attitude during decision-making under quota

Risk attitude is often regarded as an intrinsic parameter in the individual personality. However, ethological studies reported state-dependent strategy optimization irrespective of individual preference. To synthesize the two contrasting literatures, we developed a novel gambling task that dynamically manipulated the quota severity (required outcome to clear the task) in a course of choice trials and conducted a task-fMRI study in human participants. The participants showed their individual risk preference when they had no quota constraint (‘individual-preference mode’), while they adopted state-dependent optimal strategy when they needed to achieve a quota (‘strategy-optimization mode’). fMRI analyses illustrated that the interplay among prefrontal areas and salience-network areas reflected the quota severity and the utilization of the optimal strategy, shedding light on the neural substrates of the quota-dependent risk attitude. Our results demonstrated the complex nature of risk-sensitive decision-making and may provide a new perspective for the understanding of problematic risky behaviors in human.

Dynamic Analysis of a Two-Language Competitive Model with Control Strategies

The dynamic behavior of a two-language competitive model is analyzed systemically in this paper. By the linearization and the Bendixson-Dulac theorem on dynamical system, some sufficient conditions on the globally asymptotical stability of the trivial equilibria and the existence and the stability of the positive equilibrium of this model are presented. Nextly, in order to protect the endangered language, an optimal control problem relative to this model is explored. We derive some necessary conditions to solve the optimal control problem and present some numerical simulations using a Runge-Kutta fourth-order method. Finally, the languages competitive model is extended to this model assessing the impact of state-dependent pulse control strategy. Using the Poincaré map, differential inequality, and method of qualitative analysis, we prove the existence and stability of positive order-1 periodic solution for this control model. Numerical simulations are carried out to illustrate the main results and the feasibility of state-dependent impulsive control strategy.

Networked Control of Distributed Energy Resources Using an Adaptive Communication Policy

This work presents a model-based networked control structure with an adaptive communication policy for managing Distributed Energy Resources (DERs) over a shared, resource-constrained communication network. The central objective is to find a state-dependent strategy for establishing and terminating communication between the supervisor and the DERs in a way that minimizes network resource utilization without jeopardizing the desired stability and performance properties. To this end, a bounded robust Lyapunov-based controller that enforces constrained closed-loop stability in the absence of communication suspension is initially designed for each DER. A dynamic model of each DER is then included within the supervisor to provide estimates of the states of the DER when measurements are not transmitted through the network. To determine when communication between a given DER and the supervisor must be re-established, the evolution of the Lyapunov function is monitored within the DER's stability region such that if it begins to breach a state-dependent stability or performance threshold at any time, the sensor suite is prompted to send its data over the network to update its corresponding model in the supervisor. Communication is then suspended for as long as the Lyapunov function satisfies the specified threshold. The underlying idea is to use the Lyapunov stability constraint for each DER as the basis for adaptively switching on or off the communication with the supervisor. Finally, the results are illustrated through an application to a solid oxide fuel cell example.

Skuas at penguin carcass: patch use and state-dependent leaving decisions in a top-predator

Foraging decisions depend not only on simple maximization of energy intake but also on parallel fitness-relevant activities that change the forager's 'state'. We characterized patch use and patch leaving rules of a top-predatory seabird, the Brown Skua (Catharacta antarctica lonnbergi), which during its reproductive period in the Antarctic establishes feeding territories in penguin colonies. In feeding trials, we observed how skuas foraged at penguin carcass patches and analysed patch leaving decisions by incorporating the estimated state of foraging birds and patch availability. Patches were exploited in a characteristic temporal pattern with exponentially decreasing remaining patch sizes (RPSs) and intake rates. Patch size decreased particularly fast in small compared to large patches and exploitation ended at a mean RPS of 47.6% irrespective of initial size. We failed to identify a measure which those birds equalized upon patch departure from raw data. However, when accounting for the birds' state, we ascertained remaining patch size and intake rates to have the lowest variance at departure whereas food amount and feeding time remained variable. Statistical correction for territory size only and combined with state had lower effects, but remaining patch size remained the measure with lowest coefficient of variation. Thus, we could clearly reject a fixed-time or fixed-amount strategy for territorial skuas and rather suggest a state-dependent strategy that equalizes remaining patch size. Thus our results provide evidence that under natural conditions, territorial skuas adjust their foraging decision on actual energy requirements, i.e. offspring number and age.

Bubbles, Crashes, and Sterilized Central Bank Intervention

We study the efficacy of sterilized foreign exchange intervention in a model where exchange rates are pushed away from their fundamental level by a rational bubble. The bubble is sustained by uncertainty about the post-crash price and about the market absorption capacity. We show that the bubble will be sustained in equilibrium if there is a reasonable probability that the bubble might burst in the first period. We then examine how sterilized interventions can change the equilibrium. In particular, we compare announced with secret interventions, when the central bank intervenes only once in the life of the bubble. We find that the degree of central bank commitment in bringing the exchange rate back to its fundamental level is crucial to determine whether the central bank will prefer to announce the intervention or keep it secret. We also study the conditions under which it is optimal to follow a state-dependent strategy where intervention takes place whenever the spot rate diverges from fundamentals by a set amount. Finally, we show that the central bank must gradually increase the size of interventions under a continuous intervention strategy.

DESIGN OF H∞ FEEDBACK CONTROL SYSTEMS WITH QUANTIZED SIGNALS

In this paper, we consider design of H∞ feedback control systems with quantized signals. We first assume that a state feedback has been designed for a continuous-time LTI system so that the closed-loop system is (Hurwitz) stable and a desired H∞ disturbance attenuation level is achieved, and that the states are quantized before they are passed to the controller. We propose a state-dependent strategy for updating the quantizer's parameter, so that the system is asymptotically stable and achieves the same H∞ disturbance attenuation level. We then extend the result to the case of observer-based dynamic output feedback where the measurement outputs are quantized, and propose an output-dependent strategy for updating the quantizer's parameter.

Temporal partitioning and aggression among foragers: modeling the effects of stochasticity and individual state

In many natural systems, individuals compete with conspecifics and heterospecifics for food and in some cases, individuals have been observed to partition their foraging times or fight over food. In this study, I investigated when it is optimal for a consumer to partition time and be aggressive. I formulated an individual-based model of foraging and used game theory to find evolutionarily stable strategies (ESSs) that maximize the probability that consumers survive each day and acquire their daily food requirements. Consumers choose when to forage and when to behave aggressively during confrontations over food. Consumers are each associated with a state variable, representing the amount of food eaten, and a dominance ranking, which describes how likely they are to forage and fight for food. The ESS is sensitive to food abundance, consumer state, and the dominance ranking. When food is abundant, temporal partitioning is often an ESS where the dominant consumer forages first; however, partitioning is unlikely to be an ESS when food abundance is low. Fights over food are typically avoided but may be part of an ESS when food abundance is low, both consumers are hungry, or the time available for foraging each day is drawing to a close. Because the ESS is sensitive to consumer state, the stochastic nature of finding food often results in considerable variation in observed foraging dynamics from one day to the next, even when consumers adopt the same state-dependent strategy each day. Results are compared with empirical observations, and I discuss implications for consumer coexistence.

Body-Mass Changes in Female Collared Flycatchers: State-Dependent Strategy

Body-Mass Changes in Female Collared Flycatchers: State-Dependent Strategy

Body-mass Changes in Female Collared Flycatchers: State-dependent Strategy

Abstract Collared Flycatcher (Ficedula albicollis) females experimentally were forced to prolong their incubation to address the question whether mass constancy during incubation and subsequent mass loss after hatching is actually related to breeding stage. Compared to unmanipulated control females a week after expected hatching, experimental females did not show any significant mass loss during prolonged incubation, whereas control females that successfully hatched their eggs dropped their mass significantly. Results show that body mass in females is associated with the reproductive stage and may reflect an adaptive strategy. High and stable incubation mass can be a fasting endurance in case of adverse weather conditions when females stay on the nest instead of foraging.

Body-mass Changes in Female Collared Flycatchers: State-dependent Strategy

Collared Flycatcher (Ficedula albicollis) females experimentally were forced to prolong their incubation to address the question whether mass constancy during incubation and subsequent mass loss after hatching is actually related to breeding stage. Compared to unmanipulated control females a week after expected hatching, experimental females did not show any significant mass loss during prolonged incubation, whereas control females that successfully hatched their eggs dropped their mass significantly. Results show that body mass in females is associated with the reproductive stage and may reflect an adaptive strategy. High and stable incubation mass can be a fasting endurance in case of adverse weather conditions when females stay on the nest instead of foraging.

Some simple, but efficient, push and pull heuristics for production sequencing for certain flexible manufacturing systems

We describe three simple rules for sequencing production in a small cell. The system of interest is one where parts of several different types are first processed on a common machine, then on separate, type-specific, production lines. The objective is to maximize the throughput rate, measured in parts produced per hour. Two of the rules (e.g. random, rotation) push parts into the cell, the last rule (cards) uses circulating cards to pull parts into the cell. We show that the resulting performance is close to, and sometimes better than, the one observed for an 'optimal' state-dependent push strategy.

Phenotypic plasticity in fluctuating environments: consequences of the lack of individual optimization

I consider the problem of characterizing the optimal plastic response when there are large-scale fluctuations in the environment affecting all population members. Individuals differ in their state, and each makes a reproductive decision before the environmental conditions are known. An individual's state, its decision, and environmental conditions together determine the number of descendants left at the next decision epoch. I restrict attention to the simple problem in which the state of the descendants left at this epoch does not depend on these three factors. Because the environment is fluctuating, there is no optimization; instead the best action in one state implicitly depends on the best action in other states. I characterize an optimal state-dependent strategy, give a method of computation, and show how behavior of each following the optimal strategy may be reinterpreted as a form of individual optimization. Concepts are illustrated with an example of optimal dutch size as a function of territory quality.

Dynamic optimization in fluctuating environments

We consider the problem of finding an optimal strategy for an organism in a large population when the environment fluctuates from year to year and cannot be predicted beforehand. In fluctuating environments, geometric mean fitness is the appropriate measure and individual optimization fails. Consequently, optimal strategies cannot be found by stochastic dynamic programming alone. We consider a simplified model in which each year is divided into two non-overlapping time intervals. In the first interval, environmental conditions are the same each year; in the second, they fluctuate from year to year. During the first interval, which ends at time of year T, population members do not reproduce. The state and time dependent strategy employed during the interval determines the probability of survival till T and the probability distribution of possible states at T given survival. In the interval following T, population members reproduce. The state of an individual at T and the ensuing environmental conditions determine the number of surviving descendants left by the individual next year. In this paper, we give a general characterization of optimal dynamic strategies over the first time interval. We show that an optimal strategy is the equilibrium solution of a (non-fluctuating environment) dynamic game. As a consequence, the behaviour of an optimal individual over the first time interval maximizes the expected value of a reward R$^{\ast}$ obtained at the end of the interval. However, R$^{\ast}$ cannot be specified in advance and can only be found once an optimal strategy has been determined. We illustrate this procedure with an example based on the foraging decisions of a parasitoid.

Inducible Defenses and the Allocation of Resources: A Minimal Model

We use dynamic-optimization models to assess the relative fitness consequences of inducible versus constitutive defense strategies. Our models assess the cost of defense in terms of reduced growth and reproduction of the defended organism. Resources are assumed to be allocated to growth, defense, or reproduction via a time- and state-dependent strategy that either does (in the inducible case) or does not (in the constitutive case) depend on the current predation threat. Inducible defenses are shown to be superior whenever predator attacks are unpredictable and sufficiently sustained (once initiated) for the defense to become effective. In the case of inducible defenses, the relative delays involved in switching among growth, reproduction, and defense are shown to be important determinants of the optimal allocation strategy.