Impulse Control Problem Research Articles

Optimal control of continuous systems with impulse controls

Summary Impulse control problems, in which a continuously evolving state is modified by discrete control actions, have applications in epidemiology, medicine, and ecology. In this paper, we present a simple method for solving impulse control problems for systems of differential equations. In particular, we show how impulse control problems can be reformulated and solved as discrete optimal control problems. The method is illustrated with two examples. Published 2014. This article has been contributed to by US Government employees and their work is in the public domain in the USA.

Some Numerical Methods for Solving Stochastic Impulse Control in Natural Gas Storage Facilities

The valuation of gas storage facilities is characterized as a stochastic impulse control problem with finite horizon resulting in Hamilton-Jacobi-Bellman (HJB) equations for the value function. In this context the two catagories of solving schemes for optimal switching are discussed in a stochastic control framework. We reviewed some numerical methods which include approaches related to partial differential equations (PDEs), Markov chain approximation, nonparametric regression, quantization method and some practitioners’ methods. This paper considers optimal switching problem arising in valuation of gas storage contracts for leasing the storage facilities, and investigates the recent developments as well as their advantages and disadvantages of each scheme based on dynamic programming principle (DPP).

Role of CNR1 polymorphisms in moderating the effects of psychosocial adversity on impulsivity in adolescents.

Enhanced endocannabinoid signaling has been implicated in typically adolescent behavioral features such as increased risk-taking, impulsivity and novelty seeking. Research investigating the impact of genetic variants in the cannabinoid receptor 1 gene (CNR1) and of early rearing conditions has demonstrated that both factors contribute to the prediction of impulsivity-related phenotypes. The present study aimed to test the hypothesis of an interaction of the two most studied CNR1 polymorphisms rs806379 and rs1049353 with early psychosocial adversity in terms of affecting impulsivity in 15-year-olds from an epidemiological cohort sample followed since birth. In 323 adolescents (170 girls, 153 boys), problems of impulse control and novelty seeking were assessed using parent-report and self-report, respectively. Exposure to early psychosocial adversity was determined in a parent interview conducted at the age of 3 months. The results indicated that impulsivity increased following exposure to early psychosocial adversity, with this increase being dependent on CNR1 genotype. In contrast, while individuals exposed to early adversity scored higher on novelty seeking, no significant impact of genotype or the interaction thereof was detected. This is the first evidence to suggest that the interaction of CNR1 gene variants with the experience of early life adversity may play a role in determining adolescent impulsive behavior. However, given that the reported findings are obtained in a high-risk community sample, results are restricted in terms of interpretation and generalization. Future research is needed to replicate these findings and to identify the mediating mechanisms underlying this effect.

Synchronization of chaotic systems using fuzzy impulsive control

This paper investigates the problem of fuzzy impulsive control to synchronize two chaotic systems using a novel time-dependent Lyapunov function approach. Compared with the existing time-independent Lyapunov methods, the proposed method enables us to exploit more information on the impulsive intervals. Initially, using the Lyapunov technique and two parameterized linear matrix inequality (LMI) techniques, some less conservative synchronization criteria via a fuzzy impulsive controller using the states of both drive and response chaotic systems are derived. Subsequently, an LMI approach to designing such a fuzzy impulsive controller is developed to realize the synchronization. Finally, the proposed method is applied to the chaotic Lorenz system and Rosler system to illustrate its effectiveness.

An optimal machine maintenance problem with probabilistic state constraints

We consider a machine that is maintained via two types of maintenance action: (i) continuous (minor) maintenance that curbs natural degradation of the machine; and (ii) overhaul (major) maintenance that takes place at certain discrete time points and significantly improves the condition of the machine. We introduce an impulsive stochastic differential equation to model the condition of the machine over the time horizon. The problem we investigate is to choose the continuous maintenance rate and the overhaul maintenance times to minimize the total cost of operating and maintaining the machine, where probabilistic state constraints are imposed to ensure that the machine’s state and output meet minimum acceptable levels with high probability. This impulsive stochastic optimal control problem is first transformed into a deterministic optimal control problem with state jumps and continuous inequality constraints. We then show that this equivalent problem can be solved using a combination of the control parameterization technique, the time-scaling transformation, and the constraint transcription method. Finally, we illustrate our approach by solving a numerical example.

The Canonical Theory of the Impulse Process Optimality

The paper is devoted to the development of the canonical theory of the Hamilton–Jacobi optimality for nonlinear dynamical systems with controls of the vector measure type and with trajectories of bounded variation. Infinitesimal conditions of the strong and weak monotonicity of continuous Lyapunov-type functions with respect to the impulsive dynamical system are formulated. Necessary and sufficient conditions of the global optimality for the problem of the optimal impulsive control with general end restrictions are represented. The conditions include the sets of weak and strong monotone Lyapunov-type functions and are based on the reduction of the original problem of the optimal impulsive control a finite-dimensional optimization problem on an estimated set of connectable points.

A control systems analysis of HIV prevention model using impulsive input

We investigate a control systems analysis on HIV infection dynamics with regard to enhancement of the immune response. The HIV dynamic model is modified to include the pharmacokinetics and pharmacodynamics of antiretroviral HIV drugs, and the intake of drug is considered as impulsive control input. As it is administrated at discrete time instants, we assume that this yields an impulsive control problem for a nonlinear continuous-time system. Based on this new model, we study clinical experiments about antirretoviral treatments via numerical simulation and analyse the experimental results. It is noted that this modeling approach can help to provide a theoretical explanation of the clinical results. The analysis result in the paper could imply that the protocol of the experiment might enhance the immune response against HIV.

On question about representation of attraction sets in an impulse control problem

We study the asymptotic behavior of terminal positions attainability sets in an abstract linear control problem. We assume that energy resources available for the control accumulate over time. A control coefficient in the right-hand part of the differential equation can be a discontinuous function. We assume that admissible controls have a oneimpulse nature asymptoticly. This motivate us to study attraction sets for suitable characterizations of admissible controls set. For this purpose we used an extension construction in the class of finitely additive measures. The representations of these attraction sets was obtained. The coincidence of the attraction sets was shown. Mathematics Subject Classification: 28E99, 34H99, 93B03

EPA-1258 – Diagnosable impulse control behaviours in patients with parkinson's disease

Impulse control behaviours (ICB) are repetitive and excessive activities that are subsyndromal, unfulfilled the diagnostic criteria but potentially challenging problem. Impulse control disorder (ICD) on contrary is an established diagnosis which requires clinical attention. To determine the proportion of patients with ICD among those screened positive with ICB among patients with Parkinson's disease in an Asian population. The factors associated with ICB were also examined. Eighty consecutive patients attending neurology clinic who fulfilled the inclusion criteria i.e. diagnosis of idiopathic Parkinson's, on stable dopaminergic medication for at least 3 months were recruited in this prospective cross-sectional study. ICB were identified using the Questionnaire for Impulsive-Compulsive Disorder for Parkinson's disease (QUIP). Those who were screened positive, with controls, then underwent semi structured interview based on SCID-ICD. 11.3% were screened positive for ICB; these were significantly associated with higher education (p=0.022), advanced stage of disease (p=0.026) and higher levodopa dosage (p= 0.01). The most frequent ICB was compulsive medication use (7.5%), followed by hobbyism (6.25%), hypersexuality (5%), compulsive buying (3.75%), punding (2.5%), walkabout (2.5%), compulsive eating (1.25%) and pathological gambling (1.25%). 50% of those with positive QUIP (section1) fulfilled the criteria for ICD (50% positive-predictive value) while none with negative QUIP had ICD (100% negative predictive value). About 1 in 10 patients with Parkinson's disease displayed ICB, half of which might have a diagnosable ICD. The risk should alert the physicians of the need for routine assessment of impulse control problems in patients with Parkinson's disease.

Inventory Management with Stochastic Lead Times

This article analyzes a continuous time back-ordered inventory system with stochastic demand and stochastic delivery lags for placed orders. This problem in general has an infinite dimensional state space and is hence intractable. We first obtain the set of minimal conditions for reducing such a system's state space to one-dimension and show how this reduction is done. Next, by modeling demand as a diffusion process, we reformulate the inventory control problem as an impulse control problem. We simplify the impulse control problem to a Quasi-Variation Inequality (QVI). Based on the QVI formulation, we obtain the optimality of the (s, S) policy and the limiting distribution of the inventory level. We also obtain the long run average cost of such an inventory system. Finally, we provide a method to solve the QVI formulation. Using a set of computational experiments, we show that significant losses are incurred in approximating a stochastic lead time system with a fixed lead time system, thereby highlighting the need for such stochastic lead time models. We also provide insights into the dependence of this value loss on various problem parameters.

Continuous time mean‐variance optimal portfolio allocation under jump diffusion: An numerical impulse control approach

We present efficient partial differential equation methods for continuous time mean‐variance portfolio allocation problems when the underlying risky asset follows a jump‐diffusion. The standard formulation of mean‐variance optimal portfolio allocation problems, where the total wealth is the underlying stochastic process, gives rise to a one‐dimensional (1D) nonlinear Hamilton–Jacobi–Bellman (HJB) partial integrodifferential equation (PIDE) with the control present in the integrand of the jump term, and thus is difficult to solve efficiently. To preserve the efficient handling of the jump term, we formulate the asset allocation problem as a 2D impulse control problem, 1D for each asset in the portfolio, namely the bond and the stock. We then develop a numerical scheme based on a semi‐Lagrangian timestepping method, which we show to be monotone, consistent, and stable. Hence, assuming a strong comparison property holds, the numerical solution is guaranteed to converge to the unique viscosity solution of the corresponding HJB PIDE. The correctness of the proposed numerical framework is verified by numerical examples. We also discuss the effects on the efficient frontier of realistic financial modeling, such as different borrowing and lending interest rates, transaction costs, and constraints on the portfolio, such as maximum limits on borrowing and solvency. © 2013 Wiley Periodicals, Inc. Numer Methods Partial Differential Eq 30: 664–698, 2014

Brownian Inventory Models with Convex Holding Cost, Part 1: Average-Optimal Controls

We consider an inventory system in which inventory level fluctuates as a Brownian motion in the absence of control. The inventory continuously accumulates cost at a rate that is a general convex function of the inventory level, which can be negative when there is a backlog. At any time, the inventory level can be adjusted by a positive or negative amount, which incurs a fixed cost and a proportional cost. The challenge is to find an adjustment policy that balances the holding cost and adjustment cost to minimize the long-run average cost. When both upward and downward fixed costs are positive, our problem is an impulse control problem. When both fixed costs are zero, our problem is a singular or instantaneous control problem. For the impulse control problem, we prove that a four-parameter control band policy is optimal among all feasible policies. For the singular control problem, we prove that a two-parameter control band policy is optimal. We use a lower-bound approach, widely known as the “verification theorem”, to prove the optimality of a control band policy for both the impulse and singular control problems. Our major contribution is to prove the existence of a “smooth” solution to the free boundary problem under some mild assumptions on the holding cost function. The existence proof leads naturally to a numerical algorithm to compute the optimal control band parameters. We demonstrate that the lower-bound approach also works for a Brownian inventory model which prohibits an inventory backlog. A companion paper (“Brownian inventory models with convex holding cost, part 2: discount-optimal controls”, Stochastic Systems, Vol. 3, No. 2, pp. 500–573) explains how to adapt the lower-bound approach to study a Brownian inventory model under a discounted cost criterion.

On certain problems of the structure of ultrafilters related to extensions of abstract control problems

We study the procedures for constructing ultrafilters in measurable spaces that are in a general sense based on Cartesian products. Our constructions intend to use ultrafilters as generalized elements in extension constructions for abstract reachability problems with asymptotic constraints. Constrains of this kind may arise, for instance, with sequential relaxation of boundary and intermediate conditions in control problems. In this case, families of sets of admissible regular controls in practically interesting cases represent filter bases, which makes it natural to use ultrafilters (maximal filters) that are admissible, in a certain sense, with respect to these families. We assume that counterparts of reachability sets considered in this work are sets in a topological space; the latter may be non-metrizable which occurs, for instance, in typical applications of the pointwise convergence topology. Such a topology may also be useful in impulse control problems, where one studies sheafs of motions as counterparts of reachability sets for exact and approximate satisfaction of trajectory constraints.

Irreversible investments with delayed reaction: an application to generation re-dispatch in power system operation

In this article we consider how the operator of an electric power system should activate bids on the regulating power market in order to minimize the expected operation cost. Important characteristics of the problem are reaction times of actors on the regulating market and ramp-rates for production changes in power plants. Neglecting these will in general lead to major underestimation of the operation cost. Including reaction times and ramp-rates leads to an impulse control problem with delayed reaction. Two numerical schemes to solve this problem are proposed. The first scheme is based on the least-squares Monte Carlo method developed by Longstaff and Schwartz (Rev Financ Stud 14:113–148, 2001). The second scheme which turns out to be more efficient when solving problems with delays, is based on the regression Monte Carlo method developed by Tsitsiklis and van Roy (IEEE Trans Autom Control 44(10):1840–1851, 1999) and (IEEE Trans Neural Netw 12(4):694–703, 2001). The main contribution of the article is the idea of using stochastic control to find an optimal strategy for power system operation and the numerical solution schemes proposed to solve impulse control problems with delayed reaction.

Viscosity solution and impulse control of the diffusion model with reinsurance and fixed transaction costs

We consider an optimal impulse control problem on reinsurance, dividend and reinvestment of an insurance company. To close reality, we add fixed and proportional transaction costs to this problem. The value of the company is associated with expected present value of net dividends pay out minus the net reinvestment capitals until ruin time. We focus on non-cheap proportional reinsurance. We prove that the value function is a unique solution to associated Hamilton–Jacobi–Bellman equation, and establish the regularity property of the viscosity solution under a weak assumption. We solve the non-uniformly elliptic equation associated with the impulse control problem. Finally, we derive the value function and the optimal strategy of the control problem.

Perplexities of treatment resistance in eating disorders.

BackgroundTreatment resistance is an omnipresent frustration in eating disorders. Attempts to identify the features of this resistance and subsequently develop novel treatments have had modest effects. This selective review examines treatment resistant features expressed in core eating disorder psychopathology, comorbidities and biological features. Novel treatments addressing resistance are discussed.DescriptionThe core eating disorder psychopathology of anorexia nervosa becomes a coping mechanism likely via vulnerable neurobiological features and conditioned learning to deal with life events. Thus it is reinforcing and ego syntonic resulting in resistance to treatment. The severity of core features such as preoccupations with body image, weight, eating and exercising predicts greater resistance to treatment. Bulimia nervosa patients are less resistant to treatment with treatment failure related to greater body image concerns, impulsivity, depression, severe diet restriction and poor social adjustment. For those with binge eating disorder overweight in childhood and high emotional eating predicts treatment resistance. There is suggestive data that a diagnosis of an anxiety disorder and severe perfectionism may confer treatment resistance in anorexia nervosa and substance use disorders or personality disorders with impulse control problems may produce resistance to treatment in bulimia nervosa. Traits such as perfectionism, cognitive inflexibility and negative affect with likely genetic influences may also affect treatment resistance. Pharmacotherapy and novel therapies have been developed to address treatment resistance. Atypical antipsychotic drugs have shown some effect in treatment resistant anorexia nervosa and topiramate and high doses of SSRIs are helpful for treatment of resistant binge eating disorder patients. There are insufficient randomized controlled trials to evaluate the novel psychotherapies which are primarily based on the core psychopathological features of the eating disorders.ConclusionTreatment resistance in eating disorders is usually predicted by the severity of the core eating disorder psychopathology which develops from an interaction between environmental risk factors with genetic traits and a vulnerable neurobiology. Future investigations of the biological features and neurocircuitry of the core eating disorders psychopathology and behaviors may provide information for more successful treatment interventions.

Application of doubly reflected BSDEs to an impulse control problem

We establish existence results for adapted solutions of infinite horizon backward stochastic differential equations with two reflected barriers. We also apply these results to get the existence of an optimal impulse control strategy for the infinite horizon impulse control problem. The properties of the Snell envelope reduce our problem to the existence of a pair of continuous processes.

Learning Disabilities, Juvenile Delinquency and the Family: The Role of “Intensive Parenting”

Many scholars have examined the possible link and prevalence between learning disabilities and juvenile delinquency (Brier, 1989; Broder, Dunivant, Smith, & Sutton, 1981; Malmgren, Abbott, & Hawkins 1999; McNamara & Willoby, 2010; Morris & Morris, 2006; Oshima, Huang, Jonson-Reid, Drake, 2010; Quinn, Rutherford, Leone, Osher, Poirier, 2005), but questions regarding the nature of this relationship still remain. This research tradition however, has generated several main hypotheses to account for the higher incidence of delinquency among individuals with learning disabilities. Most discussion of causal linkages emphasizes school failure, differential treatment, or the susceptibility hypothesis (Dunivant, 1982; Larson, 1988; Smith, 1998).While these hypotheses focus on a variety of factors and causal linkages, one domain of influence that has been neglected is that of the family. Parents may be frustrated by their child's disability and all that it entails, or they may be frustrated by the child's inability to do well in school. The family may facilitate getting him/her the help to achieve various developmental tasks, as well as act as advocate for their child both academically and socially. These and other parental tasks and reactions can influence the socialization practices in the family. Despite the likely influence of the family, little research has explored the family impact and the role parenting processes play in the learning disability-juvenile delinquency connection.In 1976, the Office of Juvenile Justice and Delinquency Prevention began an investigation of the possibility of a learning disability-juvenile-delinquency link. This marked the beginning of an increased period of research on the topic, as well as proposed explanations for the possible link. Although it would appear that a link exists, it is unclear whether the link is causal or, alternatively, spurious, if juvenile delinquency and learning disabilities both have common factors. Malmgren, Abbott and Hawkins (1999) suggest that it may be too early to postulate an explanatory theory that there is a direct link, and suggest, instead, a continuation of exploratory research on the link itself.According to federal government figures, public schools in America have identified over 6 million children, ages 3 to 21 years of age, being served under the Individuals with Disabilities Education Act, Part B, in 2009-2010 (National Center for Educational Statistics, 2012). Of these individuals it has been found that young individuals with learning disabilities are more likely than individuals without disabilities to engage in delinquent behavior (Brier, 1989; Larson, 1988; McNamara & Willoughby, 2010; Oshima & Jonson-Reid, 2010; Pickar & Tori 1986). Individuals with disabilities are also disproportionately represented in juvenile detention facilities and correctional facilities (Morris & Morris, 2006; Shandra & Hogan, 2012).Theories detailing the causes of juvenile delinquency are widespread and focus on many biological, psychological, and sociological variables. Criminological theories of delinquency under sociology emphasize variables including race, class, and gender. However, these theories overlook individual characteristics such as learning disabilities.Juvenile delinquents and students with learning disabilities possess many of the same characteristics. Poor academic achievement, short attention spans, impulse control problems, and lack of motivation are characteristics associated with both learning disability and delinquency. Other shared characteristics include negative self concepts, low frustration tolerance, greater prevalence of males than females, poor academic problem solving, and weak social skills (Gallico, Burns, & Grob, 1988; Smith, 1998; Winters 1997; White, Moffitt, Earls, Robins & Silva, 1990). Researchers have used three main hypotheses to establish a connection between individuals with learning disabilities and juvenile delinquency. …

Alexithymia—an independent risk factor for impulsive‐compulsive disorders in Parkinson's disease

Impulsive-compulsive disorders (ICDs) are frequent side effects of dopaminergic medication in Parkinson's disease (PD). Alexithymia, a personality trait characterized by difficulties identifying and describing feelings and an externally oriented thinking style, has been linked to various impulse-control problems in the general population. In PD, the prevalence of alexithymia is approximately twice as high as in the general population. However, whether alexithymia is associated with ICDs in PD is currently unknown. We examined the relationship between self-reported ICDs and alexithymia in a sample of 91 PD patients (89 on dopaminergic medication). Additional self-report measures assessed impulsivity, depression, anxiety, behavioral inhibition/approach, and emotion-regulation strategies. We observed that alexithymia, and particularly difficulty identifying feelings and difficulty describing feelings, was significantly correlated with ICDs, even when controlling for impulsivity, anxiety, and depression. In addition, a group analysis revealed that PD patients with clinical and moderate levels of alexithymia had significantly more ICDs than non-alexithymic patients, suggesting that even moderately high alexithymia levels increase the risk for ICDs in PD. Our results identify alexithymia as an independent risk factor for ICDs in PD. Thus, the inclusion of alexithymia in the neuropsychiatric assessment of patients with PD may help identify patients at risk for ICDs.

On the solution of general impulse control problems using superharmonic functions

In this paper, a characterization of the solution of impulse control problems in terms of superharmonic functions is given. In a general Markovian framework, the value function of the impulse control problem is shown to be the minimal function in a convex set of superharmonic functions. This characterization also leads to optimal impulse control strategies and can be seen as the corresponding characterization to the description of the value function for optimal stopping problems as a smallest superharmonic majorant of the reward function. The results are illustrated with examples from different fields, including multiple stopping and optimal switching problems.