Stochastic Control Approach Research Articles

Well-posedness and large deviation principle for a class of SPDEs

In this paper, we study a class of stochastic partial differential equations with general locally monotone coefficients under thevariational framework. By proving the existence of martingale solutions and pathwise uniqueness, we first get the existence and uniqueness of strong solutions. Then we prove the large deviation principle by using the stochastic control and weakconvergence approach. The mainresults are applicable to various concrete SPDE models, such as stochastic Cahn-Hilliard equation, stochastic Kuramoto-Sivashinsky equation and stochastic 3D tamedNavier-Stokes equation.

Stochastic Online Generation Control of Cascaded Run-of-the-River Hydropower for Mitigating Solar Power Volatility

To exploit the massive solar energy available in the region, photovoltaic plants have been built in the mountain areas in Southwest China, coexisting with many small cascaded run-of-the-river hydropower plants, interconnected by short-distance transmission lines. Network constraints inside these systems are often negligible. However, due to the volatile nature of solar power and the weak connection to the external power grid, without proper coordination, solar power curtailment and water spillage are inevitable. To address this issue, this paper proposes an ultra-short-term stochastic generation control method for cascaded hydropower to mitigate solar power volatility. Two technical challenges are specifically addressed: to tackle the time-varying stochastic volatility of solar power, the Itô process model is introduced; to characterize the spatial-temporal hydraulic coupling of the cascaded hydropower plants and river operation constraints, a state-space dynamic river model derived from shallow water equations is included. Followingly, a stochastic control (SC) approach for hydropower generation based on stochastic programming is proposed, where hydropower generation commands are parameterized as affine functions of measured solar power, enabling quick response to solar power volatility. Simulation of a real-life cascaded hydro-solar system and a modified system verifies the proposed control method.

Stochastic Model Predictive Control with Adaptive Chance Constraints based on empirical Cumulative Distributions

Individual chance constraints can be used to systematically seek trade-offs between control performance and constraint violation for a given disturbance description. This paper presents a Stochastic Model Predictive Control (SMPC) approach with adaptive individual chance constraints that relies on online adaptation of the disturbance description using the empirical cumulative distribution (ECDF). Individual chance constraints are ensured by using a suitable worst-case confidence interval derived from the ECDF. The confidence interval may, however, be excessively conservative due to the empirical nature of the ECDF. To reduce this conservatism, the proposed approach accounts for the updated disturbance information, which is sampled online from the one-step ahead prediction error. Hence, the initial ECDF can be obtained from a reduced number of samples since the conservative handling of the chance constraint is continuously mitigated. This will also allow for using simpler models of the stochastic system disturbances. Convergence and recursive feasibility of the proposed adaptive approach are established. A DC-DC converter benchmark problem is used to illustrate the usefulness of the proposed approach.

A Stochastic Control Approach to Defined Contribution Plan Decumulation: 'The Nastiest, Hardest Problem in Finance'

We pose the decumulation strategy for a Defined Contribution (DC) pension plan as a problem in optimal stochastic control. The controls are the withdrawal amounts and the asset allocation strategy. We impose maximum and minimum constraints on the withdrawal amounts, and impose no-shorting no-leverage constraints on the asset allocation strategy. Our objective function measures reward as the expected total withdrawals over the decumulation horizon, and risk is measured by Expected Shortfall (ES) at the end of the decumulation period. We solve the stochastic control problem numerically, based on a parametric model of market stochastic processes. We find that, compared to a fixed constant withdrawal strategy, with minimum withdrawal set to the constant withdrawal amount, the optimal strategy has a significantly higher expected average withdrawal, at the cost of a very small increase in ES risk. Tests on bootstrapped resampled historical market data indicate that this strategy is robust to parametric model mis-specification.

A Stochastic Control Approach to Fight COVID-19

In this paper, we present several optimal control approaches which are designed to control and regulate the evolution of infection numbers in the COVID-19 disease. In our setting, the number of infected people at a certain location is modeled by a continuous-time stochastic process which can be affected by a related stochastic control process. We use mathematical tools from stochastic analysis and optimal control theory. In particular, we prove an innovative stochastic maximum principle for continuous-state branching processes with immigration (so-called CBI processes) and apply the result to a stochastic control problem stemming from epidemiology.

Scenario-Based Probabilistic Reachable Sets for Recursively Feasible Stochastic Model Predictive Control

This letter presents a stochastic model predictive control approach (MPC) for linear discrete-time systems subject to unbounded and correlated additive disturbance sequences, which makes use of the scenario approach for offline computation of probabilistic reachable sets. These sets are used in a tube-based MPC formulation, resulting in low computational requirements. Using a recently proposed MPC initialization scheme and nonlinear tube controllers, we provide recursive feasibility and closed-loop chance constraint satisfaction, as well as hard input constraint guarantees, which are typically challenging in tube-based formulations with unbounded noise. The approach is demonstrated in simulation for the control of an overhead crane system.

Optimal investment with derivatives and pricing in an incomplete market

This paper investigates the optimal investment strategy and the pricing of derivatives in an incomplete financial market with one risk-free asset, one stock and one non-redundant derivative security. In the exponential utility maximization criterion, a dynamic relationship between optimal positions for the stock and derivative security is established and the dynamic derivatives pricing formulae depending on the current optimal positions are also obtained by employing the stochastic control approach. The explicit representations of the corresponding solutions are demonstrated and their properties are discussed. Finally, numerical results are presented to characterize the behavior of the derivative security price.

Optimal investment strategies and risk-sharing arrangements for a hybrid pension plan

A continuous time stochastic model is used to study a hybrid pension plan, where both the contribution and benefit levels are adjusted depending on the performance of the plan, with risk sharing between different generations. The pension fund is invested in a risk-free asset and multiple risky assets. The objective is to seek an optimal investment strategy and optimal risk-sharing arrangements for plan trustees and participants so that this proposed hybrid pension system provides adequate and stable income to retirees while adjusting contributions effectively, as well as keeping its sustainability in the long run. These goals are achieved by minimizing the expected discount disutility of intermediate adjustment for both benefits and contributions and that of terminal wealth in finite time horizon. Using the stochastic optimal control approach, closed-form solutions are derived under quadratic loss function and exponential loss function. Numerical analysis is presented to illustrate the sensitivity of the optimal strategies to parameters of the financial market and how the optimal benefit changes with respect to different risk aversions. Through numerical analysis, we find that the optimal strategies do adjust the contributions and retirement benefits according to fund performance and model objectives so the intergenerational risk sharing seem effectively achieved for this collective hybrid pension plan.

Knock Intensity Distribution and a Stochastic Control Framework for Knock Control

Abstract One of the main factors limiting the efficiency of spark-ignited (SI) engines is the occurrence of engine knock. In high temperature and high pressure in-cylinder conditions, the fuel–air mixture auto-ignites creating pressure shock waves in the cylinder. Knock can significantly damage the engine and hinder its performance; as such, conservative knock control strategies are generally implemented which avoid such operating conditions at the cost of lower thermal efficiencies. Significant improvements in the performance of conventional knock controllers are possible if the properties of the knock process are better characterized and exploited in knock controller designs. One of the methods undertaken to better characterize knocking instances is to employ a probabilistic approach, in which the likelihood of knock is derived from the statistical distribution of knock intensity (KI). In this paper, it is shown that KI values at a fixed operating point for single fuel and dual fuel engines are accurately described using a mixed lognormal distribution. The fitting accuracy is compared against those for a randomly generated mixed-lognormally distributed dataset, and shown to exceed a 95% accuracy threshold for almost all of the operating points tested. Additionally, this paper discusses a stochastic knock control approach that leverages the mixed lognormal distribution to adjust spark timing based on KI measurements. This more informed knock control strategy would allow for improvements in engine performance and fuel efficiency by minimizing knock occurrences.

Robust optimal insurance and investment strategies for the government and the insurance company under mispricing phenomenon

This paper considers the robust optimal insurance-investment problem for the policyholder who is specified as a government and the insurance company with considering mispricing phenomenon. The government is allowed to purchase proportional insurance from the insurance company whose surplus process is assumed to follow the classical Cramér-Lundberg (C-L) model. Suppose the insurance company can invest in different financial markets and the government is limited to invest in one financial market. Thus, the insurance company is allowed to invest in a pair of mispriced stocks which offer statistical arbitrage opportunities, a risk-free asset, and a market index while the government is assumed to invest in a risk-free asset and a risky asset whose price process satisfies the geometric Brownian motion. Furthermore, we take the joint interest of the government and the insurance company into account. The government and the insurance company are both ambiguity-averse, aiming to maximize their expected joint product exponential utility of terminal wealth. By applying stochastic optimal control approach, we derive the robust equilibrium insurance-investment strategies and the corresponding equilibrium value function explicitly. In addition, we present some special cases of the model as extended results. Finally, numerical simulations are presented to illustrate the effects of model parameters on the optimal strategies.

Optimal proportional reinsurance and investment for stochastic factor models

In this work we investigate the optimal proportional reinsurance-investment strategy of an insurance company which wishes to maximize the expected exponential utility of its terminal wealth in a finite time horizon. Our goal is to extend the classical Cramér–Lundberg model introducing a stochastic factor which affects the intensity of the claims arrival process, described by a Cox process, as well as the insurance and reinsurance premia. The financial market is supposed not influenced by the stochastic factor, hence it is independent on the insurance market. Using the classical stochastic control approach based on the Hamilton–Jacobi–Bellman equation we characterize the optimal strategy and provide a verification result for the value function via classical solutions to two backward partial differential equations. Existence and uniqueness of these solutions are discussed. Results under various premium calculation principles are illustrated and a new premium calculation rule is proposed in order to get more realistic strategies and to better fit our stochastic factor model. Finally, numerical simulations are performed to obtain sensitivity analyses.

A stochastic control approach to managed futures portfolios

We study a stochastic control approach to managed futures portfolios. Building on the (Schwartz, 1997) stochastic convenience yield model for commodity prices, we formulate a utility maximization problem for dynamically trading a single-maturity futures or multiple futures contracts over a finite horizon. By analyzing the associated Hamilton–Jacobi–Bellman (HJB) equation, we solve the investor’s utility maximization problem explicitly and derive the optimal dynamic trading strategies in closed form. We provide numerical examples and illustrate the optimal trading strategies using WTI crude oil futures data.

Irreversible investment with fixed adjustment costs: a stochastic impulse control approach

We consider an optimal stochastic impulse control problem over an infinite time horizon motivated by a model of irreversible investment choices with fixed adjustment costs. By employing techniques of viscosity solutions and relying on semiconvexity arguments, we prove that the value function is a classical solution to the associated quasi-variational inequality. This enables us to characterize the structure of the continuation and action regions and construct an optimal control. Finally, we focus on the linear case, discussing, by a numerical analysis, the sensitivity of the solution with respect to the relevant parameters of the problem.

Optimal selection and release problem in software testing process: A continuous time stochastic control approach

This paper studies a joint selection of test cases and release problem for a software under test with predetermined classes of test cases and release time deadline. The software test manager can make three alternative choices dynamically during software testing progress before the deadline: continue testing and select a class of test cases, release the software, or scrap the software, with the objective of minimizing the cumulative testing cost plus penalty cost after releasing or scrapping the software. We formulate the problem as a continuous time stochastic control model and provide a mathematically rigorous method to establish the concavity of the optimal cost function. Based on this property, we are able to characterize that the optimal release policy has a threshold structure. Moreover, the thresholds are founded to be monotone in the residual time length in the case of homogeneous release cost. Besides, we put forward a method based on low convex envelope and discover that the optimal selection policy also has a threshold or other simple structure, if the running cost or the removal cost is the same for all classes. Finally, we present an approximation algorithm of computing the optimal cost function, by which some numerical examples are studied to justify our theoretical results and the robustness of our policy. We also conduct a case study to compare our dynamic selection and release testing policy with two other commonly used testing policies and find that our policy is the best in most instances.

Robust optimal proportional reinsurance and investment strategy for an insurer with defaultable risks and jumps

In this paper, we consider a robust optimal proportional reinsurance and investment problem in a model with default risks and jumps for an insurer whose objective is to maximize the expected exponential utility of terminal wealth. The surplus process of the insurer is assumed to follow a Brownian motion with drift (which is an approximation of the classical compound Poisson risk model). Assume that the insurer is allowed to purchase proportional reinsurance and invest in a financial market which consists of a risk-free asset, a defaultable bond and a risky asset whose price process is governed by a jump–diffusion model. Using stochastic control approach, we establish the robust Hamilton–Jacobi–Bellman equations for the post-default case and the pre-default case, respectively. Furthermore, we derive the closed-form expressions of the optimal strategies and the corresponding value functions in both cases. Finally, we provide numerical examples to illustrate the effects of some model parameters on the robust optimal strategies.

Strong γc-γcl H∞ stabilization for networked control systems under denial of service attacks

This paper is concerned with the strong γc-γcl H∞ stabilization problem for networked control systems (NCSs) subject to denial of service (DoS) attacks, which are common attack behaviors that affect the packet transmission of measurement or control signals. The purpose of the problem under consideration is to design a stable dynamic output feedback (DOF) controller (strong stabilizing controller) with the prescribed H∞ performance norm bound γc to tolerate multiple packet dropouts caused by DoS attacks, such that, the closed-loop system is mean-square stable and captures the H∞ disturbance attenuation norm bound γcl. Based on the Lyapunov functional and the stochastic control approach, some sufficient conditions with the form of matrix inequalities for the existence of the desired stable DOF controller are established. Then, by an orthogonal complement space technique, the controller gain is parameterized. Next, an iterative linear matrix inequality (LMI) algorithm is developed to obtain the controller gain. Finally, the usefulness of the proposed method is indicated by a numerical simulation example.

Optimal time-consistent reinsurance-investment strategy with delay for an insurer under a defaultable market

In this paper, we consider a reinsurance-investment problem with delay for an insurer under the mean-variance criterion in a defaultable market. The financial market consists of a risk-free bond, a stock and a defaultable bond. The insurer's surplus process is described by a jump-diffusion risk model and the price process of the stock is assumed to follow a constant elasticity of variance (CEV) model. In particular, we take the delay of feedback time for strategy into account. Applying stochastic control approach, we derive the time-consistent reinsurance-investment strategy in post-default case and pre-default case explicitly via a game theoretic framework, respectively. Finally, numerical examples and sensitivity analyses are provided to show the impact of financial parameters on the optimal strategies.

On the pricing of Asian options with geometric average of American type with stochastic interest rate: A stochastic optimal control approach

In this work, through stochastic optimal control in continuous time the optimal decision making in consumption and investment is modeled by a rational economic agent, representative of an economy, who is a consumer and an investor adverse to risk; this in a finite time horizon of stochastic length. The assumptions of the model are: a consumption function of HARA type, a representative company that has a stochastic production process, the agent invests in a stock and an American-style Asian put option with floating strike equal to the geometric average subscribed on the stock, both modeled by controlled Markovian processes; as well as the investment of a principal in a bank account. The model is solved with dynamic programming in continuous time, particularly the Hamilton-Jacobi-Bellman PDE is obtained, and a function in separable variables is proposed as a solution to set the optimal trajectories of consumption and investment. In the solution analysis is determined: in equilibrium, the process of short interest rate that is driven by a square root process with reversion to the mean; and through a system of differential equations of risk premiums, a PDE is deduced equivalent to the Black-Scholes-Merton but to value an American-style Asian put option.

Continuous-time mean-variance asset-liability management with stochastic interest rates and inflation risks

This paper investigates a continuous-time Markowitz mean-variance asset-liability management (ALM) problem under stochastic interest rates and inflation risks. We assume that the company can invest in $n + 1$ assets: one risk-free bond and $n$ risky stocks. The risky stock's price is governed by a geometric Brownian motion (GBM), and the uncontrollable liability follows a Brownian motion with drift, respectively. The correlation between the risky assets and the liability is considered. The objective is to minimize the risk (measured by variance) of the terminal wealth subject to a given expected terminal wealth level. By applying the Lagrange multiplier method and stochastic control approach, we derive the associated Hamilton-Jacobi-Bellman (HJB) equation, which can be converted into six partial differential equations (PDEs). The closed-form solutions for these six PDEs are derived by using the homogenization approach and the variable transformation technique. Then the closed-form expressions for the efficient strategy and efficient frontier are obtained. In addition, a numerical example is presented to illustrate the results.

Generation and evaluation of excavation schedules for hard rock tunnels in preconstruction and construction

Abstract Uncertain product characteristics in construction projects make it difficult for planners to develop schedules that reduce expected costs, durations, and associated risks. To overcome these challenges in hard rock tunnel projects, this research introduces a methodology that adapts stochastic programming and feedback control approaches for their excavation. Such approaches require rapid and consistent implementation using up-to-date information provided in a probabilistic manner throughout the entire excavation; therefore, the authors tailored dynamic programming and tunneling risk analysis methods for the methodology to address multiple sets of rock mass properties (RMPs), transitions among excavation methods at the excavation method level, decision-making times, and schedule adjustment policies (SAPs). In preconstruction and construction, the methodology allows construction planners of hard rock tunnels to generate a total-cost-optimal excavation schedule for each set of RMPs and evaluate the excavation costs and durations of schedules for multiple sets of RMPs in a timely and consistent manner by considering SAPs. Further research is required to take into account multiple advances of excavation methods for schedule generation and evaluation. Database subject headings Automated schedule generation, hard rock tunnel, uncertainties in rock mass properties, feedback control, stochastic programming, earthwork risk analysis.