Stochastic Control Model Research Articles

Stochastic inventory control models for single item replenishment with variable demand under constrained and unconstrained conditions

Stochastic inventory control models for single item replenishment with variable demand under constrained and unconstrained conditions

Stochastic optimal control model for COVID-19: mask wearing and active screening/testing

Stochastic optimal control model for COVID-19: mask wearing and active screening/testing

Enhancing Make-to-Order Manufacturing Agility: When Flexible Capacity Meets Dynamic Pricing

The rise of online marketplaces has raised customer expectations regarding customization and lead time. It poses significant challenges to manufacturing firms and prompts a move from make-to-stock to a more flexible make-to-order system. Compared to make-to-stock settings, make-to-order systems cannot smooth fluctuations in demand using available stock. While viewing dynamic pricing as a useful strategy to balance supply with demand, many manufacturing firms can also create capacity flexibility. In that scenario, system costs could be cut by managing capacity and demand simultaneously. In this paper, we consider a make-to-order production environment with base and surge capacity as well as the ability to adjust product pricing. Our main focus is on operational decision-making, assuming that the base capacity and surge capacity are fixed, but activating the surge capacity incurs a setup cost. Initially, we propose a stochastic control model to reflect this complex decision problem. However, our initial model leads to an intractable dynamic programming problem. To overcome this, we convert the problem to a more tractable diffusion control problem. This approach helps to reveal the conditions under which utilizing flexible capacity is more advantageous than relying solely on fixed capacity. When flexible capacity is advantageous, we provide a solution to the diffusion control problem that can guide optimal capacity and price adjustments. We discover an interesting interplay between capacity adjustment and dynamic pricing. In particular, we find that the price, which aims at reducing congestion, may not monotonically increase with the congestion level when capacity adjustments incur a fixed cost.

Farm debt and the over-exploitation of natural capital

This paper uses a stochastic optimal control model to show how standard loan contracts create incentives for farmers to focus on short-term financial performance at the expense of farms’ long-term natural capital. These incentives are a manifestation of the debt overhang problem. Extending this model shows how sustainability-linked loans can be used to weaken these incentives in a way that potentially benefits farmers and their bankers. The magnitude of the economic benefits generated by these loans depends on farm characteristics. The paper investigates the optimal design of sustainability-linked loans.

High-dimensional stochastic control models for newsvendor problems and deep learning resolution

AbstractThis paper studies continuous-time models for newsvendor problems with dynamic replenishment, financial hedging and Stackelberg competition. These factors are considered simultaneously and the high-dimensional stochastic control models are established. High-dimensional Hamilton-Jacobi-Bellman (HJB) equations are derived for the value functions. To circumvent the curse of dimensionality, a deep learning algorithm is proposed to solve the HJB equations. A projection is introduced in the algorithm to avoid the gradient explosion during the training phase. The deep learning algorithm is implemented for HJB equations derived from the newsvendor models with dimensions up to six. Numerical outcomes validate the algorithm’s accuracy and demonstrate that the high-dimensional stochastic control models can successfully mitigate the risk.

The effect of delay feedback control on dynamics of a stochastic Internet congestion control model

The effect of delay feedback control on dynamics of a stochastic Internet congestion control model

Moment-based approximations for stochastic control model of type (s, S)

In this study, we propose an approximation for a renewal reward process that describes a stochastic control model of type (s, S) based on the first three moments of demand random variables. Various asymptotic expansions for this model exist in the literature. All these studies rely on the condition of knowing the distribution function of demand random variables and require obtaining the asymptotic expansion of the renewal function produced by them. However, obtaining a renewal function can be challenging for certain distribution families, and in some cases, the mathematical structure of the renewal function is difficult to apply. Therefore, in this study, simple and compact approximations are presented for the stochastic control model of type (s, S). The findings of this study rely on Kambo’s method, through which we obtain approximations for the ergodic distribution, and the n t h order ergodic moments of this process. To conclude the study, the accuracy of the proposed approximate formulas are examined through a specialized illustrative example. Moreover, it has been noted that the proposed approximation is more accurate than the approximations existing in the literature.

Eurosystem stability: A stochastic dynamic optimization approach to public debt

In this paper we focus on the dynamics of tax revenue and debt that should be necessary for the European countries to comply with the rules on which the Stability and Growth Pact (SGP) is founded, when public budgets are expounded to unexpected expenses.We specify a stochastic control model with infinite horizon formalizing the optimal rules implied by the SGP and compute tax revenue and debt satisfying the model. We find that, even though in long run equilibrium, they are non-stationary processes. From a technical point of view, we make use of a matrix discounted Riccati equation and show that, even in the presence of a unitary root, a real solution exists.The implementation of the optimal model with real data initial conditions suggests for European countries a more intense use of debt rather than tax revenue to finance the public budget in the absence of financial turmoil. Furthermore, there emerge also some small margins for this policy during the Covid-19 period.

The heterogeneous relationship of owner-occupied and investment property with household portfolio choice

The dual nature of property as both a consumption and investment good presents a challenge for household portfolios. Prior theoretical literature predicts a constraint imposed by property on investment decisions, and empirical studies support this notion. However, previous research often overlooks investigating the heterogeneity of this constraint and fails to differentiate between owner-occupied and investment property. Building on a stochastic control model, we analyse the UK’s Wealth and Assets Survey panel and find that along the distribution of how household allocate their wealth a one percentage point (pp) increase in the share of owner-occupied property in the total portfolio is associated with a 0.07 pp decrease in the share of stocks in liquid assets. However, this association varies significantly based on the value of the owner-occupied property share. For low values of the owner-occupied property share, the association with stockholdings is negligible. As the share of owner-occupied property increases, the negative association with stockholdings becomes more pronounced: when the owner-occupied property share reaches 90%, a further 1 pp increase corresponds to a 0.14 pp decrease in the share of stocks in liquid assets. By contrast, buy-to-let property shows no significant relationship with stockholdings, supporting the idea that the constraint on portfolio decisions is primarily driven by the role of property as a consumption good.

Managing Customer Churn via Service Mode Control

We introduce a novel stochastic control model for the problem of a service firm interacting over time with one of its customers who probabilistically churns depending on the customer’s satisfaction. The firm has two service modes available, and they determine the drift and volatility of the Brownian reward process. The firm’s objective is to maximize the rewards generated over the customer’s lifetime. Meanwhile, the customer might churn probabilistically if the customer’s satisfaction, modeled as an Orstein–Uhlenbeck process controlled by the firm’s service mode, is below a certain threshold. We build upon Markov processes with spatial delay to solve this problem, and we explicitly characterize the firm’s optimal policy, which is either myopic or a sandwich policy. A sandwich policy is one in which the firm deploys the service mode with inferior reward rate when the customer satisfaction level is in a specific interval near the satisfaction threshold and uses the myopically optimal service mode for all other satisfaction levels. Specifically, we find that the firm should use the safe service mode when the customer is marginally satisfied and the risky service mode when the customer is marginally unsatisfied. We find numerically that the customer lifetime value under the optimal policy is large relative to that under the myopic policy. Our results are robust to a variety of alternative model specifications. Funding: J. Lu gratefully acknowledges financial support from Natural Science Foundation of China (NSFC) [Project 72192805]. Supplemental Material: The online appendix is available at https://doi.org/10.1287/moor.2021.0179 .

Uncertain stochastic hybrid differential game system with V-n jumps: Saddle point equilibrium strategies and application to advertising duopoly game

This paper focuses on the saddle point equilibrium strategies for uncertain stochastic hybrid differential game systems under chance theory. We consider systems with dynamics described by both an uncertain differential equation with V-n jumps (UDEV) and a stochastic differential equation (SDE), with linear and quadratic performance index functions. Based on chance theory and the equation of optimality of uncertain stochastic hybrid optimal control model, the saddle equilibrium equation for general uncertain stochastic hybrid differential game system is established. Furthermore, three classes of differential game models are discussed by using equilibrium equation. Finally, an advertising duopoly game problem as an application is provided, the corresponding optimal profit difference value and advertising strategies are analyzed.

Optimal Control of Industrial Pollution under Stochastic Differential Models

Considering that the amount of waste generated by an industrial enterprise is affected by many uncertain factors, such as the quality of raw materials and the state of equipment. The process is not deterministic, as assumed in most existing studies. In this paper, we propose a stochastic impulse control model to characterize the process of pollution control. The Quasi-Variational Inequality (QVI) method is implemented to solve the optimization problem. Our results show that the optimal control strategy for an industrial enterprise is to perform at a fixed intensity when the pollution reaches the threshold level. In addition, sensitivity analysis of parameters is implemented to illustrate the impact of higher growth rates and volatility on the optimal control strategy. The paper provides a decision basis for industrial enterprises to do pollution control efficiently.

Debt redemption fund and fiscal incentives

We investigate a stochastic optimal control model for the management of public debt when part of the national debt is transferred to a Redemption Fund, which is allowed to finance itself in the market at a low interest rate, in exchange of the commitment to a fiscal consolidation plan by the national government both for the national debt and the Fund. The Fund is an interesting option for countries with high debt–GDP ratio and weak fundamentals. However, we show that it is not a free lunch in a sense that a country with a higher public debt should transfer a larger amount of resources to cut down the Fund. We calibrate the model to the experience of Euro zone countries with a high debt, showing that it is possible to design the mechanism allowing to pay back the Redemption Fund and stabilizing both the national debt and the fiscal surplus with a reduction of the fiscal effort.

Validation of Stochastic Optimal Control Models for Goal-Directed Human Movements on the Example of Human Driving Behavior

Stochastic Optimal Control models represent the state-of-the-art in modeling goal-directed human movements. The linear-quadratic sensorimotor (LQS) model based on signal-dependent noise processes in state and output equation is the current main representative. With our newly introduced Inverse Stochastic Optimal Control algorithm building upon two bi-level optimizations, we can identify its unknown model parameters, namely cost function matrices and scaling parameters of the noise processes, for the first time. In this paper, we use this algorithm to identify the parameters of a deterministic linear-quadratic, a linear-quadratic Gaussian and a LQS model from human measurement data to compare the models’ capability in describing goal-directed human movements. Human steering behavior in a simplified driving task shown to posses similar features as point-ot-point human hand reaching movements serves as our example movement. The results show that the identified LQS model outperforms the others with statistical significance. Particularly, the average human steering behavior is modeled significantly better by the LQS model. This validates the positive impact of signal-dependent noise processes on modeling human average behavior.

SDEs with a Discontinuity at Final Time in a Stochastic Optimal Control Model of an Emissions Trading System

SDEs with a Discontinuity at Final Time in a Stochastic Optimal Control Model of an Emissions Trading System

Deterministic and stochastic optimal control models for plant growth using locust fertilizer

Deterministic and stochastic optimal control models for plant growth using locust fertilizer

The Role of Ca2+ Sparks in Force Frequency Relationships in Guinea Pig Ventricular Myocytes.

Calcium sparks are the elementary Ca2+ release events in excitation-contraction coupling that underlie the Ca2+ transient. The frequency-dependent contractile force generated by cardiac myocytes depends upon the characteristics of the Ca2+ transients. A stochastic computational local control model of a guinea pig ventricular cardiomyocyte was developed, to gain insight into mechanisms of force-frequency relationship (FFR). This required the creation of a new three-state RyR2 model that reproduced the adaptive behavior of RyR2, in which the RyR2 channels transition into a different state when exposed to prolonged elevated subspace [Ca2+]. The model simulations agree with previous experimental and modeling studies on interval-force relations. Unlike previous common pool models, this local control model displayed stable action potential trains at 7 Hz. The duration and the amplitude of the [Ca2+]myo transients increase in pacing rates consistent with the experiments. The [Ca2+]myo transient reaches its peak value at 4 Hz and decreases afterward, consistent with experimental force-frequency curves. The model predicts, in agreement with previous modeling studies of Jafri and co-workers, diastolic sarcoplasmic reticulum, [Ca2+]sr, and RyR2 adaptation increase with the increased stimulation frequency, producing rising, rather than falling, amplitude of the myoplasmic [Ca2+] transients. However, the local control model also suggests that the reduction of the L-type Ca2+ current, with an increase in pacing frequency due to Ca2+-dependent inactivation, also plays a role in the negative slope of the FFR. In the simulations, the peak Ca2+ transient in the FFR correlated with the highest numbers of SR Ca2+ sparks: the larger average amplitudes of those sparks, and the longer duration of the Ca2+ sparks.

Stochastic Optimal Control of Economic Growth Model under Research and Development Investment with Kalman Filtering Approaches

This article examines an economic growth model that expresses the interaction between production, technology stock, and research and development (R&D) investments. The goal of this study is to maximize production. Considering the presence of Gaussian white noises, this model is reformulated as a stochastic optimal control problem, where the R&D investment rate is defined as the control input. We aim to explore the efficiency of Kalman filtering approaches for solving this problem. Here, the extended Kalman filter (EKF) and unscented Kalman filter (UKF) are applied for state estimation. The state equation linearization is made in the EKF, while the unscented transform is taken in the UKF for generating a set of sigma points. These approaches aim to estimate the state dynamics from different perspectives. With these state estimates, two different computational algorithms are proposed, the EKF for state-control (EKF4SC) and UKF for state-control (UKF4SC) algorithms. The optimal control policy is designed to minimize the cost function. For illustration, the model's parameters are considered in the simulation experiment. The simulation results showed that the UKF has higher accuracy in the state estimation with the smallest mean squares of error compared with the EKF. Moreover, the optimal control policy based on the state estimate generated from the UKF could optimize the cost function of the problem. Hence, the results show the efficiency of the algorithms proposed. In conclusion, the application of the Kalman filtering algorithms to the economic growth model is presented. The significance of this study is to provide a stochastic optimal control model for the economic growth problem and to suggest an efficient computational approach for solving this stochastic optimal control problem.

Towards control of dam and reservoir systems with forward–backward stochastic differential equations driven by clustered jumps

AbstractWe deal with a new maximum principle‐based stochastic control model for river management through operating a dam and reservoir system. The model is based on coupled forward–backward stochastic differential equations (FBSDEs) derived from jump‐driven streamflow dynamics and reservoir water balance. A continuous‐time branching process with immigration driven by a tempered stable subordinator efficiently describes clustered inflow streamflow dynamics. This is a completely new attempt in hydrology and control engineering. Applying a stochastic maximum principle to the dynamics based on an objective functional for designing cost‐efficient control of dam and reservoir systems leads to the FBSDEs as a system of optimality equations. The FBSDEs under a linear‐quadratic ansatz lead to a tractable model, while they are solved numerically in the other cases using a least‐squares Monte‐Carlo method. Optimal controls are found in the former, while only sub‐optimal ones are computable in the latter due to a hard state constraint. Model parameters are successfully identified from a real data of a river in Japan having a dam and reservoir system. We also show that the linear‐quadratic case can capture the real operation data of the system with underestimation of the outflow discharge. More complex cases with a realistic time horizon are analyzed numerically to investigate impacts of considering the environmental flows and seasonal operational purposes. Key challenges towards more sophisticated modeling and analysis with jump‐driven FBSDEs are discussed as well.

Optimal Control Strategies for the Premium Policy of an Insurance Firm with Jump Diffusion Assets and Stochastic Interest Rate

In this paper, we present a stochastic optimal control model to optimize an insurance firm problem in the case where its cash-balance process is assumed to be described by a stochastic differential equation driven by Teugels martingales. Noticing that the insurance firm is able to control its cash-balance dynamics by regulating the underlying premium rate, the aim of the policy maker is to select an appropriate premium in order to minimize the total deviation of the state process to some pre-set target level. As a part of stochastic maximum principle approach, a verification theorem is used to fulfill this achievement.