Control Of Stochastic Systems Research Articles

A Novel Accelerated Multistage Learning Control Mechanism via Virtual Performance Reduction.

This study uses a multistage learning mechanism concept to investigate the accelerated learning control for stochastic systems. In this mechanism, the learning iterations are divided into successive stages, with each stage comprising several iterations. The learning gain is constant in each stage to accelerate the learning process and decreases it from one stage to another to eliminate the noise effect asymptotically. The critical issue is determining the switching iteration when a new stage starts. This study resolves this issue by calculating a virtual performance index of the mean-squared input error and its estimated upper bound. Specifically, the ideal, practical, and improved multistage learning control schemes are proposed to determine the switching iteration and generate the learning gain sequence. The ideal scheme achieves the best performance at the cost of a large computation burden, and the practical scheme saves computation cost, but the performance is not excellent. The improved scheme significantly approximates the best performance by introducing additional stretching parameters to the performance index. Illustrative simulations are provided to verify the theoretical results.

Adaptive output feedback control of stochastic systems with mismatched uncertainties input–output quantization

Inspired by the fact that remote control systems always demand to be quantified, and considering that there are always stochastic disturbances in the actual situation, this study discusses a sort of systems with unknown constants due to the existence of parameters that are untoward to measure. Therefore, in this research, a sort of stochastic nonlinear systems with input–output quantization mismatched uncertainties is studied by utilizing backstepping method. The difficulty is that the output signal becomes non-differential after being quantized. In addition, only the quantized output signal in the system can be regarded as known during the analysis. Therefore, the update of unknown parameters and the design of controller will grow arduous. In order to solve the above dilemma, a state observer and a projection operator are introduced. Moreover, in the design of the controller, the power form of Lyapunov function and differential method is employed to avoid the derivation of non-differentiable functions. Since the controller proposed in this study contains a small number of parameters that need to be designed, it is practical and can eliminate the problem of computational explosion. Simulation results demonstrate the effectiveness of the proposed scheme.

Controllability of partially observed stochastic semilinear fractional control systems

Controllability of partially observed stochastic semilinear fractional control systems

Design stabilisers for multi-input affine control stochastic systems via stochastic control Lyapunov functions

This paper studies the problem of state feedback stabilisation, in the sense of a weak solution, for a class of multi-input affine control stochastic systems whose drift and diffusion terms are dependent on the control and for which classical methods are not applicable. Based on the generalised stochastic Lyapunov theorem and on the technique of stochastic control Lyapunov functions, sufficient conditions for the existence of a continuous stabilising feedback control are proposed, and a state feedback controller can be developed to guarantee the closed-loop system globally asymptotically stable in probability. This work generalises previous results on the stabilisation of single-input affine control stochastic systems. The obtained results are illustrated by a numerical example.

Disturbance observer based fixed-time control of stochastic systems

In this paper, the novel fixed-time anti-disturbance control scheme is proposed for a class of stochastic systems subjected to multiple disturbances and faults, and the disturbances consist of derivative-bounded disturbances and multiply noise. Based on the pole placement method, the fixed-time disturbance observer (Fixed-time DO) is devised to estimate derivative-bounded disturbances and an adaptive law is employed to approach the time-varying fault. Then a composite fixed-time controller is constructed to make the system converge to equilibrium position within a pre-specified time. At the same time, simulation examples illustrate that the proposed controller is effective and the convergence time is not related to the initial states of the system.

On convergence of occupational measures sets of a discrete-time stochastic control system, with applications to averaging of hybrid systems

We establish that, under certain conditions, the set of random occupational measures generated by the state-control trajectories of a discrete-time stochastic system as well as the set of their mathematical expectations converge to a non-random, convex and compact set. We apply these results to the averaging a hybrid system with a slow continuous-time component and a fast discrete-time component. It is shown that the solutions of the hybrid system are approximated by the solutions of a differential inclusion. The novelty of our results is that we allow the state-control space of the fast component to be non-denumerable.

Learning to Order for Inventory Systems with Lost Sales and Uncertain Supplies

We consider a stochastic lost-sales inventory control system with lead time L over a planning horizon T. Supply is uncertain, and it is a function of the order quantity (because of random yield/capacity, etc.). We aim to minimize the T-period cost, a problem that is known to be computationally intractable even under known distributions of demand and supply. In this paper, we assume that both the demand and supply distributions are unknown and develop a computationally efficient online learning algorithm. We show that our algorithm achieves a regret (i.e., the performance gap between the cost of our algorithm and that of an optimal policy over T periods) of [Formula: see text] when [Formula: see text]. We do so by (1) showing that our algorithm’s cost is higher by at most [Formula: see text] for any [Formula: see text] compared with an optimal constant-order policy under complete information (a widely used algorithm) and (2) leveraging the latter’s known performance guarantee from the existing literature. To the best of our knowledge, a finite sample [Formula: see text] (and polynomial in L) regret bound when benchmarked against an optimal policy is not known before in the online inventory control literature. A key challenge in this learning problem is that both demand and supply data can be censored; hence, only truncated values are observable. We circumvent this challenge by showing that the data generated under an order quantity q2 allow us to simulate the performance of not only q2 but also, q1 for all [Formula: see text], a key observation to obtain sufficient information even under data censoring. By establishing a high-probability coupling argument, we are able to evaluate and compare the performance of different order policies at their steady state within a finite time horizon. Because the problem lacks convexity, commonly used learning algorithms, such as stochastic gradient decent and bisection, cannot be applied, and instead, we develop an active elimination method that adaptively rules out suboptimal solutions. This paper was accepted by Victor Martínez-de-Albéniz, operations management. Funding: This work is supported by the National Science Foundation [Grant CCF-2312205]. Z. Zhou also acknowledges the New York University’s 2024 Center for Global Economy and Business [Faculty Research Grant] and New York University [Research Catalyst Prize]. Supplemental Material: The online appendix is available at https://doi.org/10.1287/mnsc.2022.02476 .

Multi-objective continuous review inventory policy using MOPSO and TOPSIS methods

In the dynamic business environment of today, decision-making involves considering various conflicting aspects. Inventory planning problems aim to determine how much and when to order products to satisfy customer demand at the lowest possible cost while maintaining a desirable service level. These problems can be formulated as a MOPSO algorithm, which is used to handle multiple objectives in a continuous review stochastic inventory control system (r, Q). Unfortunately, most multi-objective inventory models have been solved by aggregating objectives using specific weights or by optimizing only one objective and treating the others as constraints. Considering the complexity of real-world inventory control problems, which involves conflicting objectives such as minimizing cost and maximizing service level, the need arises to employ more precise optimizers that can generate better and more diverse non-dominated solutions of reorder point and order size system. In this paper multi-criteria decision-making framework that combines MOPSO algorithm and TOPSIS method to generate a Pareto front of non-dominated solutions and rank them based on decision makers' preferences. Initially, the original MOPSO is applied to the multi-objective inventory control problem, and then the mutation operator is integrated into the MOPSO to maintain diversity in the swarm and explore the entire search space. Next, the leader selection strategy called the geographically-based system (Grids) is replaced by the crowding distance factor to choose the global optimal particle as a leader. Additionally, the ε-dominance concept is employed to limit the archive size and maintain more diversity and convergence in the MOPSO for optimizing the inventory control problem. In conclusion, this work not only pioneers a cutting-edge approach to multi objective inventory control but also underscores its practical value. By facilitating the generation of superior solutions that cater to diverse decision-maker preferences, our methodology resonates deeply with real-world challenges and sets a new benchmark for effective inventory planning.

Optimal Impulse Control for Systems Deriven by Stochastic Delayed Differential Equations

In this paper we study the problem of optimal impulse control for stochastic systems with delay in the case when the value function of the impulse problem depends only on the initial data of the given process through its initial value (value at zero) and some weighted averages. A verification theorem for such impulse control problem is given. As an example the optimal stream of dividends with transaction costs is solved.

A model of the radar situation of the coastal vessel traffic control system

Since the elements of the ship traffic control system at sea (VTS) are geographically distributed, and the observed spaces have well-defined zones, the radar facilities (radars) that make up their technological basis are evaluated by the ability to detect, classify (recognize), and ensure tracking of ships at a certain range in the spatial zone. The solution of these problems is probabilistic in nature, and the indicators are represented by random functions. However, due to the complexity of accounting for random factors in such stochastic control systems, calculations of their efficiency indicators are usually carried out deterministically for the "average" values of random parameters. The development of a mathematical model for evaluating the effectiveness of VTS in its ability to detect marine targets is a scientific task, the solution of which is possible using elements of the theory of random functions and processes.

New exploration on approximate controllability of fractional neutral‐type delay stochastic differential inclusions with non‐instantaneous impulse

This paper aims to derive a new set of sufficient conditions for the existence and approximate controllability of neutral‐type fractional stochastic integrodifferential inclusions with infinite delay and non‐instantaneous impulse in a separable Hilbert space using the Atangana–Baleanu Caputo fractional derivative. We investigate the existence of a mild solution for the Atangana–Baleanu Caputo fractional neutral‐type delay integrodifferential stochastic system while taking into account the non‐instantaneous impulses. For this purpose, the Atangana–Baleanu Caputo fractional neutral‐type impulsive delay stochastic system is transferred into an equivalent fixed point problem via an integral operator, and then, the Bohnenblust–Karlin fixed point approach is applied. Further, the approximate controllability results of the proposed nonlinear stochastic impulsive control system are established under the consideration that the corresponding linear system is approximately controllable. The set of sufficient conditions is established by using the concepts of stochastic analysis, fractional calculus, fixed point technique, semigroup theory of bounded linear operators, and the theory of multivalued maps. To illustrate the abstract results, we provide an example at the end of the paper.

The second-order maximum principle for partially observed optimal controls

In this paper, the stochastic control system under partial observation for singular optimal controls in the classical sense is studied. Note that both the system and the observation contain noise, in which the control variable is allowed to enter into all the coefficient terms. By adding some regularity assumptions, the pointwise second-order maximum principle under partial observation can be obtained by Malliavin calculus. Eventually, an example is given to demonstrate how to find the optimal control.

A stochastic maximum principle for forward–backward stochastic control systems with quadratic generators and sample-wise constraints

This paper examines the stochastic maximum principle (SMP) for a forward–backward stochastic control system where the backward state equation is characterized by the backward stochastic differential equation (BSDE) with quadratic growth and the forward state at the terminal time is constrained in a convex set with probability one. With the help of the theory of BSDEs with quadratic growth and the bounded mean oscillation (BMO) martingales, we employ the terminal perturbation approach and Ekeland’s variational principle to obtain a dynamic stochastic maximum principle. The main result has a wide range of applications in mathematical finance and we investigate a robust recursive utility maximization problem with bankruptcy prohibition as an example.

Understanding the Capability of PD Control for Uncertain Stochastic Systems

In this article, we focus on the global stabilizability problem for a class of uncertain stochastic control systems, where both the drift term and the diffusion term are nonlinear functions of the state variables and the control variables. We will show that the widely applied proportional-derivative (PD) control in engineering practice has the ability to globally stabilize such systems in the mean square sense, provided that the upper bounds of partial derivatives of the nonlinear functions satisfy a certain algebraic inequality. It will also be proved that the stabilizing PD parameters can only be selected from a two dimensional bounded convex set, which differs significantly from the existing results for PD controlled stochastic systems where the diffusion term does not depend on the control variables. Moreover, a particular polynomial on these bounds is introduced, which can be used to determine under what conditions the system is not stabilizable by the PD control, and thus demonstrating the fundamental limitations of PD control.

The Partial Controllability of Linear Stochastic Control Systems with Terminal Constraints and Its Applications to Game-Based Control Systems with Jumps

The Partial Controllability of Linear Stochastic Control Systems with Terminal Constraints and Its Applications to Game-Based Control Systems with Jumps

States estimation and resilient control for non-Gaussian stochastic distribution control system under sensor attacks

This paper studies states estimation and resilient control schemes based on model predictive control (MPC) algorithm for a class of Takagi–Sugeno (T–S) fuzzy stochastic distribution control (SDC) system subjected to sparse sensor attacks. Firstly, a T–S fuzzy model is used to approximate the dynamics of a non-Gaussian SDC system, where the outputs of the system is the output probability density functions (PDFs). Secondly, in order to estimate the states and attacks in the system, a fuzzy Luenberger observer is designed. In addition, based on the estimated states and attacks, the designed MPC resilient control method achieves a satisfied tracking performance. Finally, the feasibility of the estimation algorithm and resilient controller are verified by simulation.

Stochastic optimal control and piecewise parameterization and optimization method for inventory control system improvement

In recent decades, stochastic control systems have been widely used in industrial production, biomedicine, aerospace, military strategy and so on. In this paper, an approximate optimal strategy derived from a stochastic linear quadratic (SLQ) optimal control problem is considered, and a piecewise parameterization and optimization (PPAO) method is proposed. Firstly, using the principle of dynamic programming, the control form of SLQ optimal control problem is relevant to a Riccati differential equation. It is well known that the Riccati differential equation is difficult to be solved analytically. Thus, we present a PPAO method for finding an approximate optimal strategy for stochastic control problems. Here, a piecewise parameter control can be obtained by solving first-order differential equations rather than Riccati differential equations. Finally, the inventory control problems with different dimensions are used to justify the feasibility of PPAO method, and the results compared with the original optimal control are given. The results show that parametric control greatly simplifies the control form, and a small model error is ensured.

Optimal control of deterministic and stochastic Eco-epidemic food adulteration model

In this paper, we develop a deterministic and stochastic mathematical model to study the effect of contaminated foods on human health by spreading media awareness. A predator–prey SI model with media-induced response function is considered. In the deterministic case, the model has three equilibria: trivial, disease-free, and endemic. While the endemic equilibria exist for low media impact provided R0>1. Here, the trivial equilibrium is always unstable. However, the disease-free equilibrium exhibits local and global asymptotically stable behavior provided that R0<1. The conditions for the local behavior of endemic equilibrium are also determined along with the existence of Hopf bifurcation. Further, the impact of external interference is studied for the stochastic model. The stochastic model has a unique global positive solution which is ultimately bounded. The stochastic model also demonstrates asymptotic behavior around disease-free and endemic equilibria. Additionally, optimal control theory has been employed to explore deterministic and stochastic optimal control systems in depth. Finally, the analytical results are validated using some numerical simulations.

On Strategic Measures and Optimality Properties in Discrete-Time Stochastic Control with Universally Measurable Policies

This paper concerns discrete-time infinite-horizon stochastic control systems with Borel state and action spaces and universally measurable policies. We study optimization problems on strategic measures induced by the policies in these systems. The results are then applied to risk-neutral and risk-sensitive Markov decision processes to establish the measurability of the optimal value functions and the existence of universally measurable, randomized or nonrandomized, ϵ-optimal policies, for a variety of average cost criteria and risk criteria. We also extend our analysis to a class of minimax control problems and establish similar optimality results under the axiom of analytic determinacy. Funding: This work was supported by grants from DeepMind, the Alberta Machine Intelligence Institute (AMII), and Alberta Innovates-Technology Futures (AITF).

An alternative method for estimating Hurst exponent of control signals based on system dynamics

One of the indices for evaluating the performance of control systems based on the routine operation data is the Hurst index. It is shown that the Hurst index value is very close to that of the minimum variance index which measures the performance of a control loop in terms of minimum variance benchmark. The advantage of Hurst index over minimum variance index is that it does not require the system model and its parameters. The Hurst index is based on the Hurst exponent, which measures the time series correlations. This paper analytically scrutinizes the effect of parameters and dynamics of a stochastic control system on the output Hurst exponent, which helps better comprehend the Hurst index and its relationship with the minimum variance index. For this purpose, the control system output is modeled as an Autoregressive Moving Average (ARMA) process, and the relationship between the parameters of the ARMA process and the output Hurst exponent is examined. Furthermore, the characteristics of the autocorrelation function of a signal under minimum variance conditions are extracted. This paper also proposes a new technique for estimating the Hurst exponent of control signals based on the autocorrelation function. Simulation examples illustrate the efficiency of the proposed approaches.