Finite Time Horizon Research Articles

An optimal selective maintenance policy for multi-state series production systems by considering environment impact

Series production systems are prevalent in the manufacturing industry. The productivity of these systems is influenced by various factors, including system conditions, on-site environment, maintenance planning, and others. Efforts to develop an effective maintenance plan that considers the on-site environment are crucial for maximizing net revenue over the system’s lifetime, which has practical significance. Against this backdrop, a selective maintenance model is constructed for a multi-state series production system over a finite time horizon. This model describes the operation and maintenance processes at any decision period and analyzes the impact of the on-site environment on system productivity. Subsequently, an optimal maintenance strategy is formulated with the goal of maximizing net revenue throughout the system’s lifetime. To efficiently solve this optimal strategy, a hybrid solution approach is developed by integrating a deterministic method with a heuristic method. Finally, a numerical example involving a real chip mounter system is presented to verify the developed maintenance strategy. This is followed by a comparative experiment to validate the performance of the hybrid solution approach. Additionally, sensitivity analysis is conducted to test the strategy’s robustness. The results from the numerical example, comparative experiment, and sensitivity analysis demonstrate that the developed strategy is effective and robust, and the hybrid approach is efficient.

Optimal maintenance over a finite time horizon for a system under imperfect inspection and dynamic working environment

Inspections play a crucial role in determining the state of a system and informing timely maintenance decisions. However, due to the limitations of adopted inspection technologies, inspection errors are often inevitable. More importantly, the likelihood of such errors is often influenced by the system's working environment that varies over time and space. Despite this, most studies on imperfect inspections largely overlook the effects of such dynamic working environments on inspection errors. To address this gap, this paper proposes a maintenance policy accounting for imperfect inspections for a degrading system with hidden failures under a dynamic working environment. We develop two models based on a recursive method. The first model examines a case where the working environment affects only inspection errors, whereas the second one accounts for its impact on both inspection errors and the system's degradation process. Our focus is to determine the optimal interval and number of inspections that minimize the expected maintenance cost over a finite time horizon. Numerical examples are provided to illustrate the proposed method to ensure the effectiveness of maintenance actions. The optimization outcomes not only offer a valuable tool for engineers seeking to reduce maintenance costs but also shed light on broader managerial implications.

Battery Temperature Prediction Using an Adaptive Neuro-Fuzzy Inference System

Maintaining batteries within a specific temperature range is vital for safety and efficiency, as extreme temperatures can degrade a battery’s performance and lifespan. In addition, battery temperature is the key parameter in battery safety regulations. Battery thermal management systems (BTMSs) are pivotal in regulating battery temperature. While current BTMSs offer real-time temperature monitoring, their lack of predictive capability poses a limitation. This study introduces a novel hybrid system that combines a machine learning-based battery temperature prediction model with an online battery parameter identification unit. The identification unit continuously updates the battery’s electrical parameters in real time, enhancing the prediction model’s accuracy. The prediction model employs an Adaptive Neuro-Fuzzy Inference System (ANFIS) and considers various input parameters, such as ambient temperature, the battery’s current temperature, internal resistance, and open-circuit voltage. The model accurately predicts the battery’s future temperature in a finite time horizon by dynamically adjusting thermal and electrical parameters based on real-time data. Experimental tests are conducted on Li-ion (NCA and LFP) cylindrical cells across a range of ambient temperatures to validate the system’s accuracy under varying conditions, including state of charge and a dynamic load current. The proposed models prioritise simplicity to ensure real-time industrial applicability.

Boundary Optimal Control for Parabolic Distributed Parameter Systems With Value Iteration.

A reinforcement learning-based boundary optimal control algorithm for parabolic distributed parameter systems is developed in this article. First, a spatial Riccati-like equation and an integral optimal controller are derived in infinite-time horizon based on the principle of the variational method, which avoids the complex semigroups and operator theories. Using state data along the system trajectory, a value iteration algorithm via the Bellman optimality principle is proposed to obtain the solution of the spatial Riccati-like equation and the optimal control law. The convergence of the value iteration algorithm is proved. Subsequently, an approximation scheme based on weighted residuals is developed to implement the value iteration algorithm, where radial basis functions are chosen as the basic functions to approximate the solution of the spatial Riccati-like equation. Simulations on the diffusion-reaction process demonstrate the effectiveness of the developed method.

Variational Inequalities on Unbounded Domains for Zero-Sum Singular Controller vs. Stopper Games

We study a class of zero-sum games between a singular controller and a stopper over a finite-time horizon. The underlying process is a multidimensional (locally nondegenerate) controlled stochastic differential equation (SDE) evolving in an unbounded domain. We prove that such games admit a value and provide an optimal strategy for the stopper. The value of the game is shown to be the maximal solution in a suitable Sobolev class of a variational inequality of min-max type with an obstacle constraint and a gradient constraint. Although the variational inequality and the game are solved on an unbounded domain, we do not require boundedness of either the coefficients of the controlled SDE or of the cost functions in the game. Funding: A. Bovo was partially supported by the Doctoral Studentship from the University of Leeds.

Characterization of a Cournot–Nash Equilibrium for a Fishery Model with Fuzzy Utilities

The article deals with the extensions of discrete-time games with infinite time horizon and their application in a fuzzy context to fishery models. The criteria for these games are the total discounted utility and the average utility in a fishing problem. However, in the fuzzy case, game theory is not the best way to represent a real fishing problem because players do not always have enough information to accurately estimate their utility in the context of fishing. For this reason, in this paper, trapezoidal-type fuzzy utility values are considered for a fishing model, and the terms of the Nash equilibrium are given in the fuzzy context, i.e., this equilibrium is represented using the partial order of the α-cuts of the fuzzy numbers; to the best of the authors’ knowledge, there is no work with this type of treatment. To obtain each equilibrium, a suitable fully determined fuzzy game is used in combination with the dynamic programming technique applied to this game in the context of fishing. The main results are (i) the Nash equilibria of the fuzzy games coincide with the Nash equilibria of the nonfuzzy games and are explicitly determined in a fishery model and (ii) the values of the fuzzy games are of trapezoidal type and are also explicitly given in the fishery model.

Reflected BSDEs driven by G-Brownian motion with time-varying Lipschitz coefficients

In this paper, we consider the reflected backward stochastic differential equations driven by [Formula: see text]-Brownian motion (reflected [Formula: see text]-BSDEs) with time-varying Lipschitz coefficients. We obtain the uniqueness result by establishing a priori estimates. For the existence, the solution can be approximated by a family of reflected [Formula: see text]-BSDEs with Lipschitz conditions and by penalized [Formula: see text]-BSDEs with time-varying coefficients. The latter approximation is useful to get the comparison theorem. Finally, we study the reflected [Formula: see text]-BSDEs with infinite time horizon.

Scalable Computation of Robust Control Invariant Sets of Nonlinear Systems

Ensuring robust constraint satisfaction for an infinite time horizon is a challenging, yet crucial task when deploying safety-critical systems. We address this issue by synthesizing robust control invariant sets of perturbed nonlinear sampled-data systems. This task can be encoded as a non-convex program that we approximate by a tailored, computationally efficient successive convexification algorithm. Based on the zonotopic representation of invariant sets, we obtain an updated candidate for the invariant set and the invariance-enforcing controller by solving a single convex program. To obtain a possibly large region of safe operation, our algorithm is designed so that the sequence of candidate invariant sets is volume-wise monotonically increasing. We demonstrate the efficacy and scalability of our approach using a broad range of nonlinear control systems from the literature with up to 20 dimensions.

Edge/Cloud Infinite-Time Horizon Resource Allocation for Distributed Machine Learning and General Tasks

Edge/Cloud Infinite-Time Horizon Resource Allocation for Distributed Machine Learning and General Tasks

Commitment under uncertainty: production-inventory policies associated with environmental considerations

We consider a manufacturer that produces in response to a stochastic demand and emits pollution during the production process. Industrial pollutants released into the air are characterised by spatial variability and heterogeneity, and the precision of instruments for measuring stochastic pollution stocks varies widely across pollutants. Consequently, commitment (open-loop) strategies to control production and associated emissions are considered more practical than contingent (feedback) approaches when manufacturers are concerned about environmental consequences. In this article, we derive an optimal, open-loop control policy over an infinite time horizon and show a condition that ensures that such a policy induces asymptotic convergence of expected inventory stock and pollution stock trajectories to unique steady states. Moreover, we find that the variance in the pollution stock also converges asymptotically to a unique steady state, which is critical for prevention of irreversible environmental consequences. We show that environmental uncertainty affects inventories, leading, in the long run, to lower expected steady-state inventory stocks. We compare feedback and open-loop production policies and show that though there are losses associated with the inability to track inventory and pollution stocks accurately and thus applying the open-loop control instead of feedback, the expected long-term inventory and pollution stocks are not affected.

A differential game model for sponsored content

We consider a communication platform distinguished for its high-quality content, where advertising can take two different forms: traditional and sponsored (also known as native advertising in the marketing literature). Native advertising is a widely used marketing tool that aims to mimic the regular topics of the platform on which it is placed. Due to this striking resemblance, native advertising may be very effective, but at the same time, it may negatively influence the perceived credibility of the media outlet. In our model, a firm allocates investments to both traditional and native advertising on such a platform. Meanwhile, the media outlet must grapple with the trade-off between the profit accrued from publishing native advertising and the ensuing decline in credibility. We formalise this problem as a hierarchical infinite-time horizon linear state differential game, played à la Stackelberg, where the media outlet acts as the leader while the firm is the follower. Finally, we characterise a time-consistent open-loop equilibrium and obtain the conditions that make it optimal for the media outlet to accept native advertising.

Concentration of measure for graphon particle system

Abstract We study heterogeneously interacting diffusive particle systems with mean-field-type interaction characterized by an underlying graphon and their finite particle approximations. Under suitable conditions, we obtain exponential concentration estimates over a finite time horizon for both 1- and 2-Wasserstein distances between the empirical measures of the finite particle systems and the averaged law of the graphon system.

A kernel-based approximate dynamic programming approach: Theory and application

This article proposes a novel kernel-based Dynamic Programming (DP) approximation method to tackle the typical curse of dimensionality of stochastic DP problems over the finite time horizon. Such a method utilizes kernel functions in combination with Support Vector Machine (SVM) regression to determine an approximate cost function for the entire state space of the underlying Markov Decision Process (MDP), by leveraging cost function computed for selected representative states. Kernel functions are used to define the so-called kernel matrix, while the parameter vector of the given kernel-based cost function approximation is computed by moving backwards in time from the terminal condition and by applying SVM regression. This way, the difficulty of selecting a proper set of features is also tackled. The proposed method is then extended to the infinite time horizon case. To show the effectiveness of the proposed approach, the resulting Recursive Residual Approximate Dynamic Programming (RR-ADP) algorithm is applied to the sensor scheduling design in multi-process remote state estimation problems.

Inverse Optimal Safety Filters

CBF-QP (control barrier function quadratic programs) safety filters are pointwise minimizers of the control effort at a given state, i.e., myopically optimal at each time. But are they optimal over the entire infinite time horizon? What does it mean for a controlled system to be “optimally safe” as opposed to, conventionally “optimally stable”? When disturbances, deterministic and stochastic, have unknown upper bounds, how should safety be defined to allow a graceful degradation under disturbances? Can safety filters be designed to guarantee such weaker safety properties as well as the optimality of safety over the infinite time horizon? We pose and answer these questions for general systems affine in control and disturbances and illustrate the answers using several examples. In the process, using the existing QP safety filters, as well as more general safety-ensuring feedbacks, we generate entire families of safety filters which are optimal over the infinite horizon though they are conservative (favoring safety over ‘liveness’) relative to the standard QP.

Expected power utility maximization with delay for insurers under the 4/2 stochastic volatility model

In this paper, we are interested in the optimal investment and reinsurance strategies of an insurer with delay under the $ 4/2 $ stochastic volatility model. Indeed, the objective of the insurer is to maximize the expected power utility of the terminal wealth and the average wealth on finite time horizon. The other objective is to maximize the growth rate of expected power utility per unit time on infinite time horizon. The wealth of the insurer is described by an approximation of the classical Cramér–Lundberg process. Then, these problems can be formulated as stochastic control problems with delay. A pair of forward-backward stochastic differential equations that are derived via the stochastic maximum principle has an explicit solution obtained by solving a Riccati differential equation. So, the optimal strategy of the finite time horizon problem can be constructed explicitly. And, by investigating asymptotics of the Riccati equation, the infinite time horizon problem can be solved explicitly. Finally, we present some numerical results to illustrate our model, optimal strategies and sensitivities of some parameters.

Multiplicative Barrier Certificates for Probabilistic Safety of Markov Jump Systems

This work offers a formal framework for providing safety certificates over discrete-time Markov jump systems (MJSs). Our proposed scheme centers around a new concept of multiplicative barrier certificates (MBCs), enabling us to establish a probabilistic lower bound for the safety property in infinite time horizons within this class of models. In particular, while a summative barrier certificate, as commonly proposed in the relevant literature, may not be available to ensure the safety of MJS models, we introduce an alternative approach in a multiplicative form, as opposed to the conventional “decreasing in mean” formulation. We demonstrate that in scenarios where a summative barrier does not exist, there is potential for the existence of a multiplicative counterpart. We also provide a systematic methodology grounded on the counterexample-guided inductive synthesis (CEGIS) scheme to systematically construct the proposed multiplicative certificates. We showcase the efficacy of our approach through its application in an air traffic management system.

Nonzero-Sum Stochastic Impulse Games with an Application in Competitive Retail Energy Markets

We study a nonzero-sum stochastic differential game with both players adopting impulse controls, on a finite time horizon. The objective of each player is to maximize her total expected discounted profits. The resolution methodology relies on the connection between Nash equilibrium and the corresponding system of quasi-variational inequalities (QVIs in short). We prove, by means of the weak dynamic programming principle for the stochastic differential game, that the equilibrium expected payoff of each player is a constrained viscosity solution to the associated QVIs system in the class of linear growth functions. We also introduce a family of equilibrium expected payoffs converging to our equilibrium expected payoff of each player, and which is characterized as the unique constrained viscosity solutions of an approximation of our QVIs system. This convergence result is useful for numerical purpose. We apply a probabilistic numerical scheme which approximates the solution of the QVIs system to the case of the competition between two electricity retailers. We show how our model reproduces the qualitative behavior of electricity retail competition.

Model Predictive Bang-Bang Controller Synthesis via Approximate Value Functions

In this paper, we propose a novel method for addressing Optimal Control Problems (OCPs) with input-affine dynamics and cost functions. This approach adopts a Model Predictive Control (MPC) strategy, wherein a controller is synthesized to handle an approximated OCP within a finite time horizon. Upon reaching this horizon, the controller is re-calibrated to tackle another approximation of the OCP, with the approximation updated based on the final state and time information. To tackle each OCP instance, all non-polynomial terms are Taylor-expanded about the current time and state and the resulting Hamilton-Jacobi-Bellman (HJB) PDE is solved via Sum-of-Squares (SOS) programming, providing us with an approximate polynomial value function that can be used to synthesize a bang-bang controller.

Competition for agents’ opinions in small dynamic systems with limited control

This paper explores a competitive opinion dynamic system representing a social network evolving over time. The agents compare their own opinions with the average ones and form new opinions. There are players who control agents’ opinions in a social network, and their aims are to make opinions in the society closer to the desired levels over a finite time horizon by minimizing their costs. The feature of this model is that players can influence agents in a limited number of time moments. We show how players can choose the moments of influence from Pareto optimal set on numerical examples. The results of numerical simulations are provided in the work.

Multi-objective perishable multi-item green inventory models with uncertain finite time horizons and constraints by neutrosophic optimisation approach

Multi-objective perishable multi-item green inventory models with uncertain finite time horizons and constraints by neutrosophic optimisation approach