Terminal Cost Research Articles

Inverse problems for mean field games

The theory of mean field games (MFGs) studies the limiting behaviors of large systems where the agents interact with each other in a certain symmetric way. The running and terminal costs are critical for the agents to decide the strategies. However, in practice they are often partially known or totally unknown for the agents, while the total cost is known at the end of the game. To address this challenging issue, we propose and study several inverse problems for MFGs. When the Lagrangian is a kinetic energy, we first establish unique identifiability results, showing that one can recover either the running cost or the terminal cost from knowledge of the total cost. If the running cost is limited to the time-independent class, we can further prove that one can simultaneously recover both the running and the terminal costs. Finally, we extend the results to the setup with general Lagrangians.

Learning-Based Model Predictive Control for Autonomous Racing

In this paper, we present the adaptation of the terminal component learning-based model predictive control (TC-LMPC) architecture for autonomous racing to the Formula Student Driverless (FSD) context. We test the TC-LMPC architecture, a reference-free controller that is able to learn from previous iterations by building an appropriate terminal safe set and terminal cost from collected trajectories and input sequences, in a vehicle simulator dedicated to the FSD competition. One major problem in autonomous racing is the difficulty in obtaining accurate highly nonlinear vehicle models that cover the entire performance envelope. This is more severe as the controller pushes for incrementally more aggressive behavior. To address this problem, we use offline and online measurements and machine learning (ML) techniques for the online adaptation of the vehicle model. We test two sparse Gaussian process regression (GPR) approximations for model learning. The novelty in the model learning segment is the use of a selection method for the initial training dataset that maximizes the information gain criterion. The TC-LMPC with model learning achieves a 5.9 s reduction (3%) in the total 10-lap FSD race time.

From the Optimal Singular Stochastic Control to the Optimal Stopping for Regime-Switching Processes

This work generalizes the connection between the optimal singular stochastic control problem and the optimal stopping problem for regime-switching processes. Via the optimal singular stochastic control, the optimal stopping time and the continuation region are characterized. Moreover, we prove the existence of optimal singular stochastic control for a finite horizon singular control problem with the cost function containing the terminal cost. We prove it directly by the compactification method, which is based on an elaborate application of the properties of probability measures over the càdlàg space. Such a problem was left open in Haussmann and Suo [SIAM J. Control Optim., 33 (1995), pp. 916–936] and [SIAM J. Control Optim., 33 (1995), pp. 937–959]). In addition, our compactification method can remove the convexity condition on the coefficients used in Dufour and Miller [SIAM J. Control Optim., 43 (2004), pp. 708–730].

Variational Problems Involving a Generalized Fractional Derivative with Dependence on the Mittag–Leffler Function

In this paper, we investigate the necessary conditions to optimize a given functional, involving a generalization of the tempered fractional derivative. The exponential function is replaced by the Mittag–Leffler function, and the kernel depends on an arbitrary increasing function. The Lagrangian depends on time, the state function, its fractional derivative, and we add a terminal cost function to the formulation of the problem. Since this new fractional derivative is presented in a general form, some previous works are our own particular cases. In addition, for different choices of the kernel, new results can be deduced. Using variational techniques, the fractional Euler–Lagrange equation is proved, as are its associated transversality conditions. The variational problem with additional constraints is also considered. Then, the question of minimizing functionals with an infinite interval of integration is addressed. To end, we study the case of the Herglotz variational problem, which generalizes the previous one. With this work, several optimization conditions are proven that can be useful for different optimization problems dealing with various fractional derivatives.

Efficient self-driving strategy with preceding car following and road slopes based on learning predictive control

Self-driving vehicles has been being popular since it not only improves traffic safety and flow but also the efficiency of energy consumption. Technical advance of self-driving vehicles significantly becomes requiring the energy efficiency of the electric vehicles (EVs) since the most car has very limited power capacity which is based on sole battery source. In this research, a hybrid learning predictive control architecture is proposed to reduce the energy consumption of the EVs. This paper combines the model predictive control (MPC) to the deep reinforcement learning (DRL) networks to solve the optimization problems with constrained environments to maximize energy efficiency of the EVs. Especially, the computed cost of the predictive horizon is transferred into the DRL networks as the state value and the reward is inversely fed into MPC controller to find the optimal control strategy. Resultingly, the proposed architecture can reduce the terminal control cost function using precedent state value. The proposed algorithm was tested in the PGDrive simulation environment and the experimental results show the efficiency of the proposed energy saving architecture.

Optimal control of jump-diffusion processes with random parameters

Let $X(t)$ be a controlled jump-diffusion process starting at $x \in [a,b]$ and whose infinitesimal parameters vary according to a con\-tinuous-time Markov chain. The aim is to minimize the expected value of a cost function with quadratic control costs until $X(t)$ leaves the interval $(a,b)$, and a termination cost that depends on the final value of $X(t)$. Exact and explicit solutions are obtained for important processes.

Simultaneous moving horizon estimation and control for nonlinear systems subject to bounded disturbances

SummaryIn this work, we address the output–feedback control problem for nonlinear systems under bounded disturbances using a moving horizon approach. The controller is posed as an optimisation‐based problem that simultaneously estimates the state trajectory and computes future control inputs. It minimizes a criterion that involves finite backward and forward horizons with respect to the unknown initial state, measurement noises and control input variables. The main novelty of this work relies on linking the lengths of the forward and backward windows with the closed‐loop stability, assuming detectability and decoding sufficient conditions to assure system stabilizability. It leads to a formulation that does not require to be a Control Lyapunov Function for the terminal cost of the controller. Simulation examples are carried out to compare the performance of solving simultaneously and independently the estimation and control problems. Furthermore, the examples show how the controller influences the length of the estimation window through its gain.

A learning- and scenario-based MPC design for nonlinear systems in LPV framework with safety and stability guarantees

ABSTRACT This paper presents a learning- and scenario-based model predictive control (MPC) design approach for systems modelled in the linear parameter-varying (LPV) framework. Using input-output data collected from the system, a state-space LPV model with uncertainty quantification is first learned through the variational Bayesian inference Neural Network (BNN) approach. The learned probabilistic model is assumed to contain the true dynamics of the system with a high probability and is used to generate scenarios that ensure safety for a scenario-based MPC. Moreover, to guarantee stability and enhance the performance of the closed-loop system, a parameter-dependent terminal cost and controller, as well as a terminal robust positive invariant set are designed. Numerical examples will be used to demonstrate that the proposed control design approach can ensure safety and achieve desired control performance.

Computationally Efficient Continuous-Time Model Predictive Control of a 2-DOF Helicopter via B-Spline Parameterization.

This paper investigates one way to reduce the computational burden of continuous-time model predictive control (MPC) laws by representing the input/output signals and related models using B-spline functions. Such an approximation allows to implement the resulting feedback control law more efficiently, requiring less online computational effort. As a result, the proposed controller formulates the control signals as continuous polynomial spline functions. All constraints assumed over the prediction horizon are then expressed as constraints acting on the B-splines control polygon vertices. The performance of the proposed theoretical framework has been demonstrated with several real-time experiments using the well-known 2-DOF laboratory helicopter setup. The aim of the presented experiments was to track given step-like reference trajectories for pitch and yaw angles under notable parameter uncertainties. In order to suppress the influence of uncertainties, the control algorithm is implemented in an adaptive mode, equipped with the recursive least squares (RLS) estimation of model parameters and with the adaptation of stabilizing terminal set and terminal cost calculations. Thanks to the presented framework, it is possible to significantly reduce the computational burden, measured by the number of decision variables and input constrains, indicating the potential of the proposed concept for real-time applications, even when using embedded control hardware.

Cost-Effectiveness of Pembrolizumab as an Adjuvant Treatment in Colombia for Melanoma Patients with Lymph Node Involvement After Complete Resection.

The KEYNOTE-054 trial found that adjuvant treatment with pembrolizumab improved recurrence-free survival versus placebo in completely resected high-risk stage III melanoma patients. We assessed the cost-effectiveness of adjuvant pembrolizumab in Colombia compared with watchful waiting, a widely used strategy despite the high risk of recurrence with surgery alone. A four-health state [recurrence-free (RF), locoregional recurrence (LR), distant metastases (DM), and death) Markov model was developed to assess the lifetime medical costs and outcomes (3% annual discount), along with cost-effectiveness ratios (ICERs). The transitions from the RF and LR states were modeled using KEYNOTE-054 data, and those from the DM state were modeled using data from the KEYNOTE-006 trial and a network meta-analysis of advanced treatments received after adjuvant pembrolizumab and watchful waiting. The health state utilities were derived from KEYNOTE-054 Euro-QoL data and literature. Costs are expressed in 2021 Colombian pesos (COP). Over a 46-year time horizon, patients on adjuvant pembrolizumab and watchful waiting were estimated to gain 9.69 and 7.56 quality-adjusted life-years (QALYs), 10.83 and 8.65 life-years (LYs), and incur costs of COP 663,595,726 and COP 563,237,206, respectively. The proportion of LYs spent in RF state was 84.63% for pembrolizumab and 72.13% for watchful waiting, yielding lower subsequent treatment, disease management, and terminal care costs for pembrolizumab. Adjuvant pembrolizumab improved survival by 2.18 LYs and 2.13 QALYs versus watchful waiting. The ICER per QALY was COP 47,081,917, primarily driven by recurrence rates and advanced melanoma treatments. The deterministic sensitivity analysis results were robust and consistent across various reasonable inputs and alternative scenarios. At a willingness-to-pay threshold of COP 69,150,201 per QALY, the probability of pembrolizumab being cost-effective was 65.70%. Pembrolizumab is cost-effective as an adjuvant treatment compared to watchful waiting among patients with high-risk stage III melanoma after complete resection in Colombia.

Integrated Path Tracking Controller of Underground Articulated Vehicle Based on Nonlinear Model Predictive Control

This paper proposes an integrated path tracking controller for articulated vehicles. A nonlinear model-predictive control (NMPC)-based reference state tracker is designed as an upper-level controller to solve the vehicle’s longitudinal velocity and steering rate. A terminal cost is introduced into the NMPC to improve the controller’s stability. A lower-level controller is developed to translate upper-level solutions into vehicle actuators’ signals, including steering and driving controllers. The steering controller translates the steering rate into the linear velocity of the cylinder to calculate the required fluid volume and ultimately into the rotation speed of the steering motor. The neural network method is applied in the driving controller to ensure accuracy under different loadings. In order to investigate the effects of the path tracking controller, an articulated dump truck is adapted for the field tests by adding the steering-by-wire system and driving-by-wire system, respectively. Experimental verifications of the lower-level controller are performed. The results show that the controller can accurately satisfy the demand. Finally, the tracking performance of the integrated path tracking controller is analyzed experimentally under different reference velocities. The results indicate that tracking accuracy can be guaranteed.

Efficient min–max MPC: Achieving a large domain of attraction with short horizon

AbstractThis article considers the regulation problem of a constrained linear system with bounded disturbance. The objective is to overcome the offline and online problems of the standard dual‐mode min–max model predictive control (MPC). The offline problem is related to the construction of a polyhedral terminal set, which is prohibitively complex especially for high order systems. We propose to replace the polyhedral set with the ellipsoidal one, which is much easier to construct. The online problem is related to the computational burden, which grows exponentially with the prediction horizon . We show how to employ a new terminal cost function, that results in a new min–max MPC with a large domain of attraction even with or . Hence the computational complexity is drastically reduced. Two numerical examples with comparison to earlier solutions from the literature illustrate the effectiveness of the proposed approach.

A Nonlinear MPC Scheme for Output Tracking Without Terminal Ingredients

In this paper, we propose a novel nonlinear model predictive control (MPC) scheme without terminal ingredients for output tracking. The control objective is to steer the output of the controlled system towards some desired set, or towards the best reachable set, in case of unreachable targets. The considered MPC formulation combines a stabilizing MPC scheme without terminal ingredients with an output tracking formulation using an artificial steady-state. In addition, we propose an offset-cost that is applicable to general non-convex steady-state manifolds. The resulting MPC scheme is simple to implement since it does not require the design of terminal ingredients (i.e., terminal controller, terminal cost, and terminal region) and is applicable to general (possibly unreachable) output specifications. We show the benefits of the proposed approach by means of a numerical example where we compare it to the state of art tracking MPC formulation.

MP39-06 DISPARITY IN COST AND HEALTH SERVICE USE IN TERMINAL PHASE AMONG PROSTATE CANCER PATIENTS

You have accessJournal of UrologyCME1 Apr 2023MP39-06 DISPARITY IN COST AND HEALTH SERVICE USE IN TERMINAL PHASE AMONG PROSTATE CANCER PATIENTS Ravishankar Jayadevappa, S Bruce Malkowicz, Thomas Guzzo, and Sumedha Chhatre Ravishankar JayadevappaRavishankar Jayadevappa More articles by this author , S Bruce MalkowiczS Bruce Malkowicz More articles by this author , Thomas GuzzoThomas Guzzo More articles by this author , and Sumedha ChhatreSumedha Chhatre More articles by this author View All Author Informationhttps://doi.org/10.1097/JU.0000000000003277.06AboutPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookLinked InTwitterEmail Abstract INTRODUCTION AND OBJECTIVE: Research shows considerable racial and ethnic variation in the structure and process of care that has implications for care choices and cost. Objective is to assess the racial and ethnic disparity in the cost and health service use in terminal phase among prostate cancer patients. METHODS: Population-based cohort study using data from Surveillance, Epidemiological, and End Results-Medicare (SEER-Medicare). Cohort consisted of patients diagnosed with prostate cancer between 1996 and 2013, and deceased as of 12/31/2013. One-year period prior to death was considered as terminal phase. We operationalized cost as sum of Medicare reimbursement for inpatient, outpatient and provider services. Cost during the terminal phase was assessed for four racial and ethnic groups: African American, white, Hispanic and other. We used GLM model for cost and Poisson models for health service use. Independent variable were socio-demographic and clinical characteristics. RESULTS: Between 1996 and 2013, 788,217 Medicare beneficiaries were diagnosed with prostate cancer, of which, 314,861 (39%) were deceased by end of year 2013 and constituted the study cohort. In our cohort, African American patient were youngest (mean age 71.6 years, standard deviation (Std) 8.9), and white patients were oldest (mean 74.9, std 8.5). GLM model indicated that race, age, and comorbidity were associated with cost. The health service use varied between racial and ethnic groups. A longitudinal comparison of mean cost in terminal phase for the four racial and ethnic groups is presented in Figure 1. The mean cost for all racial and ethnic groups increased over time; and African American patients had highest cost in all years, compared to other groups. CONCLUSIONS: This study is first of its kind to assess longitudinally the cost of terminal phase and associated disparity. The terminal phase cost almost doubled between 1995 and 2013, for all racial and ethnic groups, and continued to be highest for African American patients. Future studies need to address the overuse, underuse and misuse of care that may contribute to the high cost burden in the terminal phase and the associated racial and ethnic disparity in cost and health service use. Source of Funding: DoD W81XWH-19-1-0461 © 2023 by American Urological Association Education and Research, Inc.FiguresReferencesRelatedDetails Volume 209Issue Supplement 4April 2023Page: e536 Advertisement Copyright & Permissions© 2023 by American Urological Association Education and Research, Inc.MetricsAuthor Information Ravishankar Jayadevappa More articles by this author S Bruce Malkowicz More articles by this author Thomas Guzzo More articles by this author Sumedha Chhatre More articles by this author Expand All Advertisement PDF downloadLoading ...

Computationally Efficient Stability-Based Nonlinear Model Predictive Control Design for Quadrotor Aerial Vehicles

This article presents the design of a stability-guaranteed nonlinear model predictive controller for quadrotor-type microaerial vehicles to operate robustly on fast trajectories. The basic controller structure operates without having to use terminal conditions in the optimization problem. As a result, the controller is computationally less demanding and provides more stable closed-loop performance than traditional nonlinear predictive control schemes. This article presents a detailed stability analysis without terminal costs or terminal constraints and proves the asymptotic stability and necessary conditions for the recursive feasibility of the system. This article derives the growth-bound sequence that enables obtaining the shortest possible prediction horizon for stability. The proposed analysis provides the necessary conditions to implement the controller while using the shortest stabilizing prediction horizon compared to major traditional predictive control schemes reported in the literature. This particular feature enables the proposed controller to perform fast optimization and, hence, the capability to implement fast trajectories using feedback regularization. In order to demonstrate the validity of this new proposed control scheme, first, several MATLAB simulations are conducted to demonstrate the improved performance of the controller especially when the quadrotor vehicle follows fast trajectories. Real-time lab experiments are also conducted to validate the performance of the proposed scheme for point stabilization (hovering) and trajectory tracking problems. The results show that the proposed scheme can stabilize the system with a relatively short prediction horizon, a fast convergence rate, and a small tracking error.

Dual-UAV Payload Transportation Using Optimized Velocity Profiles via Real-Time Dynamic Programming

In this paper, a real-time dynamic programming (RTDP) approach was developed for the first time to jointly carry a slung load using two unmanned aerial vehicles (UAVs) with a trajectory optimized for time and energy consumption. The novel strategy applies RTDP algorithm, where the journey was discretized into horizons consisting of distance intervals, and for every distance interval, an optimal policy was obtained using a dynamic programming sweep. The RTDP-based strategy is applied for dual-UAV collaborative payload transportation using coordinated motion where UAVs act as actuators on the payload. The RTDP algorithm provides the optimal velocity decisions for the slung load transportation to either minimize the journey time or the energy consumption. The RTDP approach involves minimizing a cost function which is derived after simplifying the combined model of the dual-UAV-payload system. The cost function derivation was also accommodated to dynamically distribute the load/energy between two multi-rotor platforms during a transportation mission. The cost function is used to calculate transition costs for all stages and velocity decisions. A terminal cost is used at the last distance interval during the first phase of the journey when the velocity at the end of the current horizon is not known. In the second phase, the last stage or edge of the horizon includes the destination, hence final velocity is known which is used to calculate the transition cost of the final stage. Once all transition costs are calculated, the minimum cost is traced back from the final stage to the current stage to find the optimal velocity decision. The developed approach was validated in MATLAB simulation, software in the loop Gazebo simulation, and real experiments. The numerical and Gazebo simulations showed the successful optimization of journey time or energy consumption based on the selection of the factor λ. Both simulation and real experiments results show the effectiveness and the applicability of the proposed approach.

Agri-Food Enterprises and Relationship Marketing During COVID-19: A Primary Data Research through Commitment-Trust Theory

This paper examines relationship marketing through the views of agri-food enterprises during COVID-19 using the commitment-trust theory as a framework. For this purpose, a primary research was conducted in 266 Greek agri-food enterprises at all stages of the agri-food sector and its activities to investigate relationship marketing in times of crisis. Through a confirmatory structural equation model (SEM), compatible with the commitment-trust theory, the results show that relationship termination costs explain the relationship commitment while communication explains trust. Moreover, there is an indirect effect of shared values on cooperation through trust. It is important to note that despite the COVID-19 crisis, there is no effect of uncertainty on trust. The above outcomes shed light on agri-food enterprises and relationship marketing in periods of crisis. With this knowledge, this paper provides recommendations to agri-food enterprises to focus on cooperation and shared values rather than being dominated by uncertainty.

Model predictive control with implicit terminal ingredients

This paper proposes a new formulation of model predictive control for linear discrete time systems under state and control constraints. The main novelty of the proposed formulation is the introduction, and a computationally adequate utilization, of the implicit terminal constraint set and the implicit terminal cost function. From a numerical point of view, the deployment of the implicit terminal ingredients enables a computationally tractable use of model predictive control for higher dimensional control problems. From a practical point of view, the design of the proposed model predictive control algorithm is very constructive for a class of linear systems with polyhedral constraints. From a structural point of view, the proposed model predictive control retains a priori guarantees of the highly desirable positive invariance and exponential stability properties.

Optimizing truck platooning transportation planning: an application to forestry products supply chains

The Fourth Industrial Revolution offers new opportunities for improving the efficiency and agility of supply chain operations such as transportation. This study explores the impact of integrating truck platooning technology in forestry products supply networks. Companies need to know how and where to use truck platooning in transportation networks to get optimum benefits from truck platooning in supply chains. To this end, a Mixed Integer Linear Programming model was developed. Decisions to be made include the selection of the potential terminal locations, the number of ordinary and platooning trucks needed in the transportation network, the origin and destination of products, and their flow in direct and backhaul routes. The objective is to minimize the overall transportation cost, including terminal location costs, fixed costs for ordinary and platoon trucks, fuel, and driver costs. A total of 27 randomly generated instances are used to test the optimization model. We considered several scenarios to analyze different combinations of using or not backhauling, having or not truck platoons in the network, and allowing or not the truck platoons to visit the forest areas. The results show that the potential savings of combining the two types of trucks are in the range of 1%–12% in the scenarios in which truck platooning transportation is allowed only between terminal and mill nodes. This savings could reach more than 20% when the truck platoons are allowed to visit forest areas, depending on how many forest areas are visited. The number of drivers can be reduced by 3% to more than 30%. In addition, using truck platooning and backhauling together could reduce fuel consumption by 15.6% on average.

Process as a battery: Electricity price aware optimal operation of zeolite crystallization in a continuous oscillatory baffled reactor

The electrification of chemical processes links the production costs and the CO2-footprint of the process industry more and more tightly to the price of electric power and the availability of power from renewable sources. Both for economic and for ecologic reasons, it is desirable to adapt the consumption to the availability of power from renewables. This work considers the optimal dynamic operation of a zeolite crystallization plant in a continuous oscillatory baffled reactor. The demand-side management problem is first investigated using a steady-state model. For the dynamic optimization problem, a moving horizon approach is introduced, and the effect of the terminal sell-off on the dynamic optimization is minimized by the usage of a terminal cost and additional horizons. The performance of the scheme is tested using different scenarios for the electric energy prices. Additionally, the influence of the uncertainty of the predictions of the price of electric power is addressed.