Active Set Changes Research Articles

Primal–dual differential dynamic programming: A model-based reinforcement learning for constrained dynamic optimization

The main objective of this study is to develop primal–dual differential dynamic programming (DDP), a model-based reinforcement learning (RL) framework that can handle constrained dynamic optimization problems. DDP has advantages of being able to provide a closed-loop policy and having computational complexity that grows linearly with respect to the time horizon. To take advantage, the DDP should consider optimality and feasibility for the disturbed state during closed-loop operations. Previous DDPs consider the feasibility only for the nominal state condition and can handle limited types of constraints. In this paper, we propose a primal–dual DDP incorporating modified augmented Lagrangian that can handle general nonlinear constraints. We pay special attention to obtain the feasible policy when active set changes due to the state perturbations, using path-following predictor–corrector approach. The developed framework method was applied to van der Pol oscillator and batch crystallization process, thereby validating the key aspects of this study.

An intelligent approach of controlled variable selection for constrained process self-optimizing control

ABSTRACT Self-optimizing control (SOC) is a technique for selecting appropriate controlled variables (CVs) and maintaining them constant such that the plant runs at its best. Some tough challenges in this subject, such as how to select CVs when the active constraint set changes remains unsolved since the notion of SOC was presented. Previous work had some drawbacks such as structural complexity and control inaccuracy when dealing with constrained SOC problems due to the elaborate control structures or the limitation of local SOC. In order to overcome the deficiency of previous methods, this paper developed a constrained global SOC (cgSOC) approach to implement self-optimizing controlled variable selection and control structure design. The constrained variables that may change between inactive and active are represented as a nonlinear function of available measurement variables under optimal operations. The unknown function is then intelligently learnt over the whole operating region through neural network training. The difference between the nonlinear function and the actual constrained variables measured in real-time is then used as CVs. When the CVs are controlled at zero in real-time, near-optimal operation can be ensured globally whenever active constraint changes. The efficacy of the proposed approach is demonstrated through an evaporator case study.

Optimization-Constrained Differential Equations with Active Set Changes

Foundational theory is established for nonlinear differential equations with embedded nonlinear optimization problems exhibiting active set changes. Existence, uniqueness, and continuation of solutions are shown, followed by lexicographically smooth (implying Lipschitzian) parametric dependence. The sensitivity theory found here accurately characterizes sensitivity jumps resulting from active set changes via an auxiliary nonsmooth sensitivity system obtained by lexicographic directional differentiation. The results in this article hold under easily verifiable regularity conditions (linear independence of constraints and strong second-order sufficiency), which are shown to imply generalized differentiation index one of a nonsmooth differential-algebraic equation system obtained by replacing the optimization problem with its optimality conditions and recasting the complementarity conditions as nonsmooth algebraic equations. The theory in this article is computationally relevant, allowing for implementation of dynamic optimization strategies (i.e., open-loop optimal control), and recovers (and rigorously formalizes) classical results in the absence of active set changes. Along the way, contributions are made to the theory of piecewise differentiable functions.

Contraction Properties of the Advanced Step Real-Time Iteration for NMPC

In this paper we study the contraction properties of the recently introduced Advanced Step Real-Time Iteration (AS-RTI) under active-set changes. Compared to the well-known Real-Time Iteration, in order to improve optimality, in the AS-RTI some extra calculations on a problem with a predicted state are carried out. This enables us to trade off controller performance for computational load in a simple manner. Under standard assumptions, we prove the contraction of the iterates and boundedness of the numerical error. To achieve these goals we use generalized equations, Robinson’s strong regularity and recently presented results for abstract real-time algorithms. Finally, we present a numerical benchmark, where the performance of different variants of the AS-RTI is demonstrated on an industrial case study.

Online Process Optimization with Active Constraint Set Changes using Simple Control Structures

The purpose of this paper is to show that online process optimization can be achieved using simple control structures without the need for solving numerical optimization problems online. In particular, we show that changes in active constraint regions can be handled using simple logics such as selectors, without needing to identify the exact location of the active constraint regions a priori, nor using a detailed model online. Good case studies are used to clearly demonstrate how optimal operation can be achieved with changes in active constraint regions using simple feedback control structures.

Real-Time Optimization of a Reactor–Separator–Recycle Process II: Dynamic Evaluation

The dynamic performance of a fully automatic real-time optimization (RTO) scheme is evaluated for a reactor–separator–recycle process in mode I (minimize column boilup at given throughput) and mode II (maximize production) operation. In both modes, the reactor composition set point is the optimized unconstrained degree of freedom. The RTO requires a simple override control scheme to reconfigure the inventory loops as the active constraint set changes between mode I and mode II. The overrides reorient inventory loops from in the direction of process flow (CS I) in mode I to opposite the process flow (CS II) upstream of the bottleneck constraint (column maximum boilup), limiting mode II production. The set of selected signals in the override scheme allows automatic distinction between mode I and mode II optimization. Closed loop results are reported for realistic sustained disturbance scenarios that include sudden fast changes as well as gradual equipment degradation. The RTO scheme significantly improves e...

Generalized sensitivity analysis of nonlinear programs using a sequence of quadratic programs

ABSTRACTLocal sensitivity information is obtained for KKT points of parametric NLPs that may exhibit active set changes under parametric perturbations; under appropriate regularity conditions, computationally relevant generalized derivatives of primal and dual variable solutions of parametric NLPs are calculated. Ralph and Dempe obtained directional derivatives of solutions of parametric NLPs exhibiting active set changes from the unique solution of an auxiliary quadratic program. This article uses lexicographic directional derivatives, a newly developed tool in nonsmooth analysis, to generalize the classical NLP sensitivity analysis theory of Ralph and Dempe. By viewing said auxiliary quadratic program as a parametric NLP, the results of Ralph and Dempe are applied to furnish a sequence of coupled QPs, whose unique solutions yield generalized derivative information for the NLP. A practically implementable algorithm is provided. The theory developed here is motivated by widespread applications of nonlinear programming sensitivity analysis, such as in dynamic control and optimization problems.

Integrating self-optimizing control and real-time optimization using zone control MPC

The combination of real-time optimization (RTO) and model predictive control (MPC) methodologies is widely used in the chemical and petrochemical industry to optimize continuous processes. However, often the setpoint updates computed by the RTO are not frequent enough to capture all disturbances. This leads to suboptimal process operation, because the system is not re-optimized after a disturbance, until it has reached its new (suboptimal) steady state. An efficient way to handle this issue is to include economic considerations in the design of the lower MPC layer. This is the main idea of self-optimizing control (SOC), where controlled variables are selected, such that keeping them constant results in near-optimal operation without requiring an RTO update. Thus, we argue that SOC is complementary to RTO, and we develop a new MPC strategy with zone control and SOC variable targets. In particular, we present an approach toward solving the problem of low-frequency setpoint updates, also when the active set changes. We demonstrate the performance of our approach in steady state and dynamic simulations, and compare it to a classic RTO/MPC combination. The results show that our approach improves the coordination between the RTO and MPC layers. As it gives better performance in between RTO executions, it also leads to a higher overall profit.

Advances in sensitivity-based nonlinear model predictive control and dynamic real-time optimization

Recent results in the development of efficient large-scale nonlinear programming (NLP) algorithms have led to fast, on-line realizations of optimization-based methods for nonlinear model predictive control (NMPC) and dynamic real-time optimization (D-RTO), with predictive nonlinear dynamic (e.g., first principle) models. For NMPC, optimization-based controllers are developed that lead to well-understood stability and robustness properties, even for large, complex plant models. The realization of NMPC requires the application of a fast NLP solver for time-critical, on-line optimization, as well as efficient NLP sensitivity tools that require 2–3 orders of magnitude less computation than the NLP solution. This leads to advanced step NMPC (asNMPC), which essentially eliminates computational delay. We also extend these capabilities to dynamic real-time optimization (D-RTO) with more general stage costs that are economically based. This overview also extends input to state stability (ISS) properties for asNMPC to handle active set changes, and also for D-RTO through convex regularizations. Two large scale distillation case studies, based on nonlinear first principle models, are presented that demonstrate the effectiveness of these approaches.

Fast economic model predictive control based on NLP-sensitivities

This study adapts the advanced step NMPC framework to Economic NMPC. Here, sufficient conditions for nominal stability are derived for NMPC controllers that incorporate economic stage costs with appropriate regularization. To guarantee these conditions, we derive a constructive strategy to calculate the regularization term directly. Moreover, we extend the sensitivity components in the advanced step NMPC framework to consider a rigorous path-following algorithm. This approach accounts for active set changes and allows much weaker constraint qualifications. Moreover, using an ℓ1 formulation of the NMPC problem satisfies these constraint qualifications and allows more reliable solution of the moving horizon optimization problem, even in the presence of noise. Finally, all of these concepts are demonstrated on a detailed case study with a continuously stirred tank reactor.

Measurement-based optimization of batch and repetitive processes using an integrated two-layer architecture

This paper is concerned with optimal control of batch and repetitive processes in the presence of uncertainty. An integrated two-layer optimization strategy is proposed, whereby within-run corrections are performed using a neighboring-extremal update strategy and run-to-run corrections are based on a constraint-adaptation scheme. The latter is appealing since a feasible operating strategy is guaranteed upon convergence, and its combination with neighboring-extremal updates improves the reactivity and convergence speed. Moreover, these two layers are consistent in that they share the same objective function. The proposed optimization scheme is declined into two versions, namely an indirect version based on the Pontryagin maximum principle and a direct version that applies a control parameterization and nonlinear programming techniques. Although less rigorous, the latter approach can deal with singular extremals and path constraints as well as handle active-set changes more conveniently. Two case studies are considered. The indirect approach is demonstrated for a level-control problem in an experimental two-tank system, whereas the direct approach is illustrated in numerical simulation on a fed-batch reactor for acetoacetylation of pyrrole. The results confirm that faster adaptation is possible with the proposed integrated two-layer scheme compared to either constraint adaptation or neighboring-extremal update alone.

Self-optimizing control with active set changes

In this paper we extend the “nullspace method” by [1] to cover changes in active set. The extension is based on recent results from explicit model predictive control by [3]. The nullspace method is a method for selecting controlled variables, assuming that the set of active constraints does not change. With the extension presented here, we show that by applying the nullspace method for several different regions, where the regions are found by a parametric program, we can use the value of the controlled variables for each region to decide when to switch regions.The proposed method is demonstrated on a simple model of an ammonia production plant, and the results are comparable to real-time optimization.

A factorization with update procedures for a KKT matrix arising in direct optimal control

Quadratic programs obtained for optimal control problems of dynamic or discrete-time processes usually involve highly block structured Hessian and constraints matrices, to be exploited by efficient numerical methods. In interior point methods, this is elegantly achieved by the widespread availability of advanced sparse symmetric indefinite factorization codes. For active set methods, however, conventional dense matrix techniques suffer from the need to update base matrices in every active set iteration, thereby loosing the sparsity structure after a few updates. This contribution presents a new factorization of a KKT matrix arising in active set methods for optimal control. It fully respects the block structure without any fill-in. For this factorization, matrix updates are derived for all cases of active set changes. This allows for the design of a highly efficient block structured active set method for optimal control and model predictive control problems with long horizons or many control parameters.

L2CXCV: A Fortran 77 package for least squares convex/concave data smoothing

Fortran 77 software is given for least squares smoothing to data values contaminated by random errors subject to one sign change in the second divided differences of the smoothed values, where the location of the sign change is also unknown of the optimization problem. A highly useful description of the constraints is that they follow from the assumption of initially increasing and subsequently decreasing rates of change, or vice versa, of the process considered. The underlying algorithm partitions the data into two disjoint sets of adjacent data and calculates the required fit by solving a strictly convex quadratic programming problem for each set. The piecewise linear interpolant to the fit is convex on the first set and concave on the other one. The partition into suitable sets is achieved by a finite iterative algorithm, which is made quite efficient because of the interactions of the quadratic programming problems on consecutive data. The algorithm obtains the solution by employing no more quadratic programming calculations over subranges of data than twice the number of the divided differences constraints. The quadratic programming technique makes use of active sets and takes advantage of a B-spline representation of the smoothed values that allows some efficient updating procedures. The entire code required to implement the method is 2920 Fortran lines. The package has been tested on a variety of data sets and it has performed very efficiently, terminating in an overall number of active set changes over subranges of data that is only proportional to the number of data. The results suggest that the package can be used for very large numbers of data values. Some examples with output are provided to help new users and exhibit certain features of the software. Important applications of the smoothing technique may be found in calculating a sigmoid approximation, which is a common topic in various contexts in applications in disciplines like physics, economics, biology and engineering. Distribution material that includes single and double precision versions of the code, driver programs, technical details of the implementation of the software package and test examples that demonstrate the use of the software is available in an accompanying ASCII file.

Adaptation of CDMA soft handoff thresholds using fuzzy inference system

This paper proposes a new procedure to adjust soft handoff thresholds (T-DROP, T-ADD) by using a fuzzy inference system (FIS). The aims are to increase the value of T-DROP in order to release the traffic channel (TCH) at high traffic loads for increasing the carried traffic, and to decrease the value of T-DROP in order to add the TCH in active set at low traffic loads for increasing the quality of TCH. The system performance indicators consist of the quality of TCH (E/sub b//N/sub 0/), outage probability, new call blocking probability (P/sub B/), handoff call blocking probability (P/sub HO/), expected number of base stations (BS) in active set (NO/sub BS/), expected number of BS changes in active set (NO/sub update/), trunking resource efficiency (TRE) and carried traffic (T/sub C/). By comparison of all parameters among three algorithms, soft handoff (SHO) using the FIS tends to have higher performance (T/sub C/, NO/sub BS/, TRE, P/sub B/, P/sub HO/, NO/sub update/) than those of IS-95A and IS-95B/cdma2000 SHO at high traffic loads or lower thresholds while the quality of TCH (E/sub b//N/sub 0/, outage probability) is still acceptable.

CXFTV2: A Fortran subroutine for the discrete least squares convex approximation

A Fortan subroutine calculates the least squares approximation to n data values containing random errors subject to non-negative second divided differences (convexity). The method employs a dual active set quadratic programming technique that allows several concavities of an iterate to be corrected simultaneously, which is a distinctive feature of this calculation. A B-spline representation of the iterates reduces each active set calculation to an unconstrained minimization with fewer variables that requires only O( n) computer operations. Details in these techniques including the data structure that establishes the implementation of the method are specified. Numerical testing on a variety of data sets indicates that the subroutine is particularly efficient, terminating after a small number of active set changes, the subroutine being suitable for large numbers of data. A numerical example and its output is provided to help the use of the software.

Sensitivity analysis for chemical process optimization

A sensitivity analysis methodology based on continuation techniques is applied to chemical engineering processes. The method is based on the trace of the optimal solution path of a process model under multiple model parameter variations of finite magnitude. The first-order stationarity conditions form a system of parameterized nonlinear equations. The solution of the equation set for different values of the parameters is obtained using continuation methods (PITCON). The behavior of the optimal solution path is studied around singular points. Simple tests are used to detect singularities due to violation of the strict complementarity, linear independence constraint qualification or second-order optimality conditions. The example cases involve single and multiple unit models with equation sets of moderate to large size. The method can handle active constraint set changes and can determine the range of model parameter variations in a given direction for which the model is feasible. It can be used to determine the most sensitive unit in a plant for a given disturbance and calculate multiple solutions to the stationarity conditions for the same parameter values.

On iterative algorithms for linear least squares problems with bound constraints

Three new iterative methods for the solution of the linear least squares problem with bound constraints are presented and their performance analyzed. The first is a modification of a method proposed by Lötstedt, while the two others are characterized by a technique allowing for fast active set changes, resulting in noticeable improvements in the speed with which constraints active at the solution are identified. The numerical efficiency of those algorithms is experimentally studied, with particular emphasis on the dependence on the starting point and the use of preconditioning for ill-conditioned problems.

The efficient set mathematics when mean-variance problems are subject to general linear constraints

In this paper we develop the efficient set mathematics for the case where mean-variance portfolio problems are subject to general linear constraints. We derive closed-form expressions for the optimal portfolio, the associated multipliers, and the portfolio's expected return and variance in each interval where the active set changes; identify where kinks on the efficient frontier occur; and show that securities plot on a security market hyperplane. The analysis extends our knowledge of portfolio theory, clarifies the zero-beta Capital Asset Pricing Model and the conditions under which securities plot on the Security Market Line (SML), and provides insight into the ambiguities associated with using the SML criterion to measure investment performance.