Nonsmooth Optimization Algorithm Research Articles

Robust adaptive fault-tolerant control of a tandem coaxial ducted fan aircraft with actuator saturation

This paper is concerned with the robust adaptive fault-tolerant control of a tandem coaxial ducted fan aircraft under system uncertainty, mismatched disturbance, and actuator saturation. For the proposed aircraft, comprehensive controllability analysis is performed to evaluate the controllability of each state as well as the margin to reject mismatched disturbance without any knowledge of the controller. Mismatched disturbance attenuation is ensured through a structured H-infinity controller tuned by a non-smooth optimization algorithm. Embedded with the H-infinity controller, an adaptive control law is proposed in order to mitigate matched system uncertainty and actuator fault. Input saturation is also considered by the modified reference model. Numerical simulation of the novel ducted fan aircraft is provided to illustrate the effectiveness of the proposed method. The simulation results reveal that the proposed adaptive controller achieves better transient response and more robust performance than classic Model Reference Adaptive Control (MRAC) method, even with serious actuator saturation.

A new proximal chebychev center cutting plane algorithm for nonsmooth optimization and its convergence

Motivated by the proximal-like bundle method [K. C. Kiwiel, Journal of Optimization Theory and Applications, 104(3) (2000), 589-603.], we establish a new proximal Chebychev center cutting plane algorithm for a type of nonsmooth optimization problems. At each step of the algorithm, a new optimality measure is investigated instead of the classical optimality measure. The convergence analysis shows that an \begin{document}$\varepsilon$\end{document} -optimal solution can be obtained within \begin{document}$O(1/\varepsilon^3)$\end{document} iterations. The numerical result is presented to show the validity of the conclusion and it shows that the method is competitive to the classical proximal-like bundle method.

Control Synthesis and Classification for Unicycle Dynamics using the Gradient and Value Sampling Particle Filters

Value functions arising from dynamic programming can be used to synthesize optimal control inputs for general nonlinear systems with state and/or input constraints; however, the inputs generated by steepest descent on these value functions often lead to chattering behavior. In [Traft & Mitchell, 2016] we proposed the Gradient Sampling Particle Filter (GSPF), which combines robot state estimation and nonsmooth optimization algorithms to alleviate this problem. In this paper we extend the GSPF to velocity controlled unicycle (or equivalently differential drive) dynamics. We also show how the algorithm can be adapted to classify whether an exogenous input—such as one arising from shared human-in-the-loop control—is desirable. The two algorithms are demonstrated on a ground robot.

A limited-memory quasi-Newton algorithm for bound-constrained non-smooth optimization

We consider the problem of minimizing a continuous function that may be non-smooth and non-convex, subject to bound constraints. We propose an algorithm that uses the L-BFGS quasi-Newton approximation of the problem's curvature together with a variant of the weak Wolfe line search. The key ingredient of the method is an active-set selection strategy that defines the subspace in which search directions are computed. To overcome the inherent shortsightedness of the gradient for a non-smooth function, we propose two strategies. The first relies on an approximation of the ε-minimum norm subgradient, and the second uses an iterative corrective loop that augments the active set based on the resulting search directions. While theoretical convergence guarantees have been elusive even for the unconstrained case, we present numerical results on a set of standard test problems to illustrate the efficacy of our approach, using an open-source Python implementation of the proposed algorithm.

System identification and robust stabilization using structured controller for a novel ducted fan flying robot

This paper proposes complete procedures for system identification and control of a novel ducted fan flying robot on near hover flight condition. A control-oriented model structure is derived from a comprehensive mechanism model and identified by ground tests and flight experiments. In order to achieve state decoupling and reference tracking, a two-loop control architecture is employed. Non-smooth optimization algorithm is applied to efficiently tune controller parameters against multiple control requirements. Experimental results show that designed structured controller can provide better performance than classical proportional -integral -derivative (PID) controller.

Topology optimization of freely vibrating continuum structures based on nonsmooth optimization

The non-differentiability of repeated eigenvalues is one of the key difficulties to obtain the optimal solution in the topology optimization of freely vibrating continuum structures. In this paper, the bundle method, which is a very promising one in the nonsmooth optimization algorithm family, is proposed and implemented to solve the problem of eigenfrequency optimization of continuum. The bundle method is well-known in the mathematical programming community, but has never been used to solve the problems of topology optimization of continuum structures with respect to simple or multiple eigenfrequencies. The advantage of this method is that the specified information of iteration history may be collected and utilized in a very efficient manner to ensure that the next stability center is closer to the optimal solution, so as to avoid the numerical oscillation in the iteration history. Moreover, in the present method, both the simple and multiple eigenfrequencies can be managed within a unified computational scheme. Several numerical examples are tested to validate the proposed method. Comparisons with nonlinear semidefinite programming method and 0–1 formulation based heuristic method show the advantages of the proposed method. It is showed that, the method can deal with the nonsmoothness of the repeated eigenvalues in topology optimization in a very effective and efficient manner without evaluating the multiplicity of the eigenvalues.

A Riemannian Gradient Sampling Algorithm for Nonsmooth Optimization on Manifolds

In this paper, an optimization method for nonsmooth locally Lipschitz functions on complete Riemannian manifolds is presented. The method is based on approximating the subdifferential of the cost function at every iteration by the convex hull of transported gradients from tangent spaces at randomly generated nearby points to the tangent space of the current iterate and can hence be seen as a generalization of the well known gradient sampling algorithm to a Riemannian setting. A convergence result is obtained under the assumption that the cost function is bounded below and continuously differentiable on an open set of full measure and that the employed vector transport and retraction satisfy certain conditions, which hold, for instance, for the exponential map and parallel transport. Then with probability one the algorithm produces iterates at which the cost function is differentiable, and each cluster point of the iterates is a Clarke stationary point. Modifications yielding only $\varepsilon$-stationary ...

Generalized Derivatives of Differential–Algebraic Equations

Nonsmooth equation-solving and optimization algorithms which require local sensitivity information are extended to systems with nonsmooth parametric differential---algebraic equations embedded. Nonsmooth differential---algebraic equations refers here to semi-explicit differential---algebraic equations with algebraic equations satisfying local Lipschitz continuity and differential right-hand side functions satisfying Caratheodory-like conditions. Using lexicographic differentiation, an auxiliary nonsmooth differential---algebraic equation system is obtained whose unique solution furnishes the desired parametric sensitivities. More specifically, lexicographic derivatives of solutions of nonsmooth parametric differential---algebraic equations are obtained. Lexicographic derivatives have been shown to be elements of the plenary hull of the Clarke (generalized) Jacobian and thus computationally relevant in the aforementioned algorithms. To accomplish this goal, the lexicographic smoothness of an extended implicit function is proved. Moreover, these generalized derivative elements can be calculated in tractable ways thanks to recent advancements in nonsmooth analysis. Forward sensitivity functions for nonsmooth parametric differential---algebraic equations are therefore characterized, extending the classical sensitivity results for smooth parametric differential---algebraic equations.

Robust Flight Control Using Nonsmooth Optimization for a Tandem Ducted Fan Vehicle

AbstractThis work addresses the aerodynamic modeling and near‐hover‐flight control design for an unconventional aerial robot of the tandem ducted fan configuration, which is intended to be prototypical of a flight service vehicle. The main model elements of this novel unmanned vehicle, which exhibit highly nonlinear and unstable open‐loop modes, are presented. A frequency‐domain controllability analysis concerning the plant's behavior around the hovering flight condition is then adopted to determine the expected control performance, which is of important practical significance to controllability improvement through vehicle design changes. A robust controller that stabilizes the unmanned vehicle under wind disturbances is designed using a newly developed nonsmooth optimization algorithm, which rigorously and efficiently tunes the arbitrarily predefined structured controller against multiple control requirements. A successive two‐loop architecture is employed in the designed controller. In this architecture, the inner loop provides stability augmentation and decoupling, and the outer loop guarantees the desired velocity tracking performance. Simulation results under stochastic wind gusts are presented to verify the performance of the proposed controllers. Preliminary flight tests are also carried out to demonstrate the performance of the system.

Proximal point algorithms for nonsmooth convex optimization with fixed point constraints

The problem of minimizing the sum of nonsmooth, convex objective functions defined on a real Hilbert space over the intersection of fixed point sets of nonexpansive mappings, onto which the projections cannot be efficiently computed, is considered. The use of proximal point algorithms that use the proximity operators of the objective functions and incremental optimization techniques is proposed for solving the problem. With the focus on fixed point approximation techniques, two algorithms are devised for solving the problem. One blends an incremental subgradient method, which is a useful algorithm for nonsmooth convex optimization, with a Halpern-type fixed point iteration algorithm. The other is based on an incremental subgradient method and the Krasnosel’skiĭ–Mann fixed point algorithm. It is shown that any weak sequential cluster point of the sequence generated by the Halpern-type algorithm belongs to the solution set of the problem and that there exists a weak sequential cluster point of the sequence generated by the Krasnosel’skiĭ–Mann-type algorithm, which also belongs to the solution set. Numerical comparisons of the two proposed algorithms with existing subgradient methods for concrete nonsmooth convex optimization show that the proposed algorithms achieve faster convergence.

Robust Power-Splitting SWIPT Beamforming for Broadcast Channels

This letter considers the multiple-input single-output (MISO) broadcast system for simultaneous wireless information and power transfer (SWIPT) using receiver power splitting and aims to optimize jointly the beamforming vectors and the power splitting ratios for minimizing the transmit power of the base station (BS) subject to the individual signal-to-interference-plus-noise ratio (SINR) and the energy-harvesting constraints at the mobile stations (MSs). However, the CSI is assumed imperfect but has a deterministic uncertainty region. Unlike existing attempts that resort to iterations guided by semidefinite relaxation (SDR), we propose a reverse convex nonsmooth optimization algorithm, which provides the near-optimal rank-one solution.

Numerical methods for matching for teams and Wasserstein barycenters

Equilibrium multi-population matching (matching for teams) is a problem from mathematical economics which is related to multi-marginal optimal transport. A special but important case is the Wasserstein barycenter problem, which has applications in image processing and statistics. Two algorithms are presented: a linear programming algorithm and an efficient nonsmooth optimization algorithm, which applies in the case of the Wasserstein barycenters. The measures are approximated by discrete measures: convergence of the approximation is proved. Numerical results are presented which illustrate the efficiency of the algorithms.

Covariance matrix adaptation evolution strategy based design of fixed structure robust H∞ loop shaping controller

This paper proposes the application of Covariance Matrix Adaptation Evolution Strategy (CMA-ES) in fixed structure H∞ loop shaping controller design. Integral Time Absolute Error (ITAE) performance requirement is incorporated as a constraint with an objective of maximization of stability margin in the fixed structure H∞ loop shaping controller design problem. Pneumatic servo system, separating tower process and F18 fighter aircraft system are considered as test systems. The CMA-ES designed fixed structure H∞ loop-shaping controller is compared with the traditional H∞ loop shaping controller, non-smooth optimization and Heuristic Kalman Algorithm (HKA) based fixed structure H∞ loop shaping controllers in terms of stability margin. 20% perturbation in the nominal plant is used to validate the robustness of the CMA-ES designed H∞ loop shaping controller. The effect of Finite Word Length (FWL) is considered to show the implementation difficulties of controller in digital processors. Simulation results demonstrated that CMA-ES based fixed structure H∞ loop shaping controller is suitable for real time implementation with good robust stability and performance.

Parallel computing subgradient method for nonsmooth convex optimization over the intersection of fixed point sets of nonexpansive mappings

Nonsmooth convex optimization problems are solved over fixed point sets of nonexpansive mappings by using a distributed optimization technique. This is done for a networked system with an operator, who manages the system, and a finite number of users, by solving the problem of minimizing the sum of the operator’s and users’ nondifferentiable, convex objective functions over the intersection of the operator’s and users’ convex constraint sets in a real Hilbert space. We assume that each of their constraint sets can be expressed as the fixed point set of an implementable nonexpansive mapping. This setting allows us to discuss nonsmooth convex optimization problems in which the metric projection onto the constraint set cannot be calculated explicitly. We propose a parallel subgradient algorithm for solving the problem by using the operator’s attribution such that it can communicate with all users. The proposed algorithm does not use any proximity operators, in contrast to conventional parallel algorithms for nonsmooth convex optimization. We first study its convergence property for a constant step-size rule. The analysis indicates that the proposed algorithm with a small constant step size approximates a solution to the problem. We next consider the case of a diminishing step-size sequence and prove that there exists a subsequence of the sequence generated by the algorithm which weakly converges to a solution to the problem. We also give numerical examples to support the convergence analyses.

A derivative-free trust-region algorithm for composite nonsmooth optimization

The derivative-free trust-region algorithm proposed by Conn et al. (SIAM J Optim 20:387–415, 2009) is adapted to the problem of minimizing a composite function $$\varPhi (x)=f(x)+h(c(x))$$ , where $$f$$ and $$c$$ are smooth, and $$h$$ is convex but may be nonsmooth. Under certain conditions, global convergence and a function-evaluation complexity bound are proved. The complexity result is specialized to the case when the derivative-free algorithm is applied to solve equality-constrained problems. Preliminary numerical results with minimax problems are also reported.

On the convergence and worst-case complexity of trust-region and regularization methods for unconstrained optimization

A nonlinear stepsize control framework for unconstrained optimization was recently proposed by Toint (Optim Methods Softw 28:82---95, 2013), providing a unified setting in which the global convergence can be proved for trust-region algorithms and regularization schemes. The original analysis assumes that the Hessians of the models are uniformly bounded. In this paper, the global convergence of the nonlinear stepsize control algorithm is proved under the assumption that the norm of the Hessians can grow by a constant amount at each iteration. The worst-case complexity is also investigated. The results obtained for unconstrained smooth optimization are extended to some algorithms for composite nonsmooth optimization and unconstrained multiobjective optimization as well.

Numerical Analysis of $\mathcal{V}\mathcal{U}$-Decomposition, $\mathcal{U}$-Gradient, and $\mathcal{U}$-Hessian Approximations

Advances in bundle methods for nonsmooth optimization have lead to the development of $\mathcal{V}\mathcal{U}$-decompositions, the $\mathcal{U}$-gradient, and the $\mathcal{U}$-Hessian. These variational analysis constructs have proven extremely valuable and lead to the development of the superlinearly convergent $\mathcal{V}\mathcal{U}$-algorithm for nonsmooth optimization. In this paper we examine these constructs from the viewpoint of derivative-free optimization. We show that, given a finite max function $f(x) = \max_{i=0, 1, \ldots m} f_i(x)$ and a black-box which returns function values for each $f_i$, it is possible to construct approximations of the $\mathcal{V}\mathcal{U}$-decompositions, $\mathcal{U}$-gradient and $\mathcal{U}$-Hessian. The approximations do not require excessive black-box calls, and the accuracy of the approximations is directly related to the accuracy of the approximate gradient and Hessians for each $f_i$.

Prices stabilization for inexact unit-commitment problems

A widespread and successful approach to tackle unit-commitment problems is constraint decomposition: by dualizing the linking constraints, the large-scale nonconvex problem decomposes into smaller independent subproblems. The dual problem consists then in finding the best Lagrangian multiplier (the optimal “price”); it is solved by a convex nonsmooth optimization method. Realistic modeling of technical production constraints makes the subproblems themselves difficult to solve exactly. Nonsmooth optimization algorithms can cope with inexact solutions of the subproblems. In this case however, we observe that the computed dual solutions show a noisy and unstable behaviour, that could prevent their use as price indicators. In this paper, we present a simple and easy-to-implement way to stabilize dual optimal solutions, by penalizing the noisy behaviour of the prices in the dual objective. After studying the impact of a general stabilization term on the model and the resolution scheme, we focus on the penalization by discrete total variation, showing the consistency of the approach. We illustrate our stabilization on a synthetic example, and real-life problems from EDF (the French Electricity Board).

Spherical Mesh Adaptive Direct Search for Separating Quasi-Uncorrelated Sources by Range-Based Independent Component Analysis

It is seemingly paradoxical to the classical definition of the independent component analysis (ICA), that in reality, the true sources are often not strictly uncorrelated. With this in mind, this letter concerns a framework to extract quasi-uncorrelated sources with finite supports by optimizing a range-based contrast function under unit-norm constraints (to handle the inherent scaling indeterminacy of ICA) but without orthogonality constraints. Albeit the appealing contrast properties of the range-based function (e.g., the absence of mixing local optima), the function is not differentiable everywhere. Unfortunately, there is a dearth of literature on derivative-free optimizers that effectively handle such a nonsmooth yet promising contrast function. This is the compelling reason for the design of a nonsmooth optimization algorithm on a manifold of matrices having unit-norm columns with the following objectives: to ascertain convergence to a Clarke stationary point of the contrast function and adhere to the necessary unit-norm constraints more naturally. The proposed nonsmooth optimization algorithm crucially relies on the design and analysis of an extension of the mesh adaptive direct search (MADS) method to handle locally Lipschitz objective functions defined on the sphere. The applicability of the algorithm in the ICA domain is demonstrated with simulations involving natural, face, aerial, and texture images.

CARTopt: a random search method for nonsmooth unconstrained optimization

A random search algorithm for unconstrained local nonsmooth optimization is described. The algorithm forms a partition on $\mathbb{R}^{n}$ using classification and regression trees (CART) from statistical pattern recognition. The CART partition defines desirable subsets where the objective function f is relatively low, based on previous sampling, from which further samples are drawn directly. Alternating between partition and sampling phases provides an effective method for nonsmooth optimization. The sequence of iterates {z k } is shown to converge to an essential local minimizer of f with probability one under mild conditions. Numerical results are presented to show that the method is effective and competitive in practice.