Nonsmooth Optimization Algorithm Research Articles

Linear quantile mixed models

Dependent data arise in many studies. Frequently adopted sampling designs, such as cluster, multilevel, spatial, and repeated measures, may induce this dependence, which the analysis of the data needs to take into due account. In a previous publication (Geraci and Bottai in Biostatistics 8:140–154, 2007), we proposed a conditional quantile regression model for continuous responses where subject-specific random intercepts were included to account for within-subject dependence in the context of longitudinal data analysis. The approach hinged upon the link existing between the minimization of weighted absolute deviations, typically used in quantile regression, and the maximization of a Laplace likelihood. Here, we consider an extension of those models to more complex dependence structures in the data, which are modeled by including multiple random effects in the linear conditional quantile functions. We also discuss estimation strategies to reduce the computational burden and inefficiency associated with the Monte Carlo EM algorithm we have proposed previously. In particular, the estimation of the fixed regression coefficients and of the random effects’ covariance matrix is based on a combination of Gaussian quadrature approximations and non-smooth optimization algorithms. Finally, a simulation study and a number of applications of our models are presented.

Nonsmooth Optimization for Efficient Beamforming in Cognitive Radio Multicast Transmission

It is known that the design of optimal transmit beamforming vectors for cognitive radio multicast transmission can be formulated as indefinite quadratic optimization programs. Given the challenges of such nonconvex problems, the conventional approach in literature is to recast them as convex semidefinite programs (SDPs) together with rank-one constraints. Then, these nonconvex and discontinuous constraints are dropped allowing for the realization of a pool of relaxed candidate solutions, from which various randomization techniques are utilized with the hope to recover the optimal solutions. However, it has been shown that such approach fails to deliver satisfactory outcomes in many practical settings, wherein the determined solutions are found to be unacceptably far from the actual optimality. On the contrary, we in this contribution tackle the aforementioned optimal beamforming problems differently by representing them as SDPs with additional reverse convex (but continuous) constraints. Nonsmooth optimization algorithms are then proposed to locate the optimal solutions of such design problems in an efficient manner. Our thorough numerical examples verify that the proposed algorithms offer almost global optimality whilst requiring relatively low computational load.

An Effective Nonsmooth Optimization Algorithm for Locally Lipschitz Functions

To construct an effective minimization algorithm for locally Lipschitz functions, we show how to compute a descent direction satisfying Armijo’s condition. We present a finitely terminating algorithm to construct an approximating set for the Goldstein subdifferential leading to the desired descent direction. Using this direction, we propose a minimization algorithm for locally Lipschitz functions and prove its convergence. Finally, we implement our algorithm with matrix laboratory (MATLAB) codes and report our testing results. The comparative numerical results attest to the efficiency of the proposed algorithm.

Beamforming Optimization in Multi-User Amplify-and-Forward Wireless Relay Networks

Optimization problems of beamforming in multi-user amplify-and-forward (AF) wireless relay networks are indefinite (nonconvex) quadratic programs, which require effective computational solutions. Solutions to these problems have often been obtained by relaxing the original problems to semi-definite programs (SDPs) of convex optimization. Most existing works have claimed that these relaxed SDPs actually provide the optimal beamforming solutions. This paper, however, shows that this is not the case in many practical scenarios where SDPs fail to provide even a feasible beamforming solution. To fill this gap, we develop in this paper a nonsmooth optimization algorithm, which provides the optimal solution at low computational complexity.

Numerical and experimental investigation of the behavior of extended end-plate connections in steel structures

It is well known that one of the most popular methods of connecting members in structural steel work is the bolted end-plate connection. Bolted end-plates are simple in their use and construction. But they are extremely complex in term of analysis and behavior since the connection behavior significantly affects the structural frame response and therefore it has to be included to the global analysis and the design of frame. The present paper deals with the structural behavior of full-scale stiffened and un-stiffened cantilever connections of typical I sections. The connection between the extended end-plate to the column flange is achieved by means of high strength bolts in each case. In order to obtain experimentally the actual tension force induced within each bolt, strain gauges were installed inside each one of the top bolts. Thus, the connection behavior is characterized by the tension force in the bolt, the extended end-plate behavior, the moment-rotation relation and the beam and column strains. Thereby, it is important to predict the global behavior of column-beam connections by means of their geometrical and mechanical properties. The experimental test results are compared to those obtained by means of a numerical approach based on the finite element method and is coupled to the theory of non-smooth mechanics. All the arising non-linearities in the connection are described through a non-monotone multi-valued reaction-displacement law. Thus, the problem is formulated as a hemivariational inequality leading to a sub-stationarity problem of the potential or the complementary energy of the connection. This simulation problem is solved by applying a non-convex non-smooth optimization algorithm. The comparison of the results of the experimental testing program with the numerical simulation proves the effectiveness of the proposed numerical method.

On Nesterov’s nonsmooth Chebyshev–Rosenbrock functions

We discuss two nonsmooth functions on R n introduced by Nesterov. We show that the first variant is partly smooth in the sense of Lewis and that its only stationary point is the global minimizer. In contrast, we show that the second variant has 2 n − 1 Clarke stationary points, none of them local minimizers except the global minimizer, but also that its only Mordukhovich stationary point is the global minimizer. Nonsmooth optimization algorithms from multiple starting points generate iterates that approximate all 2 n − 1 Clarke stationary points, not only the global minimizer, but it remains an open question as to whether the nonminimizing Clarke stationary points are actually points of attraction for optimization algorithms.

Structured MIMO ∞Design for Dual-Stage Inertial Stabilization: Case Study for HIFOO

AbstractThe paper reports on a control design for an experimental platform that contains two aligned motorized gimbals stabilizing an optical payload around a single axis using measurements from an inertial angular rate sensor (MEMS gyro). Such a laboratory system represents a simplified testbed for the more practically useful line-of-sight stabilization. The paper defends the choice of the control configuration and argues that when using MIMO design procedures, enforcing some structure upon the transfer function matrix makes the controller implementation and fine-tuning easier. Such structural constraints were taken into consideration in the computational ∞design by using the publicly available HIFOO solver based on algorithms for non-convex and non-smooth optimization. Results are contrasted with the controllers designed with classical methods and full MIMO ∞optimization.

A mesh adaptive direct search algorithm for multiobjective optimization

This work studies multiobjective optimization ( MOP ) of nonsmooth functions subject to general constraints. We first present definitions and optimality conditions as well as some single-objective formulations of MOP, parameterized with respect to some reference point in the space of objective functions. Next, we propose a new algorithm called M ultiM ads (multiobjective mesh adaptive direct search) for MOP. M ultiM ads generates an approximation of the Pareto front by solving a series of single-objective formulations of MOP generated using the NBI (natural boundary intersection) framework. These single-objective problems are solved using the M ads (mesh adaptive direct search) algorithm for constrained nonsmooth optimization. The Pareto front approximation is shown to satisfy some first-order necessary optimality conditions based on the Clarke calculus. M ultiM ads is then tested on problems from the literature with different Pareto front landscapes and on a styrene production process simulation problem from chemical engineering.

Variational formulation and nonsmooth optimization algorithms in elastodynamic contact problems for cracked body

A mathematical model of an elastodynamic contact problem for a body with a crack with unilateral restrictions and friction on the crack faces is presented in classical and weak forms. Different variational formulations of unilateral contact problems with friction based on the principles of Hamilton–Ostrogradskii and Tupin, and boundary variational principles are considered. In particular, boundary variational functionals that are used with boundary integral equations are established. Nonsmooth optimization algorithms of Udzawa type for the solution of these unilateral contact problems with friction are developed. The convergence of the proposed algorithms is studied numerically.

Multiple robust H∞ controller design using the nonsmooth optimization method

AbstractThis paper proposes a systematic technique to design multiple robust H∞ controllers. The proposed technique achieves a desired robust performance objective, which is impossible to achieve with a single robust controller, by dividing the uncertainty set into several subsets and by designing a robust controller to each subset. To achieve this goal with a small number of divisions of the uncertainty set, an optimization problem is formulated. Since the cost function of this optimization problem is not a smooth function, a numerical nonsmooth optimization algorithm is proposed to solve this problem. This method avoids the use of Lyapunov variables, and therefore it leads to a moderate size optimization problem. A numerical example shows that the proposed multiple robust control method can improve the closed‐loop performance when a single robust controller cannot achieve satisfactory performance. Copyright © 2009 John Wiley & Sons, Ltd.

Nonsmooth Optimization Algorithms in Some Problems of Fracture Dynamics

Mathematical statement of elastodynamic contact problem for cracked body with considering unilateral restrictions and friction of the crack faces is done in classical and weak forms. Different variational formulations of unilateral contact problems with friction based on boundary variational principle are considered. Nonsmooth optimization algorithms of Udzawa’s type for solution of unilateral contact problem with friction have been developed. Convergence of the proposed algorithms has been studied numerically.

Variational formulation and Nonsmooth Optimization Algorithms in Elastostatic Contact Problems for Cracked Body

Variational formulation and Nonsmooth Optimization Algorithms in Elastostatic Contact Problems for Cracked Body

Identifying Structure of Nonsmooth Convex Functions by the Bundle Technique

We consider the problem of minimizing nonsmooth convex functions, defined piecewise by a finite number of functions each of which is either convex quadratic or twice continuously differentiable with positive definite Hessian on the set of interest. This is a particular case of functions with primal-dual gradient structure, a notion closely related to the so-called ${\cal VU}$ space decomposition: At a given point, nonsmoothness is locally restricted to the directions of the subspace ${\cal V}$, while along the subspace ${\cal U}$ the behavior of the function is twice differentiable. Constructive identification of the two subspaces is important, because it opens the way to devising fast algorithms for nonsmooth optimization (by following iteratively the manifold of smoothness on which superlinear ${\cal U}$-Newton steps can be computed). In this work we show that, for the class of functions in consideration, the information needed for this identification can be obtained from the output of a standard bundle method for computing proximal points, provided a minimizer satisfies the nondegeneracy and strong transversality conditions.

Multiobjective Optimization Through a Series of Single-Objective Formulations

This work deals with bound constrained multiobjective optimization (MOP) of nonsmooth functions for problems where the structure of the objective functions either cannot be exploited, or are absent. Typical situations arise when the functions are computed as the result of a computer simulation. We first present definitions and optimality conditions as well as two families of single-objective formulations of MOP. Next, we propose a new algorithm called for the biobjective optimization (BOP) problem (i.e., MOP with two objective functions). The property that Pareto points may be ordered in BOP and not in MOP is exploited by our algorithm. generates an approximation of the Pareto front by solving a series of single-objective formulations of BOP. These single-objective problems are solved using the recent (mesh adaptive direct search) algorithm for nonsmooth optimization. The Pareto front approximation is shown to satisfy some first order necessary optimality conditions based on the Clarke calculus. Finally, is tested on problems from the literature designed to illustrate specific difficulties encountered in biobjective optimization, such as a nonconvex or disjoint Pareto front, local Pareto fronts, or a nonuniform Pareto front.

Piecewise Partially Separable Functions and a Derivative-free Algorithm for Large Scale Nonsmooth Optimization

This paper introduces the notion of piecewise partially separable functions and studies their properties. We also consider some of many applications of these functions. Finally, we consider the problem of minimizing of piecewise partially separable functions and develop an algorithm for its solution. This algorithm exploits the structure of such functions. We present the results of preliminary numerical experiments.

A new nonsmooth optimization algorithm for minimum sum-of-squares clustering problems

The minimum sum-of-squares clustering problem is formulated as a problem of nonsmooth, nonconvex optimization, and an algorithm for solving the former problem based on nonsmooth optimization techniques is developed. The issue of applying this algorithm to large data sets is discussed. Results of numerical experiments have been presented which demonstrate the effectiveness of the proposed algorithm.

Constrained non-smooth optimization in dead-beat control of linear servosystems

Motivated by the polynomial approach to the dead-beat ripple-free control of a continuous linear servosystem, the paper presents a new effective algorithm for non-smooth constrained optimization with design objectives applicable to both control and process variables. Constraints on closed-loop poles are introduced and their role in optimization and rejection of a finite-length continuous plant disturbances is examined. The solution is based on a new way of parametrization of the set of all admissible dead-beat ripple-free controllers and the constrained non-smooth optimization profits from its linear programming formulation. Copyright © 2005 John Wiley & Sons, Ltd.

A comparison of deterministic and probabilistic optimization algorithms for nonsmooth simulation-based optimization

In solving optimization problems for building design and control, the cost function is often evaluated using a detailed building simulation program. These programs contain code features that cause the cost function to be discontinuous. Optimization algorithms that require smoothness can fail on such problems. Evaluating the cost function is often so time-consuming that stochastic optimization algorithms are run using only a few simulations, which decreases the probability of getting close to a minimum. To show how applicable direct search, stochastic, and gradient-based optimization algorithms are for solving such optimization problems, we compare the performance of these algorithms in minimizing cost functions with different smoothness. We also explain what causes the large discontinuities in the cost functions.

Analysis of steel bolted connections by means of a nonsmooth optimization procedure

A nonsmooth optimization procedure is herein proposed for the investigation of the bearing resistance of steel bolted connections. Taking into account the presence of nonlinear effects such as e.g. plasticity and interface interaction, and describing the resistance in bearing strength by means of a nonmonotone multi-valued reaction-displacement law obtained by experimental testing, the problem is formulated as a hemivariational inequality one. The latter is equivalent to a substationarity problem of the potential energy of the connection under investigation. This problem can be effectively treated numerically by applying an appropriately chosen nonconvex, nonsmooth optimization algorithm and in particular, the NSOLIB optimization algorithm based on the proximal bundle method has been applied. Two numerical examples whose results are compared to experimental testing results demonstrate the effectiveness of the proposed method.

Mathematical Investigation of Nonsmooth Optimization Algorithm in Elastodynamic Contact Problems with Friction for Bodies with Cracks

Article Mathematical Investigation of Nonsmooth Optimization Algorithm in Elastodynamic Contact Problems with Friction for Bodies with Cracks was published on December 1, 2003 in the journal International Journal of Nonlinear Sciences and Numerical Simulation (volume 4, issue 4).