Nondifferentiable Optimization Problems Research Articles

On embedding the volume algorithm in a variable target value method

We employ the volume algorithm as a subgradient deflection strategy in a variable target value method for solving nondifferentiable optimization problems. Focusing on Lagrangian duals for LPs, we exhibit primal nonconvergence of the original method, establish convergence of the proposed algorithm in the dual space, and present related computational results.

On embedding the volume algorithm in a variable target value method

We employ the volume algorithm as a subgradient deflection strategy in a variable target value method for solving nondifferentiable optimization problems. Focusing on Lagrangian duals for LPs, we exhibit primal nonconvergence of the original method, establish convergence of the proposed algorithm in the dual space, and present related computational results.

Optimal capacitor placement in distribution systems using a combination fuzzy-GA method

This work presents a combination fuzzy-GA method to resolve the capacitor placement problem. The problem formulation considers three distinct objective functions related to minimize the total cost for energy loss and capacitors to be installed, as well as decreasing the deviation of bus voltage and improving the margin loading of feeders. The novel formulation is a multi-objective and non-differentiable optimization problem. These objective functions are first formulated in fuzzy sets to assess their imprecise nature before introducing a fuzzy satisfying method based on the GA to derive the optimal solution. The proposed approach is implemented in a software package and its effectiveness is verified through numerical examples on the Tai-power system.

An improved stepsize of the subgradient algorithm for solving the lagrangian relaxation problem

The Lagrangian relaxation approach has been successfully applied to many large-scale mathematical programming problems. The Lagrangian relaxation problem itself is a non-differentiable optimization problem. One of the methods for solving such problem is the subgradient algorithm. In this paper, we propose an improved stepsize of the subgradient algorithm for solving the Lagrangian relaxation problem. Our version of the algorithm may significantly improve the rate of convergence of subgradient algorithm when applied to the solution of Lagrangian relaxation problem. An illustrative numerical example is also given.

A Solution Method for a Special Class of Nondifferentiable Unconstrained Optimization Problems

We consider quasidifferentiable functions in the sense of Demyanov and Rubinov, i. e. functions, which are directionally differentiable and whose directional derivative can be expressed as a difference of two sublinear functions, so that its “subdifferential”, called the quasidifferential, consists of a pair of sets. For these functions a generalized gradient algorithm is proposed. Its behaviour is studied in detail for the special class of continuously subdifferentiable functions. Numerical test results are given. Finally, the general quasidifferentiable case is simulated by means of “perturbed” subdifferentials, where we make use of the non-uniqueness in the quasidifferential representation.

Convex nondifferentiable optimization: A survey focused on the analytic center cutting plane method

We present a survey of nondifferentiable optimization problems and methods with special focus on the analytic center cutting plane method. We propose a self-contained convergence analysis that uses the formalism of the theory of self-concordant functions, but for the main results, we give direct proofs based on the properties of the logarithmic function. We also provide an in-depth analysis of two extensions that are very relevant to practical problems: the case of multiple cuts and the case of deep cuts. We further examine extensions to problems including feasible sets partially described by an explicit barrier function, and to the case of nonlinear cuts. Finally, we review several implementation issues and discuss some applications.

VIOLATION-GUIDED NEURAL-NETWORK LEARNING FOR CONSTRAINED FORMULATIONS IN TIME-SERIES PREDICTIONS

Time-series predictions by artificial neural networks (ANNs) are traditionally formulated as unconstrained optimization problems. As an unconstrained formulation provides little guidance on search directions when a search gets stuck in a poor local minimum, we have proposed to use a constrained formulation in order to use constraint violations to provide additional guidance. In this paper, we formulate ANN learning with cross-validations for time-series predictions as a non-differentiable nonlinear constrained optimization problem. Based on our theory of Lagrange multipliers for discrete constrained optimization, we propose an efficient learning algorithm, called violation guided back-propagation (VGBP), that computes an approximate gradient using back-propagation (BP), that introduces annealing to avoid blind acceptance of trial points, and that applies a relax-and-tighten (R&T) strategy to achieve faster convergence. Extensive experimental results on well-known benchmarks, when compared to previous work, show one to two orders-of-magnitude improvement in prediction quality, while using less weights.

OPTIMAL DESIGN OF A TWO-STAGE MOUNTING ISOLATION SYSTEM BY THE MAXIMUM ENTROPY APPROACH

In this paper, the optimization of the parameters for two-stage mounting isolation system is studied. The maximum entropy approach is applied to deal with the non-differentiable optimization problems. Results of numerical examples confirm the efficiency and accuracy of the proposed method, which is also suitable for the optimization of other kind of isolation system, such as Ruzicka isolation system (Ding wen-jing 1988 Theory on Vibration Reduction, Tsinghua University Publishing House, Ruzicka 1967 Journal of Engineering and Industries ASME 89), etc

Multiobjective optimal feeder reconfiguration

A formulation for the feeder reconfiguration to reduce power loss, increase system security and improve power quality by considering operational constraints is presented. The problem formulation proposed is a constrained, multiobjective and nondifferentiable optimisation problem. The optimal feeder configuration can simultaneously attain the above objectives. However, these objectives are noncommensurable such that it is difficult to solve this class of problem by conventional approaches that optimise a single objective. Thus, these objective functions are first formulated in fuzzy sets to represent their imprecise nature. A fuzzy satisfied method based on evolutionary programming is then introduced to determine the optimal solution. The proposed approach is implemented in a software package and its effectiveness is verified through numerical examples on the Tai-power system.

Necessary optimality conditions in nondifferentiable vector optimization

The purpose of the present paper is to give necessary optimality conditions for weak Pareto minimun, peints of nondifferentiable vector optimization problems vcing generalized definitions of the upper and lower Dini-Hadamard derivatives. We give two different approaches for such definitions, a global one and a componentwise one

Limited Memory Space Dilation and Reduction Algorithms

In this paper, we present variants of Shor and Zhurbenko's r-algorithm, motivated by the memoryless and limited memory updates for differentiable quasi-Newton methods. This well known r-algorithm, which employs a space dilation strategy in the direction of the difference between two successive subgradients, is recognized as being one of the most effective procedures for solving nondifferentiable optimization problems. However, the method needs to store the space dilation matrix and update it at every iteration, resulting in a substantial computational burden for large-sized problems. To circumvent this difficulty, we first propose a memoryless update scheme, which under a suitable choice of parameters, yields a direction of motion that turns out to be a convex combination of two successive anti-subgradients. Moreover, in the space transformation sense, the new update scheme can be viewed as a combination of space dilation and reduction operations. We prove convergence of this new method, and demonstrate how it can be used in conjunction with a variable target value method that allows a practical, convergent implementation of the method. We also examine a memoryless variant that uses a fixed dilation parameter instead of varying degrees of dilation and/or reduction as in the former algorithm, as well as another variant that examines a two-step limited memory update. These variants are tested along with Shor's r-algorithm and also a modified version of a related algorithm due to Polyak that employs a projection onto a pair of Kelley's cutting planes. We use a set of standard test problems from the literature as well as randomly generated dual transportation and assignment problems in our computational experiments. The results exhibit that the proposed space dilation and reduction method and the modification of Polyak's method are competitive, and offer a substantial advantage over the r-algorithm and over the other tested limited memory variants with respect to accuracy as well as effort.

Nonmonotone and Monotone Active-Set Methods for Image Restoration, Part 2: Numerical Results

Active-set methods based on augmented Lagrangian smoothing of nondifferentiable optimization problems arising in image restoration are studied through numerical experiments. Implemented algorithms for solving both one-dimensional and two-dimensional image restoration problems are described. A new, direct way for solving the constrained optimization problem appearing in the inner iteration of the monotone algorithms is presented. Several numerical examples are included.

Nonmonotone and Monotone Active-Set Methods for Image Restoration, Part 1: Convergence Analysis

Active-set methods based on augmented Lagrangian smoothing of nondifferentiable optimization problems arising in image restoration are discussed. One-dimensional image restoration problems and two different formulations of two-dimensional image restoration problems are given. Both nonmonotone and monotone active-set algorithms are described and finite-step convergence of the algorithms is considered.

A New Algorithm for Computing Process Flexibility

In the design of a chemical process (CP), certain design specifications (for example, those related to process economics, process performance, safety, and the environment) must be satisfied. During the operation of the plant, because design models have uncertainties associated with them, we need to ensure that, within the region of uncertainty, all design specifications are satisfied. In recent years, research has focused on the investigation of the process flexibility (Biegler et al. Systematic Methods of Chemical Process Design; Prentice Hall: Englewood Cliffs, NJ, 1997) based on the feasibility function (Halemane and Grossmann AIChE J. 1983, 29 (3), 425) which is a measure of the CP's ability to meet design specifications under uncertainty. Several researchers have proposed methods for calculating the process feasibility function, which involves solving a very complex multiextremal and nondifferentiable optimization problem. Current methods for calculation of the flexibility function use an enumeration procedure (explicit or implicit), which in the worst case can require a large number of iterations. To try to address this issue, in this paper, we have introduced an efficient approach, which avoids enumeration. Through examples, we have shown that the new method leads to a small number of iterations and has low CPU requirements.

Simultaneous Optimal Design of Structural and Control Systems Including Discrete Design Variables.

We formulate a simultaneous optimal design problem of structural and control systems including discrete design variables. The problem generally becomes nonlinear and/or nondifferentiable optimization problem whose design variables are continuous or discrete. Since the nondifferentiability of the problem, we can not apply standard descent-based nonlinear optimization techniques such as steepest descent method, Quasi-Newton method etc.. Even if a subproblem, which is generated by fixing discrete design variables of the original problem, is corresponding to BMI(Bilinear Matrix Inequality), it is difficult to obtain the(global)optimal solution efficiently because of NP hardness nature of BMI. In this paper, we apply SA(Simulated Annealing)which is known as the one of the probabilistic optimization method. We present a new controller parametrization of the controller based on Youla parametrization. This parametrization does not need numerical computational procedure, such as solving Riccati equation or LMI to obtain stabilizing controller. Moreover, the class of the controller is at least larger than that of DVFB control(which is also a stabilizing control law without numerical computational procedure). We give a design example and show the effectiveness of the proposed design method.

Regularization by Functions of Bounded Variation and Applications to Image Enhancement

Optimization problems regularized by bounded variation seminorms are analyzed. The optimality system is obtained and finite-dimensional approximations of bounded variation function spaces as well as of the optimization problems are studied. It is demonstrated that the choice of the vector norm in the definition of the bounded variation seminorm is of special importance for approximating subspaces consisting of piecewise constant functions. Algorithms based on a primal—dual framework that exploit the structure of these nondifferentiable optimization problems are proposed. Numerical examples are given for denoising of blocky images with very high noise.

An algorithm for a minimum fuel control problem

This paper describes an algorithm for a general minimum fuel control problem. The objective function of the problem is represented by the functional: F0 x,u = F0,1 x,u F0,2 x,u where F0,1 is continuously differentiable with respect to states x and controls u, while F0,2 includes the term tf m g t, x t u t ur t d t . 0 i 1 i i i A direction of descent of the algorithm is found by solving a convex (possibly non-differentiable) optimization problem. An efficient version of a proximity algorithm is used to solve this sub-problem. State and terminal constraints are treated via a feasible directions approach and an exact penalty function respectively. The algorithm is globally convergent under minimal assumptions imposed on the problem. Every accumulation point of a sequence generated by the algorithm satisfies the combined strong-weak version of the maximum principle condition.

Solving Euclidean Distance Multifacility Location Problems Using Conjugate Subgradient and Line-Search Methods

Subgradient methods are popular for solving nondifferentiable optimization problems because of their relative ease in implementation, but are not always robust and require a careful design of strategies in order to yield an effective procedure for any given class of problems. In this paper, we present an approach for solving the Euclidean distance multifacility location problem (EMFLP) using conjugate or deflected subgradient based algorithms along with suitable line-search strategies. The subgradient deflection method considered is the Average Direction Strategy (ADS) imbedded within the Variable Target Value Method (VTVM). We also investigate the generation of two types of subgradients to be employed in conjunction with ADS. The first type is a simple valid subgradient that assigns zero values to contributions corresponding to the nondifferentiable terms in the objective function, and so, the subgradient is composed by summing the contributions corresponding to the differentiable terms alone. The second type expends more effort to derive a low-norm member of the subdifferential in order to enhance the prospect of obtaining a descent direction. Furthermore, a special Newton-based line-search that exploits the nondifferentiability of the problem is also designed to be implemented in the developed algorithm in order to study its impact on the convergence behavior. Various combinations of the above strategies are composed and evaluated on a set of test problems. The results show that a modification of the VTVM method along with the first or a certain combination of the two subgradient generation strategies, and the use of a suitable line-search technique, provides promising results. An alternative block-halving step-size strategy used within VTVM in conjunction with the proposed line-search method yields a competitive second choice performance.

An active set strategy based on the augmented Lagrangian formulation for image restoration

Lagrangian and augmented Lagrangian methods for nondifferentiable optimization problems that arise from the total bounded variation formulation of image restoration problems are analyzed. Conditional convergence of the Uzawa algorithm and unconditional convergence of the first order augmented Lagrangian schemes are discussed. A Newton type method based on an active set strategy defined by means of the dual variables is developed and analyzed. Numerical examples for blocky signals and images perturbed by very high noise are included.

Nondifferentiable optimal control problems for matrix defferential equations

Nondifferentiable optimal control problems for matrix defferential equations