Nondifferentiable Constraints Research Articles

Score-matching neural networks for improved multi-band source separation

We present the implementation of a score-matching neural network that represents a data-driven prior for non-parametric galaxy morphologies. The gradients of this prior can be incorporated in the optimization of galaxy models to aid with tasks like deconvolution, inpainting or source separation. We demonstrate this approach with modification of the multi-band modeling framework scarlet that is currently employed as deblending method in the pipelines of the HyperSuprimeCam survey and the Rubin Observatory. The addition of the prior avoids the requirement of non-differentiable constraints, which can lead to convergence failures we discovered in scarlet. We present the architecture and training details of our score-matching neural network and show with simulated Rubin-like observations that using a data-driven prior outperforms the baseline scarlet method in accuracy of total flux and morphology estimates, while maintaining excellent performance for colors. We also demonstrate significant improvements in the robustness to inaccurate initializations. The trained score models used for this analysis are publicly available at https://github.com/SampsonML/galaxygrad.

A nonlinear multi-period hydrothermal optimal power flow model for hydropower systems

Despite their social and climate benefits, renewable energy sources also present challenges such as intermittency, volatility and randomness, which may considerably reduce security and reliability of the system. Hydro units have the ability to provide flexible and rapid energy reserves, serving as an effective way of handling these challenges. Yet, many Hydrothermal Optimal Power Flow (HOPF) models in the literature often provide a simplified and sometimes inaccurate representation of the hydraulic aspects. In most HPOF models proposed in the literature, the representation of hydraulic aspects disregards important physical and operational hydro constraints. Such simplifications can curtail the potential benefits of hydro generation, especially concerning system security and reliability. In this paper, we investigate the impact of hydro constraints in the operation of hydrothermal systems by means of HOPF models. For such a purpose, we propose a nonlinear multi-period active–reactive HOPF model where hydro constraints are represented in detail, by means of a nonlinear framework. This model presents innovative formulations for hydro units, specifically designed to address pivotal non-differentiable constraints. The proposed multi-period HOPF model is used as a benchmark to assess inaccuracies present in linear or linearized HOPF model variants. The results using the IEEE 24-bus and 300-bus systems, adapted so they become hydropower systems, clearly identify significant impact of some hydro constraints which have been neglected in most models described in the literature, in the hydrothermal operational policy and scheduling.

CAT: Constrained Adversarial Training for Anatomically-Plausible Semi-Supervised Segmentation.

Deep learning models for semi-supervised medical image segmentation have achieved unprecedented performance for a wide range of tasks. Despite their high accuracy, these models may however yield predictions that are considered anatomically impossible by clinicians. Moreover, incorporating complex anatomical constraints into standard deep learning frameworks remains challenging due to their non-differentiable nature. To address these limitations, we propose a Constrained Adversarial Training (CAT) method that learns how to produce anatomically plausible segmentations. Unlike approaches focusing solely on accuracy measures like Dice, our method considers complex anatomical constraints like connectivity, convexity, and symmetry which cannot be easily modeled in a loss function. The problem of non-differentiable constraints is solved using a Reinforce algorithm which enables to obtain a gradient for violated constraints. To generate constraint-violating examples on the fly, and thereby obtain useful gradients, our method adopts an adversarial training strategy which modifies training images to maximize the constraint loss, and then updates the network to be robust to these adversarial examples. The proposed method offers a generic and efficient way to add complex segmentation constraints on top of any segmentation network. Experiments on synthetic data and four clinically-relevant datasets demonstrate the effectiveness of our method in terms of segmentation accuracy and anatomical plausibility.

Solving Economic Emission Load Dispatch Using Oppositional Slime Mould Algorithm With Wavelet Mutation

This paper introduces a modified slime mould algorithm to enhance the exploitation and exploration of the algorithm. As stated in no free lunch theorem no single algorithm can proficiently solve the optimization problems at hand. Hence, hybridization of more than one heuristic algorithm is employed to tackle the situation. The proposed strategy employs opposition-based learning and wavelet mutation-based strategy to the basic slime mould algorithm for exploring the search space while avoiding local optima. These methods are tested on standard benchmark functions, CEC-14 functions and different single and multiobjective thermal load dispatch problems including small, medium and large-scale problems while satisfying non-differentiable constraints namely valve point loading, prohibited operating zone and ramp rate limit which make objective function non-convex and discontinuous. The pareto dominance technique is employed to achieve the optimal solution in multiobjective problems. This paper also compares the performance of the proposed technique with other methods available in the literature for different economic load dispatch problems and emission dispatch problems. The Wilcoxon signed-rank test performed to confirm the competitiveness of the proposed method for independent samples.

Reaching Through Latent Space: From Joint Statistics to Path Planning in Manipulation

We present a novel approach to path planning for robotic manipulators, in which paths are produced via iterative optimisation in the latent space of a generative model of robot poses. Constraints are incorporated through the use of constraint satisfaction classifiers operating on the same space. Optimisation leverages gradients through our learned models that provide a simple way to combine goal reaching objectives with constraint satisfaction, even in the presence of otherwise non-differentiable constraints. Our models are trained in a task-agnostic manner on randomly sampled robot poses. In baseline comparisons against a number of widely used planners, we achieve commensurate performance in terms of task success, planning time and path length, performing successful path planning with obstacle avoidance on a real 7-DoF robot arm.

Fair classification via Monte Carlo policy gradient method

Artificial intelligence is steadily increasing its impact on everyday life. Therefore, the societal issues of artificial intelligence have become an important concern in the AI research. The presence of data that reflects human biases towards historically discriminated groups defined by sensitive features such as race and gender, results in machine learning models which discriminate against these groups. In order to tackle the impact of bias in data, researchers developed a variety of specialized machine learning algorithms which are able to satisfy different fairness constraints imposed on the model. Group fairness constraints do not fit standard machine learning formulations easily due to their non-differentiable nature. In this paper we developed a technique for learning a fair classifier by Monte Carlo policy gradient method which naturally deals with such non-differentiable constraints. Our methodology focuses on direct optimization of both group fairness metric and predictive performance of the model. In addition, we propose two different variance reduction techniques of gradient estimation. We compare our models to seven other related and state-of-the-art models and demonstrate that they are able to achieve better trade-off between accuracy and unfairness. To the best of our knowledge, this is the first fair classification algorithm which solves the issue of non-differentiable constraints by reinforcement learning techniques.

Fast inverse design of microstructures via generative invariance networks.

The problem of the efficient design of material microstructures exhibiting desired properties spans a variety of engineering and science applications. The ability to rapidly generate microstructures that exhibit user-specified property distributions can transform the iterative process of traditional microstructure-sensitive design. We reformulate the microstructure design process using a constrained generative adversarial network (GAN) model. This approach explicitly encodes invariance constraints within GANs to generate two-phase morphologies for photovoltaic applications obeying design specifications: specifically, user-defined short-circuit current density and fill factor combinations. Such invariance constraints can be represented by differentiable, deep learning-based surrogates of full physics models mapping microstructures to photovoltaic properties. Furthermore, we propose a multi-fidelity surrogate that reduces expensive label requirements by a factor of five. Our framework enables the incorporation of expensive or non-differentiable constraints for the fast generation of microstructures (in 190 ms) with user-defined properties. Such proposed physics-aware data-driven methods for inverse design problems can be used to considerably accelerate the field of microstructure-sensitive design.

Maximum and minimum entropy states yielding local continuity bounds

Given an arbitrary quantum state (σ), we obtain an explicit construction of a state ρε*(σ) [respectively, ρ*,ε(σ)] which has the maximum (respectively, minimum) entropy among all states which lie in a specified neighborhood (ε-ball) of σ. Computing the entropy of these states leads to a local strengthening of the continuity bound of the von Neumann entropy, i.e., the Audenaert-Fannes inequality. Our bound is local in the sense that it depends on the spectrum of σ. The states ρε*(σ) and ρ*,ε(σ) depend only on the geometry of the ε-ball and are in fact optimizers for a larger class of entropies. These include the Rényi entropy and the minimum- and maximum-entropies, providing explicit formulas for certain smoothed quantities. This allows us to obtain local continuity bounds for these quantities as well. In obtaining this bound, we first derive a more general result which may be of independent interest, namely, a necessary and sufficient condition under which a state maximizes a concave and Gâteaux-differentiable function in an ε-ball around a given state σ. Examples of such a function include the von Neumann entropy and the conditional entropy of bipartite states. Our proofs employ tools from the theory of convex optimization under non-differentiable constraints, in particular Fermat’s rule, and majorization theory.

Constraint handling for gradient-based optimization of compositional reservoir flow

The development of adjoint procedures for general compositional flow problems is much more challenging than for oil-water problems, due to significantly higher complexity of the underlying physics. The treatment of nondifferentiable constraints, an example of which is a maximum gas rate specification in injection or production wells, when the control variables are well-bottom-hole pressures, poses an additional major challenge. A new formal approach for handling these constraints is presented and compared against a formal treatment within the optimizer employing constraint lumping and a simpler heuristic treatment in the forward model. The three constraint-handling methods are benchmarked for three example cases of increasing complexity. Moreover, the new approach allows the optimizer to converge to optimal solutions exhibiting higher objective values, since unlike the formal lumping-based methods, where a pressure reduction suggested by the optimizer propagates through the smoothing function to all well rates, it handles constraints individually on a per well and per time step basis. The numerical examples show that the new formal constraint-handling approach allows the optimizer to converge significantly faster than formal lumping-based techniques independently of the initial guess used for the optimization.

Differential geometry tools for multidisciplinary design optimization, Part I: Theory

Analysis within the field of Multidisciplinary Design Optimization (MDO) generally falls under the headings of architecture proofs and sensitivity information manipulation. We propose a differential geometry (DG) framework for further analyzing MDO systems, and here, we outline the theory undergirding that framework: general DG, Riemannian geometry for use in MDO, and the translation of MDO into the language of DG. Calculating the necessary quantities requires only basic sensitivity information (typically from the state equations) and the use of the implicit function theorem. The presence of extra or non-differentiable constraints may limit the use of the framework, however. Ultimately, the language and concepts of DG give us new tools for understanding, evaluating, and developing MDO methods; in Part I, we discuss the use of these tools and in Part II, we provide a specific application.

A cell evolution method for reliability-based design optimization

The reliability-based design optimization (RBDO) presents to be a systematic and powerful approach for process designs under uncertainties. The traditional double-loop methods for solving RBDO problems can be computationally inefficient because the inner reliability analysis loop has to be iteratively performed for each probabilistic constraint. To solve RBDOs in an alternative and more effective way, Deb et al. [1] proposed recently the use of evolutionary algorithms with an incorporated fastPMA. Since the imbedded fastPMA needs the gradient calculations and the initial guesses of the most probable points (MPPs), their proposed algorithm would encounter difficulties in dealing with non-differentiable constraints and the effectiveness could be degraded significantly as the initial guesses are far from the true MPPs. In this paper, a novel population-based evolutionary algorithm, named cell evolution method, is proposed to improve the computational efficiency and effectiveness of solving the RBDO problems. By using the proposed cell evolution method, a family of test cells is generated based on the target reliability index and with these reliability test cells the determination of the MPPs for probabilistic constraints becomes a simple parallel calculation task, without the needs of gradient calculations and any initial guesses. Having determined the MPPs, a modified real-coded genetic algorithm is applied to evolve these cells into a final one that satisfies all the constraints and has the best objective function value for the RBDO. Especially, the nucleus of the final cell contains the reliable solution to the RBDO problem. Illustrative examples are provided to demonstrate the effectiveness and applicability of the proposed cell evolution method in solving RBDOs. Simulation results reveal that the proposed cell evolution method outperforms comparative methods in both the computational efficiency and solution accuracy, especially for multi-modal RBDO problems.

On interval-subgradient and no-good cuts

Interval-gradient cuts are (nonlinear) valid inequalities derived from continuously differentiable nonconvex constraints. In this paper we define interval-subgradient cuts, a generalization to nondifferentiable constraints, and show that no-good cuts with 1-norm are a special case of interval-subgradient cuts. We then briefly discuss what happens if other norms are used.

On a variational problem with non-differentiable constraints

We study the regularity of minimizers and critical points of the Dirichlet energy under an integral constraint given by a non-differentiable function. We obtain existence of a Lipschitz continuous minimizer for a relaxed problem. In two dimensions, some regularity can also be proved for critical points.

Univariate Global Optimization with Multiextremal Non-Differentiable Constraints Without Penalty Functions

This paper proposes a new algorithm for solving constrained global optimization problems where both the objective function and constraints are one-dimensional non-differentiable multiextremal Lipschitz functions. Multiextremal constraints can lead to complex feasible regions being collections of isolated points and intervals having positive lengths. The case is considered where the order the constraints are evaluated is fixed by the nature of the problem and a constraint i is defined only over the set where the constraint i−1 is satisfied. The objective function is defined only over the set where all the constraints are satisfied. In contrast to traditional approaches, the new algorithm does not use any additional parameter or variable. All the constraints are not evaluated during every iteration of the algorithm providing a significant acceleration of the search. The new algorithm either finds lower and upper bounds for the global optimum or establishes that the problem is infeasible. Convergence properties and numerical experiments showing a nice performance of the new method in comparison with the penalty approach are given.

Design of planar articulated mechanisms using branch and bound

This paper considers an optimization model and a solution method for the design of two-dimensional mechanical mechanisms. The mechanism design problem is modeled as a nonconvex mixed integer program which allows the optimal topology and geometry of the mechanism to be determined simultaneously. The underlying mechanical analysis model is based on a truss representation allowing for large displacements. For mechanisms undergoing large displacements elastic stability is of major concern. We derive conditions, modeled by nonlinear matrix inequalities, which guarantee that a stable equilibrium is found and that buckling is prevented. The feasible set of the design problem is described by nonlinear differentiable and non-differentiable constraints as well as nonlinear matrix inequalities.To solve the mechanism design problem a branch and bound method based on convex relaxations is developed. To guarantee convergence of the method, two different types of convex relaxations are derived. The relaxations are strengthened by adding valid inequalities to the feasible set and by solving bound contraction sub-problems. Encouraging computational results indicate that the branch and bound method can reliably solve mechanism design problems of realistic size to global optimality.

Optimal Design for LC Power Harmonic Filters

This study describes a novel method for designing LC-type (single-tuned) filters to reduce harmonic distortion. The proposed method minimizes the total cost of the filters, such that the harmonic distortion is within an acceptable range. The filter design problem is formulated as a combinatorial optimization problem with a nondifferentiable objective function and multiple constraints. The objective function of the filter design problem is the filter cost function, which includes both purchasing and installation costs. The harmonic current limit for the given utility, the system impedance variations, and resonance conditions are also considered in the problem formulation. Furthermore, a solution methodology based on an optimization technique-simulated annealing-is proposed to determine the size of filters that can be made at minimum cost. Test cases with promised results are reported.

OPTIMIZATION OF COMPOSITE LAMINATES WITH CUTOUTS USING GENETIC ALGORITHM, VARIABLE METRIC AND COMPLEX SEARCH METHODS

Optimization of composite laminates with cutouts is a complex problem, involving non-differentiable objective function and constraints. Choice of the optimization method is generally based on the nature and complexity of the objective function, constraints and how easily and/or accurately the first derivatives can be found. Many researchers have attempted and applied different classical optimization techniques for non-convex optimization problems. This paper clearly brings out the advantages of a non-traditional optimization method-Genetic algorithm (GA) over conventional techniques, the limitations of conventional techniques and GA's ability to approach the global optimum in an n-dimensional search space, for composite laminates.

A globally convergent method for semi-infinite linear programming

This paper presents a globally convergent method for solving a general semi-infinite linear programming problem. Some important features of this method include: It can solve a semi-infinite linear program having an unbounded feasible region. It requires an inexact solution to a nonlinear subproblem at each iteration. It allows unbounded index sets and nondifferentiable constraints. The amount of work at each iteration k does not increase with k.

Nondifferentiable reverse convex programs and facetial convexity cuts via a disjunctive characterization

Disjunctive Programs can often be transcribed as reverse convex constrained problems with nondifferentiable constraints and unbounded feasible regions. We consider this general class of nonconvex programs, called Reverse Convex Programs (RCP), and show that under quite general conditions, the closure of the convex hull of the feasible region is polyhedral. This development is then pursued from a more constructive standpoint, in that, for certain special reverse convex sets, we specify a finite linear disjunction whose closed convex hull coincides with that of the special reverse convex set. When interpreted in the context of convexity/intersection cuts, this provides the capability of generating any (negative edge extension) facet cut. Although this characterization is more clarifying than computationally oriented, our development shows that if certain bounds are available, then convexity/intersection cuts can be strengthened relatively inexpensively.

Forward differential dynamic programming

The dynamic programming formulation of the forward principle of optimality in the solution of optimal control problems results in a partial differential equation with initial boundary condition whose solution is independent of terminal cost and terminal constraints. Based on this property, two computational algorithms are described. The first-order algorithm with minimum computer storage requirements uses only integration of a system of differential equations with specified initial conditions and numerical minimization in finite-dimensional space. The second-order algorithm is based on the differential dynamic programming approach. Either of the two algorithms may be used for problems with nondifferentiable terminal cost or terminal constraints, and the solution of problems with complicated terminal conditions (e.g., with free terminal time) is greatly simplified.