Box Constraints Research Articles

Dyadic Partition-Based Training Schemes for TV/TGV Denoising

AbstractDue to their ability to handle discontinuous images while having a well-understood behavior, regularizations with total variation (TV) and total generalized variation (TGV) are some of the best-known methods in image denoising. However, like other variational models including a fidelity term, they crucially depend on the choice of their tuning parameters. A remedy is to choose these automatically through multilevel approaches, for example by optimizing performance on noisy/clean image pairs. In this work, we consider such methods with space-dependent parameters which are piecewise constant on dyadic grids, with the grid itself being part of the minimization. We prove existence of minimizers for fixed discontinuous parameters under mild assumptions on the data, which lead to existence of finite optimal partitions. We further establish that these assumptions are equivalent to the commonly used box constraints on the parameters. On the numerical side, we consider a simple subdivision scheme for optimal partitions built on top of any other bilevel optimization method for scalar parameters, and demonstrate its improved performance on some representative test images when compared with constant optimized parameters.

Solving higher dimensional expensive black box global optimization problems using sparse directional search on surrogates

AbstractWe present a new algorithm Directional Optimization Search with Surrogate (DOSS), for optimizing problems with box constraints and a computationally expensive black-box objective function. DOSS is a radial basis function (RBF) based method that mainly focuses on higher dimensional and computationally expensive objective functions that can be multimodal. DOSS introduces three new techniques not previously used in earlier RBF algorithms, including using a combination of the coordinates knowledge level, fewer initial sampling points, and the surrogate’s gradient information. Numerical results on a test set including 14 test problems with 36, 48, and 60 dimensions show that DOSS outperforms two recently published algorithms RBFOpt and TuRBO, and earlier RBF algorithms such as DYCORS. TuRBO is a Gaussian process based optimization algorithm, which outperformed earlier state-of-the-art methods. DOSS algorithm also has a good performance on real-world optimization problems, including robot pushing and rover trajectory planning problems. Almost sure convergence for the DOSS algorithm is also proven in this paper. An implementation of DOSS is available at https://doi.org/10.5281/zenodo.13731558.

Box Constraints and Weighted Sparsity Regularization for Identifying Sources in Elliptic PDEs

We explore the possibility for using boundary data to identify sources in elliptic PDEs. Even though the associated forward operator has a large null space, it turns out that box constraints, combined with weighted sparsity regularization, can enable rather accurate recovery of sources with constant magnitude/strength. In addition, for sources with varying strength, the support of the inverse solution will be a subset of the support of the true source. We present both an analysis of the problem and a series of numerical experiments. Our work only addresses discretized problems. The reason for introducing the weighting procedure is that standard (unweighted) sparsity regularization fails to provide adequate results for the source identification task considered in this paper. This investigation is also motivated by applications, e.g. recovering mass distributions from measurements of gravitational fields and inverse scattering. We develop the methodology and the analysis in terms of Euclidean spaces, and our results can therefore be applied to many problems. For example, the results are equally applicable to models involving the screened Poisson equation as to models using the Helmholtz equation, with both large and small wave numbers.

A Quasi-Newton Subspace Trust Region Algorithm for Nonmonotone Variational Inequalities in Adversarial Learning over Box Constraints

The first-order optimality condition of convexly constrained nonconvex nonconcave min-max optimization problems with box constraints formulates a nonmonotone variational inequality (VI), which is equivalent to a system of nonsmooth equations. In this paper, we propose a quasi-Newton subspace trust region (QNSTR) algorithm for the least squares problems defined by the smoothing approximation of nonsmooth equations. Based on the structure of the nonmonotone VI, we use an adaptive quasi-Newton formula to approximate the Hessian matrix and solve a low-dimensional strongly convex quadratic program with ellipse constraints in a subspace at each step of the QNSTR algorithm efficiently. We prove the global convergence of the QNSTR algorithm to an ϵ\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\epsilon $$\\end{document}-first-order stationary point of the min-max optimization problem. Moreover, we present numerical results based on the QNSTR algorithm with different subspaces for a mixed generative adversarial networks in eye image segmentation using real data to show the efficiency and effectiveness of the QNSTR algorithm for solving large-scale min-max optimization problems.

Proximal Gauss-Newton Method for Box-constrained Parameter Identification of a Nonlinear Railway Suspension System

The identification of railway vehicle components’ characteristics from measured data is a challenging task with compelling applications in health monitoring, fault detection, and system prognosis. Usually, though, such systems are highly nonlinear, and naive identification techniques may lead to unstable methods and inaccurate results. In this paper, we show that these issues can be easily tackled with the recently introduced proximal Gauss–Newton method, which we employ to identify the parameters of a railway nonlinear suspension system. In the proposed model, the parameters are subject to safety bounds in form of box constraints, which allows preventing nonphysical solutions. The suspension system we consider is highly nonlinear due to the presence of an airspring in the secondary suspension, which we introduce in a simplified Berg model. Numerical examples, featuring data corrupted by various noise levels, demonstrate the accuracy and efficiency of our proposed method. Comparisons with state-of-the-art approaches are also provided.

CMA-ES-based topology optimization accelerated by spectral level-set-boundary modeling

Topology optimization commonly encounters several challenges, such as ill-posedness, grayscale issues, interdependencies among design variables, multimodality, and the curse of dimensionality. Furthermore, addressing the latter two concurrently presents considerable difficulty. In this study, we introduce a framework aimed at mitigating all the above obstacles simultaneously. The objective is to achieve optimal configurations in a notably reduced timeframe eliminating the need for the initial trial-and-error iterations. The topology optimization approach we propose is implemented via precise structural boundary modeling utilizing a body-fitted mesh generated using a Fourier series expanded level-set method. This methodology expedites the exploration of optimal solutions. We employ the covariance matrix adaptation-evolution strategy to address multimodality, thereby enhancing the optimization process. The implementation of the Fourier-series-expanded level-set method reduces the number of design variables while maintaining accuracy in finite-element analyses by replacing design variables from discretized level-set functions with the coefficients of the Fourier series expansion. To facilitate the exploration of optimal solutions, a method is also introduced for handling box constraints through an adaptive penalty function. To demonstrate the effectiveness of the proposed scheme, we address three distinct problems: mean compliance minimization, heat flux manipulation, and the control of electromagnetic wave scattering. Despite each system being governed by different equations, topology optimization method consistently yields notable acceleration in computational efficiency across all scenarios, and remarkably without requiring initial guesses.

Zero gradient sum algorithm of arbitrary initial value with constraints and communication delay based on directed graph

The distributed convex optimisation issue with inequality constraints and box constraints on the directed communication topology is examined in this paper. In the case of the communication delay between agents, we propose a piecewise zero-gradient-sum triggered control algorithm that allows arbitrary initial values. Using the log-barrier penalty method, we only need to study an unconstrained approximation problem of the original problem. Firstly, each agent's state converges to the optimal value of the related local objective function in a fixed time. Simultaneously, the upper bound of fixed time in this paper is smaller than some existing results. Then, in the case of communication delay, time-triggered and event-triggered methods with non-uniform sampling interval are proposed in this paper, which make the communication cost lower and the application broader. Using the Lyapunov function method, the sufficient conditions for each agent's state to converge to the optimal point are given, and the Zeno behaviour is excluded. Finally, to confirm the algorithm's effectiveness, we provide two numerical examples and one machine learning example as demonstrations.

Spin-It Faster: Quadrics Solve All Topology Optimization Problems That Depend Only On Mass Moments

The behavior of a rigid body primarily depends on its mass moments, which consist of the mass, center of mass, and moments of inertia. It is possible to manipulate these quantities without altering the geometric appearance of an object by introducing cavities in its interior. Algorithms that find cavities of suitable shapes and sizes have enabled the computational design of spinning tops, yo-yos, wheels, buoys, and statically balanced objects. Previous work is based, for example, on topology optimization on voxel grids, which introduces a large number of optimization variables and box constraints, or offset surface computation, which cannot guarantee that solutions to a feasible problem will always be found. In this work, we provide a mathematical analysis of constrained topology optimization problems that depend only on mass moments. This class of problems covers, among others, all applications mentioned above. Our main result is to show that no matter the outer shape of the rigid body to be optimized or the optimization objective and constraints considered, the optimal solution always features a quadric-shaped interface between material and cavities. This proves that optimal interfaces are always ellipsoids, hyperboloids, paraboloids, or one of a few degenerate cases, such as planes. This insight lets us replace a difficult topology optimization problem with a provably equivalent non-linear equation system in a small number (<10) of variables, which represent the coefficients of the quadric. This system can be solved in a few seconds for most examples, provides insights into the geometric structure of many specific applications, and lets us describe their solution properties. Finally, our method integrates seamlessly into modern fabrication workflows because our solutions are analytical surfaces that are native to the CAD domain.

On exact and inexact RLT and SDP-RLT relaxations of quadratic programs with box constraints

Quadratic programs with box constraints involve minimizing a possibly nonconvex quadratic function subject to lower and upper bounds on each variable. This is a well-known NP-hard problem that frequently arises in various applications. We focus on two convex relaxations, namely the reformulation–linearization technique (RLT) relaxation and the SDP-RLT relaxation obtained by combining the Shor relaxation with the RLT relaxation. Both relaxations yield lower bounds on the optimal value of a quadratic program with box constraints. We show that each component of each vertex of the RLT relaxation lies in the set {0,12,1}\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\{0,\\frac{1}{2},1\\}$$\\end{document}. We present complete algebraic descriptions of the set of instances that admit exact RLT relaxations as well as those that admit exact SDP-RLT relaxations. We show that our descriptions can be converted into algorithms for efficiently constructing instances with (1) exact RLT relaxations, (2) inexact RLT relaxations, (3) exact SDP-RLT relaxations, and (4) exact SDP-RLT but inexact RLT relaxations. Our preliminary computational experiments illustrate that our algorithms are capable of generating computationally challenging instances for state-of-the-art solvers.

Direct stellarator coil design using global optimization: application to a comprehensive exploration of quasi-axisymmetric devices

Many stellarator coil design problems are plagued by multiple minima, where the locally optimal coil sets can sometimes vary substantially in performance. As a result, solving a coil design problem a single time with a local optimization algorithm is usually insufficient and better optima likely do exist. To address this problem, we propose a global optimization algorithm for the design of stellarator coils and outline how to apply box constraints to the physical positions of the coils. The algorithm has a global exploration phase that searches for interesting regions of design space and is followed by three local optimization algorithms that search in these interesting regions (a ‘global-to-local’ approach). The first local algorithm (phase I), following the globalization phase, is based on near-axis expansions and finds stellarator coils that optimize for quasisymmetry in the neighbourhood of a magnetic axis. The second local algorithm (phase II) takes these coil sets and optimizes them for nested flux surfaces and quasisymmetry on a toroidal volume. The final local algorithm (phase III) polishes these configurations for an accurate approximation of quasisymmetry. Using our global algorithm, we study the trade-off between coil length, aspect ratio, rotational transform and quality of quasi-axisymmetry. The database of stellarators, which comprises approximately 200 000 coil sets, is available online and is called QUASR, for ‘quasi-symmetric stellarator repository’.

On the regularity of multipliers and second-order optimality conditions of KKT-type for semilinear parabolic control problems

A class of optimal control problems governed by semilinear parabolic equations with mixed constraints and a box constraint for control variable is considered. We show that if the so-called generalized separation condition is satisfied, then both optimality conditions of KKT-type and regularity of multipliers are fulfilled. Moreover, we show that if the initial value is good enough and boundary ∂Ω has a property of positive geometric density, then multipliers and optimal solutions are Hölder continuous.

On Constrained Mixed-Integer DR-Submodular Minimization

Diminishing returns (DR)–submodular functions encompass a broad class of functions that are generally nonconvex and nonconcave. We study the problem of minimizing any DR-submodular function with continuous and general integer variables under box constraints and, possibly, additional monotonicity constraints. We propose valid linear inequalities for the epigraph of any DR-submodular function under the constraints. We further provide the complete convex hull of such an epigraph, which, surprisingly, turns out to be polyhedral. We propose a polynomial-time exact separation algorithm for our proposed valid inequalities with which we first establish the polynomial-time solvability of this class of mixed-integer nonlinear optimization problems. Funding: This work was supported by the Office of Naval Research Global [Grant N00014-22-1-2602].

Surrogate optimisation strategies for intraocular lens formula constant optimisation.

To investigate surrogate optimisation (SO) as a modern, purely data-driven, nonlinear adaptive iterative strategy for lens formula constant optimisation in intraocular lens power calculation. A SO algorithm was implemented for optimising the root mean squared formula prediction error (rmsPE, defined as predicted refraction minus achieved refraction) for the SRKT, Hoffer Q, Holladay, Haigis and Castrop formulae in a dataset of N = 888 cataractous eyes with implantation of the Hoya Vivinex hydrophobic acrylic aspheric lens. A Gaussian Process estimator was used as the model, and the SO was initialised with equidistant datapoints within box constraints, and the number of iterations restricted to either 200 (SRKT, Hoffer Q, Holladay) or 700 (Haigis, Castrop). The performance of the algorithm was compared to the classical gradient-based Levenberg-Marquardt algorithm. The SO algorithm showed stable convergence after fewer than 50/150 iterations (SRKT, HofferQ, Holladay, Haigis, Castrop). The rmsPE was reduced systematically to 0.4407/0.4288/0.4265/0.3711/0.3449 dioptres. The final constants were A = 119.2709, pACD = 5.7359, SF = 1.9688, -a0 = 0.5914/a1 = 0.3570/a2 = 0.1970, C = 0.3171/H = 0.2053/R = 0.0947 for the SRKT, Hoffer Q, Holladay, Haigis and Castrop formula and matched the respective constants optimised in previous studies. The SO proves to be a powerful adaptive nonlinear iteration algorithm for formula constant optimisation, even in formulae with one or more constants. It acts independently of a gradient and is in general able to search within a (box) constrained parameter space for the best solution, even where there are multiple local minima of the target function.

Slater conditions without interior points for programs in Lebesgue spaces with pointwise bounds and finitely many constraints

We consider optimization problems in Lebesgue spaces with pointwise box constraints and finitely many additional linear constraints. We prove that the existence of a Slater point which lies strictly between the pointwise bounds and which satisfies the linear constraints is sufficient for the existence of Lagrange multipliers. Surprisingly, the Slater point is also necessary for the existence of Lagrange multipliers in a certain sense. We also demonstrate how to handle additional finitely many nonlinear constraints.

Normalized penalty gradient flow: A continuous-time approach for solving constrained nonconvex nonsmooth optimization problems

To avoid calculating penalty parameters, this paper introduces a continuous-time approach that combines the normalized gradient flow with the penalty method to solve the nonconvex nonsmooth optimization problem with a convex constraint set. Subsequently, this approach is extended to solve nonconvex nonsmooth optimization with a box constraint set and affine equality constraints. Compared with the current methods, the proposed approach does not require calculating the penalty parameters, allows the objective function to be nonconvex or nonsmooth, and allows the initial point to be arbitrarily selected. It is proved that the solution trajectory with any initial point converges to the critical point set of the optimization problem. Finally, the effectiveness of the proposed approach is authenticated through several numerical experiments.

Learnable binary MIMO detection with negative penalty based on inexact ADMM

AbstractIn this letter, a new learnable binary MIMO detection algorithm in massive MIMO systems with one‐bit analog‐to‐digital converters is proposed. A negative penalty is introduced to transform a non‐convex constraint to a convex box constraint and a learnable iterative algorithm is presented based on inexact alternating direction methods of multipliers. The proposed MIMO detection method performs better than the existing binary MIMO detection methods even with lower computational complexity.

A warm-start FE-dABCD algorithm for elliptic optimal control problems with constraints on the control and the gradient of the state

In this paper, elliptic control problems with the integral constraint on the gradient of the state and the box constraint on the control are considered. The optimality conditions for the problem are proved. To numerically solve the problem, a finite element duality-based inexact majorized accelerated block coordinate descent (FE-dABCD) algorithm is proposed. Specifically, both the state and the control are discretized by piecewise linear functions. An inexact majorized ABCD algorithm is employed to solve the discretized problem via its dual, which is a multi-block unconstrained convex optimization problem, but the primal variables are also generated in each iteration. Thanks to the inexactness of the FE-dABCD algorithm, the subproblems at each iteration are allowed to be solved inexactly. For the smooth subproblem, we use the preconditioned generalized minimal residual (GMRES) method to solve it. For the two nonsmooth subproblems, one of them has a closed form solution through introducing an appropriate proximal term, and another one is solved by the line search Newton's method. Based on these efficient strategies, we prove that our proposed FE-dABCD algorithm enjoys O(1k2) iteration complexity. Moreover, to make the algorithm more efficient and further reduce its computation cost, based on the mesh-independence of ABCD method, we propose an FE-dABCD algorithm with a warm-start strategy (wFE-dABCD). Some numerical experiments are done and the numerical results show the efficiency of the FE-dABCD algorithm and wFE-dABCD algorithm.

Auxetic Properties of the Stiffest Elastic Bodies as a Result of Topology Optimization and Microstructures Recovery Based on Homogenization Method

Herein, several semianalytical formulae for the optimal distribution of elastic moduli in 2D or 3D domains occupied by nonhomogeneous, least compliant bodies are presented and three different methods for recovering microstructures predicted by the isotropic material design method are proposed, which is a stress‐based variant of topology optimization for isotropic bodies. In all five presented variants of the topology optimization of elastic bodies, the numerical implementation of these formulae requires only the use of any algorithm of searching for the minimum of functional in multidimensions without any constraints, even box constraints on design parameters. For each variant considered, the formulae defining this functional are defined analytically. All results concern the design of composite structures that maximize their stiffness (equivalently minimize their compliance) while satisfying a certain isoperimetric condition introducing the upper limit of the available material resource. The article pays special attention to the fact that the stiffest, heterogeneous isotropic elastic material has almost always auxetic properties. Based on the asymptotic homogenization method and adopted bending‐free lattice model, an algorithm for the numerical recovering of nonhomogeneous isotropic microstructure in the entire domain occupied by the plate is proposed, with a fairly easy‐to‐implement ability to model an auxetic behavior.

Numerical solution of box constrained separable convex quadratic programming problems

In this paper, minimization problems with a separable convex quadratic objective function subject to a linear equality constraint/linear equality constraints, and bounds on the variables (box constraints) are considered. A necessary and sufficient condition for a feasible solution to be an optimal solution to each of these problems has been established. A convergent polynomial algorithm for solving the problem with a linear equality constraint and bounded variables is proposed, and some numerical results, obtained by this algorithm, are presented.

Douglas–Rachford algorithm for control- and state-constrained optimal control problems

<abstract><p>We consider the application of the Douglas–Rachford (DR) algorithm to solve linear-quadratic (LQ) control problems with box constraints on the state and control variables. We have split the constraints of the optimal control problem into two sets: one involving the ordinary differential equation with boundary conditions, which is affine, and the other, a box. We have rewritten the LQ control problems as the minimization of the sum of two convex functions. We have found the proximal mappings of these functions, which we then employ for the projections in the DR iterations. We propose a numerical algorithm for computing the projection onto the affine set. We present a conjecture for finding the costates and the state constraint multipliers of the optimal control problem, which can, in turn, be used to verify the optimality conditions. We conducted numerical experiments with two constrained optimal control problems to illustrate the performance and the efficiency of the DR algorithm in comparison with the traditional approach of direct discretization.</p></abstract>