Infeasible Point Research Articles

A feasible smoothing accelerated projected gradient method for nonsmooth convex optimization

Smoothing accelerated gradient methods achieve faster convergence rates than that of the subgradient method for some nonsmooth convex optimization problems. However, Nesterov's extrapolation may require gradients at infeasible points, and thus they cannot be applied to some structural optimization problems. We introduce a variant of smoothing accelerated projected gradient methods where every variable is feasible. The O(k−1log⁡k) convergence rate is obtained using the Lyapunov function. We conduct a numerical experiment on the robust compliance optimization of a truss structure.

Preserving multi-dimensional information: A hypersphere method for parameter space analysis

BackgroundPhysiological modelling often involves models described by large numbers of variables and significant volumes of clinical data. Mathematical interpretation of such models frequently necessitates analysing data points in high-dimensional spaces. Existing algorithms for analysing high-dimensional points either lose important dimensionality or do not describe the full position of points. Hence, there is a need for an algorithm which preserves this information. MethodsThe most-distant uncovered point (MDUP) hypersphere method is a binary classification approach which defines a collection of equidistant N-dimensional points as the union of hyperspheres. The method iteratively generates hyperspheres at the most distant point in the interest region not yet contained within any hypersphere, until the entire region of interest is defined by the union of all generated hyperspheres. This method is tested on a 7-dimensional space with up to 35.8 million points representing feasible and infeasible spaces of model parameters for a clinically validated cardiovascular system model. ResultsFor different numbers of input points, the MDUP hypersphere method tends to generate large spheres away from the boundary of feasible and infeasible points, but generates the greatest number of relatively much smaller spheres around the boundary of the region of interest to fill this space. Runtime scales quadratically, in part because the current MDUP implementation is not parallelised. ConclusionsThe MDUP hypersphere method can define points in a space of any dimension using only a collection of centre points and associated radii, making the results easily interpretable. It can identify large continuous regions, and in many cases capture the general structure of a region in only a relative few hyperspheres. The MDUP method also shows promise for initialising optimisation algorithm starting conditions within pre-defined feasible regions of model parameter spaces, which could improve model identifiability and the quality of optimisation results.

Bayesian Optimization via Exact Penalty

Constrained optimization problems pose challenges when the objective function and constraints are nonconvex and their evaluation requires expensive black-box simulations. Recently, hybrid optimization methods that integrate statistical surrogate modeling with numerical optimization algorithms have shown great promise, as they inherit the properties of global convergence from statistical surrogate modeling and fast local convergence from numerical optimization algorithms. However, the computational efficiency is not satisfied by practical needs under limited budgets and in the presence of equality constraints. In this article, we propose a novel hybrid optimization method, called exact penalty Bayesian optimization (EPBO), which employs Bayesian optimization within the exact penalty framework. We model the composite penalty function by a weighted sum of Gaussian processes, where the qualitative components of the constraint violations are smoothed by their predictive means. The proposed method features (i) closed-form acquisition functions, (ii) robustness to initial designs, (iii) the capability to start from infeasible points, and (iv) effective handling of equality constraints. We demonstrate the superiority of EPBO to state-of-the-art competitors using a suite of benchmark synthetic test problems and two real-world engineering design problems.

An exact algorithm for linear integer programming problems with distributionally robust chance constraints

The distributionally robust chance constrained integer programming problems are notoriously hard to solve due to the stochastic feature and the discrete nature of integer variables. In this paper, an exact solution method is developed for linear integer programming problems with the distributionally robust chance constraint. By exploring the geometric properties, some domain cut techniques based on feasible points and infeasible points are derived. Thus cut off some sub-boxes which do not contain any optimal solution and the optimality gap is reduced successively in the solution iterations. Then the optimal solution will be found in a finite number of iterations. Encouraging computational results are also reported in the paper.

The bifurcation of constrained optimization optimal solutions and its applications

<abstract> <p>The appearance and disappearance of the optimal solution for the change of system parameters in optimization theory is a fundamental problem. This paper aims to address this issue by transforming the solutions of a constrained optimization problem into equilibrium points (EPs) of a dynamical system. The bifurcation of EPs is then used to describe the appearance and disappearance of the optimal solution and saddle point through two classes of bifurcation, namely the pseudo bifurcation and saddle-node bifurcation. Moreover, a new class of pseudo-bifurcation phenomena is introduced to describe the transformation of regular and degenerate EPs, which sheds light on the relationship between the optimal solution and a class of infeasible points. This development also promotes the proposal of a tool for predicting optimal solutions based on this phenomenon. The study finds that the bifurcation of the optimal solution is closely related to the bifurcation of the feasible region, as demonstrated by the 5-bus and 9-bus optimal power flow problems.</p> </abstract>

Feasibility and flexibility regions estimation at TSO–DSO interconnection node using grid structure optimization

Major integration of renewable energy has raised the requirement of frequency and voltage control ancillary services. This necessitates to utilize distributed energy resources for grid flexibility solutions. Aggregated DERs’ flexibility at TSO–DSO node, identified by flexibility operating regions, can provide these services to the grid. Estimating the actual shape of the flexibility region for both slow and fast response systems, is important for real-time applications. However, correct estimation of this region is challenging due to its dependence on various factors, its non-convex shape, and high computational requirements. Proposed novel methodology named ‘Grid Structure Optimization’ can estimate the actual shape of the flexibility region for both slow and fast response systems without any prior shape approximation. The proposed approach is tested on modified IEEE 9 and 33 bus systems considering impacts of Q/P ratio, market dispatch point, cost and type of market. Results highlight the efficacy of this approach for real-time applications as the shape of the region completely avoids the possibility of infeasible points in the region. The results also indicate that the cost willingness is a factor of non-convexity on the feasibility region. Furthermore, the proposed approach is demonstrated to identify DER’s flexibility for contingency-based frequency control ancillary services.

A new projective exact penalty function for a general constrained optimization

A new projective exact penalty function method is proposed for the equivalent reduction of constrained optimization problems to unconstrained ones. In the method, the original objective function is extended to infeasible points by summing its value at the projection of an infeasible point on the feasible set with the distance to the set. The equivalence means that local and global minimums of the problems coincide. Nonconvex sets with multivalued projections are admitted, and the objective function may be lower semicontinuous. The particular case of convex problems is included. So the method does not assume the existence of the objective function outside the allowable area and does not require the selection of the penalty coefficient.

Solver-free heuristics to retrieve feasible points for offshore wind farm collection system

A set of solver-free heuristics for the offshore wind collection system problem is presented. Currently, algorithms of this type are not able to cope with relevant constraints when explicitly minimizing for investment cost. The first algorithm is a two-step decision process of which the output is a tree network satisfying cable thermal limits, but vulnerable to violate planarity constraints. A cable crossings repair heuristic is then introduced in order to fix infeasible points from the first heuristic. Finally, the negative cycle cancelling refining heuristic takes over to improve feasible points. The latter iteratively swaps connections, intending to find cycles with negative costs to reduce investment expenditure. The applicability of the workflow is empirically demonstrated through a set of large-scale real-world offshore farms. Results indicate that: (i) feasible points can be retrieved in a computing time of seconds; and (ii) warm starting can help global optimization solvers to converge significantly faster for hard problems.

Multi-objective optimization path planning with moving target

<div align="left"><span>Path planning or finding a collision-free path for mobile robots between starting position and its destination is a critical problem in robotics. This study is concerned with the multi objective optimization path planning problem of autonomous mobile robots with moving targets in dynamic environment, with three objectives considered: path security, length and smoothness. Three modules are presented in the study. The first module is to combine particle swarm optimization algorithm (PSO) with bat algorithm (BA). The purpose of PSO is to optimize two important parameters of BA algorithm to minimize distance and smooth the path. The second module is to convert the generated infeasible points into feasible ones using a new local search algorithm (LS). The third module obstacle detection and avoidance (ODA) algorithm is proposed to complete the path, which is triggered when the mobile robot detects obstacles in its field of vision. ODA algorithm based on simulating human walking in a dark room. Several simulations with varying scenarios are run to test the validity of the proposed solution. The results show that the mobile robots are able to travel clearly and completely safe with short path, proving the effectiveness of this method.</span><span> </span> </div>

Hybrid Surrogate-Based Constrained Optimization With a New Constraint-Handling Method.

Surrogate-based-constrained optimization for some optimization problems involving computationally expensive objective functions and constraints is still a great challenge in the optimization field. Its difficulties are of two primary types. One is how to handle the constraints, especially, equality constraints; another is how to sample a good point to improve the prediction of the surrogates in the feasible region. Overcoming these difficulties requires a reliable constraint-handling method and an efficient infill-sampling strategy. To perform inequality- and equality-constrained optimization of expensive black-box systems, this work proposes a hybrid surrogate-based-constrained optimization method (HSBCO), and the main innovation is that a new constraint-handling method is proposed to map the feasible region into the origin of the Euclidean subspace. Thus, if the constraint violation of an infeasible solution is large, then it is far from the origin in the Euclidean subspace. Therefore, all constraints of the problem can be transformed into an equivalent equality constraint, and the distance between an infeasible point and the origin in the Euclidean subspace represents the constraint violation of the infeasible solution. Based on the distance, the objective function of the problem can be penalized by a Gaussian penalty function, and the original constrained optimization problem becomes an unconstrained optimization problem. Thus, the feasible solutions of the original minimization problem always have a lower objective function value than any infeasible solution in the penalized objective space. To improve the optimization performance, kriging-based efficient global optimization (EGO) is used to find a locally optimal solution in the first phase of HSBCO, and starting from this locally optimal solution, RBF-model-based global search and local search strategies are introduced to seek global optimal solutions. Such a hybrid optimization strategy can help the optimization process converge to the global optimal solution within a given maximum number of function evaluations, as demonstrated in the experimental results on 23 test problems. The method is shown to achieve the global optimum more closely and efficiently than other leading methods.

Optimal synthesis of four-bar linkages for path generation using the individual repairing method

Abstract. Constraints are one of the main factors that hinder the optimal synthesis of linkages to obtain the global optimum. At present, the most commonly used constraint handling method is called the penalty function (PF) method, but this technique has proven to be inefficient and unreliable in path synthesis. In this paper, a novel and effective constraint handling method called the individual repairing (IR) method is developed for the optimization model of four-bar mechanisms for path synthesis. The function of the IR method is to check the conformity of each individual to the Grashof condition, the sequence condition, and the variable range condition after population initialization and each iteration as well as to repair all unreasonable individuals according to the corresponding repairing technique to make them all satisfy the constraint conditions. In other words, the new constraint handling method establishes a transformation method so that each infeasible point in the search space uniquely corresponds to a feasible point. Based on the IR method, the optimization of path synthesis can be effectively carried out. Finally, the validity and reliability of the new constraint handling method are verified using three examples.

Inclusive Trial Designs in Acute Spinal Cord Injuries: Prediction–Based Stratification of Clinical Walking Outcome and Projected Enrolment Frequencies

Background New therapeutic approaches in neurological disorders are progressing into clinical development. Past failures in translational research have underlined the critical importance of selecting appropriate inclusion criteria and primary outcomes. Narrow inclusion criteria provide sensitivity, but increase trial duration and cost to the point of infeasibility, while broader requirements amplify confounding, increasing the risk of trial failure. This dilemma is perhaps most pronounced in spinal cord injury (SCI), but applies to all neurological disorders with low frequency and/or heterogeneous clinical manifestations. Objective Stratification of homogeneous patient cohorts to enable the design of clinical trials with broad inclusion criteria. Methods Prospectively–gathered data from patients with acute cervical SCI were analysed using an unbiased recursive partitioning conditional inference tree (URP–CTREE) approach. Performance in the 6-minute walk test at 6 months after injury was classified based on standardized neurological assessments within the first 15 days of injury. Functional and neurological outcomes were tracked throughout rehabilitation up to 6 months after injury. Results URP–CTREE identified homogeneous outcome cohorts in a study group of 309 SCI patients. These cohorts were validated by an internal, yet independent, validation group of 172 patients. The study group cohorts identified demonstrated distinct recovery profiles throughout rehabilitation. The baseline characteristics of the analysed groups were compared to a reference group of 477 patients. Conclusion URP–CTREE enables inclusive trial design by revealing the distribution of outcome cohorts, discerning distinct recovery profiles and projecting potential patient enrolment by providing estimates of the relative frequencies of cohorts to improve the design of clinical trials in SCI and beyond.

Safe Bayesian Optimization Using Interior-Point Methods—Applied to Personalized Insulin Dose Guidance

This paper considers the problem of Bayesian optimization for systems with safety-critical constraints, where both the objective function and the constraints are unknown, but can be observed by querying the system. In safety-critical applications, querying the system at an infeasible point can have catastrophic consequences. Such systems require a safe learning framework, such that the performance objective can be optimized while satisfying the safety-critical constraints with high probability. In this paper we propose a safe Bayesian optimization framework that ensures that the points queried are always in the interior of the partially revealed safe region, thereby guaranteeing constraint satisfaction with high probability. The proposed interior-point Bayesian optimization framework can be used with any acquisition function, making it broadly applicable. The performance of the proposed method is demonstrated using a personalized insulin dosing application for patients with type 1 diabetes.

The Stochastic Augmented Lagrangian method for domain adaptation

Among various topics explored in the transfer learning community, domain adaptation (DA) has been of primary interest and successfully applied in diverse fields. However, theoretical understanding of learning convergence in DA has not been sufficiently explored. To address such an issue, this paper presents the Stochastic Augmented Lagrangian method (SALM) to solve the optimization problem associated with domain adaptation. In contrast to previous works, the SALM is able to find the optimal Lagrangian multipliers, as opposed to manually selecting the multipliers which could result in significantly suboptimal solutions. Additionally, the SALM is the first algorithm which can find a feasible point with arbitrary precision for domain adaptation problems with bounded penalty parameters. We also observe that with unbounded penalty parameters, the proposed algorithm is able to find an approximate stationary point of infeasibility. We validate our theoretical analysis with several experimental results using benchmark data sets including MNIST, SYNTH, SVHN, and USPS.

Comparative Study of Pseudospectral Methods for Spacecraft Optimal Attitude Maneuvers

This paper presents a comparative study of six common pseudospectral (PS) methods for solving optimal spacecraft attitude control problems. The problems of minimum time, minimum control effort, and the linear combination thereof are considered here, for a rest-to-rest spacecraft slew maneuver. In this study, three traditional PS methods (Gauss, Legendre, and Chebyshev) and three variations of the novel Birkhoff PS method (Legendre–Gauss, Legendre–Gauss–Lobatto, and Chebyshev–Gauss–Lobatto) are used. The Wilcoxon rank sum test is used to perform statistical analysis to compare results between traditional and Birkhoff PS results. Accuracy, computational efficiency, and robustness to initial conditions are used as performance metrics for each PS method. The results show that for all three optimal control problems, the Birkhoff Legendre–Gauss method is more accurate than the traditional Gauss PS method. Furthermore, the results show that the Birkhoff variants could in some cases find the minimum cost using fewer iterations compared with traditional PS methods. The Birkhoff PS methods also are less susceptible to initial guesses for solutions that could cause the results to reach infeasible points or become trapped in local minimums.

Meta-heuristic approach for solving multi-objective path planning for autonomous guided robot using PSO–GWO optimization algorithm with evolutionary programming

As path planning is an NP-hard problem it can be solved by multi-objective algorithms. In this article, we propose a multi-objective path planning algorithm which consists of three steps: (1) the first step consists of optimizing a path by the hybridization of the Grey Wolf optimizer-particle swarm optimization algorithm, it minimizes the path distance and smooths the path. (2) the second step, all optimal and feasible points generated by PSO–GWO algorithm are integrated with Local Search technique to convert any infeasible point into feasible point solution, the last step (3) depends on collision avoidance and detection algorithm, where mobile robot detects the presence of an obstacle in its sensing circle and then avoid them using collision avoidance algorithm. The proposed method is further improved by adding the mutation operators by evolutionary, it further solves path safety, length, and smooths it further for a mobile robot. Different simulations have been performed under numerous environments to test the feasibility of the proposed algorithm and it is shown the algorithm produces a more feasible path with a short distance and thus proves that it overcomes the shortcomings of other conventional techniques.

Recursive Modified Pattern Search on High-Dimensional Simplex : A Blackbox Optimization Technique

In this paper, a novel derivative-free pattern search based algorithm for Black-box optimization is proposed over a simplex constrained parameter space. At each iteration, starting from the current solution, new possible set of solutions are found by adding a set of derived step-size vectors to the initial starting point. While deriving these step-size vectors, precautions and adjustments are considered so that the set of new possible solution points still remain within the simplex constrained space. Thus, no extra time is spent in evaluating the (possibly expensive) objective function at infeasible points (points outside the unit-simplex space); which being the primary motivation of designing a customized optimization algorithm specifically when the parameters belong to a unit-simplex. While minimizing any objective function of m parameters, within each iteration, the objective function is evaluated at 2m new possible solution points. So, upto 2m parallel threads can be incorporated which makes the computation even faster while optimizing expensive objective functions over high-dimensional parameter space. Once a local minimum is discovered, in order to find a better solution, a novel ‘re-start’ strategy is considered to increase the likelihood of finding a better solution. Unlike existing pattern search based methods, a sparsity control parameter is introduced which can be used to induce sparsity in the solution in case the solution is expected to be sparse in prior. A comparative study of the performances of the proposed algorithm and other existing algorithms are shown for a few low, moderate and high-dimensional optimization problems. Upto 338 folds improvement in computation time is achieved using the proposed algorithm over Genetic algorithm along with better solution. The proposed algorithm is used to estimate the simultaneous quantiles of North Atlantic Hurricane velocities during 1981–2006 by maximizing a non-closed form likelihood function with (possibly) multiple maximums.

Handling of Constraints in Efficient Global Optimization

Although the Efficient Global Optimization (EGO) algorithm has been widely used in multi-disciplinary optimization, it is still difficult to handle multiple constraint problems. In this study, to increase the accuracy of approximation, the Least Squares Support Vector Regression (LSSVR) is suggested to replace the kriging model for approximating both objective and constrained functions while the variances of these surrogate models are still obtained by kriging. To enhance the ability to search the feasible region, two criteria are suggested. First, a Maximize Probability of Feasibility (MPF) strategy to handle the infeasible initial sample points is suggested to generate feasible points. Second, a Multi-Constraint Parallel (MCP) criterion is suggested for multiple constraints handling, parallel computation and validation, respectively. To illustrate the efficiency of the suggested EGO-based method, several deterministic benchmarks are tested and the suggested methods demonstrate a superior performance compared with two other constrained algorithms. Finally, the suggested algorithm is successfully utilized to optimize the fiber path of variable-stiffness beam and lightweight B-pillar to demonstrate the performance for engineering applications.

On the complexity of solving feasibility problems with regularized models

The complexity of solving feasibility problems is considered in this work. It is assumed that the constraints that define the problem can be divided into expensive and cheap constraints. At each iteration, the introduced method minimizes a regularized pth-order model of the sum of squares of the expensive constraints subject to the cheap constraints. Under a Hölder continuity property with constant on the pth derivatives of the expensive constraints, it is shown that finding a feasible point with precision or an infeasible point that is stationary with tolerance of minimizing the sum of squares of the expensive constraints subject to the cheap constraints has iteration complexity and evaluation complexity (of the expensive constraints) , where and if p = 1, while if p>1. Moreover, if the derivatives satisfy a Lipschitz condition and a uniform regularity assumption holds, both complexities reduce to , independently of p.

A perturbation view of level-set methods for convex optimization

Level-set methods for convex optimization are predicated on the idea that certain problems can be parameterized so that their solutions can be recovered as the limiting process of a root-finding procedure. This idea emerges time and again across a range of algorithms for convex problems. Here we demonstrate that strong duality is a necessary condition for the level-set approach to succeed. In the absence of strong duality, the level-set method identifies epsilon -infeasible points that do not converge to a feasible point as epsilon tends to zero. The level-set approach is also used as a proof technique for establishing sufficient conditions for strong duality that are different from Slater’s constraint qualification.