Infeasible Space Research Articles

Evolution-guided Bayesian optimization for constrained multi-objective optimization in self-driving labs

The development of automated high-throughput experimental platforms has enabled fast sampling of high-dimensional decision spaces. To reach target properties efficiently, these platforms are increasingly paired with intelligent experimental design. However, current optimizers show limitations in maintaining sufficient exploration/exploitation balance for problems dealing with multiple conflicting objectives and complex constraints. Here, we devise an Evolution-Guided Bayesian Optimization (EGBO) algorithm that integrates selection pressure in parallel with a q-Noisy Expected Hypervolume Improvement (qNEHVI) optimizer; this not only solves for the Pareto Front (PF) efficiently but also achieves better coverage of the PF while limiting sampling in the infeasible space. The algorithm is developed together with a custom self-driving lab for seed-mediated silver nanoparticle synthesis, targeting 3 objectives (1) optical properties, (2) fast reaction, and (3) minimal seed usage alongside complex constraints. We demonstrate that, with appropriate constraint handling, EGBO performance improves upon state-of-the-art qNEHVI. Furthermore, across various synthetic multi-objective problems, EGBO shows significative hypervolume improvement, revealing the synergy between selection pressure and the qNEHVI optimizer. We also demonstrate EGBO’s good coverage of the PF as well as comparatively better ability to propose feasible solutions. We thus propose EGBO as a general framework for efficiently solving constrained multi-objective problems in high-throughput experimentation platforms.

Preserving multi-dimensional information: A hypersphere method for parameter space analysis

BackgroundPhysiological modelling often involves models described by large numbers of variables and significant volumes of clinical data. Mathematical interpretation of such models frequently necessitates analysing data points in high-dimensional spaces. Existing algorithms for analysing high-dimensional points either lose important dimensionality or do not describe the full position of points. Hence, there is a need for an algorithm which preserves this information. MethodsThe most-distant uncovered point (MDUP) hypersphere method is a binary classification approach which defines a collection of equidistant N-dimensional points as the union of hyperspheres. The method iteratively generates hyperspheres at the most distant point in the interest region not yet contained within any hypersphere, until the entire region of interest is defined by the union of all generated hyperspheres. This method is tested on a 7-dimensional space with up to 35.8 million points representing feasible and infeasible spaces of model parameters for a clinically validated cardiovascular system model. ResultsFor different numbers of input points, the MDUP hypersphere method tends to generate large spheres away from the boundary of feasible and infeasible points, but generates the greatest number of relatively much smaller spheres around the boundary of the region of interest to fill this space. Runtime scales quadratically, in part because the current MDUP implementation is not parallelised. ConclusionsThe MDUP hypersphere method can define points in a space of any dimension using only a collection of centre points and associated radii, making the results easily interpretable. It can identify large continuous regions, and in many cases capture the general structure of a region in only a relative few hyperspheres. The MDUP method also shows promise for initialising optimisation algorithm starting conditions within pre-defined feasible regions of model parameter spaces, which could improve model identifiability and the quality of optimisation results.

Feasibility/Flexibility-based optimization for process design and operations

This paper provides an overview of concepts and computational approaches for the evaluation of feasibility/flexibility and how they can be used for process design and process operations optimization. It emphasizes more recent topics in this area, in particular feasible region evaluation, feasibility-based optimization, and optimization with flexibility requirements. The description of process feasibility and the feasibility-based optimization problem are presented as a way to efficiently incorporate multiple constraints and avoid unnecessary exploration of the infeasible space in the black-box optimization context. The relationship between flexibility analysis and robust optimization is also highlighted, and opportunities in exploring synergies therein are outlined. Applications in pharmaceutical design and process scheduling are used to provide context in the utilization of the presented approaches.

Data preprocessing strategy in constructing convolutional neural network classifier based on constrained particle swarm optimization with fuzzy penalty function

Convolutional neural networks (CNNs) have attracted increasing attention in recent years because of their powerful abilities to extract and represent spatial/temporal information. However, for general data, its features are assumed to have weak or no correlation, and directly applying CNN to classify such data could result in poor classification performance. To address this problem, a combined technique of original data representation method of fuzzy penalty function-based constrained particle swarm optimization (FCPSO) and CNN, so-called FCPSO-CNN is designed to effectively solve the classification problems for generic dataset and applied to recognize (classify) black plastic wastes in recycling problems. In more detail, CPSO is introduced to optimize feature reordering matrix under constraints and the construction of this matrix is driven by fitness function of CNN that quantifies classification performance. The Mamdani type fuzzy inference system (FIS) is employed to realize the fuzzy penalty function (FPF) which is utilized to realize the constrained problems of CPSO as well as alleviate the issues of the original penalty function method suffering from the lack of robustness. Experimental results demonstrate that FCPSO-CNN achieves the best classification accuracy on 13 out of 17 datasets; the statistical analysis also confirms the superiority of FCPSO-CNN. An interesting point is worth to mention that some feature reordering matrices in the infeasible space come with better classification accuracy. It has been found that the proposed method results in more accurate solution than one-dimensional CNN, random reordering feature-based CNN and some well-known classifiers (e.g., Naive Bayes, Multilayer perceptron, Support vector machine).

An infeasible space exploring matheuristic for the Production Routing Problem

This paper proposes a novel matheuristic algorithm for solving the Production Routing Problem (PRP). The PRP is a hard-to-solve combinatorial optimization problem with numerous practical applications in the field of freight transportation, logistics and supply chain management. A manufacturer is responsible for determining production decisions, as well as the timing and the quantity of replenishment services offered to a set of geographically dispersed customers over a multi-period time horizon. The problem calls for jointly optimizing the production, inventory, distribution and routing decisions. The paper provides a novel two-commodity flow formulation and proposes a two-phase infeasible space matheuristic algorithm for solving the examined problem. The first phase deals with a relaxation of the problem to construct production-distribution plans. In the second phase, these are completed with routing information and optimized via a local search framework which oscillates between the feasible and the infeasible solution space. The framework is equipped with mixed integer programming components for restoring infeasibility and for diversifying the conducted search. Computational experiments demonstrate that the infeasibility space exploration significantly contributes to the quality of the final solutions. The results obtained by the proposed matheuristic out-match the results of state-of-the-art approaches. More specifically, 594 and 55 new best solutions out of 1440 and 90 instances of two well-established benchmark data sets of small-medium and large instances are reported, respectively.

From Multimodality to Monomodality: Head and Neck MR-CT Registration with Synthetic Image Bridge

To develop and demonstrate the efficacy of a novel head-and-neck multimodality image registration technique using generative adversarial networks, and show its improvement over traditional direct multimodal registration. 25 head-and-neck patients received MR and CT (CTaligned) scans on the same day with the same immobilization. 19 of the MR-CT pairs were used to train a neural network to generate synthetic CTs from MR images while the registration testing was performed on the remaining 6 patients, who also had a separate CT without immobilization (CTnon-aligned). The CTnon-aligned images exhibited large neck motion. CTnon-aligned’s were deformed to the synthetic CT using b-splines and mutual information, thus effectively becoming a monomodal registration. These results were compared to CTnon-aligned directly registered to MR (multimodal). The same registrations were performed in the opposite direction. Results were evaluated using the mean distance to agreement (MDA) among spinal cord contours, landmark error, and Jacobian determinant of the estimated deformation fields. The cord contours are better matched when a synthetic CT replaces the MR, compared to a direct deformable registration with the MR. At larger initial misalignment (2-15mm MDA), the synthetic CT decreases the MDA by 1 - 10mm and 0.4 - 3mm for MR→CT and CT→MR directions, respectively. The average landmark error decreased from 10.95mm to 6.22mm when going from an MR→CTnon-aligned to a CTsynth→CTnon-aligned deformable registration. In the CT to MR direction, the average landmark error decreased from 8.19mm to 6.71mm when going from a CTnon-aligned→MR to a CTnon-aligned→CTsynth deformable registration. The proposed method demonstrated inverse consistency for both the contour and landmark analysis, while the direct method does not. Additionally, the Jacobian determinant showed feasible expansion and shrinking, with no infeasible space folding (negative values). We showed that using a synthetic CT in lieu of an MR for MR→CT and CT→MR deformable registration offers superior results to direct registration.

Embedding differential privacy in decision tree algorithm with different depths

Differential privacy (DP) has become one of the most important solutions for privacy protection in recent years. Previous studies have shown that prediction accuracy usually increases as more data mining (DM) logic is considered in the DP implementation. However, although one-step DM computation for decision tree (DT) model has been investigated, existing research has not studied the scenarios when the DP is embedded in two-step DM computation, three-step DM computation until the whole model DM computation. It is very challenging to embed DP in more than two steps of DM computation since the solution space exponentially increases with the increase of computational complexity. In this work, we propose algorithms by making use of Markov Chain Monte Carlo (MCMC) method, which can efficiently search a computationally infeasible space to embed DP into DT generation algorithm. We compare the performance when embedding DP in DT with different depths, i.e., one-step DM computation (previous work), two-step, three-step and the whole model. We find that the deep combination of DP and DT does help to increase the prediction accuracy. However, when the privacy budget is very large (e.g., ϵ = 10), this may overwhelm the complexity of DT model, and the increasing trend is not obvious. We also find that the prediction accuracy decreases with the increase of model complexity.

A Multifaceted Approach for Safety Design and Probabilistic Optimization

Safety design and probabilistic optimization are fields that are widely subject to uncertainty, thus making traditional deterministic methods highly unreliable for these two fields. Popular design optimizations methods widely used for safety design and probabilistic optimization are the performance measure approach (PMA) and the performance measure approach (RIA). In a problem where the analysis is performed from an infeasible design space, a modified reliability index approach (MRIA) is employed to address some inefficiency of the traditional RIA to be able to find the optimal solutions. The PMA uses an inverse reliability analysis, which is more computationally efficient at finding the most probable design points but has been reported to have numerical instabilities on some cases. In this paper, three benchmark examples were thoroughly studied with various initial points to examine the stability and efficiency of MRIA and PMA. A hybrid reliability approach (HRA) was then presented after determining a selection factor from the optimum conditions. The proposed HRA aims to determine which of the two optimization methods would be more appropriate during the optimization processes.

Constrained Novelty Search: A Study on Game Content Generation

Novelty search is a recent algorithm geared toward exploring search spaces without regard to objectives. When the presence of constraints divides a search space into feasible space and infeasible space, interesting implications arise regarding how novelty search explores such spaces. This paper elaborates on the problem of constrained novelty search and proposes two novelty search algorithms which search within both the feasible and the infeasible space. Inspired by the FI-2pop genetic algorithm, both algorithms maintain and evolve two separate populations, one with feasible and one with infeasible individuals, while each population can use its own selection method. The proposed algorithms are applied to the problem of generating diverse but playable game levels, which is representative of the larger problem of procedural game content generation. Results show that the two-population constrained novelty search methods can create, under certain conditions, larger and more diverse sets of feasible game levels than current methods of novelty search, whether constrained or unconstrained. However, the best algorithm is contingent on the particularities of the search space and the genetic operators used. Additionally, the proposed enhancement of offspring boosting is shown to enhance performance in all cases of two-population novelty search.

A hybrid multi-population genetic algorithm for the dynamic facility layout problem

Due to inherent complexity of the dynamic facility layout problem, it has always been a challenging issue to develop a solution algorithm for this problem. For more than one decade, many researchers have proposed different algorithms for this problem. After reviewing the shortcomings of these algorithms, we realize that the performance can be further improved by a more intelligent search. This paper develops an effective novel hybrid multi-population genetic algorithm. Using a proposed heuristic procedure, we separate solution space into different parts and each subpopulation represents a separate part. This assures the diversity of the algorithm. Moreover, to intensify the search more and more, a powerful local search mechanism based on simulated annealing is developed. Unlike the available genetic operators previously proposed for this problem, we design the operators so as to search only the feasible space; thus, we save computational time by avoiding infeasible space. To evaluate the algorithm, we comprehensively discuss the parameter tuning of the algorithms by Taguchi method. The perfectly tuned algorithm is then compared with 11 available algorithms in the literature using well-known set of benchmark instances. Different analyses conducted on the results, show that the proposed algorithm enjoys the superiority and outperformance over the other algorithms.

A Novel Hybrid Algorithm for Constrained Multi-objective Optimization

A new hybrid Optimization Algorithm is proposed to solve Constrained Multi-objective Optimization Problems (CMOPs). The algorithm is named BBO/DE which combines the exploitation ability of Biogeography-based Optimization (BBO) and the exploration ability of Differential Evolution (DE). Meanwhile distance measures and adaptive penalty functions are adopted to handle the constraints so that optimal solutions in the infeasible space can be searched effectively. In addition, the feasible archive is applied to store the non-dominated feasible solutions obtained so far and is updated based on crowding-distance. Experiment results demonstrate that the proposed hybrid algorithm BBO/DE can approximate the true Pareto front and has better distribution.

Robust classifier learning with fuzzy class labels for large-margin support vector machines

Using class label fuzzification, this study develops the idea of refreshing the attitude of the difficult training examples and gaining a more robust classifier for large-margin support vector machines (SVMs). Fuzzification relaxes the specific hard-limited Lagrangian constraints of the difficult examples, extends the infeasible space of the canonical constraints for optimization, and reconfigures the consequent decision function with a wider margin. With the margin, a classifier capable of achieving a high generalization performance can be more robust. This paper traces the rationale for such a robust performance back to the changes of governing loss function. From the aspect of loss function, the reasons are causally explained. In the study, we also demonstrate a two-stage system for experiments to show the changes corresponding to the label fuzzification. The system first captures the difficult examples in the first-stage preprocessor, and assigns them various fuzzified class labels. Three types of membership functions, including a constant, a linear, and a sigmoidal membership function, are designated in the preprocessor to manipulate the within-class correlations of the difficult examples for reference of the fuzzification. The consequent performance benchmarks confirm the robust and generalized ability due to the label fuzzification. Since the change of yi′ is fundamental, the idea may be transplanted to different prototypes of SVM.

The case for strategic oscillation

We study a “hard” optimization problem for metaheuristic search, where a natural neighborhood (that consists of moves for flipping the values of zero-one variables) confronts two local optima, separated by a maximum possible number of moves in the feasible space. Once a descent method reaches the first local optimum, all sequences of feasible moves to reach the second, which is the global optimum, must ultimately pass through solutions that are progressively worse until reaching the worst solution of all, which is adjacent to the global optimum. We show how certain alternative neighborhoods can locate the global more readily, but disclose that each of these approaches encounters serious difficulties by slightly changing the problem formulation. We also identify other possible approaches that seem at first to be promising but turn out to have deficiencies. Finally, we observe that a strategic oscillation approach for transitioning between feasible and infeasible space overcomes these difficulties, reinforcing recent published observations about the utility of solution trajectories that alternate between feasibility and infeasibility. We also sketch features of such an approach that have implications for future research.

Constraint Handling in Multiobjective Evolutionary Optimization

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> This paper proposes a constraint handling technique for multiobjective evolutionary algorithms based on an adaptive penalty function and a distance measure. These two functions vary dependent upon the objective function value and the sum of constraint violations of an individual. Through this design, the objective space is modified to account for the performance and constraint violation of each individual. The modified objective functions are used in the nondominance sorting to facilitate the search of optimal solutions not only in the feasible space but also in the infeasible regions. The search in the infeasible space is designed to exploit those individuals with better objective values and lower constraint violations. The number of feasible individuals in the population is used to guide the search process either toward finding more feasible solutions or favor in search for optimal solutions. The proposed method is simple to implement and does not need any parameter tuning. The constraint handling technique is tested on several constrained multiobjective optimization problems and has shown superior results compared to some chosen state-of-the-art designs. </para>

Heuristic Algorithm with Oscillation Strategy for a New Class of Assignment Problem

Abstract A new class of assignment problem which roots in the optimization management of slabs in steel industry is considered in this article. Compared with the generalized assignment problem, flow constraints should be considered in this problem besides the capacity constraints when assigning items to knapsacks. This problem could be reduced to the generalized assignment problem, and so is NP-hard. A heuristic with oscillation strategy and long-term memory list is proposed to solve it. The oscillation strategy makes it possible that the local search oscillates between the feasible and infeasible solution spaces to find better feasible solutions. A long-term memory list is embedded to encourage the diverse moves of items, which improves the diversity of the algorithm. In order to testify the efficiency of the heuristic, 23 instances have been randomly generated for the computational experiments. The results show that for small-size instances, the maximum deviation of the heuristic from the optimal solution is 0.55% and for larger-size instances, the heuristic could find good solutions in a very short time.

Infeasible/feasible search trajectories and directional rounding in integer programming

The notion that strategies in non-linear and combinatorial optimization can benefit by purposefully and systematically navigating between feasible and infeasible space has been around for many years, but still is sometimes dismissed as having little relevance for creating more effective methods. To reinforce the case on behalf of approaches that endorse infeasible/feasible search trajectories, it is possible to formulate simple theorems disclosing useful properties of such trajectories in the context of integer programming. These results motivate a closer examination of integer programming search processes based on directional rounding processes, including a special variant called conditional directional rounding. From these foundations a variety of new strategies emerge for exploiting connections between feasible and infeasible space.

Optimizing cane supply decisions within a sugar mill region

A sugar mill region contains large differences in sugar yield due to harvest date, harvest age of crop, geographical location and crop class. While it is desirable to harvest all cane when the likely sugar yields are at the season's peak, this is not possible due to limited capacities of harvesting, mill crushing and transport. The harvesting of cane is therefore carried out over several months. This paper focuses on the development of a large-scale integer programming model to optimize the decisions of harvest date, crop cycle length, and whether to fallow for all paddocks within a mill region. Net revenue for the mill region is maximized over a planning horizon of several years. The model, which is an extension of the generalized assignment problem with over 500 000 integer variables, is subject to the constraints of total assigned land, equity, mill crushing and geographical transport capacity. This large model is solved heuristically using a new form of local search that incorporates some of the ideas of tabu search (but there is no tabu list). It applies oscillation through feasible versus infeasible space, that is controlled by the success of the search throughout the solution process. Application to five large Australian mill regions through strong industry participation resulted in up to a 7 per cent increase in net revenue. Copyright © 1999 John Wiley & Sons, Ltd.