Auxiliary Objective Function Research Articles

A meta-heuristic extension of the Lagrangian heuristic framework

Lagrangian heuristics for discrete optimization work by modifying Lagrangian relaxed solutions into feasible solutions to an original problem. They are designed to identify feasible, and hopefully also near-optimal, solutions and have proven to be highly successful in many applications. Based on a primal-dual global optimality condition for non-convex optimization problems, we develop a meta-heuristic extension of the Lagrangian heuristic framework. The optimality condition characterizes (near-)optimal solutions in terms of near-optimality and near-complementarity measures for Lagrangian relaxed solutions. The meta-heuristic extension amounts to constructing a weighted combination of these measures, thus creating a parametric auxiliary objective function, which is a close relative to a Lagrangian function, and embedding a Lagrangian heuristic in a search procedure in the space of the weight parameters. We illustrate and make a first assessment of this meta-heuristic extension by applying it to the generalized assignment and set covering problems. Our computational experience show that the meta-heuristic extension of a standard Lagrangian heuristic can significantly improve upon solution quality.

A numerical approach to sequential multi-hypothesis testing for Bernoulli model

In this article, we deal with the problem of sequential testing of multiple hypotheses. The main goal is minimizing the expected sample size (ESS) under restrictions on the error probabilities. We take, as a criterion of minimization, a weighted sum of the ESSs evaluated at some points of interest in the parameter space, aiming at its minimization under restrictions on the error probabilities. We use a variant of the method of Lagrange multipliers based on the minimization of an auxiliary objective function (called Lagrangian) combining the objective function with the restrictions, taken with some constants called multipliers. Subsequently, the multipliers are used to make the solution comply with the restrictions. We develop a computer-oriented method of minimization of the Lagrangian function that provides, depending on the specific choice of the parameter points, optimal tests in different concrete settings, as in Bayesian, Kiefer-Weiss, and other settings. To exemplify the proposed methods for the particular case of sampling from a Bernoulli population, we develop a set of computer algorithms for designing sequential tests that minimize the Lagrangian function and for the numerical evaluation of test characteristics like the error probabilities and the ESS. We implement the algorithms in the R programming language. The program code is available in a public GitHub repository. For the Bernoulli model, we made a series of computer evaluations related to the optimality of sequential multi-hypothesis tests, in a particular case of three hypotheses. A numerical comparison with the matrix sequential probability ratio test is carried out. A method of solution of the multi-hypothesis Kiefer-Weiss problem is proposed and is applied for a particular case of three hypotheses in the Bernoulli model.

Automatic Prostate Gleason Grading Using Pyramid Semantic Parsing Network in Digital Histopathology.

PurposeProstate biopsy histopathology and immunohistochemistry are important in the differential diagnosis of the disease and can be used to assess the degree of prostate cancer differentiation. Today, prostate biopsy is increasing the demand for experienced uropathologists, which puts a lot of pressure on pathologists. In addition, the grades of different observations had an indicating effect on the treatment of the patients with cancer, but the grades were highly changeable, and excessive treatment and insufficient treatment often occurred. To alleviate these problems, an artificial intelligence system with clinically acceptable prostate cancer detection and Gleason grade accuracy was developed.MethodsDeep learning algorithms have been proved to outperform other algorithms in the analysis of large data and show great potential with respect to the analysis of pathological sections. Inspired by the classical semantic segmentation network, we propose a pyramid semantic parsing network (PSPNet) for automatic prostate Gleason grading. To boost the segmentation performance, we get an auxiliary prediction output, which is mainly the optimization of auxiliary objective function in the process of network training. The network not only includes effective global prior representations but also achieves good results in tissue micro-array (TMA) image segmentation.ResultsOur method is validated using 321 biopsies from the Vancouver Prostate Centre and ranks the first on the MICCAI 2019 prostate segmentation and classification benchmark and the Vancouver Prostate Centre data. To prove the reliability of the proposed method, we also conduct an experiment to test the consistency with the diagnosis of pathologists. It demonstrates that the well-designed method in our study can achieve good results. The experiment also focused on the distinction between high-risk cancer (Gleason pattern 4, 5) and low-risk cancer (Gleason pattern 3). Our proposed method also achieves the best performance with respect to various evaluation metrics for distinguishing benign from malignant.AvailabilityThe Python source code of the proposed method is publicly available at https://github.com/hubutui/Gleason. All implementation details are presented in this paper.ConclusionThese works prove that the Gleason grading results obtained from our method are effective and accurate.

Selection of Sensors for Efficient Transmitter Localization

We address the problem of localizing an (unauthorized) transmitter using a distributed set of sensors. Our focus is on developing techniques that perform the transmitter localization in an efficient manner, wherein the efficiency is defined in terms of the number of sensors used to localize. Localization of unauthorized transmitters is an important problem which arises in many important applications, e.g., in patrolling of shared spectrum systems for any unauthorized users. Localization of transmitters is generally done based on observations from a deployed set of sensors with limited resources, thus it is imperative to design techniques that minimize the sensors’ energy resources. In this paper, we design greedy approximation algorithms for the optimization problem of selecting a given number of sensors in order to maximize an appropriately defined objective function of localization accuracy. The obvious greedy algorithm delivers a constant-factor approximation only for the special case of two hypotheses (potential locations). For the general case of multiple hypotheses, we design a greedy algorithm based on an appropriate auxiliary objective function—and show that it delivers a provably approximate solution for the general case. We develop techniques to significantly reduce the time complexity of the designed algorithms by incorporating certain observations and reasonable assumptions. We evaluate our techniques over multiple simulation platforms, including an indoor as well as an outdoor testbed, and demonstrate the effectiveness of our designed techniques—our techniques easily outperform prior and other approaches by up to 50-60% in large-scale simulations and up to 16% in small-scale testbeds.

Constraint based local search for flowshops with sequence-dependent setup times

Permutation flowshop scheduling problem with sequence-dependent setup times (PFSP-SDST) and makespan minimisation is NP-hard. It has important practical applications, for example, in the cider industry and the print industry. There exist several metaheuristic algorithms to solve this problem. However, within practical time limits, those algorithms still either find low quality solutions or struggle with large problems. In this paper, we have proposed a simple but effective local search algorithm, called constraint based local search (CBLS) algorithm, which transforms the SDST constraints into an auxiliary objective function and uses the auxiliary objective function to guide the search towards the optimal value of the actual objective function. Our motivation comes from the constraint optimisation models in artificial intelligence (AI), where constraint-based informed decisions are of particular interest instead of random-based decisions. Our experimental results on well-known 480 instances of PFSP-SDST show that the proposed CBLS algorithm outperforms existing state-of-the-art PFSP-SDST algorithms. Moreover, our algorithm obtains new upper bounds for 204 out of 360 medium- and large-sized problem instances.

Reducing Objective Function Mismatch in Deep Clustering with the Unsupervised Companion Objective

Preservation of local similarity structure is a key challenge in deep clustering. Many recent deep clustering methods therefore use autoencoders to help guide the model's neural network towards an embedding which is more reflective of the input space geometry. However, recent work has shown that autoencoder-based deep clustering models can suffer from objective function mismatch (OFM). In order to improve the preservation of local similarity structure, while simultaneously having a low OFM, we develop a new auxiliary objective function for deep clustering. Our Unsupervised Companion Objective (UCO) encourages a consistent clustering structure at intermediate layers in the network -- helping the network learn an embedding which is more reflective of the similarity structure in the input space. Since a clustering-based auxiliary objective has the same goal as the main clustering objective, it is less prone to introduce objective function mismatch between itself and the main objective. Our experiments show that attaching the UCO to a deep clustering model improves the performance of the model, and exhibits a lower OFM, compared to an analogous autoencoder-based model.

A Novel Multisupervised Coupled Metric Learning for Low-Resolution Face Matching

This paper presents a new multisupervised coupled metric learning (MS-CML) method for low-resolution face image matching. While coupled metric learning has achieved good performance in degraded face recognition, most existing coupled metric learning methods only adopt the category label as supervision, which easily leads to changes in the distribution of samples in the coupled space. And the accuracy of degraded image matching is seriously influenced by these changes. To address this problem, we propose an MS-CML method to train the linear and nonlinear metric model, respectively, which can project the different resolution face pairs into the same latent feature space, under which the distance of each positive pair is reduced and that of each negative pair is enlarged. In this work, we defined a novel multisupervised objective function, which consists of a main objective function and an auxiliary objective function. The supervised information of the main objective function is the category label, which plays a major supervisory role. The supervised information of the auxiliary objective function is the distribution relationship of the samples, which plays an auxiliary supervisory role. Under the supervision of category label and distribution information, the learned model can better deal with the intraclass multimodal problem, and the features obtained in the coupled space are more easily matched correctly. Experimental results on three different face datasets validate the efficacy of the proposed method.

Adjoint-based well placement optimisation for Enhanced Oil Recovery (EOR) under geological uncertainty: From seismic to production

The development of new and existing oil fields is very expensive and technically challenging; this provides an important economic incentive to improve oil recovery via efficient computational optimisation. Decisions related to these activities may be significantly aided by sound and proven mathematical-oriented methods. The use of intuitive engineering judgement alone cannot guarantee sustainable profitability over long periods, especially under geological (reservoir model) uncertainty. To capture the uncertainties in the subsurface geological/reservoir model in this work, geostatistical realisations of the model are obtained using available information (permeabilities and porosities). We also apply specialised algorithms within the MATLAB Reservoir Simulation Toolbox – MRST (interfaced with PETREL™) to optimally vary the well locations and production rates, thus maximising the field's oil recovery. A new modification of an existing pseudowell-based injection well placement algorithm is presented herein with the application of adjoint-computed gradients of an auxiliary objective function (the Lorenz coefficient). The difficulty of this problem is characterised by the presence of discrete variables, a nonlinear and nonconvex objective function and constraints. The developed workflow is applied to a realistic case study, for which robust optimality is demonstrated using the worst case realisation for the determination of optimal well locations and controls. A comparative investigation of optimising injection well location vs. simultaneous optimisation of injection and production well placement is also presented. In our presented case studies, we also discover that increasing the optimisation search space does not necessarily guarantee improved results.

Testing Scenario Library Generation for Connected and Automated Vehicles, Part I: Methodology

Testing and evaluation is a critical step in the development and deployment of connected and automated vehicles (CAVs), and yet there is no systematic framework to generate testing scenario library. This study aims to provide a general framework for the testing scenario library generation (TSLG) problem with different operational design domains (ODDs), CAV models, and performance metrics. Given an ODD, the testing scenario library is defined as a critical set of scenarios that can be used for CAV test. Each testing scenario is evaluated by a newly proposed measure, scenario criticality, which can be computed as a combination of maneuver challenge and exposure frequency. To search for critical scenarios, an auxiliary objective function is designed, and a multi-start optimization method along with seed-filling is applied. The proposed framework is theoretically proved to obtain accurate evaluation results with much fewer number of tests, if compared with the on-road test method. In part II of the study, three case studies are investigated to demonstrate the proposed methodologies. Reinforcement learning based technique is applied to enhance the searching method under high-dimensional scenarios.

Alternately Updated Spectral–Spatial Convolution Network for the Classification of Hyperspectral Images

The connection structure in the convolutional layers of most deep learning-based algorithms used for the classification of hyperspectral images (HSIs) has typically been in the forward direction. In this study, an end-to-end alternately updated spectral–spatial convolutional network (AUSSC) with a recurrent feedback structure is used to learn refined spectral and spatial features for HSI classification. The proposed AUSSC includes alternating updated blocks in which each layer serves as both an input and an output for the other layers. The AUSSC can refine spectral and spatial features many times under fixed parameters. A center loss function is introduced as an auxiliary objective function to improve the discrimination of features acquired by the model. Additionally, the AUSSC utilizes smaller convolutional kernels than other convolutional neural network (CNN)-based methods to reduce the number of parameters and alleviate overfitting. The proposed method was implemented on four HSI data sets, as follows: Indian Pines, Kennedy Space Center, Salinas Scene, and Houston. Experimental results demonstrated that the proposed AUSSC outperformed the HSI classification accuracy obtained by state-of-the-art deep learning-based methods with a small number of training samples.

An Improved Multi-parametric Programming Algorithm for Flux Balance Analysis of Metabolic Networks

Flux balance analysis has proven an effective tool for analyzing metabolic networks. In flux balance analysis, reaction rates and optimal pathways are ascertained by solving a linear program, in which the growth rate is maximized subject to mass-balance constraints. A variety of cell functions in response to environmental stimuli can be quantified using flux balance analysis by parameterizing the linear program with respect to extracellular conditions. However, for most large, genome-scale metabolic networks of practical interest, the resulting parametric problem has multiple and highly degenerate optimal solutions, which are computationally challenging to handle. An improved multi-parametric programming algorithm based on active-set methods is introduced in this paper to overcome these computational difficulties. Degeneracy and multiplicity are handled, respectively, by introducing generalized inverses and auxiliary objective functions into the formulation of the optimality conditions. These improvements are especially effective for metabolic networks because their stoichiometry matrices are generally sparse; thus, fast and efficient algorithms from sparse linear algebra can be leveraged to compute generalized inverses and null-space bases. We illustrate the application of our algorithm to flux balance analysis of metabolic networks by studying a reduced metabolic model of Corynebacterium glutamicum and a genome-scale model of Escherichia coli. We then demonstrate how the critical regions resulting from these studies can be associated with optimal metabolic modes and discuss the physical relevance of optimal pathways arising from various auxiliary objective functions. Achieving more than fivefold improvement in computational speed over existing multi-parametric programming tools, the proposed algorithm proves promising in handling genome-scale metabolic models.

A RBF-based constrained global optimization algorithm for problems with computationally expensive objective and constraints

This paper presents a metamodel-based constrained optimization method, called Radial basis function-based Constrained Global Optimization (RCGO), to solve optimization problems involving computationally expensive objective function and inequality constraints. RCGO is an extension of the adaptive metamodel-based global optimization (AMGO) algorithm which can handle unconstrained black-box optimization problems. Firstly, a sequential sampling method is implemented to obtain the initial points for building the radial basis function (RBF) approximations to all computational expensive functions while enforcing a feasible solution. Then, an auxiliary objective function subject to the approximate constraints is constructed to determine the next iterative point and further improve the solution. During the process, a distance function with a group of exponents is introduced in the auxiliary function to balance the local exploitation and the global exploration. The RCGO method is tested on a series of benchmark problems, and the results demonstrate that RCGO needs fewer costly evaluations and can be applied for costly constrained problems with all infeasible start points. And the test results on the 30D problems demonstrate that RCGO has advantages in solving the problems. The proposed method is then applied to the design of a cycloid gear pump and desirable results are obtained.

Iterative minimization algorithm for efficient calculations of transition states

This paper presents an efficient algorithmic implementation of the iterative minimization formulation (IMF) for fast local search of transition state on potential energy surface. The IMF is a second order iterative scheme providing a general and rigorous description for the eigenvector-following (min-mode following) methodology. We offer a unified interpretation in numerics via the IMF for existing eigenvector-following methods, such as the gentlest ascent dynamics, the dimer method and many other variants. We then propose our new algorithm based on the IMF. The main feature of our algorithm is that the translation step is replaced by solving an optimization subproblem associated with an auxiliary objective function which is constructed from the min-mode information. We show that using an efficient scheme for the inexact solver and enforcing an adaptive stopping criterion for this subproblem, the overall computational cost will be effectively reduced and a super-linear rate between the accuracy and the computational cost can be achieved. A series of numerical tests demonstrate the significant improvement in the computational efficiency for the new algorithm.

Determining kinetic parameters of a block coal bed gas by solving inverse problem based on data of borehole gas measurements

In the framework of the developed and implemented nonlinear geomechanical model of a coal bed having block structure, describing gas inflow, the authors propose the method for quantitative estimate of gas content as well as diffusion and mass exchange coefficients based on solution of inverse problems using pressure measurements taken in shut-in borehole in the “pressure drop” mode. Besides the main function, the set problem had an auxiliary objective function with the less number of arguments. Numerical experiments with synthetic input data yield that the auxiliary objective function has a number of local minimums. The authors also put forward a technique of finding global minimum that provides the set problem solution.

Multi‐objective Approaches to the Unit Crewing Problem in Airline Crew Scheduling

ABSTRACTThe goal of the crew pairing problem is to partition a flight schedule into sequences of flights called pairings that crew members can operate at minimum cost. It is solved for each crew rank, for example, captains and first officers. In optimised crew pairings, crew often split and join other crew to operate outgoing flights. Because minimum cost pairings contain little buffer time, crew splitting after a delayed flight contributes to the propagation of delay. This effect can be avoided by unit crewing, that is, by keeping crew of different ranks together for as long as possible during a pairing. However, increasing unit crewing increases cost.We investigate sequential and parallel methods for unit crewing, explicitly considering the two objectives of minimising cost and maximising unit crewing. We apply multi‐objective techniques in the sequential approach, where the (minimum cost) pairing problem for one crew rank is solved first and both objectives feature when solving for the second crew rank. Because the quality of the pairings obtained by this method is limited by the solution for the first crew rank, we then propose to solve the two crew pairing problems simultaneously, again using multi‐objective optimisation methods. We introduce a multi‐objective optimisation model for this problem and propose a new heuristic branching technique that favours unit crewed pairings when applied to a scalarised model with an auxiliary objective function. We compare this with a direct approach with management of the number of unit crewing constraints and a Dantzig–Wolfe decomposition approach.Numerical tests on domestic New Zealand data show that the multi‐objective approaches considerably increase the level of unit crewing, without much increase in cost, even in the sequential approach. Moreover, we show that the parallel approach is superior in terms of quality of the pairing solution but computationally more expensive. The heuristic branching technique provides good quality solutions in reasonable time, as compared with the Dantzig–Wolfe method. Copyright © 2014 John Wiley & Sons, Ltd.

10.1007/s11493-008-1005-4

Combinatorial approach to solving the problem of detection of an unknown quasi-periodic fragment in a noisy numerical sequence is considered. The problem is analyzed under the following conditions: (1) the number of repeats is known; (2) the number of the sequence term corresponding to the starting instant of the fragment is a deterministic (non-random) value; and (3) the observed sequence is corrupted by additive Gaussian uncorrelated noise. It is demonstarted that the problem under consideration consists in testing the set of composite hypotheses on the mean of a random Gaussian vector. It is shown that the search for a maximum likelihood hypothesis can be reduced to the search for a maximum of an auxiliary objective function. It is proved that the problem of maximization of this function is NP-hard in the general case. An approximate polynomial algorithm for solving the problem is proposed. To improve the approximation, an algorithm of local search is proposed. Numerical simulation showed reasonable results from the applied point of view.

A posteriori joint detection of a recurring tuple of reference fragments in a quasi-periodic sequence

The problem of joint detection of a recurring tuple of reference fragments in a noisy numerical quasi-periodic sequence is solved in the framework of the a posteriori (off-line) approach. It is assumed that (i) the total number of fragments in the sequence is known, (ii) the index of the sequence member corresponding to the beginning of a fragment is a deterministic (not random) value, and (iii) a sequence distorted by an additive uncorrelated Gaussian noise is available for observation. It is shown that the problem consists of testing a set of simple hypotheses about the mean of a random Gaussian vector. A specific feature of the problem is that the cardinality of the set grows exponentially as the vector dimension (i.e., the length of the observed sequence) and the number of fragments in the sequence increase. It is established that the search for a maximum-likelihood hypothesis is equivalent to the search for arguments that maximize a special auxiliary objective function with linear inequality constraints. It is shown that this function is maximized by solving the basic extremum problem. It is proved that this problem is solvable in polynomial time. An exact algorithm for its solution is substantiated that underlies an algorithm guaranteeing optimal (maximum-likelihood) detection of a recurring tuple of reference fragments. The results of numerical simulation demonstrate the noise stability of the detection algorithm.

A posteriori joint detection of reference fragments in a quasi-periodic sequence

The problem of joint detection of quasi-periodic reference fragments (of given size) in a numerical sequence and its partition into segments containing series of recurring reference fragments is solved in the framework of the a posteriori approach. It is assumed that (i) the number of desired fragments is not known, (ii) an ordered reference tuple of sequences to be detected is given, (iii) the index of the sequence member corresponding to the beginning of a fragment is a deterministic (not random) value, and (iv) a sequence distorted by an additive uncorrelated Gaussian noise is available for observation. It is established that the problem consists of testing a set of hypotheses about the mean of a random Gaussian vector. The cardinality of the set grows exponentially as the vector dimension (i.e., the sequence length) increases. It is shown that the search for a maximum-likelihood hypothesis is equivalent to the search for arguments that minimize an auxiliary objective function. It is proved that the minimization problem for this function can be solved in polynomial time. An exact algorithm for its solution is substantiated. Based on the solution to an auxiliary extremum problem, an efficient a posteriori algorithm producing an optimal (maximum-likelihood) solution to the partition and detection problem is proposed. The results of numerical simulation demonstrate the noise stability of the algorithm.

A posteriori detecting a quasiperiodic fragment in a numerical sequence

Combinatorial approach to solving the problem of detection of an unknown quasi-periodic fragment in a noisy numerical sequence is considered. The problem is analyzed under the following conditions: (1) the number of repeats is known; (2) the number of the sequence term corresponding to the starting instant of the fragment is a deterministic (non-random) value; and (3) the observed sequence is corrupted by additive Gaussian uncorrelated noise. It is demonstarted that the problem under consideration consists in testing the set of composite hypotheses on the mean of a random Gaussian vector. It is shown that the search for a maximum likelihood hypothesis can be reduced to the search for a maximum of an auxiliary objective function. It is proved that the problem of maximization of this function is NP-hard in the general case. An approximate polynomial algorithm for solving the problem is proposed. To improve the approximation, an algorithm of local search is proposed. Numerical simulation showed reasonable results from the applied point of view.

Pole assignment with eigenvalue and stability robustness

This paper presents a computation method for pole assignment with eigenvalue and stability robustness. The robustness measure is constructed to balance the tradeoff between an eigenvalue sensitivity measure and a stability robustness measure, both defined in terms of the non-differentiable spectral norm. It is established that the robustness measure can be minimized asymptotically via a sequence of smooth unconstrained minimizations involving some auxiliary objective functions. Moreover, the sequence of minimizers converges to the set of minimizers of the robustness measure. A numerical algorithm with analytical formulae of the gradient of the auxiliary objective functions is provided. Through a numerical example and comparing with other methods, the idea of using a combined robustness measure in the design is illustrated and the effectiveness of the computation is demonstrated.