Maximum Entropy Sampling Research Articles

Generalized scaling for the constrained maximum-entropy sampling problem

Generalized scaling for the constrained maximum-entropy sampling problem

A physics-informed Bayesian optimization method for rapid development of electrical machines

Advanced slot and winding designs are imperative to create future high performance electrical machines (EM). As a result, the development of methods to design and improve slot filling factor (SFF) has attracted considerable research. Recent developments in manufacturing processes, such as additive manufacturing and alternative materials, has also highlighted a need for novel high-fidelity design techniques to develop high performance complex geometries and topologies. This study therefore introduces a novel physics-informed machine learning (PIML) design optimization process for improving SFF in traction electrical machines used in electric vehicles. A maximum entropy sampling algorithm (MESA) is used to seed a physics-informed Bayesian optimization (PIBO) algorithm, where the target function and its approximations are produced by Gaussian processes (GP)s. The proposed PIBO-MESA is coupled with a 2D finite element model (FEM) to perform a GP-based surrogate and provide the first demonstration of the optimal combination of complex design variables for an electrical machine. Significant computational gains were achieved using the new PIBO-MESA approach, which is 45% faster than existing stochastic methods, such as the non-dominated sorting genetic algorithm II (NSGA-II). The FEM results confirm that the new design optimization process and keystone shaped wires lead to a higher SFF (i.e. by 20%) and electromagnetic improvements (e.g. maximum torque by 12%) with similar resistivity. The newly developed PIBO-MESA design optimization process therefore presents significant benefits in the design of high-performance electric machines, with reduced development time and costs.

Regression-enhanced Entrotaxis as an autonomous search algorithm for seeking an unknown gas leakage source

Rapid localization of hazardous gas leak sources is a vital task regarding the security of human society. Over the past few decades, autonomous search using mobile sensing devices equipped with gas sensors has become a promising field. Cognitive search strategies, such as Infotaxis and Entrotaxis, have been successfully used to search an unknown source autonomously in turbulence. However, they are time-consuming and may not be efficient because all source parameters (locations and strength) are estimated via particle filtering, especially when the prior information of source strength is rarely known. To address this inefficiency, this paper proposes a hybrid autonomous source searching approach, named as regression-enhanced Entrotaxis, which incorporates the Entrotaxis algorithm with the regression methods. Only the position of leak source is inferred by particle filtering, while the strength is estimated by the least squares based on the historical measurements. Through this way, the searching space in particle filtering is converted from 3-dimensions (source position in x–y plane and source strength) to 2-dimensions (source position in x–y plane), speeding up the computation significantly. Then, the reward function according to maximum entropy sampling principles is utilized to guide the robot’s next move. The performance of the proposed method is compared to that of the Entrotaxis algorithm. The results of Monte Carlo simulations demonstrate that fewer particles will be used to reach a specific success rate, saving time in the search process. Besides, the proposed strategy relies on less prior information regarding source strength, which is more in line with the actual scenario. In addition, the proposed approach is adaptable to a broader search region.

Tridiagonal maximum-entropy sampling and tridiagonal masks

The NP-hard maximum-entropy sampling problem (MESP) seeks a maximum (log-) determinant principal submatrix, of a given order, from an input covariance matrix C. We give an efficient dynamic-programming algorithm for MESP when C (or its inverse) is tridiagonal and generalize it to the situation where the support graph of C (or its inverse) is a spider graph with a constant number of legs (and beyond). We give a class of arrowhead covariance matrices C for which a natural greedy algorithm solves MESP. A mask M for MESP is a correlation matrix with which we pre-process C, by taking the Hadamard product M∘C. Upper bounds on MESP with M∘C give upper bounds on MESP with C. Most upper-bounding methods are much faster to apply, when the input matrix is tridiagonal, so we consider tridiagonal masks M (which yield tridiagonal M∘C). We make a detailed analysis of such tridiagonal masks, and develop a combinatorial local-search based upper-bounding method that takes advantage of fast computations on tridiagonal matrices.

The use of the Karhunen Loève expansion in the design of computer experiments

This article proposes the use of the Karhunen Loève expansion in the design of computer experiments when the Gaussian Process (GP) model is used to model the output, Y(x). We start by finding the K-L expansion in the case of the univariate and multivariate processes. When the analytical solution does not exist, we apply several methods of approximation, such as Haar wavelet functions and Fourier expansions. Numerical examples are presented to illustrate the idea of the approximations. We use the approximated model to select the maximum entropy sampling design.

Best Principal Submatrix Selection for the Maximum Entropy Sampling Problem: Scalable Algorithms and Performance Guarantees

This paper studies a classic maximum entropy sampling problem (MESP), which aims to select the most informative principal submatrix of a prespecified size from a covariance matrix. MESP is widely applied to many areas, including healthcare, power systems, manufacturing, and data science. By investigating its Lagrangian dual and primal characterization, we derive a novel convex integer program for MESP and show that its continuous relaxation yields a near-optimal solution. The results motivate us to study efficient approximation algorithms and develop their approximation bounds for MESP, which improves the best known one in the literature.

On Computing with Some Convex Relaxations for the Maximum-Entropy Sampling Problem

Based on a factorization of an input covariance matrix, we define a mild generalization of an upper bound of Nikolov and of Li and Xie for the NP-hard constrained maximum-entropy sampling problem ( CMESP ). We demonstrate that this factorization bound is invariant under scaling and independent of the particular factorization chosen. We give a variable-fixing methodology that could be used in a branch-and-bound scheme based on the factorization bound for exact solution of CMESP , and we demonstrate that its ability to fix is independent of the factorization chosen. We report on successful experiments with a commercial nonlinear programming solver. We further demonstrate that the known “mixing” technique can be successfully used to combine the factorization bound with the factorization bound of the complementary CMESP and with the “linx bound” of Anstreicher. History: Andrea Lodi, area editor for Design & Analysis of Algorithms–Discrete. Funding: This work was supported by the Conselho Nacional de Desenvolvimento Científico e Tecnológico [Grants 305444/2019-0 and 434683/2018-3] and the Air Force Office of Scientific Research [Grant A9550-19-1-0175].

Application of an improved maximum entropy sampling method in hull form optimization

The development of approximation models based on dynamic sampling can greatly improve the construction efficiency of such models as well as the optimization efficiency of the hull form. The distribution of samples selected using the dynamic sampling method has a direct influence on the accuracy and robustness of the approximation model. In this study, an improved maximum entropy sampling method is proposed to enhance the uniformity of samples selected during dynamic sampling, as well as the efficiency, accuracy, and robustness of the Kriging model. Uniform Design (UD) experimentation method was utilized to provide a uniformly distributed sample set candidates to the maximum entropy sampling process. UD can significantly reduce the tedious computation of information entropy during the maximum-entropy-sampling-based optimization process, which greatly improves the sampling efficiency and ensures a rational sample distribution. Finally, the effectiveness of the proposed method is validated via tests on different mathematical functions and the hull form optimization of S60.

Optimal Observational Scheduling Framework for Binary and Multiple Stellar Systems* *Based in part on observations obtained at the international Gemini Observatory, a program of NSF's NOIRLab, which is managed by the Association of Universities for Research in Astronomy (AURA) under a cooperative agreement with the National Science Foundation on behalf of the Gemini Observatory partnership: the National Science Foundation (United

The optimal instant of observation of astrophysical phenomena for objects that vary on human timescales is an important problem, as it bears on the cost-effective use of usually scarce observational facilities. In this paper, we address this problem in the case of tight visual binary systems through a Bayesian framework based on the maximum entropy sampling principle. Our proposed information-driven methodology exploits the periodic structure of binary systems to provide a computationally efficient estimation of the probability distribution of the optimal observation time. We show the optimality of the proposed sampling methodology in the Bayes sense and its effectiveness through direct numerical experiments. We successfully apply our scheme to the study of two visual-spectroscopic binaries and one purely astrometric triple hierarchical system. We note that our methodology can be applied to any time-evolving phenomena, a particularly interesting application in the era of dedicated surveys, where a definition of the cadence of observations can have a crucial impact on achieving the science goals.

Bayesian optimized collection strategies for fatigue strength testing

AbstractA statistical framework is presented enabling optimal sampling and analysis of constant life fatigue data. Protocols using Bayesian maximum entropy sampling are built based on conventional staircase and stress step methods, reducing the requirement of prior knowledge for data collection. The Bayesian Staircase method shows improved parameter estimation efficiency, and the Bayesian Stress Step method shows equal accuracy to the standard method at larger step size allowing experimentalists to lessen concerns of loading history. Statistical methods for determining model suitability are shown, highlighting the influence of protocol. Experimental validation is performed, showing the applicability of the methods in laboratory testing.

Technical Note—Masking Anstreicher’s linx Bound for Improved Entropy Bounds

A fundamental NP-hard combinatorial-optimization in the area of statistical designs is the maximum-entropy sampling problem (MESP), which seeks to maximize Shannon's “differential entropy” over all subsets of a prespecified cardinality from a set of n Gaussian random variables. This problem has applications in many areas, such as the redesign of environmental-monitoring networks. Most algorithms for exact solution of MESP are branch-and-bound based, and one of the best upper bounds is based on Anstrecher's recent concave “linx relaxation” of differential entropy. A key paradigm for improving bounds is by “masking” the covariance of the random variables with a correlation matrix. The main result establishes that in the best case, the linx bound can be improved by an amount that is at least linear in n by masking. These and other recent results on the hot topic of MESP are leading to practical algorithms for exact solution of meaningful design problems in applied areas such as environmental statistics.

Mixing convex-optimization bounds for maximum-entropy sampling

The maximum-entropy sampling problem is a fundamental and challenging combinatorial-optimization problem, with application in spatial statistics. It asks to find a maximum-determinant order-s principal submatrix of an order-n covariance matrix. Exact solution methods for this NP-hard problem are based on a branch-and-bound framework. Many of the known upper bounds for the optimal value are based on convex optimization. We present a methodology for “mixing” these bounds to achieve better bounds.

Tridiagonal Maximum-Entropy Sampling and Tridiagonal Masks

The NP-hard maximum-entropy sampling problem (MESP) seeks a maximum (log-)determinant principal submatrix, of a given order, from a positive-semidefinite input matrix C. We give an efficient dynamic-programming algorithm for MESP when C (or its inverse) is tridiagonal. A mask M for MESP is a correlation matrix with which we pre-process C, by taking the Hadamard product M ◦C. Upper bounds on MESP with M ◦C give upper bounds on MESP with C. Most upper-bounding methods are much faster to apply, when the input matrix is tridiagonal, so we consider tridiagonal masks M (which yield tridiagonal M ◦ C). We analyze such tridiagonal masks, and develop a combinatorial local-search based upper-bounding method that takes advantage of fast computations on tridiagonal matrices.

Efficient Solution of Maximum-Entropy Sampling Problems

Maximum-entropy sampling is a difficult nonlinear discrete optimization problem that arises in spatial statistics, for example, in the design of weather-monitoring networks. An exact algorithm for maximum-entropy sampling was first described in 1995, and subsequent papers have devised a variety of methods that obtain bounds and, in some cases, exact solutions. In “An Efficient Algorithm for Maximum-Entropy Sampling,” Anstreicher describes a new bound for the maximum-entropy sampling problem that is superior to all previously known bounds and is also efficiently computable. A branch-and-bound algorithm that incorporates the new bound solves challenging benchmark instances to optimality for the first time.

An R Package for Generating Covariance Matrices for Maximum-Entropy Sampling from Precipitation Chemistry Data

We present an open-source R package (MESgenCov v 0.1.0) for temporally fitting multivariate precipitation chemistry data and extracting a covariance matrix for use in the MESP (maximum-entropy sampling problem). We provide multiple functionalities for modeling and model assessment. The package is tightly coupled with NADP/NTN (National Atmospheric Deposition Program/National Trends Network) data from their set of 379 monitoring sites, 1978–present. The user specifies the sites, chemicals, and time period desired, fits an appropriate user-specified univariate model for each site and chemical selected, and the package produces a covariance matrix for use by MESP algorithms.

Greedy Maximization of Functions with Bounded Curvature under Partition Matroid Constraints

We investigate the performance of a deterministic GREEDY algorithm for the problem of maximizing functions under a partition matroid constraint. We consider non-monotone submodular functions and monotone subadditive functions. Even though constrained maximization problems of monotone submodular functions have been extensively studied, little is known about greedy maximization of non-monotone submodular functions or monotone subadditive functions.
 We give approximation guarantees for GREEDY on these problems, in terms of the curvature. We find that this simple heuristic yields a strong approximation guarantee on a broad class of functions.
 We discuss the applicability of our results to three real-world problems: Maximizing the determinant function of a positive semidefinite matrix, and related problems such as the maximum entropy sampling problem, the constrained maximum cut problem on directed graphs, and combinatorial auction games.
 We conclude that GREEDY is well-suited to approach these problems. Overall, we present evidence to support the idea that, when dealing with constrained maximization problems with bounded curvature, one needs not search for (approximate) monotonicity to get good approximate solutions.

A discussion on adaptive designs for computer experiments

Maximum entropy sampling (MES) criteria provide a useful framework for studying sequential designs for computer experiments in a Bayesian framework. However, there is some technical difficulty in making the procedure fully adaptive in the sense of making proper use of previous output as well as input data. In the simple Gaussian set-up only previous input values need to be used. The approach discussed uses a full hierarchical model for the Gaussian process. The idea is to take advantage of the Karhumen-Loéve (K-L) expansion to approximate the process covariance function using an orthogonal function basis. It is argued that this may make it easier to use Bayesian hierarchical models, rather than estimating the covariance parameters directly, using the traditional approach. The article paper shows how to reduce the full MES method to a simple one by using a special empirical Bayes approximation, rather than using time-consuming integration. A simple simulator example is presented to show that full adaptation is beneficial.

Generalized maximum-entropy sampling

We introduce the generalized constrained maximum-entropy sampling problem (GCMESP) as a common generalization of the ordinary constrained maximum-entropy sampling problem (CMESP) and the constrained D-optimality problem (CDOPTP). Exact algorithms for both CMESP and CDOPTP are based on branch-and-bound methods. We extend a spectral upper-bounding method for the CMESP to the GCMESP.

Maximum-entropy sampling and the Boolean quadric polytope

We consider a bound for the maximum-entropy sampling problem (MESP) that is based on solving a max-det problem over a relaxation of the Boolean quadric polytope (BQP). This approach to MESP was first suggested by Christoph Helmberg over 15 years ago, but has apparently never been further elaborated or computationally investigated. We find that the use of a relaxation of BQP that imposes semidefiniteness and a small number of equality constraints gives excellent bounds on many benchmark instances. These bounds can be further tightened by imposing additional inequality constraints that are valid for the BQP. Duality information associated with the BQP-based bounds can be used to fix variables to 0/1 values, and also as the basis for the implementation of a “strong branching” strategy. A branch-and-bound algorithm using the BQP-based bounds solves some benchmark instances of MESP to optimality for the first time.

Entrotaxis as a strategy for autonomous search and source reconstruction in turbulent conditions

This paper proposes a strategy for performing an efficient autonomous search to find an emitting source of sporadic cues of noisy information. We focus on the search for a source of unknown strength, releasing particles into the atmosphere where turbulence can cause irregular gradients and intermittent patches of sensory cues. Bayesian inference, implemented via the sequential Monte Carlo method, is used to update posterior probability distributions of the source location and strength in response to sensor measurements. Posterior sampling is then used to approximate a reward function, leading to the manoeuvre to where the entropy of the predictive distribution is the greatest. As it is developed based on the maximum entropy sampling principle, the proposed framework is termed as Entrotaxis. We compare the performance and search behaviour of Entrotaxis with the popular Infotaxis algorithm, for searching in sparse and turbulent conditions where typical gradient-based approaches become inefficient or fail. The algorithms are assessed via Monte Carlo simulations with simulated data and an experimental dataset. Whilst outperforming the Infotaxis algorithm in most of our simulated scenarios, by achieving a faster mean search time, the proposed strategy is also more computationally efficient during the decision making process.