Black-box Problems Research Articles

Solving the Black Box Problem: A Normative Framework for Explainable Artificial Intelligence

Many of the computing systems programmed using Machine Learning are opaque: it is difficult to know why they do what they do or how they work. Explainable Artificial Intelligence aims to develop analytic techniques that render opaque computing systems transparent, but lacks a normative framework with which to evaluate these techniques’ explanatory successes. The aim of the present discussion is to develop such a framework, paying particular attention to different stakeholders’ distinct explanatory requirements. Building on an analysis of “opacity” from philosophy of science, this framework is modeled after accounts of explanation in cognitive science. The framework distinguishes between the explanation-seeking questions that are likely to be asked by different stakeholders, and specifies the general ways in which these questions should be answered so as to allow these stakeholders to perform their roles in the Machine Learning ecosystem. By applying the normative framework to recently developed techniques such as input heatmapping, feature-detector visualization, and diagnostic classification, it is possible to determine whether and to what extent techniques from Explainable Artificial Intelligence can be used to render opaque computing systems transparent and, thus, whether they can be used to solve the Black Box Problem.

Prediction and Interpretable Visualization of Retrosynthetic Reactions Using Graph Convolutional Networks.

Recently, many research groups have been addressing data-driven approaches for (retro)synthetic reaction prediction and retrosynthetic analysis. Although the performances of the data-driven approach have progressed because of recent advances of machine learning and deep learning techniques, problems such as improving capability of reaction prediction and the black-box problem of neural networks persist for practical use by chemists. To spread data-driven approaches to chemists, we focused on two challenges: improvement of retrosynthetic reaction prediction and interpretability of the prediction. In this paper, we propose an interpretable prediction framework using graph convolutional networks (GCN) for retrosynthetic reaction prediction and integrated gradients (IG) for visualization of contributions to the prediction to address these challenges. As a result, from the viewpoint of balanced accuracies, our model showed better performances than the approach using an extended-connectivity fingerprint. Furthermore, IG-based visualization of the GCN prediction successfully highlighted reaction-related atoms.

Kriging-based unconstrained global optimization through multi-point sampling

ABSTRACTThe generalized efficient global optimization (GEGO) method is able to solve expensive black-box problems. However, selecting one sampling point per cycle may result in large time consumption and loss of convergence accuracy. To this end, a kriging-based multi-point unconstrained global optimization (KMUGO) method is proposed. It extends the GEGO method with the improved constant liar (CL) strategy. For each cycle, the kriging model is first constructed or updated by the existing sampled data. Then, the enhanced alternative CL strategy is used to find multiple points, which will be further screened to identify the final expensive-evaluation points. Test results of numerical problems and an engineering simulation case show that KMUGO can deliver better convergence than GEGO, multi-point sampling method-based kriging (MPSK), hybrid and adaptive meta-model-based global optimization (HAM) and the kriging-based global optimization method using multi-point infill search criterion (KGOMISC).

Surrogate Optimization of Computationally Expensive Black-Box Problems with Hidden Constraints (SHEBO) v1.0

Surrogate Optimization of Computationally Expensive Black-Box Problems with Hidden Constraints (SHEBO) v1.0

An Improved Surrogate Based Optimization Method for Expensive Black-box Problems

For expensive black-box problems, surrogate modelling techniques are generally used to decrease the computational source. In this study, an improved surrogate based optimization (SBO) method is presented to solve the real-world engineering applications with expensive black-box objective responses. An optimized ensemble of surrogates combing three typical surrogate modelling techniques is adapted to efficiently predict the objective response. Meanwhile, the hierarchical design space reduction (HSR) strategy is employed for obtaining the smaller design subspace for improving the optimization efficiency. During the search, all test problems are considered as the real-world engineering applications whereas the actual global optima as well as the function characteristics are unknown in advance. The results show that the proposed method is superior in identifying the global optimum.

Surrogate Optimization of Computationally Expensive Black-Box Problems with Hidden Constraints

We introduce the algorithm SHEBO (surrogate optimization of problems with hidden constraints and expensive black-box objectives), an efficient optimization algorithm that employs surrogate models to solve computationally expensive black-box simulation optimization problems that have hidden constraints. Hidden constraints are encountered when the objective function evaluation does not return a value for a parameter vector. These constraints are often encountered in optimization problems in which the objective function is computed by a black-box simulation code. SHEBO uses a combination of local and global search strategies together with an evaluability prediction function and a dynamically adjusted evaluability threshold to iteratively select new sample points. We compare the performance of our algorithm with that of the mesh-based algorithms mesh adaptive direct search (MADS, NOMAD [nonlinear optimization by mesh adaptive direct search] implementation) and implicit filtering and SNOBFIT (stable noisy optimization by branch and fit), which assigns artificial function values to points that violate the hidden constraints. Our numerical experiments for a large set of test problems with 2–30 dimensions and a 31-dimensional real-world application problem arising in combustion simulation show that SHEBO is an efficient solver that outperforms the other methods for many test problems.

Data driven stability analysis of black-box switched linear systems

Can we conclude the stability of an unknown dynamical system from the knowledge of a finite number of snapshots of trajectories? We tackle this black-box problem for switched linear systems. We show that, for any given random set of observations, one can give probabilistic stability guarantees. The probabilistic nature of these guarantees implies a trade-off between their quality and the desired level of confidence. We provide an explicit way of computing the best stability-like guarantee, as a function of both the number of observations and the required level of confidence. Our proof techniques rely on geometrical analysis, chance-constrained optimization, and stability analysis tools for switched systems, including the joint spectral radius.

Against Interpretability: a Critical Examination of the Interpretability Problem in Machine Learning

The usefulness of machine learning algorithms has led to their widespread adoption prior to the development of a conceptual framework for making sense of them. One common response to this situation is to say that machine learning suffers from a “black box problem.” That is, machine learning algorithms are “opaque” to human users, failing to be “interpretable” or “explicable” in terms that would render categorization procedures “understandable.” The purpose of this paper is to challenge the widespread agreement about the existence and importance of a black box problem. The first section argues that “interpretability” and cognates lack precise meanings when applied to algorithms. This makes the concepts difficult to use when trying to solve the problems that have motivated the call for interpretability (etc.). Furthermore, since there is no adequate account of the concepts themselves, it is not possible to assess whether particular technical features supply formal definitions of those concepts. The second section argues that there are ways of being a responsible user of these algorithms that do not require interpretability (etc.). In many cases in which a black box problem is cited, interpretability is a means to a further end such as justification or non-discrimination. Since addressing these problems need not involve something that looks like an “interpretation” (etc.) of an algorithm, the focus on interpretability artificially constrains the solution space by characterizing one possible solution as the problem itself. Where possible, discussion should be reformulated in terms of the ends of interpretability.

Two‐fold calibration approach for microscopic traffic simulation models

Calibrating the microsimulation traffic models can be defined as a black-box optimisation problem with some non-concave objective functions. In this regard, the stochastic optimisation algorithms are suitable choices to explore the search space and prevent getting stuck in local optimums. However, considering only the traffic attributes-related objectives may fail to calibrate the model in terms of safety. Therefore, by defining two different objectives, a two-fold calibration approach is proposed such that the simulation model reproduces the real-world transportation network more accurately, both in terms of safety and operation. Moreover, the performance of two different approaches to solve this multi-objective optimisation problem are evaluated. It is shown that by aggregating the objectives in one single formula (i.e. a priori methods), the information exchange among solutions is not captured, which may lead to non-optimal solutions. While this limitation is overcome by a posteriori methods since different objectives can be optimised separately and simultaneously. In this regard, the performance of posteriori-based multi-objective particle swarm optimisation (MOPSO) algorithm in calibrating VISSIM is compared with some priori-based optimisation algorithms (e.g. PSO, genetic algorithm, and whale optimisation algorithm). The results show that posteriori-based MOPSO leads to a more accurate solution set in terms of both objectives.

Analysis-of-Marginal-Tail-Means (ATM): A Robust Method for Discrete Black-Box Optimization

We present a new method, called analysis-of-marginal-tail-means (ATM), for effective robust optimization of discrete black-box problems. ATM has important applications in many real-world engineering problems (e.g., manufacturing optimization, product design, and molecular engineering), where the objective to optimize is black-box and expensive, and the design space is inherently discrete. One weakness of existing methods is that they are not robust: these methods perform well under certain assumptions, but yield poor results when such assumptions (which are difficult to verify in black-box problems) are violated. ATM addresses this by combining both rank- and model-based optimization, via the use of marginal tail means. The trade-off between rank- and model-based optimization is tuned by first identifying important main effects and interactions from data, then finding a good compromise which best exploits additive structure. ATM provides improved robust optimization over existing methods, particularly in problems with (i) a large number of factors, (ii) unordered factors, or (iii) experimental noise. We demonstrate the effectiveness of ATM in simulations and in two real-world engineering problems: the first on robust parameter design of a circular piston, and the second on product family design of a thermistor network.

An algorithmic framework for the optimization of computationally expensive bi-fidelity black-box problems

We introduce an algorithm for the optimization of problems whose objective functions are evaluated by computationally expensive black-box simulations and for which an analytic description of the objective and its derivatives are not available. We consider the case where two levels of simulation model fidelity are available, namely a high fidelity model that is computationally very expensive to evaluate, and a low fidelity model that is less accurate and computationally cheaper but still time consuming. The computational effort is alleviated by using computationally cheap surrogate models that approximate the simulations at both fidelity levels. The local correlation between both fidelity surrogate models determines when the low fidelity model can be trusted for making sampling decisions for the high fidelity model. In the numerical experiments we investigate how well our algorithm responds to problems whose objective function fidelity levels are well correlated and badly correlated. We study how different initial design strategies and parameter settings impact the performance of the algorithm. The results show that our algorithm actively learns from the local correlation computations how well suited the low fidelity model is for making sampling decisions and it ignores the low fidelity model if the correlation is too low.

Rethinking Automated Investment Adviser Disclosure

Today, investment advisers, including robo-advisers, push dense disclosures to consumer investors who may not be able to fully comprehend their import. Tomorrow, robo-advisers and their innovative technology, if embraced by regulators and consumers, can facilitate and cultivate a trusted collaborative relationship between investor and adviser where the adviser learns from the investor and provides education tailored to the investor, helping to solve vexing consumer protection issues and lessen the impending retirement crisis. More and more of us are comfortable making decisions without human involvement and entirely through a digital interface. Despite consumers’ increasing comfort relying upon machines to direct and guide their decisions, most Americans nevertheless lack basic foundational knowledge in finance and investing. Even those Americans who are able to save for retirement lack financial sophistication and may not be able to understanding the complex products in which they are invested. Robo advice may not be the solution for these problems, as robo-advisers present a potential black box problem where the digital investment advisers may not be capable of explanation beyond the information input and advice output. In such a world, it is not only the new technology that should be questioned. The existing standards through which the new technology is regulated should similarly be explored. An understanding of potential benefits and potential harms may provide lessons that better allow us to protect consumer investors and better prepare for the so-called retirement crisis. This essay explores the parameters of the pre-existing regulatory regime as applied to a new landscape of investing with a focus on the key regulatory tool of disclosure. This short essay examines the challenges of delivering appropriate disclosures in a new world of robo-advisory advice. It suggests a reconceptualization of the aim of disclosure to capitalize on robo-advisers’ ability to truly know their clients and shift the burden of investor education from the customer to a fiduciary level adviser capable of tailoring advice and education to each investor at a level she is capable of understanding. The essay concludes with a recommendation: robo-advisers and investor collaboration via an active and iterative disclosure regime that encourages investor comprehension, understanding, and learning.

A Kriging-based bi-objective constrained optimization method for fuel economy of hydrogen fuel cell vehicle

The Kriging-based single-objective optimization for expensive black-box problems has been preventing the engineering application. The main challenge is how to reduce time consumption and improve convergence accuracy. To this end, a Kriging-based bi-objective constrained optimization (KBCO) algorithm is proposed. For each cycle, KBCO firstly uses the sampled design points to build or consecutively update Kriging models of expensive objective and constraint functions. And then, the predictive objective, root mean square error (RMSE) and maximum feasible probability produced by Kriging models are used to construct two objectives, which will be optimized by the NSGA-II solver to generate the Pareto optimal solutions. Finally, the Pareto front data will be further screened to obtain new expensive-evaluation sampling points and append them to sample data. Several numerical tests and a fuel economy simulation case for hydrogen fuel cell vehicle verify the feasibility and effectiveness of the KBCO method.

UNIPOPT: Univariate projection-based optimization without derivatives

A derivative-free optimization framework UNIPOPT is developed to solve multi-dimensional box-constrained black-box problems. The method is based on projecting the objective function onto a univariate space, which leads to a point-to-set map. The lower envelope of the map contains the global minimum. A sensitivity theorem is employed to predict the points on the lower envelope and a trust-region based algorithm is used to correct the predictions. A model-based algorithm is used to optimize the univariate lower envelope such that its minimum corresponds to that of the original problem. We provide theoretical results related to the prediction and correction steps and prove the convergence of the overall algorithm. UNIPOPT is applied to an extensive suite of test problems comprising of both convex nonsmooth and nonconvex smooth black-box problems, and its performance is compared to the existing model-based approaches. The algorithm is able to locate global solutions for more test problems compared to the other solvers.

The Right Direction Needed to Develop White-Box Deep Learning in Radiology, Pathology, and Ophthalmology: A Short Review.

The popularity of deep learning (DL) in the machine learning community has been dramatically increasing since 2012. The theoretical foundations of DL are well-rooted in the classical neural network (NN). Rule extraction is not a new concept, but was originally devised for a shallow NN. For about the past 30 years, extensive efforts have been made by many researchers to resolve the “black box” problem of trained shallow NNs using rule extraction technology. A rule extraction technology that is well-balanced between accuracy and interpretability has recently been proposed for shallow NNs as a promising means to address this black box problem. Recently, we have been confronting a “new black box” problem caused by highly complex deep NNs (DNNs) generated by DL. In this paper, we first review four rule extraction approaches to resolve the black box problem of DNNs trained by DL in computer vision. Next, we discuss the fundamental limitations and criticisms of current DL approaches in radiology, pathology, and ophthalmology from the black box point of view. We also review the conversion methods from DNNs to decision trees and point out their limitations. Furthermore, we describe a transparent approach for resolving the black box problem of DNNs trained by a deep belief network. Finally, we provide a brief description to realize the transparency of DNNs generated by a convolutional NN and discuss a practical way to realize the transparency of DL in radiology, pathology, and ophthalmology.

Prediction of Pareto Dominance Using an Attribute Tendency Model for Expensive Multi-Objective Optimization

This paper proposes a novel surrogate-assisted multi-objective evolutionary algorithm, MOEA-ATCM, to solve expensive or black-box multi-objective problems with small evaluation budgets. The proposed approach encompasses a state-of-the-art MOEA based on a nondominated sorting genetic algorithm assisted by multi-fidelity optimization methods. A high-fidelity attribute tendency (AT) surrogate model was used to construct a linear decision space by introducing the knowledge of the objective space. A coarse model (CM) based on the AT model and correlation analyses of the objective functions and decision attributes were used to predict the Pareto dominance for candidates in the new decision space constructed by the AT model. Two major roles of MOEA-ATCM were identified: (1) the development of a new multi-fidelity surrogate-model-based method to predict Pareto dominance in a decision space that was then applied to MOEA, which does not need to dynamically update surrogate models in the optimization process and (2) the development of a Pareto dominance prediction method to obtain good nondominated solutions of expensive or black box problems with relatively few objective function evaluations. The advantages of MOEA-ATCM were verified by mathematical benchmark problems and a real-world multi-objective parameter optimization problem.

Deterministic global derivative-free optimization of black-box problems with bounded Hessian

Obtaining guaranteed lower bounds for problems with unknown algebraic form has been a major challenge in derivative-free optimization. In this work, we present a deterministic global optimization method for black-box problems where the derivatives are not available or it is computationally expensive to obtain. However, a global upper bound on the diagonal Hessian elements is known. An edge-concave underestimator (Hasan in J Glob Optim 71:735–752, 2018) can be then constructed with vertex polyhedral solution. Evaluating this underestimator only at the vertices leads to a valid lower bound on the original black-box problem. We have implemented this lower bounding technique within a branch-and-bound framework and assessed its computational performance in locating $$\epsilon $$-global optimal solution for several box-constrained, nonconvex black-box functions.

A novel global optimization algorithm and data-mining methods for turbomachinery design

A new multi-objective, multi-disciplinary global optimization strategy is proposed to address the high-dimensional, computationally expensive black box problem (HEB) in turbomachinery design. The strategy consists of an adaptive sampling hybrid optimization algorithm (ASHOA), two data-mining techniques, a 3D blade parameterization method, and the aerodynamic/mechanical codes. Firstly, the ASHOA is established by integrating a novel adaptive sampling Kriging metamodel and a new hybrid optimization method. Secondly, two data-mining methods (analysis of variance (ANOVA) and self-organizing map (SOM)) are applied to set the initial design space and optimization objectives of the transonic centrifugal compressor. A refined design space and objective parameters of the optimization problem are eventually obtained. Finally, the optimization process of a transonic centrifugal compressor is carried out based on the refined design space and objectives using ASHOA. The results show that the search efficiency of the optimization strategy is 2–10 times higher when compared to other excellent optimization algorithms. For the optimized compressor, both isentropic efficiency and total pressure ratio at design condition are improved by 1.61% and 4.13%, respectively, and the maximum stress decreases by 9.68%.

Foreword

Automated algorithm selection and configuration are key enabling approaches for improving the state of the art in solving a broad range of important problems, by exploiting performance complementarity between multiple algorithms for the same problem (selection) and by realising the latent performance potential in parameterised algorithms (configuration). Compared to traditional, manual approaches, these techniques are not only more efficient and rely less on human expertise, but also provide a more principled basis for algorithm selection and configuration, enable fairer comparisons between algorithms, and facilitate new insights into which algorithmic techniques and components work best and under which circumstances.Work on automated algorithm selection, configuration, and related areas spans multiple, weakly connected communities, including artificial intelligence, evolutionary computation, mathematical optimisation and operations research. This special issue follows a Dagstuhl seminar on the same topic, held in October 2016, and is intended as a further step toward creating synergy between those communities.For this special issue, we selected, from a substantial number of submissions, seven papers that jointly cover a broad range of topics and approaches, including various methods for algorithm selection and configuration, search landscape analysis, software engineering aspects, as well as applications to prominent discrete and continuous, single- and multiobjective optimisation problems.The survey paper by Kerschke et al. provides an overview of research in automated algorithm selection, ranging from early and seminal works to recent and promising application areas. Unlike earlier surveys, it covers applications to discrete and continuous problems; it also situates algorithm selection in the context of a wide spectrum of conceptually related approaches, such as algorithm configuration, scheduling, and portfolio selection, and discusses open challenges.Alyaha and Rowe present a study of landscape characteristics for three NP-hard combinatorial optimisation problems: number partitioning and two variants of knapsack problems. A comparative analysis of landscapes induced by different neighbourhood operators, penalty functions, and problem parameters led to improved problem understanding and to a heuristic for selecting the most appropriate local search operator.The paper by Saalem et al. introduces an approach for assessing the effectiveness of exploratory landscape features with respect to their impact on the performance of algorithm selection methods. A model-based framework for continuous black-box problem comparison using Gaussian process (GP) regression is presented, leading to a flexible surrogate model for problem landscapes. The GP substantially facilitates problem comparison while efficiently measuring model quality.Kerschke et al. present an automated algorithm selection method for single-objective continuous black-box optimisation problems, based on sophisticated exploratory landscape analysis and machine learning techniques. The efficiency of the selector is illustrated on the Black-Box Optimisation Benchmark (BBOB), by improving the performance over the single best solver from a representative set of solvers by a factor of two on average.In the paper by Wessing et al., an improved initialisation procedure for a well-known automated configuration procedure, irace, is shown to outperform classical uniform sampling of algorithm configurations. Techniques from the design and analysis of computer experiments are applied, that is, several Latin hypercube sampling methods that are able to handle categorical and numerical parameters that may be conditional (nested) on the value of other (branching) parameters.Blot et al. investigate the automatic configuration of multiobjective local search algorithms for permutation problems—specifically, for the bi-objective permutation flowshop and travelling salesman problems. They consider two performance metrics as configuration objectives and study several approaches for automated configuration according to these two objectives, presenting clear evidence that multiobjective algorithms are best configured using a multiobjective configurator.Finally, Swan et al. introduce a new approach to the automation of the design of metaheuristics, the so-called Automated Open-Closed Principle (AOCP), which addresses the problem of state dependencies between configurable components in flexible algorithm frameworks. AOCP extends current approaches to automated algorithm configuration by offering a principled software engineering approach to ensure the automated assembly of algorithms from an extensible palette of components.By now, automated algorithm selection and configuration are mature research areas, as witnessed not only by the sophistication of the methods and the success of their many applications, but also by a large and fast-growing body of literature. Still, we see much room for further work, spanning the gamut from theoretical to empirical studies, from new methodology to applications. We hope that this special issue will inspire interest and future work in these dynamic and exciting research areas.

Abstract WP68: Interpretable Deep Learning-based Characterization of Intracranial Hemorrhage on Head CT

Background: Machine learning algorithms have proven accurate in the detection of intracranial hemorrhage (ICH) on head CT. Most reported algorithms, however, are limited to binary detection and global lesion volume estimation. We developed a pipeline to additionally perform subtype classification for clinical risk stratification, while also providing visual displays to improve interpretability of algorithm output (“black box problem”) and enable clinical systems integration. Methods: We developed a convolutional neural network (CNN) for ICH detection and subtype classification. The algorithm was trained on 2,209 head CT exams (hemorrhage-negative controls and the 5 major ICH subtypes) and outputs class-activation heatmaps to aid interpretability. We quantify intraparenchymal hemorrhage (IPH) volume using a modified U-Net CNN trained on a cohort of 110 manually-segmented hematomas to precisely delineate lesion boundaries. Algorithm output was stored as DICOM segmentation objects which can be overlaid on native images within our existing viewer applications, enabling the user to dynamically adjust characteristics such as color and transparency. Results: Diagnostic performance for ICH detection was high with ROC area-under-curve of 0.98. Fig 1A shows a heatmap highlighting different ICH foci. IPH subtype classification performance is summarized in Fig 1B with area-under-curve of 0.95. Fig 1C shows IPH segmentation and volume estimation overlays with automatically generated text for the medical record. Dice similarity coefficient for segmentation was 0.87. Conclusion: We describe a machine-learning approach for ICH detection, subtype classification, and IPH-specific volume estimation. Utilizing easily interpretable heatmaps, dynamically-adjustable overlays, and automated result text generation, our pipeline is capable of providing accurate information optimized for clinical integration.