Black-box Algorithms Research Articles

A multiscale computational framework for the development of spines in molluscan shells.

From mathematical models of growth to computer simulations of pigmentation, the study of shell formation has given rise to an abundant number of models, working at various scales. Yet, attempts to combine those models have remained sparse, due to the challenge of combining categorically different approaches. In this paper, we propose a framework to streamline the process of combining the molecular and tissue scales of shell formation. We choose these levels as a proxy to link the genotype level, which is better described by molecular models, and the phenotype level, which is better described by tissue-level mechanics. We also show how to connect observations on shell populations to the approach, resulting in collections of molecular parameters that may be associated with different populations of real shell specimens. The approach is as follows: we use a Quality-Diversity algorithm, a type of black-box optimization algorithm, to explore the range of concentration profiles emerging as solutions of a molecular model, and that define growth patterns for the mechanical model. At the same time, the mechanical model is simulated over a wide range of growth patterns, resulting in a variety of spine shapes. While time-consuming, these steps only need to be performed once and then function as look-up tables. Actual pictures of shell spines can then be matched against the list of existing spine shapes, yielding a potential growth pattern which, in turn, gives us matching molecular parameters. The framework is modular, such that models can be easily swapped without changing the overall working of the method. As a demonstration of the approach, we solve specific molecular and mechanical models, adapted from available theoretical studies on molluscan shells, and apply the multiscale framework to evaluate the characteristics of spines from three distinct populations of Turbo sazae.

A general explicable forecasting framework for weather events based on ordinal classification and inductive rules combined with fuzzy logic

This paper presents a method for providing explainability in the integration of artificial intelligence (AI) and data mining techniques when dealing with meteorological prediction. Explainable artificial intelligence (XAI) refers to the transparency of AI systems in providing explanations for their predictions and decision-making processes, and contribute to improve prediction accuracy and enhance trust in AI systems. The focus of this paper relies on the interpretability challenges in ordinal classification problems within weather forecasting. Ordinal classification involves predicting weather phenomena with ordered classes, such as temperature ranges, wind speed, precipitation levels, and others. To address this challenge, a novel and general explicable forecasting framework, that combines inductive rules and fuzzy logic, is proposed in this work. Inductive rules, derived from historical weather data, provide a logical and interpretable basis for forecasting; while fuzzy logic handles the uncertainty and imprecision in the weather data. The system predicts a set of probabilities that the incoming sample belongs to each considered class. Moreover, it allows the expert decision-making process to be strengthened by relying on the transparency and physical explainability of the model, and not only on the output of a black-box algorithm. The proposed framework is evaluated using two real-world weather databases related to wind speed and low-visibility events due to fog. The results are compared to both ML classifiers and specific methods for ordinal classification problems, achieving very competitive results in terms of ordinal performance metrics while offering a higher level of explainability and transparency compared to existing approaches.

Crossover can guarantee exponential speed-ups in evolutionary multi-objective optimisation

Evolutionary algorithms are popular algorithms for multi-objective optimisation (also called Pareto optimisation) as they use a population to store trade-offs between different objectives. Despite their popularity, the theoretical foundation of multi-objective evolutionary optimisation (EMO) is still in its early development. Fundamental questions such as the benefits of the crossover operator are still not fully understood. We provide a theoretical analysis of the well-known EMO algorithms GSEMO and NSGA-II to showcase the possible advantages of crossover: we propose classes of “royal road” functions on which these algorithms cover the whole Pareto front in expected polynomial time if crossover is being used. But when disabling crossover, they require exponential time in expectation to cover the Pareto front. The latter even holds for a large class of black-box algorithms using any elitist selection and any unbiased mutation operator. Moreover, even the expected time to create a single Pareto-optimal search point is exponential. We provide two different function classes, one tailored for one-point crossover and another one tailored for uniform crossover, and we show that some immune-inspired hypermutations cannot avoid exponential optimisation times. Our work shows the first example of an exponential performance gap through the use of crossover for the widely used NSGA-II algorithm and contributes to a deeper understanding of its limitations and capabilities.

A DEIM Tucker tensor cross algorithm and its application to dynamical low-rank approximation

We introduce a Tucker tensor cross approximation method that constructs a low-rank representation of a d-dimensional tensor by sparsely sampling its fibers. These fibers are selected using the discrete empirical interpolation method (DEIM). Our proposed algorithm is referred to as DEIM fiber sampling (DEIM-FS). For a rank-r approximation of an O(Nd) tensor, DEIM-FS requires access to only dNrd−1 tensor entries, a requirement that scales linearly with the tensor size along each mode. We demonstrate that DEIM-FS achieves an approximation accuracy close to the Tucker-tensor approximation obtained via higher-order singular value decomposition at a significantly reduced cost. We also present DEIM-FS (iterative) that does not require access to singular vectors of the target tensor unfolding and can be viewed as a black-box Tucker tensor algorithm. We employ DEIM-FS to reduce the computational cost associated with solving nonlinear tensor differential equations (TDEs) using dynamical low-rank approximation (DLRA). The computational cost of solving DLRA equations can become prohibitive when the exact rank of the right-hand side tensor is large. This issue arises in many TDEs, especially in cases involving non-polynomial nonlinearities, where the right-hand side tensor has full rank. This necessitates the storage and computation of tensors of size O(Nd). We show that DEIM-FS results in significant computational savings for DLRA by constructing a low-rank Tucker approximation of the right-hand side tensor on the fly. Another advantage of using DEIM-FS is to significantly simplify the implementation of DLRA equations, irrespective of the type of TDEs. We demonstrate the efficiency of the algorithm through several examples including solving high-dimensional partial differential equations.

BlastAssist: a deep learning pipeline to measure interpretable features of human embryos.

BlastAssist: a deep learning pipeline to measure interpretable features of human embryos.

Unsupervised framework for evaluating and explaining structural node embeddings of graphs

Abstract An embedding is a mapping from a set of nodes of a network into a real vector space. Embeddings can have various aims like capturing the underlying graph topology and structure, node-to-node relationship, or other relevant information about the graph, its subgraphs or nodes themselves. A practical challenge with using embeddings is that there are many available variants to choose from. Selecting a small set of most promising embeddings from the long list of possible options for a given task is challenging and often requires domain expertise. Embeddings can be categorized into two main types: classical embeddings and structural embeddings. Classical embeddings focus on learning both local and global proximity of nodes, while structural embeddings learn information specifically about the local structure of nodes’ neighbourhood. For classical node embeddings, there exists a framework which helps data scientists to identify (in an unsupervised way) a few embeddings that are worth further investigation. Unfortunately, no such framework exists for structural embeddings. In this article, we propose a framework for unsupervised ranking of structural graph embeddings. The proposed framework, apart from assigning an aggregate quality score for a structural embedding, additionally gives a data scientist insights into properties of this embedding. It produces information which predefined node features the embedding learns, how well it learns them, and which dimensions in the embedded space represent the predefined node features. Using this information, the user gets a level of explainability to an otherwise complex black-box embedding algorithm.

Validating EmotiBit, an open-source multi-modal sensor for capturing research-grade physiological signals from anywhere on the body

Peripheral physiological signals provide a powerful window into understanding the body and mind. Peripheral physiology devices that are currently available on the market, however, face a number of challenges. Consumer-grade devices (e.g. FitBit, Apple Watch) are easy to wear, but provide limited access to unprocessed data. Combined with black-box signal processing algorithms, this makes it difficult to interpret the data for scientific research purposes. Research-grade devices (e.g. Empatica, Shimmer, BIOPAC) provide greater access to high-quality data but remain in closed ecosystems and at price points that are out of reach for many. To bridge the gaps in available biometric solutions, our labs have created an open-source physiological sensing platform called EmotiBit (http://www.emotibit.com/), measuring EDA, multi-wavelength PPG, temperature, and 9-axis IMU. This study compares EmotiBit biometric signals to gold-standard devices by Brain Products and finds that the physiological signals exhibit a high degree of similarity, validating their use in research.

Adaptation of the Scaling Factor Based on the Success Rate in Differential Evolution

Differential evolution is a popular heuristic black-box numerical optimization algorithm which is often used due to its simplicity and efficiency. Parameter adaptation is one of the main directions of study regarding the differential evolution algorithm. The main reason for this is that differential evolution is highly sensitive to the scaling factor and crossover rate parameters. In this study, a novel adaptation technique is proposed which uses the success rate to replace the popular success history-based adaptation for scaling factor tuning. In particular, the scaling factor is sampled with a Cauchy distribution, whose location parameter is set as an nth order root of the current success rate, i.e., the ratio of improved solutions to the current population size. The proposed technique is universal and can be applied to any differential evolution variant. Here it is tested with several state-of-the-art variants of differential evolution, and on two benchmark sets, CEC 2017 and CEC 2022. The performed experiments, which include modifications of algorithms developed by other authors, show that in many cases using the success rate to determine the scaling factor can be beneficial, especially with relatively small computational resource.

Trusting AI made decisions in healthcare by making them explainable.

In solving the trust issues surrounding machine learning algorithms whose reasoning cannot be understood, advancements can be made toward the integration of machine learning algorithms into mHealth applications. The aim of this paper is to provide a transparency layer to black-box machine learning algorithms and empower mHealth applications to maximize their efficiency. Using a machine learning testing framework, we present the process of knowledge transfer between a white-box model and a black-box model and the evaluation process to validate the success of the knowledge transfer. The presentation layer of the final output of the base white-box model and the knowledge-infused white-box model shows clear differences in reasoning. The correlation between the base black-box model and the new knowledge-infused model is very high, indicating that the knowledge transfer was successful. There is a clear need for transparency in digital health and health in general. Adding solutions to the toolbox of explainable artificial intelligence is one way to gradually decrease the obscurity of black-box models.

Look inside 3D point cloud deep neural network by patch-wise saliency map

The 3D point cloud deep neural network (3D DNN) has achieved remarkable success, but its black-box nature hinders its application in many safety-critical domains. The saliency map technique is a key method to look inside the black-box and determine where a 3D DNN focuses when recognizing a point cloud. Existing point-wise point cloud saliency methods are proposed to illustrate the point-wise saliency for a given 3D DNN. However, the above critical points are alternative and unreliable. The findings are grounded on our experimental results which show that a point becomes critical because it is responsible for representing one specific local structure. However, one local structure does not have to be represented by some specific points, conversely. As a result, discussing the saliency of the local structure (named patch-wise saliency) represented by critical points is more meaningful than discussing the saliency of some specific points. Based on the above motivations, this paper designs a black-box algorithm to generate patch-wise saliency map for point clouds. Our basic idea is to design the Mask Building-Dropping process, which adaptively matches the size of important/unimportant patches by clustering points with close saliency. Experimental results on several typical 3D DNNs show that our patch-wise saliency algorithm can provide better visual guidance, and can detect where a 3D DNN is focusing more efficiently than a point-wise saliency map. Finally, we apply our patch-wise saliency map to adversarial attacks and backdoor defenses. The results show that the improvement is significant.

Multispecies Benchmark Analysis for LC-MS/MS Validation and Performance Evaluation in Bottom-Up Proteomics.

We present an instrument-independent benchmark procedure and software (LFQ_bout) for the validation and comparative evaluation of the performance of LC-MS/MS and data processing workflows in bottom-up proteomics. The procedure enables a back-to-back comparison of common and emerging workflows, e.g., diaPASEF or ScanningSWATH, and evaluates the impact of arbitrary and inadequately documented settings or black-box data processing algorithms. It enhances the overall performance and quantification accuracy by recognizing and reporting common quantification errors.

Healthy Mistrust: Medical Black Box Algorithms, Epistemic Authority, and Preemptionism.

In the ethics of algorithms, a specifically epistemological analysis is rarely undertaken in order to gain a critique (or a defense) of the handling of or trust in medical black box algorithms (BBAs). This article aims to begin to fill this research gap. Specifically, the thesis is examined according to which such algorithms are regarded as epistemic authorities (EAs) and that the results of a medical algorithm must completely replace other convictions that patients have (preemptionism). If this were true, it would be a reason to distrust medical BBAs. First, the author describes what EAs are and why BBAs can be considered EAs. Then, preemptionism will be outlined and criticized as an answer to the question of how to deal with an EA. The discussion leads to some requirements for dealing with a BBA as an EA.

Machine Learning as a Tool for Hypothesis Generation

Abstract While hypothesis testing is a highly formalized activity, hypothesis generation remains largely informal. We propose a systematic procedure to generate novel hypotheses about human behavior, which uses the capacity of machine learning algorithms to notice patterns people might not. We illustrate the procedure with a concrete application: judge decisions about whom to jail. We begin with a striking fact: the defendant’s face alone matters greatly for the judge’s jailing decision. In fact, an algorithm given only the pixels in the defendant’s mug shot accounts for up to half of the predictable variation. We develop a procedure that allows human subjects to interact with this black-box algorithm to produce hypotheses about what in the face influences judge decisions. The procedure generates hypotheses that are both interpretable and novel: they are not explained by demographics (e.g., race) or existing psychology research, nor are they already known (even if tacitly) to people or experts. Though these results are specific, our procedure is general. It provides a way to produce novel, interpretable hypotheses from any high-dimensional data set (e.g., cell phones, satellites, online behavior, news headlines, corporate filings, and high-frequency time series). A central tenet of our article is that hypothesis generation is a valuable activity, and we hope this encourages future work in this largely “prescientific” stage of science.

An efficient method for generalised Wiener series estimation of nonlinear systems using Gaussian processes

System identification of dynamical systems aims to predict the output of a system for a given input by inferring model details from data. This is particularly challenging for nonlinear vibrating systems with long memory, which often have complex dynamics for which the physics is not fully understood. However, in many engineering applications, it is important to capture as much of the nonlinearity in the response as possible. The Wiener series is a classical data-driven technique that decomposes the system response into a set of orthogonal functionals of increasing order. Unlike standard black-box algorithms, such as neural networks, the series is highly interpretable and can offer insight into the nonlinearities present. However, current methods require vast quantities of data to calculate higher order terms in the Wiener series. In this paper, a novel generalised formulation of the Wiener series is developed by framing the series explicitly as a QR decomposition. Then by placing a Gaussian process prior over the kernels, their inherent structure is exploited to significantly reduce the quantity of data required for inference. Initial investigations were carried out using two systems: first pre-defined kernels and second an idealised nonlinear oscillatory system with unknown kernels. It is demonstrated that the Gaussian Process Generalised Wiener Series (GPGWS) enables the calculation of up to and including the third order kernel for systems with long memory and improves the robustness to noise.

On the Evaluation of Coal Strength Alteration Induced by CO2 Injection Using Advanced Black-Box and White-Box Machine Learning Algorithms

Summary The injection of carbon dioxide (CO2) into coal seams is a prominent technique that can provide carbon sequestration in addition to enhancing coalbed methane extraction. However, CO2 injection into the coal seams can alter the coal strength properties and their long-term integrity. In this work, the strength alteration of coals induced by CO2 exposure was modeled using 147 laboratory-measured unconfined compressive strength (UCS) data points and considering CO2 saturation pressure, CO2 interaction temperature, CO2 interaction time, and coal rank as input variables. Advanced white-box and black-box machine learning algorithms including Gaussian process regression (GPR) with rational quadratic kernel, extreme gradient boosting (XGBoost), categorical boosting (CatBoost), adaptive boosting decision tree (AdaBoost-DT), multivariate adaptive regression splines (MARS), K-nearest neighbor (KNN), gene expression programming (GEP), and group method of data handling (GMDH) were used in the modeling process. The results demonstrated that GPR-Rational Quadratic provided the most accurate estimates of UCS of coals having 3.53%, 3.62%, and 3.55% for the average absolute percent relative error (AAPRE) values of the train, test, and total data sets, respectively. Also, the overall determination coefficient (R2) value of 0.9979 was additional proof of the excellent accuracy of this model compared with other models. Moreover, the first mathematical correlations to estimate the change in coal strength induced by CO2 exposure were established in this work by the GMDH and GEP algorithms with acceptable accuracy. Sensitivity analysis revealed that the Spearman correlation coefficient shows the relative importance of the input parameters on the coal strength better than the Pearson correlation coefficient. Among the inputs, coal rank had the greatest influence on the coal strength (strong nonlinear relationship) based on the Spearman correlation coefficient. After that, CO2 interaction time and CO2 saturation pressure have shown relatively strong nonlinear relationships with model output, respectively. The CO2 interaction temperature had the smallest impact on coal strength alteration induced by CO2 exposure based on both Pearson and Spearman correlation coefficients. Finally, the leverage technique revealed that the laboratory database used for modeling CO2-induced strength alteration of coals was highly reliable, and the suggested GPR-Rational Quadratic model and GMDH correlation could be applied for predicting the UCS of coals exposed to CO2 with high statistical accuracy and reliability.

Feel the force: From local surface pressure measurement to flow reconstruction in fluid–structure interaction

Drawing inspiration from the lateral lines of fish, the inference of flow characteristics via surface-based data has drawn considerable attention. The current approaches often rely on analytical methods tailored exclusively for potential flows or utilize black-box machine learning algorithms to estimate a specific set of flow parameters. In contrast to a black-box machine learning approach, we demonstrate that it is possible to identify certain modes of fluid flow and then reconstruct the entire flow field from these modes. We use dynamic mode decomposition to parametrize complex, dynamic features across the entire flow field. We then leverage deep neural networks to infer the dynamic modes of the pressure and velocity fields within a large, unsteady flow domain, employing solely a time series of pressure measurements collected on the surface of an immersed obstacle. Our methodology is successfully demonstrated to diverse fluid–structure interaction scenarios, including cases with both free oscillations in the wake of a cylinder and forced oscillations of tandem cylinders, demonstrating its versatility and robustness.

Transparency in Algorithmic Decision-making: Interpretable Models for Ethical Accountability

Concerns regarding their opacity and potential ethical ramifications have been raised by the spread of algorithmic decisionmaking systems across a variety of fields. By promoting the use of interpretable machine learning models, this research addresses the critical requirement for openness and moral responsibility in these systems. Interpretable models provide a transparent and intelligible depiction of how decisions are made, as opposed to complicated black-box algorithms. Users and stakeholders need this openness in order to understand, verify, and hold accountable the decisions made by these algorithms. Furthermore, interpretability promotes fairness in algorithmic results by making it easier to detect and reduce biases. In this article, we give an overview of the difficulties brought on by algorithmic opacity, highlighting how crucial it is to solve these difficulties in a variety of settings, including those involving healthcare, banking, criminal justice, and more. From linear models to rule-based systems to surrogate models, we give a thorough analysis of interpretable machine learning techniques, highlighting their benefits and drawbacks. We suggest that incorporating interpretable models into the design and use of algorithms can result in a more responsible and moral application of AI in society, ultimately benefiting people and communities while lowering the risks connected to opaque decision-making processes.

Diversity-aware fairness testing of machine learning classifiers through hashing-based sampling

Context:There are growing concerns about algorithmic fairness, as some machine learning (ML)-based algorithms have been found to exhibit biases against protected attributes such as gender, race, age and so on. Individual fairness requires an ML classifier to produce similar outputs for similar individuals. Verification Based Testing (Vbt) is a state-of-the-art black-box testing algorithm for individual fairness that leverages constraint solving to generate test cases. Objective:Generating diverse test cases is expected to facilitate efficient detection of diverse discriminatory data instances (i.e., cases that violate individual fairness). Hashing-based sampling techniques draw a sample approximately uniformly at random from the set of solutions of given Boolean constraints. We propose Vbt-X, which improves Vbt with hashing-based sampling, aiming to improve its testing performance. Method:We realize hashing-based sampling for Vbt. The challenge is that the off-the-shelf hashing-based sampling techniques cannot be integrated in a straightforward manner because the constraints in Vbt are generally not Boolean. Moreover, we propose several enhancement techniques to make Vbt-X more efficient. Results:To evaluate our method, we conduct experiments, where Vbt-X is compared to Vbt, Sg and ExpGA (other well-known fairness testing algorithms) over a set of configurations consisting of several datasets, protected attributes, and ML classifiers. The results show that, with each configuration, Vbt-X detects more discriminatory data instances with higher diversity than Vbt and Sg. Vbt-X detects discriminatory data instances with higher diversity than ExpGA, though the number of discriminatory data instances detected by Vbt-X is lesser than ExpGA. Conclusion:Our proposed method performs better than other state-of-the-art black-box fairness testing algorithms, particularly in terms of diversity. Our method can serve to efficiently identify flaws in ML classifiers with respect to individual fairness for subsequent improvements of an ML classifier. On the other hand, although our method is specific to individual fairness, it could work for testing other aspects of a software system such as security and counterfactual explanations with some technical adaptations, which remains for future work.

Instance Weighted Incremental Evolution Strategies for Reinforcement Learning in Dynamic Environments.

Evolution strategies (ESs), as a family of black-box optimization algorithms, recently emerge as a scalable alternative to reinforcement learning (RL) approaches such as Q-learning or policy gradient and are much faster when many central processing units (CPUs) are available due to better parallelization. In this article, we propose a systematic incremental learning method for ES in dynamic environments. The goal is to adjust previously learned policy to a new one incrementally whenever the environment changes. We incorporate an instance weighting mechanism with ES to facilitate its learning adaptation while retaining scalability of ES. During parameter updating, higher weights are assigned to instances that contain more new knowledge, thus encouraging the search distribution to move toward new promising areas of parameter space. We propose two easy-to-implement metrics to calculate the weights: instance novelty and instance quality. Instance novelty measures an instance's difference from the previous optimum in the original environment, while instance quality corresponds to how well an instance performs in the new environment. The resulting algorithm, instance weighted incremental evolution strategies (IW-IESs), is verified to achieve significantly improved performance on challenging RL tasks ranging from robot navigation to locomotion. This article thus introduces a family of scalable ES algorithms for RL domains that enables rapid learning adaptation to dynamic environments.

Machine learning-based clinical decision support using laboratory data.

Artificial intelligence (AI) and machine learning (ML) are becoming vital in laboratory medicine and the broader context of healthcare. In this review article, we summarized the development of ML models and how they contribute to clinical laboratory workflow and improve patient outcomes. The process of ML model development involves data collection, data cleansing, feature engineering, model development, and optimization. These models, once finalized, are subjected to thorough performance assessments and validations. Recently, due to the complexity inherent in model development, automated ML tools were also introduced to streamline the process, enabling non-experts to create models. Clinical Decision Support Systems (CDSS) use ML techniques on large datasets to aid healthcare professionals in test result interpretation. They are revolutionizing laboratory medicine, enabling labs to work more efficiently with less human supervision across pre-analytical, analytical, and post-analytical phases. Despite contributions of the ML tools at all analytical phases, their integration presents challenges like potential model uncertainties, black-box algorithms, and deskilling of professionals. Additionally, acquiring diverse datasets is hard, and models' complexity can limit clinical use. In conclusion, ML-based CDSS in healthcare can greatly enhance clinical decision-making. However, successful adoption demands collaboration among professionals and stakeholders, utilizing hybrid intelligence, external validation, and performance assessments.