Multidimensional Parameter Spaces Research Articles

Design of multicomponent thermosetting polymers with enhanced tensile properties through active learning

Multicomponent thermosets can simultaneously achieve high strength, modulus, and toughness inaccessible by single-component materials, but efficiently designing those with exceptional mechanical properties relies on serendipity because knowledge-guided methodologies often fail in high-dimensional spaces and multi-objective problems. We proposed a multi-objective Bayesian-optimization-based active learning method enhanced by a dual-driver acquisition function to accelerate the formulation design of multicomponent epoxy resins. After several iterative experiments, we identified a multicomponent formulation of ultra-strong and high-tough samples that was much better than currently available epoxy resins. The molecular dynamic simulations and synchrotron radiation wide-angle X-ray scattering revealed that the exceptional mechanical properties of multicomponent thermosetting polymers originate from the larger meshes of crosslinking networks and the homogeneity of cured structures. The closed-loop, dual-driver Bayesian optimization strategy can be generalized for the fast discovery of high-performance materials that involve time-intensive experiments and multi-dimensional parameter spaces.

Optimizing data acquisition: a Bayesian approach for efficient machine learning model training

Acquiring a substantial number of data points for training accurate machine learning (ML) models is a big challenge in scientific fields where data collection is resource-intensive. Here, we propose a novel approach for constructing a minimal yet highly informative database for training ML models in complex multi-dimensional parameter spaces. To achieve this, we mimic the underlying relation between the output and input parameters using Gaussian process regression (GPR). Using a set of known data, GPR provides predictive means and standard deviation for the unknown data. Given the predicted standard deviation by GPR, we select data points using Bayesian optimization to obtain an efficient database for training ML models. We compare the performance of ML models trained on databases obtained through this method, with databases obtained using traditional approaches. Our results demonstrate that the ML models trained on the database obtained using Bayesian optimization approach consistently outperform the other two databases, achieving high accuracy with a significantly smaller number of data points. Our work contributes to the resource-efficient collection of data in high-dimensional complex parameter spaces, to achieve high precision ML predictions.

Optimal schedules for annealing algorithms.

Annealing algorithms such as simulated annealing and population annealing are widely used both for sampling the Gibbs distribution and solving optimization problems (i.e., finding ground states). For both statistical mechanics and optimization, additional parameters beyond temperature are often needed such as chemical potentials, external fields, or Lagrange multipliers enforcing constraints. In this paper we derive a formalism for optimal annealing schedules in multidimensional parameter spaces using methods from nonequilibrium statistical mechanics. The results are closely related to work on optimal control of thermodynamic systems [Sivak and Crooks, Phys. Rev. Lett. 108, 190602 (2012)0031-900710.1103/PhysRevLett.108.190602]. Within the formalism, we compare the efficiency of population annealing and multiple weighted runs of simulated annealing ("annealed importance sampling") and discuss the effects of nonergodicity on both algorithms. Theoretical results are supported by numerical simulations of spin glasses.

Combining evolutionary strategies and novelty detection to go beyond the alignment limit of the Z3 3HDM

We present a novel artificial intelligence approach for beyond the Standard Model parameter space scans by augmenting an evolutionary strategy with novelty detection. Our approach leverages the power of evolutionary strategies, previously shown to quickly converge to the valid regions of the parameter space, with a novelty reward to continue exploration once converged. Taking the Z3 3HDM as our physics case, we show how our methodology allows us to quickly explore highly constrained multidimensional parameter spaces, providing up to eight orders of magnitude higher sampling efficiency when compared with pure random sampling and up to four orders of magnitude when compared to random sampling around the alignment limit. In turn, this enables us to explore regions of the parameter space that have been hitherto overlooked, leading to the possibility of novel phenomenological realizations of the Z3 three Higgs doublet model that had not been considered before. Published by the American Physical Society 2024

Investigating Model Dependencies for Obscured Active Galactic Nuclei: A Case Study of NGC 3982

Abstract X-ray spectroscopy of heavily obscured active galactic nuclei (AGN) offers a unique opportunity to study the circumnuclear environment of accreting supermassive black holes. However, individual models describing the obscurer have unique parameter spaces that give distinct parameter posterior distributions when fit to the same data. To assess the impact of model-specific parameter dependencies, we present a case study of the nearby heavily obscured low-luminosity AGN NGC 3982, which has a variety of column density estimations reported in the literature. We fit the same broadband XMM-Newton+NuSTAR spectra of the source with five unique obscuration models and generate posterior parameter distributions for each. By using global parameter exploration, we traverse the full prior-defined parameter space to accurately reproduce complex posterior shapes and inter-parameter degeneracies. The unique model posteriors for the line-of-sight column density are broadly consistent, predicting Compton-thick N H > 1.5 × 1024 cm−2 at the 3σ confidence level. The posterior median intrinsic X-ray luminosity in the 2–10 keV band was found to differ substantially, however, with values in the range log L 2–10 keV/ erg s−1 = 40.9–42.1 for the individual models. We additionally show that the posterior distributions for each model occupy unique regions of their respective multidimensional parameter spaces and how such differences can propagate into the inferred properties of the central engine. We conclude by showcasing the improvement in parameter inference attainable with the High Energy X-ray Probe, with its uniquely broad, simultaneous, and high-sensitivity bandpass of 0.2–80 keV.

Toward Self-Driven Autonomous Material and Device Acceleration Platforms (AMADAP) for Emerging Photovoltaics Technologies.

ConspectusIn the ever-increasing renewable-energy demand scenario, developing new photovoltaic technologies is important, even in the presence of established terawatt-scale silicon technology. Emerging photovoltaic technologies play a crucial role in diversifying material flows while expanding the photovoltaic product portfolio, thus enhancing security and competitiveness within the solar industry. They also serve as a valuable backup for silicon photovoltaic, providing resilience to the overall energy infrastructure. However, the development of functional solar materials poses intricate multiobjective optimization challenges in a large multidimensional composition and parameter space, in some cases with millions of potential candidates to be explored. Solving it necessitates reproducible, user-independent laboratory work and intelligent preselection of innovative experimental methods.Materials acceleration platforms (MAPs) seamlessly integrate robotic materials synthesis and characterization with AI-driven data analysis and experimental design, positioning them as enabling technologies for the discovery and exploration of new materials. They are proposed to revolutionize materials development away from the Edisonian trial-and-error approaches to ultrashort cycles of experiments with exceptional precision, generating a reliable and highly qualitative data situation that allows training machine learning algorithms with predictive power. MAPs are designed to assist the researcher in multidimensional aspects of materials discovery, such as material synthesis, precursor preparation, sample processing and characterization, and data analysis, and are drawing escalating attention in the field of energy materials. Device acceleration platforms (DAPs), however, are designed to optimize functional films and layer stacks. Unlike MAPs, which focus on material discovery, a central aspect of DAPs is the identification and refinement of ideal processing conditions for a predetermined set of materials. Such platforms prove especially invaluable when dealing with "disordered semiconductors," which depend heavily on the processing parameters that ultimately define the functional properties and functionality of thin film layers. By facilitating the fine-tuning of processing conditions, DAPs contribute significantly to the advancement and optimization of disordered semiconductor devices, such as emerging photovoltaics.In this Account, we review the recent advancements made by our group in automated and autonomous laboratories for advanced material discovery and device optimization with a strong focus on emerging photovoltaics, such as solution-processing perovskite solar cells and organic photovoltaics. We first introduce two MAPs and two DAPs developed in-house: a microwave-assisted high-throughput synthesis platform for the discovery of organic interface materials, a multipurpose robot-based pipetting platform for the synthesis of new semiconductors and the characterization of thin film semiconductor composites, the SPINBOT system, which is a spin-coating DAP with the potential to optimize complex device architectures, and finally, AMANDA, a fully integrated and autonomously operating DAP. Notably, we underscore the utilization of a robot-based high-throughput experimentation technique to address the common optimization challenges encountered in extensive multidimensional composition and parameter spaces pertaining to organic and perovskite photovoltaics materials. Finally, we briefly propose a holistic concept and technology, a self-driven autonomous material and device acceleration platform (AMADAP) laboratory, for autonomous functional solar materials discovery and development. We hope to discover how AMADAP can be further strengthened and universalized with advancing development of hardware and software infrastructures in the future.

Knowledge-based modeling of simulation behavior for Bayesian optimization

Numerical simulations consist of many components that affect the simulation accuracy and the required computational resources. However, finding an optimal combination of components and their parameters under constraints can be a difficult, time-consuming and often manual process. Classical adaptivity does not fully solve the problem, as it comes with significant implementation cost and is difficult to expand to multi-dimensional parameter spaces. Also, many existing data-based optimization approaches treat the optimization problem as a black-box, thus requiring a large amount of data. We present a constrained, model-based Bayesian optimization approach that avoids black-box models by leveraging existing knowledge about the simulation components and properties of the simulation behavior. The main focus of this paper is on the stochastic modeling ansatz for simulation error and run time as optimization objective and constraint, respectively. To account for data covering multiple orders of magnitude, our approach operates on a logarithmic scale. The models use a priori knowledge of the simulation components such as convergence orders and run time estimates. Together with suitable priors for the model parameters, the model is able to make accurate predictions of the simulation behavior. Reliably modeling the simulation behavior yields a fast optimization procedure because it enables the optimizer to quickly indicate promising parameter values. We test our approach experimentally using the multi-scale muscle simulation framework OpenDiHu and show that we successfully optimize the time step widths in a time splitting approach in terms of minimizing the overall error under run time constraints.

A multidimensional objective prior distribution from a scoring rule

The construction of objective priors is, at best, challenging for multidimensional parameter spaces. A common practice is to assume independence and set up the joint prior as the product of marginal distributions obtained via “standard” objective methods, such as Jeffreys or reference priors. However, the assumption of independence a priori is not always reasonable, and whether it can be viewed as strictly objective is still open to discussion. In this paper, by extending a previously proposed objective approach based on scoring rules for the one dimensional case, we propose a novel objective prior for multidimensional parameter spaces which yields a dependence structure. The proposed prior has the appealing property of being proper and does not depend on the chosen model; only on the parameter space considered.

Nonlinear dynamics and onset of chaos in a physical model of a damper pressure relief valve

Hydraulic valves, for the purpose of targeted pressure relief and damping, are a ubiquitous part of modern suspension systems. This paper deals with the analytical computation of fixed points of the dynamical system of a single-stage shock absorber valve, which can be applied for the direct computation of its system variables at equilibrium without brute-force numerical integration. The obtained analytical expressions are given for the original dimensional version of the model derived from continuity and motion equations for a realistic valve setup. Furthermore, a large part of the study is devoted to a systematic sensitivity analysis of the model towards crucial parameter changes. Numerical investigation of a potential loss of stability and following nonlinear oscillations is performed in multi-dimensional parameter spaces, which reveals sustained valve vibrations at increased valve mass and applied pretension force. The dynamical behavior is analyzed by phase space orbits, as well as Fourier-transformed valve displacement data to identify dominant frequencies. Chaotic indicators, such as Lyapunov exponents and the Smaller Alignment Index (SALI), are utilized to understand the nature of the oscillatory motion and to distinguish between order and chaos.

First direct measurement of electron and positron bunch characteristics at the positron source of the SuperKEKB B-factory

Electron (e−) and positron (e+) bunch characteristics were directly measured for the first time by using wideband beam monitors (WBMs) and a detection system at the e+ source of the SuperKEKB B-factory. Secondarily generated e− and e+ bunches after the e+-production target were identified in their dynamical capture process at locations of the WBMs under a two-bunch acceleration scheme. The longitudinal and transverse bunch characteristics, the time intervals between the e− and e+ bunches, the bunch lengths, transverse bunch positions, and bunch charges were simultaneously separately measured for each bunch as functions of the capture phase to investigate their dynamical capture process. The results show that quite symmetric behaviors of the e− and e+ bunch characteristics were observed. The new WBMs open up a new window for direct measurements of both e− and e+ bunches during their dynamical capture process and in the optimization procedure of the e+ bunch intensity in multidimensional parameter spaces at any e+ sources.

Optimization of the injection beam line at the Cooler Synchrotron COSY using Bayesian Optimization

The complex non-linear processes in multi-dimensional parameter spaces, that are typical for an accelerator, are a natural application for machine learning algorithms. This paper reports on the use of Bayesian optimization for the optimization of the Injection Beam Line (IBL) of the Cooler Synchrotron storage ring COSY at the Forschungszentrum Jülich, Germany. Bayesian optimization is a machine learning method that optimizes a continuous objective function using limited observations. The IBL is composed of 15 quadrupoles and 28 steerers. The goal is to increase the beam intensity inside the storage ring. The results showed the effectiveness of the Bayesian optimization in achieving better/faster results compared to manual optimization.

Efficiently exploring multidimensional parameter spaces beyond the Standard Model

We propose a method to ease the challenges of exploring multidimensional parameter spaces in beyond-the-Standard Model theories. We evaluate the model likelihood for any choice of parameters by sampling the theory parameters intelligently and building a kernel density estimator. By reducing the number of expensive Monte Carlo simulations, this method provides an efficient way to test complex theories. We illustrate our technique to set new limits on a short-lived heavy neutrino $N$, proposed as an explanation of anomalies in neutrino experiments. Using a search for lepton pairs in the T2K near detector, we find exclusion limits on the model parameters in a vast region of parameter space, fully exploiting the advantages of our new method. With a single Monte Carlo simulation, we obtain the differential event rate for arbitrary choices of model parameters, allowing us to cast limits on any slice of the model parameter space. We conclude that $N$ particles with lifetimes greater than $c{\ensuremath{\tau}}^{0}\ensuremath{\gtrsim}3\text{ }\text{ }\mathrm{cm}$ are excluded by T2K data. We also derive model-independent constraints in terms of the total rate, lifetime, and $N$ mass and provide an approximated analytical formula. This method can be applied in other branches of physics to explore the landscape of theory parameters efficiently.

Exploring parameter spaces with artificial intelligence and machine learning black-box optimization algorithms

Constraining beyond the Standard Model theories usually involves scanning highly multidimensional parameter spaces and checking observable predictions against experimental bounds and theoretical constraints. Such a task is often timely and computationally expensive, especially when the model is severely constrained and thus leading to very low random sampling efficiency. In this work we tackled this challenge using artificial intelligence and machine learning search algorithms used for black-box optimization problems. Using the constrained minimal supersymmetric standard model and the phenomenological minimal supersymmetric standard model parameter spaces, we consider both the Higgs mass and the dark matter relic density constraints to study their sampling efficiency and parameter space coverage. We find our methodology to produce orders of magnitude improvement of sampling efficiency while reasonably covering the parameter space.

Software tools for neutron wavelength-resolved Laue diffraction in multi-dimensional diffraction and parameter spaces

Software tools for neutron wavelength-resolved Laue diffraction in multi-dimensional diffraction and parameter spaces

Self-Optimization of Continuous Flow Electrochemical Synthesis Using Fourier Transform Infrared Spectroscopy and Gas Chromatography.

A continuous-flow electrochemical synthesis platform has been developed to enable self-optimization of reaction conditions of organic electrochemical reactions using attenuated total reflection Fourier transform infrared spectroscopy (ATR FT-IR) and gas chromatography (GC) as online real-time monitoring techniques. We have overcome the challenges in using ATR FT-IR as the downstream analytical methods imposed when a large amount of hydrogen gas is produced from the counter electrode by designing two types of gas-liquid separators (GLS) for analysis of the product mixture flowing from the electrochemical reactor. In particular, we report an integrated GLS with an ATR FT-IR probe at the reactor outlet to give a facile and low-cost solution to determining the concentrations of products in gas-liquid two-phase flow. This approach provides a reliable method for quantifying low-volatile analytes, which can be problematic to be monitored by GC. Two electrochemical reactions the methoxylation of 1-formylpyrrolidine and the oxidation of 3-bromobenzyl alcohol were investigated to demonstrate that the optimal conditions can be located within the pre-defined multi-dimensional reaction parameter spaces without intervention of the operator by using the stable noisy optimization by branch and FIT (SNOBFIT) algorithm.

Efficient exploration and calibration of a semi-analytical model of galaxy formation with deep learning

ABSTRACT We implement a sample-efficient method for rapid and accurate emulation of semi-analytical galaxy formation models over a wide range of model outputs. We use ensembled deep learning algorithms to produce a fast emulator of an updated version of the galform model from a small number of training examples. We use the emulator to explore the model’s parameter space, and apply sensitivity analysis techniques to better understand the relative importance of the model parameters. We uncover key tensions between observational data sets by applying a heuristic weighting scheme in a Markov chain Monte Carlo framework and exploring the effects of requiring improved fits to certain data sets relative to others. Furthermore, we demonstrate that this method can be used to successfully calibrate the model parameters to a comprehensive list of observational constraints. In doing so, we re-discover previous galform fits in an automatic and transparent way, and discover an improved fit by applying a heavier weighting to the fit to the metallicities of early-type galaxies. The deep learning emulator requires a fraction of the model evaluations needed in similar emulation approaches, achieving an out-of-sample mean absolute error at the knee of the K-band luminosity function of 0.06 dex with less than 1000 model evaluations. We demonstrate that this is an extremely efficient, inexpensive, and transparent way to explore multidimensional parameter spaces, and can be applied more widely beyond semi-analytical galaxy formation models.

Exploration of lattice Hamiltonians for functional and structural discovery via Gaussian process-based exploration–exploitation

Statistical physics models ranging from simple lattice to complex quantum Hamiltonians are one of the mainstays of modern physics that have allowed both decades of scientific discovery and provided a universal framework to understand a broad range of phenomena from alloying to frustrated and phase separated materials to quantum systems. Traditionally, exploration of the phase diagrams corresponding to multidimensional parameter spaces of Hamiltonians was performed using a combination of basic physical principles, analytical approximations, and extensive numerical modeling. However, exploration of complex multidimensional parameter spaces is subject to the classic dimensionality problem, and the behaviors of interest concentrated on low dimensional manifolds remain undiscovered. Here, we demonstrate that a combination of exploration and exploration–exploitation with Gaussian process modeling and Bayesian optimization allows effective exploration of the parameter space for lattice Hamiltonians and effectively maps the regions at which specific macroscopic functionalities or local structures are maximized. We argue that this approach is general and can be further extended well beyond the lattice Hamiltonians to effectively explore the parameter space of more complex off-lattice and dynamic models.

Autonomous materials discovery driven by Gaussian process regression with inhomogeneous measurement noise and anisotropic kernels

A majority of experimental disciplines face the challenge of exploring large and high-dimensional parameter spaces in search of new scientific discoveries. Materials science is no exception; the wide variety of synthesis, processing, and environmental conditions that influence material properties gives rise to particularly vast parameter spaces. Recent advances have led to an increase in the efficiency of materials discovery by increasingly automating the exploration processes. Methods for autonomous experimentation have become more sophisticated recently, allowing for multi-dimensional parameter spaces to be explored efficiently and with minimal human intervention, thereby liberating the scientists to focus on interpretations and big-picture decisions. Gaussian process regression (GPR) techniques have emerged as the method of choice for steering many classes of experiments. We have recently demonstrated the positive impact of GPR-driven decision-making algorithms on autonomously-steered experiments at a synchrotron beamline. However, due to the complexity of the experiments, GPR often cannot be used in its most basic form, but rather has to be tuned to account for the special requirements of the experiments. Two requirements seem to be of particular importance, namely inhomogeneous measurement noise (input-dependent or non-i.i.d.) and anisotropic kernel functions, which are the two concepts that we tackle in this paper. Our synthetic and experimental tests demonstrate the importance of both concepts for experiments in materials science and the benefits that result from including them in the autonomous decision-making process.

Backtesting of Algorithmic Cryptocurrency Trading Strategies

This thesis presents a tool for backtesting algorithmic trading strategies for cryptocurrencies. The tool, called quantbacktest, provides a convenient way to automatically run comparisons of multi-dimensional parameter spaces for algorithmic trading strategies. The tool supports any algorithmic strategy to be simulated and any parameter spaces to be tested and optimized with minimal adjustments. Also, arbitrary trading frequencies can be tested, from intraday to long-term strategies. Many standard return metrics, risk metrics, and robustness test functionalities come out-of-the-box in CSV format and as diagrams. Users can provide signals and price data via CSV or Excel files. Signal processing does not require a technical (code-level) understanding of the backtesting tool on behalf of the user.

Reliable Surrogate Modeling of Antenna Input Characteristics by Means of Domain Confinement and Principal Components

A reliable design of contemporary antenna structures necessarily involves full-wave electromagnetic (EM) analysis which is the only tool capable of accounting, for example, for element coupling or the effects of connectors. As EM simulations tend to be CPU-intensive, surrogate modeling allows for relieving the computational overhead of design tasks that require numerous analyses, for example, parametric optimization or uncertainty quantification. Notwithstanding, conventional data-driven surrogates are not suitable for handling highly nonlinear antenna characteristics over multidimensional parameter spaces. This paper proposes a novel modeling approach that employs a recently introduced concept of domain confinement, as well as principal component analysis. In our approach, the modeling process is restricted to the region containing high-quality designs with respect to the performance figures of antennas under design, identified using a set of pre-optimized reference designs. The model domain is spanned by the selected principal components of the reference design set, which reduces both its volume and dimensionality. As a result, a reliable surrogate can be constructed over wide ranges of both operating conditions and antenna parameters, using small training datasets. Our technique is demonstrated using two antenna examples and is favorably compared to both conventional and constrained modeling approaches. Application case studies (antenna optimization) are also discussed.